Large Language Models (LLMs) excel as general-purpose tools but often require contextual data for effective enterprise application. Flink SQL facilitates this integration by providing access to structured and unstructured data. Structured data can be enriched for LLMs using SQL joins or lookup joins, efficiently retrieving information from external sources while caching results. For unstructured data, retrieval-augmented generation (RAG) uses embeddings stored in vector databases. Flink SQL supports arrays for vector handling, enabling relevant domain-specific information to improve LLM output.
In building AI agents, Flink could be used to summarize research papers and connect them to internal projects through two streaming SQL jobs—updating embeddings in a vector store and retrieving relevant documents. However, current Flink SQL lacks built-in support for some advanced features. To meet this need, user-defined functions (UDFs), particularly Process Table Functions (PTFs), allow for customizable integration, potentially enriching the agent’s capabilities by incorporating external services and tools.
Source link
