Query Transformation for Structured Data
What you'll learn: How to convert natural language questions into SQL queries, filters, or other structured formats so your RAG system can retrieve from databases, not just documents.
Why This Matters
So far, you've worked with unstructured text—documents embedded as vectors. But what if your data lives in a SQL database, a graph database, or a metadata-filtered catalog? Natural language questions like "Show me sales over $10,000 from Q3 2023" need translation into structured queries the database understands.
The Core Idea
Query transformation means converting a user's natural language question into a machine-executable query format. For structured data sources:
- SQL databases: Transform to
SELECT * FROM sales WHERE amount > 10000 AND date BETWEEN ... - Metadata filters: Transform to
{"region": "US", "year": 2023} - Graph queries: Transform to Cypher or similar query languages
Think of it as a translator: The LLM speaks English, but your database only speaks SQL. Query transformation builds that bridge.
How It Works
- Analyze the question to identify filters, aggregations, or joins
- Map entities to table names, column names, or metadata fields
- Generate the structured query (often using an LLM with schema context)
- Validate and execute the query against the database
- Return results to the generation phase
For example, combining this with query routing (from lesson 2019): if the user asks about product specs, route to your vector database; if they ask for inventory counts, route to SQL with query transformation.
When to Use It
- Your RAG system needs to access relational databases, data warehouses, or knowledge graphs
- Questions involve filters, aggregations, or structured lookups
- You want to combine structured retrieval with semantic search (hybrid approach)
Key Takeaway: Query transformation lets RAG systems query structured data sources by converting natural language into SQL, filters, or other machine-readable formats—essential for real-world enterprise RAG where data isn't always text.