Semantic data aggregation with BigQuery and AI.AGG | Google Cloud Blog

Multi-level batching architecture

AI.AGG abstracts the immense engineering complexity of chunking and parallel execution. When you execute an AI.AGG query, the BigQuery engine automatically orchestrates a sequence natively within the database:

1. Row-level validation: The engine helps verify no single row exceeds the maximum context window (dropping or returning an error on payloads >10 MiB).

2. Context-safe batching: The engine dynamically groups input rows into smaller batches that safely fit within the token limits of your chosen Gemini endpoint.

3. Parallel inference: Your natural-language SQL prompt is applied to each batch in parallel, generating intermediate semantic summaries via Vertex AI.

4. Recursive synthesis: Finally, the system batches these intermediate outputs together and passes them back to the LLM with the exact same instruction. This reduction repeats recursively until a single, final aggregated string is produced per SQL GROUP BY clause.

Because AI.AGG requires…