Lumilake: An Agentic Analytics Engine for AI4Science

In the emerging agentic analytics, the fundamental unit of computation is no longer a SQL query but an agentic workflow: a DAG of agent operations that compose SQL queries, object fetches, LLM/VLM invocations, and multi-round subagent interactions. These workflows are computationally explosive—a single seven-node financial-analysis template applied to 160 stock symbols generates over 800 GPU-resident LLM/VLM calls with overlapping prefixes—yet existing systems treat each call in isolation. We present Lumilake, an agentic analytics engine that models workflows as first-class, introspectable DAGs and addresses three challenges: (i) LLM-aware query optimization that consolidates shared prefixes to maximize KV-cache reuse, (ii) a multi-tenant runtime that dispatches heterogeneous operators across mixed hardware with cross-tenant deduplication, and (iii) governance by design with semantic lineage at every agent step. Deployed at the National University of Singapore, Lumilake achieves up to 3.8× speedup over Ray on six real workflows from clinical decision support, materials spectroscopy, and financial analysis on commodity GPUs. Our engine is open-sourced at https://github.com/mlsys-io/lumilake.