All videos
0:00 / 0:00
ai

Unlocking agentic data engineering with Lakeflow + Genie

Databricks24 June 2026Watch on YouTube

Part of series

Ep. 9 · Customerlake Databricks Agentic

View the series

Description

Data engineering is a messy process, and the growth of AI apps and agents is adding more complexity. The data that AI needs is stored all over the enterprise. Previously, companies had to rely on brittle, expensive pipelines to move this information. But Lakeflow make it easy to turn raw data stored in these systems into insights and action. At the 2026 Data + AI Summit, Bilal Aslam, Sr. Director of Product Management at Databricks, shared updates to Lakeflow and introduced Genie ZeroOps, an AI agents for data and AI operations. Learn more: https://www.databricks.com/blog/introducing-genie-zeroops 00:00 —Data engineering and the "messy middle" 02:21 — Simpliyfing the data stack 03:18 — Building an open data foundation for AI with Lakeflow 04:50 — Now available: Lakeflow Designer 05:40 — Lakeflow Connect's growing ecosystem 06:18 — Ditching Kafka with Zerobus Ingest 06:57 — Updates to Lakeflow Jobs 08:47 — Challenges to using AI in data operations 12:34 — Introducing Genie ZeroOps with live demo 17:23 — What Genie ZeroOps means for data engineers Presenters: Bilal Aslam, Sr. Director of Product Management, Databricks

What you'll learn

  • Lakeflow simplifies data integration by transforming raw data from various enterprise systems into actionable insights without relying on brittle, expensive pipelines.
  • Genie ZeroOps is an AI agent that automates data and AI operations, helping data engineers solve challenges in AI implementation.
  • Lakeflow Designer, Lakeflow Connect with expanding ecosystem, and Zerobus Ingest replace traditional tools like Kafka for more efficient data operations.
  • The 'messy middle' of data engineering is solved by building an open data foundation that AI applications can use directly.
  • AI agents transform data engineering work by automating routine tasks and reducing the complexity of distributed data systems.

Frequently asked questions

What problem does Lakeflow solve in data engineering?
Lakeflow solves the issue of data being scattered across many different enterprise systems. Previously, companies had to rely on expensive and brittle pipelines to move this data. Lakeflow makes it easy to transform raw data into insights and actionable results.
What is Genie ZeroOps and what does it do?
Genie ZeroOps is an AI agent for data and AI operations introduced by Databricks. It helps data engineers with complex data operations tasks and was live demonstrated during the presentation.
What new tools and features have been made available for Lakeflow?
Lakeflow Designer is now available, Lakeflow Connect is expanding with a growing ecosystem, and Zerobus Ingest replaces traditional tools like Kafka for more efficient data ingestion.
How does Lakeflow help build a data foundation for AI?
Lakeflow builds an open data foundation where data from various sources comes together and AI applications can use it directly, without complex integration processes in between.

Topics

ai agentsdata engineeringdatabrickslakeflowdata operations