The Data Engineering Show

Episodes

• 
  How AI Is Reshaping Modern Data Teams and the Future of Platforms with Xavier Gumara Rigol 18.08.2026 17m

  What if AI could transform how your entire data team works—without replacing them? In this episode, Benjamin Wagner explores with Xavier Gumara Rigol, Head of Data at Manychat, how natural language-to-SQL tools are reshaping data analyst and data scientist roles, why building a strong data platform is essential for AI-powered self-serve analytics, and the critical strategies for evaluating text-to-insight solutions in 2026. Whether you're leading a data organization or building your next analytics capability, discover how to balance build versus buy decisions and position your team for the AI-driven future of data engineering.
• 
  Building Modern Data Platforms Without Legacy Bottlenecks ft Andrew Jones 06.08.2026 19m

  What if you could transform your data team from a bottleneck into a self-serve platform that empowers the entire organization? In this episode, host Benjamin Wagner sits down with Andrew Jones, Staff Data Engineer at LocalStack, to explore how to balance building new data infrastructure while supporting legacy systems, why reliability and data contracts are becoming non-negotiable, and how AI agents are accelerating platform development. Whether you're scaling a data organization or rethinking your data governance strategy, this conversation is packed with practical insights on managing different user personas, automating support requests, and staying open-minded as the data landscape evolves. Tune in to discover how to build resilient, self-serve data platforms that drive real business impact.
• 
  Why 99% of BI Tools Get Embedded Analytics Wrong And How Omni Fixed It ft. Chris Merrick 21.07.2026 20m

  What if the future of analytics wasn't about building another middleware layer, but about getting closer to the actual business user? In this episode, Benjamin sits down with Chris Merrick, CTO and cofounder of Omni, to explore why semantic layers matter more than ever in an agentic world, how AI is reshaping embedded analytics and customer-facing data experiences, and the key strategies for keeping complex data models aligned across federated sources. Whether you're building analytics platforms, managing data infrastructure, or trying to make AI work at scale, this conversation is packed with practical insights on balancing governed analytics with exploratory AI, unifying disparate data sources, and capturing business intelligence beyond just the metrics in your warehouse. Tune in to discover how the next generation of analytics platforms will need to think about the entire business, not just the data.
• 
  AI for Data and Data for AI: The Dual Frontier of Modern Data Engineering with Pranav Motarwar 16.06.2026 19m

  What if the data engineering skills you have today become obsolete in five years? In this episode, host Benjamin Wagner sits down with Pranav Motarwar, a data engineer who's witnessed the industry's transformation from traditional ETL to AI-powered pipelines, to explore how AI is fundamentally reshaping data engineering roles, why you need to master both "AI for data" and "data for AI" to stay relevant, and the emerging infrastructure required to handle multimodal data at scale. Whether you're a data engineer wondering about your career longevity or a builder curious about next-gen data stacks, this conversation unpacks the skills you'll need, the tools defining 2026, and why data engineers aren't disappearing - they're just evolving faster than ever.
• 
  AI Won't Replace Engineers, But This Framework Will Change How They Build with Rohit Girme 07.05.2026 18m

  What if you could build AI features with confidence while moving at the pace of innovation? In this episode, Benjamin Wagner sits down with Rohit Girma, Staff Software Engineer at Airbnb, to explore how to evaluate generative AI in production, why breaking down complex problems into smaller chunks accelerates development, and the key strategies for scaling AI-powered products beyond zero-to-one. Whether you're shipping AI features or transforming your engineering workflow, this conversation offers practical insights on building reliable AI systems, leveraging LLMs as orchestration tools, and the future of software development. Tune in to discover why humans remain essential in the scaling phase and how your team can move faster without sacrificing quality.
• 
  The Framework Canva Uses for 200M+ Designers with Paul Tune 28.04.2026 22m

  In this episode of The Data Engineering Show, Benjamin sits down with Paul Tune, Staff Research Scientist at Canva, to explore the advancement of machine learning at one of the world's leading design platforms. Learn how Canva is transitioning from traditional ML like recommendation engines for templates to cutting-edge agentic workflows that allow users and AI to collaborate on complex design tasks. Whether you're interested in the infrastructure behind distributed training or the nuances of post-training LLMs for aesthetic tasks, this deep dive offers a masterclass in scaling ML for millions of creative users.
• 
  Llama 2 & 3 Safety: Soumya Batra on Agentic AI Training 08.04.2026 22m

  What if the expertise that built foundation models could reshape how you think about AI's future? In this episode, Benjamin sits down with Soumya Batra, founder and CEO of WisePort AI and former safety lead on Llama 2 and Llama 3 at Meta, to explore how foundation models evolved from traditional NLP, why post-training holds the highest leverage for safety and controllability, and what natively agentic AI means for the next frontier of AI development. Whether you're curious about the model training lifecycle or wondering what comes after large language models, this conversation unpacks the technical strategies and vision shaping tomorrow's AI systems.
• 
  The Data Fusion Secret & Why Custom Query Engines Fail with Nikita Lapkov 24.03.2026 18m

  What if building a distributed SQL engine meant rethinking everything about how query execution works at scale? In this episode, Benjamin sits down with Nikita, Senior Software Engineer at Cloudflare, to explore how R2 SQL leverages object storage and distributed computing to power analytics across 300 global locations, why backward compatibility becomes critical when you can't control infrastructure rollouts, and the key strategies for handling joins and adaptive query execution in a stateless, point-to-point network architecture. Whether you're designing distributed systems or curious about how Cloudflare processes petabytes of data, this conversation reveals the real-world engineering challenges and innovations shaping the future of cloud data platforms.
• 
  How Zipline AI Turns Weeks of Engineering Into Minutes of SQL Queries ft. Nikhil Simha 10.03.2026 24m

  What if you could deploy ML features and real-time data pipelines without building complex infrastructure from scratch? In this episode, host Benjamin sits down with Nikhil Simha, CTO at Zipline AI and co-author of Chronon AI, to explore how Chronon, an open-source system that generates data infrastructure from simple queries, is transforming feature engineering at companies like OpenAI and Airbnb. Learn why iteration speed matters for fraud detection, how to serve thousands of signals at a massive scale, and what the future of analytical databases looks like in an AI-first world. Whether you're scaling real-time ML systems or building customer-facing analytics, this conversation is packed with practical insights on bridging the gap between data scientists and ML engineers.
• 
  The Geo-Data Problem Nobody Talks About And How Voi Solved It ft. Magnus Dahlbäck 19.02.2026 16m

  What if your data platform could power both critical business decisions and real-time product features at scale? In this episode, host Benjamin sits down with Magnus Dahlbäck, Senior Director of Data and Platform at Voi, to explore how a metrics-first approach and semantic layers transform data accessibility, why traditional ML and LLMs require different strategies for different problems, and how to balance FinOps costs while processing billions of IoT events daily. Whether you're building data infrastructure for a high-growth company or rethinking how your organization consumes data, this conversation is packed with practical strategies for unlocking data value and preparing your platform for AI. Tune in to discover how Voi ditched traditional BI tools and revolutionized their approach to enterprise analytics.
• 
  Why 99% of Data Teams Give Up on Real-Time And How Artie Changes That 03.02.2026 29m

  What happens when a team of seven engineers spends a year trying to build a production-ready CDC connector and fails? For Artie CTO and co-founder Robin Tang, it was the spark needed to build a platform that makes data streaming accessible. In this episode, Robin joins Benjamin to discuss the "DFS" (Deep First Search) approach to data sources, the engineering hurdles of real-time Postgres-to-Snowflake pipelines, and why "theoretically correct" architectures often fail in practice.
• 
  The $100M Problem: How Lyft's Data Platform Prevents ML Failures with Ritesh Varyani at Lyft 16.12.2025 25m

  What if your data platform could serve AI-native workloads while scaling reliably across your entire organization? In this episode, Benjamin sits down with Ritesh, Staff Engineer at Lyft, to explore how to build a unified data stack with Spark, Trino, and ClickHouse, why AI is reshaping infrastructure decisions, and the strategies powering one of the industry's most sophisticated data platforms. Whether you're architecting data systems at scale or integrating AI into your analytics workflow, this conversation delivers actionable insights into reliability, modernization, and the future of data engineering. Tune in to discover how Lyft is balancing open-source investments with cutting-edge AI capabilities to unlock better insights from data.
• 
  60 Billion Predictions Daily: Inside Credit Karma’s Agentic Data Layer with Maddie Daianu 19.11.2025 19m

  What does MLOps look like when you are deploying 60 billion machine learning predictions a day? Maddie Daianu, Head of Data and AI at Intuit Credit Karma, joins the Data Bros to pull back the curtain on one of the most high-volume data environments in FinTech. With a 100-person team serving 140 million members, standard data practices break down. Maddie shares how her team manages terabytes of daily data on Google Cloud and explains the massive strategic pivot they are undertaking right now: The move from "Information" to "Agency."
• 
  Block Bad Data Before the Write with Nike’s Ashok Singamaneni 07.10.2025 20m

  Nike’s Principal Data Engineer Ashok Singamaneni joins Benjamin and Eldad to discuss his open-source data quality framework, Spark Expectations. Ashok explains how the tool, which was inspired by Databricks DLT Expectations, shifts data quality checks to before the data is written to a final table. This proactive approach uses row-level, aggregation-level, and query data quality checks to fail jobs, drop bad records, or alert teams - ultimately saving huge costs on recompute and engineering effort in mission-critical data pipelines.
• 
  Postgres vs. Elasticsearch: The Unexpected Winner in High-Stakes Search for Instacart with Ankit Mittal 17.09.2025 21m

  Modernizing Search Infrastructure: How Instacart Transitioned from Elasticsearch to PostgreSQL for Enhanced Performance and Simplicity. In this episode of The Data Engineering Show, host Benjamin Wagner speaks with Ankit Mittal, former senior engineer at Instacart, about the company's innovative approach to modernizing their search infrastructure by transitioning from Elasticsearch to PostgreSQL for single-retailer search functionality.
• 
  Is Self-Service BI a False Promise? Lei Tang of Fabi.ai Thinks So 28.08.2025 21m

  AI is reshaping business intelligence by enabling true self-service analytics and transforming how organizations interact with their data through natural language processing. In this episode of The Data Engineering Show, host Benjamin interviews Lei, Co-founder and CTO of Fabi.ai, to explore how AI-native BI platforms are reshaping data analytics and empowering non-technical users to derive meaningful insights from complex datasets.
• 
  Building Uber's AI Assistant: How Genie Revolutionizes On-Call Support with Paarth Chothani from Uber 22.07.2025 25m

  In this episode of The Data Engineering Show, the bros speak with Paarth, a Staff Engineer at Uber, about his work on Genie - an innovative AI assistant that revolutionizes on-call support by combining RAG (Retrieval Augmented Generation) with agent-based automation to help engineers find solutions faster.
• 
  From Zero to 100M Users: Inside Notion’s Data Stack and AI Strategy with Sumit Gupta 10.06.2025 22m

  Dive into the future of data engineering with Sumit Gupta, Lead BI Engineer at Notion, as he shares insights with the bros on navigating the AI revolution in modern data stacks. From leveraging tools like Snowflake and dbt to automating content creation with AI, discover how traditional technical skills are evolving alongside the rise of AI. Whether you're a seasoned data professional or just starting your journey, learn why embracing AI isn't optional and how to balance technical expertise with crucial soft skills in this rapidly changing landscape. Get an insider's perspective on working at tech giants like Notion, Snowflake, and Dropbox, while exploring practical applications of AI in both professional and personal contexts.
• 
  How Rising Wave Is Redefining Real-Time Data with Postgres Power 07.05.2025 31m

  In this episode of The Data Engineering Show, the bros sit with Yingjun Wu, founder and CEO of Rising Wave, to explore the innovative world of stream processing systems. Yingjun shares his journey from academic research to creating a Postgres-compatible streaming system that drastically reduces resource usage. They discuss how Rising Wave's S3-based architecture and Postgres compatibility provide advantages over traditional systems like Flink, and explore the increasing role of Apache Iceberg in data pipelines.
• 
  Revolutionizing Data Governance with DataStrato’s Unified Open Source Approach 08.04.2025 23m

  In this episode of The Data Engineering Show, the bros sit with Lisa Cao, Product Manager at DataStrato, to explore data catalogs and Apache Gravitino, a unified metadata lake used to manage access and perform data governance for all data sources. They discuss data catalogs and how they refine the data management process.