TestGuild Automation Podcast

AI Agents in QA: How to Keep Up with AI-Driven Dev Velocity with Vilhelm von Ehrenheim 02.06.2026 35min

AI coding tools promised to make development faster — and they delivered. But here's the problem nobody talks about enough: when you speed up coding, you don't eliminate the bottleneck in the SDLC. You just move it. And for most teams, it lands squarely in QA. In this episode, Joe sits down with Vilhelm von Ehrenheim, Co-founder and Chief AI Officer of QA.tech, to dig into how agentic AI is reshaping software testing from the ground up. Vilhelm brings serious ML credibility, he helped build Motherbrain, one of the earliest production LLM systems in venture capital, and he's now applying that experience to one of the hardest problems in software delivery: testing at AI development velocity. You'll learn how QA.tech's behavioral knowledge graph gives AI agents the context they need to actually understand your application, why validating user intent beats checking element identifiers every time, how autonomous agents can review PRs, reproduce bugs from Slack messages, and generate targeted tests without a single line of test code ,and what the tester's role actually looks like when agents do the heavy lifting. If you're wondering whether your QA practice can survive the pace of AI-driven development, this one's required listening. 🔗 Book a demo now: https://testgld.link/qatechdemo

Maestro MCP, AI Mobile Testing That Fixes Its Own Tests with Leland Takamine 19.05.2026 38min

What happens when AI agents can not only write mobile app code, but also validate their own work automatically? In this episode, I sit down with Maestro Co-founder and CEO Leland Takamine to explore one of the biggest shifts happening in software testing right now: agentic mobile testing. Leland shares how his team went from solving mobile performance testing challenges to building one of the fastest-growing mobile automation frameworks used by companies like Microsoft, Meta, Amazon, and DoorDash. We dive deep into: How AI coding agents are changing mobile testing workflows What "closing the agentic feedback loop" actually means Why deterministic testing still matters in the age of AI How Maestro MCP lets AI agents validate mobile app changes automatically Why mobile test maintenance may finally become manageable The future role of testers as AI-generated code explodes Leland also gives a live demo showing an AI agent building, validating, debugging, and generating a reusable mobile test completely autonomously. If you care about AI testing, mobile automation, MCP servers, or the future of QA engineering, this episode will likely change how you think about testing workflows over the next few years. Try it out now for yourself: Maestro Studio: https://testgld.link/mstudio Maestro MCP docs: https://testgld.link/maestromcp

AI Testing Costs, How to Prevent Runaway Token Bills with Arthur Hicken 12.05.2026 38min

AI-powered testing tools are exploding across software engineering teams… but so are the hidden costs. In this episode, Joe sits down with Arthur Hicken to unpack the growing problem of runaway AI token usage, unexpected LLM billing, and the operational risks of deploying AI agents into testing and DevOps pipelines. Inspired by Arthur's article on the emerging "Token Tax," this conversation explores why many teams are underestimating the true cost of AI automation. You'll learn: Why AI-generated testing can create unexpected scaling costs How runaway AI agents and infinite loops happen Real-world examples of massive AI billing surprises Why deterministic problems shouldn't always use LLMs The hidden risks of "vibe testing" and autonomous AI remediation How QA teams can monitor, test, and control token usage Why performance testing and service virtualization matter more than ever in AI systems Practical strategies to avoid expensive AI deployment mistakes Whether you're a software tester, automation engineer, QA leader, or DevOps practitioner, this episode will help you think more strategically about AI testing before costs spiral out of control.

Stop Rewriting Tests: How to Add AI to Selenium and Playwright Without Starting Over with Alex Rodionov 05.05.2026 30min

AI-powered testing tools promise faster automation and less maintenance, but most require teams to abandon their existing frameworks. In this episode, we explore Alumnium, an opensource AI-native end-to-end testing solution created by Alex Rodionov, an engineer at Airbnb and a tech lead on the Selenium project. Instead of replacing tools like Playwright or Selenium, Alumnium adds an AI layer on top, helping teams: Reduce test maintenance by removing brittle locators Build more resilient, self-healing tests Write less code while increasing coverage Run tests across web and mobile using intent-based steps We also go beyond the hype and break down what actually matters for real teams: Why AI-driven tests can still become flaky The performance and cost tradeoffs of LLM-based execution What "context rot" is—and how it impacts long test runs How to safely introduce AI into existing test suites without rewriting everything Check it out now: https://testguild.me/alumAI

Claude AI Mobile Testing, Run Real Device Tests with AI with Frank Moyer and Chris Faulhaber 28.04.2026 31min

AI is changing how we build and test software, but most teams are still struggling to turn AI-generated tests into real production value. Use code TESTGUILD3 try for yourself free now for 3 months: https://links.testguild.com/Kobiton In this episode, we break down what actually works when it comes to AI-powered mobile test automation, especially when running tests on real devices not simulators from Claude. You'll learn: How teams are generating and running Appium tests using natural language in minutes Why AI-generated tests often fail—and how to avoid costly false positives The real impact of AI on test automation roles and responsibilities How real device testing exposes issues AI alone can't catch Practical ways to reduce test maintenance while increasing coverage We also explore a major shift happening right now: AI is making it easier to create tests—but dramatically increasing the volume of code and risk that needs to be validated. That means one thing: Testing isn't going away—it's becoming more critical than ever. If you're a QA engineer, automation engineer, or DevOps leader trying to keep up with AI-driven development, this episode will give you a clear, practical perspective on what to focus on next.

AI Testing Is Breaking Your Pipeline. Fix Quality Before It's Too Late with Eric Minick 15.04.2026 29min

AI coding tools are helping teams move faster than ever, but there's a hidden cost. In this episode, we break down new insights from a DevOps industry report revealing a growing "velocity paradox": teams are shipping more code, but experiencing more failures, rollbacks, and burnout. You'll discover why AI adoption is heavily skewed toward coding, but not testing, pipelines, or observability, and how that imbalance is creating fragile systems that break under pressure. More importantly, you'll learn what high-performing teams are doing differently to maintain quality while scaling speed. What You'll Discover: ✔️ Why AI is increasing deployment failures (and how to stop it) ✔️ The "velocity vs quality" trap hurting modern DevOps teams ✔️ How to reduce flaky tests and pipeline instability ✔️ Why observability and feature flags are now critical, not optional ✔️ Practical ways to improve your CI/CD pipeline for AI-driven development ✔️ The role of QA engineers in the age of AI (and why it's growing, not shrinking) If you're a tester, automation engineer, or DevOps leader trying to keep up

Scaling Quality Engineering: How to Deliver Faster Across Global Teams with Sunita McCoy 07.04.2026 33min

AI is changing how we build and test software, but most teams are struggling to turn that promise into real results. In this episode, we break down what it actually takes to scale quality engineering across global teams without creating bottlenecks, burnout, or broken processes. You'll learn: why most test automation and transformation initiatives fail how to separate AI hype from reality what high-performing teams are doing differently to ship faster with confidence Today's expert, Sunita McCoy, a Global Engineering Leader and Transformation Specialist, shares practical insights from leading large-scale engineering transformations, including: how to build a culture that supports AI adoption why "quality as a phase" is dead how to shift toward treating quality as a product If you're a QA leader, automation engineer, or DevOps professional trying to improve reliability, reduce risk, and future-proof your skills in the age of AI, this episode gives you a clear path forward.

Mobile Test Automation is Broken. Here's How QApilot Fixes It with Aditya Challa 31.03.2026 37min

Mobile test automation is still one of the biggest bottlenecks in modern software delivery. In this interview, QApilot's Co-founder Aditya Challa explains why most AI testing approaches fail and how to fix them. Learn more about QApilot: https://links.testguild.com/flutterqa If your mobile tests are flaky, slow, or hard to trust, you're not alone. Most teams are trying to apply LLM-based AI to problems that actually require deterministic reliability—and that's where things break down. In this video, you'll learn: Why mobile test automation breaks at scale The real issue with "99% accurate" AI in testing LLMs vs deterministic AI (and why it matters for mobile apps) How flaky tests destroy confidence in your pipeline How QApilot approaches mobile testing differently What reliable, scalable mobile automation should look like What this means for you: Fewer false positives, faster releases, and mobile tests you can actually trust. 00:00 Why Mobile Test Automation Is Still Broken 01:10 QApilot Overview 01:51 Why Mobile Testing Tools Fail 03:13 Why Appium Isn't Enough 05:09 QApilot's Approach to Mobile Testing 07:10 Scaling Mobile Testing Across Devices 08:02 Autonomous Testing + Human in the Loop 10:55 How QApilot Works (Architecture + Agents) 13:45 Real Example: Mobile App Crawling in Action 16:31 Finding Bugs Automatically (Performance + Accessibility) 18:52 Device Farms & Real Device Testing 21:50 Future of Mobile Testing (SRE + AI + Quality Layer) 27:06 Real Customer Results & Case Study 31:02 Why QApilot Focuses Only on Mobile 34:04 Where QApilot Fits in CI/CD 36:00 How to Try QApilot + Final Advice

AI Testing: How Solo Testers Stay Confident in Releases with Christine Pinto 25.03.2026 44min

Are you the only tester on your team—and expected to ensure quality across everything? In this episode, we break down the growing challenge of solo QA testing in the age of AI-driven development—where code is generated faster than ever, but confidence hasn't caught up. Christine Pinto shares real-world insights from her experience as a solo tester and now as a founder building tools designed to help testers reduce risk, collaborate better, and make smarter release decisions. You'll learn: Why "all tests passing" doesn't mean your product is safe The hidden risks of AI-generated code and test automation How to shift from test coverage to risk-based testing Practical ways solo testers can avoid burnout and isolation How to bring collaboration back into QA—even if you're the only tester Why better requirements still matter more than better AI

AI Testing from Production Logs: Generate Smarter Regression Tests with Tanvi Mittal 17.03.2026 27min

What if your production logs could automatically generate new test cases? In this episode, Joe Colantonio sits down with Tanvi Mittal to break down how AI-powered log mining is changing the way teams approach software testing, quality engineering, and DevOps. Most teams ignore production logs or use them only for debugging. But those logs contain real user behavior, real failures, and real edge cases—the exact scenarios your test suite is probably missing. 👉 Learn how to: Convert production logs into automated regression tests Use AI to detect real-world failure patterns Apply shift-right testing to catch bugs earlier (and smarter) Handle the challenge of testing non-deterministic AI systems Reduce flaky tests and automation debt with real data If you're working with Playwright, Selenium, Cypress, or AI-driven testing tools, this episode will give you a completely new way to think about test coverage.

AI Testing: How to Ensure Quality in Non-Deterministic Systems with Adam Sandman 10.03.2026 43min

How do you ensure software quality when the system you're testing doesn't give the same output twice? Go to https://links.testguild.com/inflectra and start your free 30-day trial, no credit card, no contract required. That's the core challenge facing every QA team building or testing AI-powered applications today and it's breaking all the rules we've relied on for decades. In this episode of the TestGuild Automation Podcast, I sit down with Adam Sandman, co-founder of Inflectra, to get into what non-deterministic AI testing actually means in practice, why traditional pass/fail testing no longer cuts it, and what quality professionals need to do differently right now. We cover: Why AI-generated code is raising the stakes for QA teams while budgets stay flat The fundamental difference between deterministic and non-deterministic systems — and why it changes everything about how you test How to set acceptable risk thresholds for AI systems (hint: it depends on whether you're building an e-commerce chatbot or an air traffic control system) Why testers who embrace AI as a tool — not a threat — will be the ones leading their organizations forward How a live demo failure at a conference inspired Inflectra's new non-deterministic testing tool, SureWire If you're a tester, QA manager, or automation engineer trying to figure out how to keep up with AI-driven development without losing your mind — or your job — this one's for you.

Test Automation Tools That Scale: From Zero to 1.6M Users with Sanjay Kumar 03.03.2026 29min

What does it really take to build a test automation tool that millions of testers rely on, without venture capital, paid ads, or a massive team? In this episode, we explore how SelectorsHub grew into one of the most widely used productivity tools in software testing, reaching over 1.6 million testers worldwide. You'll discover: How to build test automation tools that solve real QA pain Why community-driven development beats chasing funding How to prioritize features when you have thousands of users Whether AI testing tools will replace selector-based automation How to choose between Playwright vs Selenium using automation analysis What founders and QA leaders can learn from scaling without VC If you're an automation engineer, QA lead, DevOps professional, or tool builder looking to scale smarter, this episode delivers real-world insight without hype. Whether you're building frameworks internally or launching your own automation product, you'll walk away with a clearer strategy for solving problems testers actually care about.

AI Test Automation: Ship Twice as Fast with 10x Coverage with Karim Jouini 24.02.2026 42min

AI test automation is evolving fast — but most tools still generate brittle code that breaks with every UI change. See it for yourself now: https://links.testguild.com/Thunders In this episode of the TestGuild Podcast, Joe Colantonio sits down with Karim Jouini, founder of Thunders, to explore a radically different approach to AI testing: executing test automation in plain English without generating Selenium or Playwright code. Instead of "auto-healing selectors," Thunders interprets natural language directly — allowing teams to: Ship twice as fast Achieve 10x test coverage with the same resources Reduce regression cycles from weeks to days Eliminate massive automation maintenance overhead Karim shares real-world case studies, including: A European bank that reduced a 3-year core banking upgrade testing effort to 4 months A SaaS company that transitioned from a traditional QA team to AI-assisted product-led testing We also discuss: Whether AI test agents replace QA roles How QA managers must shift from individual contributors to AI managers The risks of adopting AI without a defined success metric The future of shift-left testing in the AI era If you're a software tester, automation engineer, QA lead, or DevOps leader trying to understand what's hype versus real ROI in AI testing — this episode breaks it down. Try it for yourself and see how AI testing fits into your pipeline. Get personal demo: https://links.testguild.com/Thunders

Performance Testing with AI w/ Akash Thakur 17.02.2026 26min

Is traditional performance testing becoming obsolete? In this episode, performance engineering expert Akash Thakur shares why AI is fundamentally transforming load testing, scripting, observability, and shift-left strategies. With 17 years of real-world enterprise experience, Akash explains how AI-augmented tools are already reducing scripting time by 30%, improving analysis speed, and helping teams move from reactive performance testing to predictive intelligence. You'll learn: How AI is accelerating performance scripting and analysis Why shift-left performance testing is finally becoming realistic The role of structured data in predictive QA models How to test AI applications (LLMs, GPUs, inference throughput) differently than traditional web apps What the future role of performance engineers looks like — architect, not script writer If you're a performance tester, SRE, QA leader, or DevOps engineer wondering how AI will impact your role — this episode gives you practical, actionable insights you can apply immediately.

Spec2TestAI: Stop Defects Before They Reach Production with Missy Trumpler 27.01.2026 34min

Most teams find defects after the damage is done — during regression, late-stage testing, or production incidents. That's expensive, stressful, and completely avoidable. Try Spec2Test AI now: https://testguild.me/spec2testdemo In this episode, Joe Colantonio sits down with Missy Trumpler, CEO of AgileAILabs, to explore how Spec2TestAI helps teams prevent defects before code ships by applying AI directly to requirements. You'll learn: Why traditional test automation still misses critical risk How predictive, requirements-based AI testing works in practice What "shift-left" actually looks like beyond the buzzword How to reduce escaped defects without writing more tests Why secure, explainable AI matters for QA and enterprise teams This conversation is especially valuable for software testers, automation engineers, and QA leaders who want earlier visibility into risk, faster feedback, and higher confidence releases. Don't miss Automation Guild 2026 - Register Now: https://testguild.me/podag26

Locust Performance Testing with AI and Observability with Lars Holmberg 13.01.2026 30min

Performance testing often fails for one simple reason: teams can't see where the slowdown actually happens. In this episode, we explore Locust load testing and why Python-based performance testing is becoming the go-to choice for modern DevOps, QA, and SRE teams. You'll learn how Locust enables highly realistic user behavior, massive concurrency, and distributed load testing — without the overhead of traditional enterprise tools. We also dive into: Why Python works so well for AI-assisted load testing How Locust fits naturally into CI/CD and GitHub Actions The real difference between load testing vs performance testing How observability and end-to-end tracing eliminate guesswork Common performance testing mistakes even experienced teams make Whether you're a software tester, automation engineer, or QA leader looking to shift-left performance testing, this conversation will help you design smarter tests and catch scalability issues before your users do.

Top 8 Automation Testing Trends for 2026 with Joe Colantonio 06.01.2026 12min

AI testing is everywhere — but clarity isn't. In this episode, Joe Colantonio breaks down the real test automation trends for 2026, based on data from 40,000+ testers, 510 live Q&A questions, and 50+ interviews with industry leaders. This isn't vendor hype or futuristic speculation. It's what working testers are actually worried about — and what they're doing next. You'll learn: Why 72.8% of testers prioritize AI, yet don't trust it alone The real reason AI testing feels harder instead of easier How integration chaos is blocking automation success Why "AI auditor" and "quality strategist" are emerging career paths What agentic AI, MCPs, and vibe testing really mean in practice How compliance, accessibility, and security will redefine QA in 2026 If you're a tester, automation engineer, or QA leader trying to stay relevant — this episode gives you the signal through the noise, and a clear path forward. If you're a software tester, automation engineer, or QA leader looking ahead to 2026, this episode lays out what's coming — and how to stay connected. Discount code: 100GUILDCOIN (https://testguild.me/podag26)

Automation Testing Podcast 2026: New Schedule, Events, Discounts with Joe Colantonio 28.12.2025 2min

This is a special end-of-year episode of the Automation Testing Podcast. With family in town and a busy holiday season, Joe didn't want to skip a week without checking in and saying thank you to the TestGuild community. In this short episode, Joe shares: A huge milestone as the podcast approaches its 13-year anniversary Why the Automation Testing Podcast is moving from Sundays to Tuesdays starting in 2026 How loyal listeners can still get $100 off a full 5-day Automation Guild 2026 pass A sneak peek at TestGuild IRL — live, in-person events coming next year Gratitude for the listeners, YouTube community, and sponsors who make TestGuild possible If you're a software tester, automation engineer, or QA leader looking ahead to 2026, this episode lays out what's coming — and how to stay connected. Discount code: 100GUILDCOIN (https://testguild.me/podag26) Questions or ideas? Email Joe directly at joe@testguild.com As always — test everything, and keep the good.

AI Testing LLMs & RAG: What Testers Must Validate with Imran Ali 21.12.2025 32min

AI is transforming how software is built, but testing AI systems requires an entirely new mindset. Don't miss AutomationGuild 2026 - Register Now: https://testguild.me/podag26 Use code TestGuildPod20 to get 20% off your ticket. In this episode, Joe Colantonio sits down with Imran Ali to break down what AI testing really looks like when you're dealing with LLMs, RAG pipelines, and autonomous QA workflows. You'll learn: Why traditional pass/fail testing breaks down with LLMs How to test non-deterministic AI outputs for consistency and accuracy Practical techniques for detecting hallucinations, grounding issues, and prompt injection risks How RAG systems change the way testers validate AI-powered applications Where AI delivers quick wins today—and where human validation still matters This conversation goes beyond hype and gets into real-world AI testing strategies QA teams are using right now to keep up with AI-generated code, faster release cycles, and DevOps velocity. If you're a tester, automation engineer, or QA leader wondering how AI changes your role,not replaces it,this episode is your roadmap.

AI Codebase Discovery for Testers with Ben Fellows 14.12.2025 44min

What if understanding your codebase was no longer a blocker for great testing? Most testers were trained to work around the code — clicking through UIs, guessing selectors, and relying on outdated docs or developer explanations. In this episode, Playwright expert Ben Fellows flip that model on its head. Using AI tools like Cursor, testers can now explore the codebase directly — asking questions, uncovering APIs, understanding data relationships, and spotting risk before a single test is written. This isn't about becoming a developer. It's about using AI to finally see how the system really works — and using that insight to test smarter, earlier, and with far more confidence. If you've ever joined a new team, inherited a legacy app, or struggled to understand what really changed in a release, this episode is for you. Registration for Automation Guild 2026 Now: https://testguild.me/podag26