Ship It Weekly - DevOps, SRE, Platform and Cloud Engineering News

Episodes

• 
  Ship It Conversations: Jay Lark of Hookbridge on Webhook Reliability, Retries, Idempotency, Replay, HMAC Security, Local Testing, and Operating Webhooks in Production 27.07.2026 31m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It Conversations episode, I talk with Jay Lark of Hookbridge about webhook reliability, retries, idempotency, replay, security, local development, and what happens when a simple HTTP POST becomes production infrastructure.Jay is a Principal DevOps Engineer and the founder of Hookbridge, a service focused on making webhook delivery more reliable and easier to operate.We start with the basic model: one service sends an HTTP POST to another when something happens. The happy path is easy. The problems begin when an endpoint is unavailable, responds too slowly, receives duplicate events, or gets them out of order.Jay explains what “at least once delivery” means and why receivers must expect duplicates. We talk about idempotency, event IDs, retry behavior, availability during deployments, and why returning a 200 does not prove downstream processing succeeded.We also dig into observability and security. Teams need enough visibility to know whether a webhook arrived, whether signature verification passed, what response was returned, and where processing failed. Jay breaks down HMAC signatures, timestamps, replay protection, and why a valid signature still does not replace normal business-logic validation.Local development is another source of friction. External providers cannot send events directly to localhost, so developers often rely on temporary tunnels, staging deployments, copied payloads, or mocks. Jay explains how Hookbridge uses a fixed URL and local client to forward real webhook traffic to a developer’s machine.We also talk about n8n, self-hosted OpenClaw systems, Hookbridge pull endpoints, and when polling may be simpler or safer than exposing another inbound endpoint.The big takeaway: design the failure path before a webhook becomes business-critical. Verify the sender, expect duplicate and out-of-order events, build enough visibility to debug failures, and have a replay strategy before the first incident.Highlights• Why webhooks are harder than “just an HTTP POST”• What at-least-once delivery means for receivers• Why idempotency, retries, and event ordering matter• What teams need for webhook observability and debugging• Where HMAC signatures, timestamps, and replay protection fit• Why local webhook development is still awkward• When polling or pull-based delivery may be a better fit• When teams should stop building webhook infrastructure themselvesLinksHookbridge: https://hookbridge.ioHookbridge local development CLI: https://www.hookbridge.io/cli.htmlHookbridge pull endpoints: https://www.hookbridge.io/pull.htmlJay Lark on LinkedIn: https://www.linkedin.com/in/jay-lark-ba7a3b5/OpenClaw: https://openclaw.aiMore episodes and show notes: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  AWS CloudFormation Express Mode, Spark 4.2 Vector Search, AI Speeds Coding but Not Delivery, Platform Governance Developers Won’t Hate, and Why Could Is Not the Same as Should 24.07.2026 18m

  This week on Ship It Weekly: AWS CloudFormation Express mode promises faster infrastructure feedback by reporting deployments complete before extended resource stabilization finishes. Apache Spark 4.2 adds native vector operations and nearest-neighbor joins, giving some teams a way to keep AI data workloads closer to the platforms they already run.GitLab’s latest research says AI is helping developers generate and commit code faster, but review, testing, governance, and deployment are not accelerating at the same pace. Then, a platform engineering case study from Sevdesk shows how minimum viable governance, useful feedback, and progressive enforcement can improve compliance without turning the platform team into another approval queue.The theme this week: making one part of the system faster does not automatically improve the whole system.In the lightning round: OpenShift 4.22.5 receives an important security update, outdated autoscaling thresholds disrupt GitHub Actions, Cloudflare experiences an incident where some POST requests fail to reach customer origins, and GitHub Code Quality becomes generally available—with automatic billing attached.The episode closes with Reid Savage reflecting on their first year managing Honeycomb’s SRE team and the difference between what a capable team could do and what it should do.LinksCloudFormation Express mode https://aws.amazon.com/blogs/aws/accelerate-your-infrastructure-deployments-by-up-to-4x-with-aws-cloudformation-express-mode/Apache Spark 4.2 https://spark.apache.org/releases/spark-release-4-2-0.htmlGitLab AI Accountability Report https://about.gitlab.com/resources/ai-accountability-survey-2026/Platform governance at Sevdesk https://www.infoq.com/presentations/platform-engineering-team-compliance/OpenShift 4.22.5 security update https://access.redhat.com/errata/RHSA-2026%3A37585GitHub Actions incident https://github.com/orgs/community/discussions/201795Cloudflare status https://www.cloudflarestatus.com/GitHub Code Quality GA https://github.blog/changelog/2026-07-20-github-code-quality-is-now-generally-available/Could vs. Should — Reid Savage https://www.honeycomb.io/blog/could-should-first-year-managing-sre-teamThis week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W30/More episodes and full show notes https://shipitweekly.fm/
• 
  Ship It Conversations: Mat Ryer of Grafana Labs on AI Observability, Agents, Evals, and Operating AI in Production 20.07.2026 45m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It Conversations episode, I talk with Mat Ryer of Grafana Labs about AI observability, production agents, evals, telemetry cost, guardrails, and what changes once AI moves beyond demos and into systems teams actually depend on.Mat is Senior Director of AI at Grafana Labs, where he focuses on how AI fits into observability and production systems.We talk about Grafana Assistant, why AI observability is not just logs, latency, and HTTP 200s, and how teams can measure whether agents are actually helping. Mat gets into evals, LLM-as-judge patterns, traces as a way to think about conversations, user feedback, tool choice, model changes, and the cost of collecting new telemetry.We also dig into UX and trust. If an AI assistant gives you a wall of text, you still have to decide whether to believe it. If it can show the graph, deep link into Grafana, apply filters, and expose the source data, that becomes a much more useful operating experience.The big takeaway: start small, enhance workflows you already have, build feedback loops, and treat production AI like something you actually have to operate.Highlights• Why AI demos are easy, but production AI is harder• Why agents need observability, evals, and guardrails• How Grafana thinks about AI Assistant and AI observability• Where LLM-as-judge patterns, traces, tool calls, and feedback fit• Why telemetry cost problems may repeat with AI workloads• Why UX matters when operators need to trust the answer• Where AI can help SRE and platform teams todayLinksMat Ryer on LinkedIn: https://www.linkedin.com/in/matryer/Mat Ryer on GitHub: https://github.com/matryerGrafana Labs: https://grafana.comGrafana Assistant: https://grafana.com/products/cloud/ai-assistant/Grafana AI Observability: https://grafana.com/docs/grafana-cloud/machine-learning/ai-observability/Grafana Adaptive Telemetry: https://grafana.com/products/cloud/adaptive-telemetry/Grafana MCP server: https://github.com/grafana/mcp-grafanaOpenTelemetry: https://opentelemetry.ioPrometheus: https://prometheus.ioGrafana Loki: https://grafana.com/docs/loki/latest/More episodes and show notes: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  GitHub API Enumeration, Grok Build CLI Data Exposure, AWS Security Hub Network Scanning, AI-Powered Patch Pressure, and Why Visibility Is Not Ownership 18.07.2026 19m

  This week on Ship It Weekly: Datadog tracked coordinated GitHub API enumeration, xAI’s Grok Build CLI reportedly uploaded repo data without redaction, AWS Security Hub added Network Scanning and exposure impact analysis, and Microsoft says AI-powered vulnerability discovery is changing patch pressure.The theme: visibility is not ownership. A GitHub API map does not revoke a token. An exposure finding does not close a port. A patch bulletin does not patch the fleet. And an AI coding tool reading your repo is still access.Brian covers GitHub as a production surface, AI coding tool data boundaries, cloud exposure based on reachability and blast radius, and why patching needs to look more like production operations than spreadsheet theater.Also, the Ship It Weekly shop is open at shop.tellerstech.com with Ship It Weekly t-shirt designs. Use coupon code SHIPTHESTORE for 20% off your order for the next few weeks.LinksDatadog: Coordinated GitHub API enumeration https://securitylabs.datadoghq.com/articles/coordinated-github-api-enumeration/The Verge: Grok Build CLI repository upload report https://www.theverge.com/ai-artificial-intelligence/965600/spacexai-grok-build-repository-uploadAWS Security Hub Network Scanning https://aws.amazon.com/about-aws/whats-new/2026/07/aws-security-hub-network-scanning/AWS Security Hub impact analysis for exposure findings https://aws.amazon.com/about-aws/whats-new/2026/07/impact-analysis-aws-security-hub/Microsoft: Windows vulnerability management and AI-powered discovery https://blogs.windows.com/windowsexperience/2026/07/09/evolving-windows-vulnerability-management-to-meet-the-speed-of-ai-powered-discovery/SRE Weekly Issue 525 https://sreweekly.com/sre-weekly-issue-525/HalluSquatting / hallucinated package risk https://www.endorlabs.com/learn/slopsquatting-when-ai-agents-hallucinate-malicious-packagesAWS Lambda Managed Instances for Java cold starts https://aws.amazon.com/blogs/compute/eliminating-java-cold-starts-with-aws-lambda-managed-instances/This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W29/Ship It Weekly shop https://shop.tellerstech.com/More episodes and full show notes https://shipitweekly.fm/
• 
  EKS Rollbacks, GitHub Actions Supply Chain Attacks, AI Agentjacking, CloudWatch Log Alarms, and Why Safety Nets Don’t Replace Ownership 10.07.2026 19m

  This week on Ship It Weekly: Amazon EKS added Kubernetes version rollbacks, Novee Security published Cordyceps research on GitHub Actions supply chain risk, Tenet Security showed how fake telemetry can hijack AI coding agents, and Amazon CloudWatch added alarms directly from log queries.The theme: safety nets are getting better, but the blast radius is getting wider. Rollback buttons, log alarms, zone-aware routing, secret scanning, and AI agent workflows all help, but they do not replace ownership.Brian covers why EKS rollbacks are useful but not a substitute for real upgrade discipline, why GitHub Actions YAML is production code with credentials, how fake Sentry telemetry can become hostile agent context, and why easier log-based alarms can also mean easier pager noise.In the lightning round: ECS Service Connect zone-aware routing, etcd 3.7, GitHub innersource advisories, secret scanning metadata improvements, and CloudWatch Application Signals service events.LinksAmazon EKS Kubernetes version rollbacks https://aws.amazon.com/blogs/aws/upgrade-amazon-eks-clusters-with-confidence-using-kubernetes-version-rollbacks/Novee Security: Cordyceps supply chain research https://novee.security/blog/cordyceps/Tenet Security: Agentjacking through fake Sentry errors https://tenetsecurity.ai/blog/agentjacking-coding-agents-with-fake-sentry-errors/Amazon CloudWatch log query alarms https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-cloudwatch-log-alarms/ECS Service Connect zone-aware routing https://aws.amazon.com/about-aws/whats-new/2026/07/ecs-service-connect-zone-aware/etcd 3.7 announcement https://etcd.io/blog/2026/announcing-etcd-3.7/GitHub innersource security advisories https://github.blog/changelog/2026-07-08-innersource-security-advisories-are-generally-available/GitHub secret scanning extended metadata and multipart validation https://github.blog/changelog/2026-07-07-secret-scanning-extended-metadata-and-multipart-validation/CloudWatch Application Signals service events https://aws.amazon.com/about-aws/whats-new/2026/06/cloudwatch-service-events/Our LinksThis week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W28/More episodes and full show notes https://shipitweekly.fm/
• 
  Ship It Conversations: Evan Phoenix of Miren on Deployment Pain, Terraform, Waypoint, and Better Defaults for Small Teams 06.07.2026 41m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It: Conversations episode, I talk with Evan Phoenix of Miren about why deployment is still painful, what teams keep getting wrong when they try to simplify it, and why small teams may need better defaults more than more platform knobs.Evan is the CEO of Miren. He previously worked on Terraform Enterprise and Waypoint at HashiCorp, and he also built Puma and Rubinius.We talk about deployment as the “final boss” of software delivery. Not because teams do not know how to ship code, but because deployment is where everything collides: runtimes, registries, secrets, networking, cloud services, databases, rollbacks, and the internal platform nobody wants to touch anymore.A big theme is opinionated tooling. Engineers often say they want flexibility, but many teams are really asking for good defaults, a clear happy path, and fewer decisions to own.We also get into Terraform Enterprise, Terraform Cloud, Terragrunt, OpenTofu, Waypoint, Kubernetes, Heroku, ECS, container registries, and how AI changes the deployment conversation. AI can generate infrastructure code, but when that setup breaks, someone still has to understand it and be on call for it.Highlights• Why deployment is still painful after years of platforms and abstractions• What Evan learned from Terraform Enterprise and Waypoint• Why Terraform structure, state, modules, and repo layout remain hard• Why OpenTofu gained traction beyond the Terraform licensing change• Why Kubernetes can be too much surface area for some teams• What small teams actually need from deployment tooling• How AI changes infrastructure and deployment workflows• Why generated infrastructure still needs ownership and accountabilityLinks• Miren: https://miren.dev• Miren on GitHub: https://github.com/mirendev• Evan Phoenix: https://evanphx.dev• Evan on Bluesky: https://bsky.app/profile/evanphx.dev• Evan on Linkedin: https://www.linkedin.com/in/evanphoenix/Things mentioned• Terraform Enterprise: https://developer.hashicorp.com/terraform/enterprise• Terraform Cloud: https://developer.hashicorp.com/terraform/cloud-docs• Terragrunt: https://terragrunt.gruntwork.io• OpenTofu: https://opentofu.org• HashiCorp Waypoint: https://github.com/hashicorp/waypoint• Knative: https://knative.devOur linksMore episodes + show notes: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  Amazon Q CVEs, Hijacked npm and Go Packages, AWS WAF HTTP/2 Issues, Lambda MicroVMs, and Why Execution Is the Boundary Now 03.07.2026 18m

  This week on Ship It Weekly: Amazon Q Developer and the AWS language servers had a pair of trust-boundary CVEs, JFrog found hijacked npm and Go packages using hidden VS Code tasks to run malware when a workspace opens, AWS WAF had HTTP/2 request-body inspection issues, and AWS introduced Lambda MicroVMs for running user-generated and AI-generated code in isolated sandboxes.The bigger theme: execution is the boundary now. The repo, the IDE, the AI assistant, the WAF, and the sandbox all sit at the point where something gets to run, inspect, block, or decide. Before execution, trust is a policy. After execution, trust is a blast radius.In the lightning round, Brian covers GitHub’s record advisory volume, Git 2.55, Valkey 9.1 on Amazon ElastiCache, and a quick Fable 5 callback now that Anthropic’s Fable 5 is back online.LinksAWS security bulletin: Amazon Q / AWS language server CVEs https://aws.amazon.com/security/security-bulletins/2026-047-aws/JFrog: Hijacked npm packages using VS Code tasks https://research.jfrog.com/post/hijacked-npm-vscode-tasks-blockchain/AWS security bulletin: AWS WAF HTTP/2 inspection issues https://aws.amazon.com/security/security-bulletins/2026-048-aws/AWS Lambda MicroVMs https://aws.amazon.com/blogs/aws/run-isolated-sandboxes-with-full-lifecycle-control-aws-lambda-introduces-microvms/GitHub Advisory Database record volume https://github.blog/security/supply-chain-security/inside-the-advisory-database-and-what-happens-when-vulnerability-volume-breaks-records/Git 2.55 highlights https://github.blog/open-source/git/highlights-from-git-2-55/Amazon ElastiCache Valkey 9.1 https://aws.amazon.com/blogs/database/announcing-valkey-9-1-for-amazon-elasticache/Claude Fable 5 and Mythos 5 model docs https://platform.claude.com/docs/en/about-claude/models/introducing-claude-fable-5-and-claude-mythos-5This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W27/More episodes and full show notes https://shipitweekly.fm/
• 
  Ship It Conversations: Kat Traxler of Vectra AI on AI Security, the Zero-Day Clock, IAM, and Cloud Risk 28.06.2026 42m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It: Conversations episode, I talk with Kat Traxler of Vectra AI about AI security, the zero-day clock, IAM, cloud risk, AI-assisted bug hunting, and why the scariest future security problems may still start with the boring fundamentals teams already struggle with today.Kat is a Principal Security Researcher at Vectra AI focused on abuse techniques and vulnerabilities in the public cloud, especially around the intersection of cloud security, AppSec, IAM, managed identities, and insecure-by-design flaws.We talk about the current AI security mood, from the excitement around faster research and bug hunting to the fear that AI could shrink the window between vulnerability disclosure and exploitation. Kat explains the “San Francisco Consensus,” why the zero-day clock is getting so much attention, and why she thinks the facts may be real while some of the conclusions are overextended.The bigger theme here is that AI is absolutely changing security work, but it does not erase the fundamentals. Attackers still take the lowest-friction path that works. For most teams, that still means credentials, IAM, misconfigurations, known vulnerabilities, and systems that were never threat-modeled as deeply as people assume.Highlights• Why AI security feels exciting and unsettling at the same time• What the “San Francisco Consensus” means and why people are talking about the zero-day clock• How AI may shrink the time between vulnerability disclosure and exploitation• Why Kat is skeptical of the full “zero-day apocalypse” narrative• Why credentials, IAM, misconfigurations, and known vulnerabilities still matter most for many teams• How AI helps narrow the search space in bug hunting and security research• Where AI is useful for code-level bugs, and where it still struggles with context and threat modeling• Why human expertise still matters when using AI for writing, research, and cloud security analysis• Why IAM remains hard because it sits at the intersection of people, access, and technology• What insecure-by-design flaws are, and why AI may not solve those anytime soonKat / Vectra AI links• Kat Traxler at Vectra AI: https://www.vectra.ai/about/author/kat-traxler• Kat’s site: https://kattraxler.cloud/• The San Francisco Consensus: https://kattraxler.cloud/the-san-francisco-consensus/• Kat on X: https://x.com/NightmareJS• Vectra AI: https://www.vectra.ai/Our linksMore episodes + show notes + links: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  containerd CRI Vulnerabilities, Datadog PostgreSQL HA on Kubernetes, AWS DevOps Agent with Datadog MCP Server, EKS Control Plane Egress, and Why Users Feel the Wait 26.06.2026 19m

  This week on Ship It Weekly: containerd disclosed a batch of CRI plugin vulnerabilities, Datadog tested PostgreSQL high availability on Kubernetes and found that failover is not useful if it cannot happen safely, AWS DevOps Agent and Datadog MCP Server moved AI incident response closer to real production workflows, and Amazon EKS added customer-routed control-plane egress.The bigger theme: the control plane keeps getting wider. Runtimes, databases, incident agents, API-server egress, credentials, the cloud console, and object metadata are all becoming part of the production blast radius. And when something breaks, users do not experience your architecture diagram. They experience waiting.In the lightning round, Brian covers GitHub self-service credential revocation for incident response, AWS Management Console Private Access without internet connectivity, Vercel Connect and short-lived agent credentials, and Amazon S3 annotations.Linkscontainerd CRI plugin vulnerabilities / AWS security bulletin https://aws.amazon.com/security/security-bulletins/2026-046-aws/Datadog: PostgreSQL high availability on Kubernetes https://www.datadoghq.com/blog/engineering/postgresql-ha-kubernetes/AWS DevOps Agent and Datadog MCP Server https://aws.amazon.com/blogs/devops/production-ready-autonomous-incident-resolution-with-aws-devops-agent-now-ga-and-datadog-mcp-server/Amazon EKS customer-routed control-plane egress https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-control-plane-egress-through-your-vpc/GitHub self-service credential revocation for incident response https://github.blog/changelog/2026-06-24-self-service-credential-revocation-for-incident-response/AWS Management Console Private Access https://aws.amazon.com/about-aws/whats-new/2026/06/aws-management-console-private/Vercel Connect https://vercel.com/blog/introducing-vercel-connectAmazon S3 annotations https://aws.amazon.com/blogs/aws/amazon-s3-annotations-attach-rich-queryable-context-directly-to-your-objects/Marc Brooker: Waiting, latency, MTTR, and the inspection paradox https://brooker.co.za/blog/2026/06/19/waiting.htmlThis week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W26/More episodes and full show notes https://www.shipitweekly.fm
• 
  Ship It Conversations: Guardsquare’s Joel DeStefano on Mobile App Security, Runtime Protection, App Hardening, and Why Scanning Isn’t Enough 21.06.2026 35m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It: Conversations episode, I talk with Joel DeStefano from Guardsquare about mobile app security, why it is different from backend and cloud security, and why scanning alone is not enough once an app is shipped into the real world.We talk about the shift in trust model that happens with mobile apps. In backend and cloud systems, teams usually have more control over the runtime, infrastructure, policies, and monitoring. With mobile, the app becomes a public artifact running on someone else’s device, in an environment you do not fully control.The bigger theme here is that mobile security is not just “scan it before release.” Scanning matters, but teams also need to think about app hardening, obfuscation, runtime protection, monitoring, and whether the app connecting back to their APIs is genuine and uncompromised.Highlights• Why mobile changes the trust model compared to backend and cloud systems• What DevOps, SRE, and platform teams should understand about mobile app risk• Why scanning is useful, but not enough by itself• The danger of assuming app store approval means an app is secure• Why “we do not store sensitive data in the app” can be a misleading security argument• How attackers can reverse engineer apps, inspect workflows, and learn how the app talks to backend APIs• What code hardening and obfuscation actually help protect against• Why runtime checks matter for rooted devices, compromised environments, debuggers, hooking frameworks, overlays, and accessibility abuse• The difference between Android and iOS security assumptions• Why the OS is not responsible for protecting your app’s business logic• How mobile security should fit into CI/CD without destroying release velocity• What should block a release versus what should become tracked risk• Why testing, hardening, runtime protection, and monitoring should work together as one strategy• How AI may speed up attackers without fundamentally changing the need for strong security fundamentals• Joel’s advice for improving mobile security posture: start with the app’s critical workflows, backend interactions, and real business riskJoel / Guardsquare links• Guardsquare: https://hubs.ly/Q04fJgkJ0• Guardsquare Blog: https://www.guardsquare.com/blogOWASP mobile security links• OWASP Mobile Application Security: https://owasp.org/www-project-mobile-app-security/• OWASP MASVS: https://mas.owasp.org/MASVS/Our linksMore episodes + show notes + links: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  PeopleSoft Zero-Day Exploited, npm v12 Install Script Changes, GitHub Agentic Tokens, Anthropic Model Risk, and Default Trust Breaking 19.06.2026 22m

  This episode of Ship It Weekly is about default trust getting punished. Brian covers Oracle’s emergency PeopleSoft advisory for CVE-2026-35273, npm v12 changing install-script defaults, GitHub Agentic Workflows moving away from long-lived personal access tokens, and Anthropic disabling Fable 5 and Mythos 5 after a U.S. export-control directive. The common thread: legacy ERP systems, package installs, CI/CD agents, and AI models all become production risks when teams trust the default without checking what that trust can actually do.In the lightning round, Brian covers Tekton CloudEvents moving to a dedicated events controller, NVIDIA Triton Inference Server 26.04 changing inference defaults, AWS Nitro Isolation Engine bringing formal verification to Graviton5-based isolation, and Homebrew 6.0 adding explicit trust for third-party taps. The bigger theme: production does not care why you trusted the default. It only cares what that default was allowed to do.The bigger theme: production does not care why you trusted the default. It only cares what that default was allowed to do.LinksOracle PeopleSoft CVE-2026-35273 advisory https://www.oracle.com/security-alerts/alert-cve-2026-35273.htmlnpm v12 breaking changes https://github.blog/changelog/2026-06-09-upcoming-breaking-changes-for-npm-v12/GitHub Agentic Workflows no longer need PATs https://github.blog/changelog/2026-06-11-agentic-workflows-no-longer-need-a-personal-access-token/Anthropic Fable 5 / Mythos 5 access statement https://www.anthropic.com/news/fable-mythos-accessTekton Pipelines releases https://github.com/tektoncd/pipeline/releasesNVIDIA Triton Inference Server 26.04 release notes https://docs.nvidia.com/deeplearning/triton-inference-server/release-notes/rel-26-04.htmlAWS Nitro Isolation Engine https://aws.amazon.com/blogs/compute/aws-nitro-isolation-engine-formally-verifying-the-hypervisor-in-the-aws-nitro-system/Homebrew 6.0.0 https://brew.sh/2026/06/11/homebrew-6.0.0/This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W25/More episodes and show notes https://shipitweekly.fm/
• 
  Ship It Conversations: Meta’s Francois Richard on AI Incident Response, SLOs, and Reliability at Scale 16.06.2026 42m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It: Conversations episode, I talk with Francois Richard, Engineering Director at Meta, about reliability at scale, how AI is changing production risk, what teams actually learn from incidents, and why recovery practice matters just as much as prevention.We talk about the proactive and reactive sides of reliability, why SLOs should represent a promise to users instead of just another dashboard number, how incident reviews should drive real system improvements, and how teams can practice recovery before production forces the lesson on them.The bigger theme here is that reliability is not just about avoiding failure. It is about knowing what happens when prevention fails. That means practicing regional failure, understanding overload behavior, improving incident response, using AI carefully during investigation, and making reliability targets match the actual lifecycle and importance of the system.Highlights• Why reliability work starts with both prevention and recovery• The difference between reactive incident response and proactive reliability engineering• How Meta thinks about disaster recovery testing and regional failure practice• Why an SLO should be treated like a promise to users, not just a dashboard metric• How SLO trends help teams decide when to invest more in reliability or take more product risk• What engineers actually learn during the “pressure cooker” of an incident• Why incident reviews should produce follow-up work, not just a nicer explanation of what broke• The difference between finding the cause of an incident and improving the system• Where AI agents can help with incident investigation, telemetry, metrics, and query building• Why AI-generated code can increase change volume while reducing human context• How faster code generation changes the kinds of reliability problems teams should expect• Why recovery practice matters, especially for region loss, traffic spikes, overload, and restart behavior• What smaller DevOps and SRE teams can learn from Meta-scale reliability patterns• Why not every system needs six nines, especially early in a product lifecycle• How to think about reliability investment based on user promise, product maturity, and operational risk• Why At Scale Systems & Reliability is focused on the infrastructure behind AI and the use of AI to operate large-scale systemsFrancois’ links• LinkedIn: https://www.linkedin.com/in/francoisrichard/At Scale links• Systems & Reliability 2026: https://bit.ly/4xd2FdG• At Scale Conferences: https://atscaleconference.com/Our linksMore episodes + show notes + links: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  Coinbase Outage, Meta AI Account Recovery, AWS AgentCore Code Injection, Apigee Tenant Isolation, and the Glue That Breaks Production 12.06.2026 23m

  This episode of Ship It Weekly is about the hidden glue holding production together.Brian covers Coinbase’s May 7 outage postmortem, where an AWS us-east-1 cooling failure exposed the difference between being “multi-AZ” on paper and actually being able to recover when stateful, low-latency systems are tied to a failed zone.Then he looks at Meta’s AI-assisted Instagram support issue and why account recovery is identity infrastructure, not just customer support. If AI can influence password resets, email changes, MFA resets, or account ownership flows, that workflow needs to be treated like a production control plane.The episode also covers AWS AgentCore CLI CVE-2026-11393, where collaborator metadata could break out into generated Python code during agent import, and an Apigee cross-tenant issue from Google’s Apigee security bulletins that shows why tenant isolation has to be tested beyond the obvious happy path.LinksCoinbase May 7 outage postmortem https://www.coinbase.com/blog/a-postmortem-of-our-may-7-2026-outageMeta AI support / Instagram account recovery reporting https://www.theverge.com/tech/945658/meta-ai-support-chatbot-exploit-instagram-accountsAWS AgentCore CLI CVE-2026-11393 https://aws.amazon.com/security/security-bulletins/2026-040-aws/AgentCore CLI GitHub advisory https://github.com/aws/agentcore-cli/security/advisories/GHSA-m4x6-gwgp-4pm7Google Apigee security bulletins https://docs.cloud.google.com/apigee/docs/security-bulletins/security-bulletinsCloudflare real-time threat intel WAF rules https://blog.cloudflare.com/realtime-threat-intel-waf-rules/AWS Lambda tenant isolation with event source mappings https://aws.amazon.com/blogs/compute/integrating-event-source-mappings-with-aws-lambda-tenant-isolation-mode/Amazon OpenSearch Serverless next generation https://aws.amazon.com/about-aws/whats-new/2026/05/amazon-opensearch-serverless-next-generation-generally-available/GitHub Enterprise Managed Users IP allow list coverage https://github.blog/changelog/2026-06-08-ip-allow-list-coverage-for-emu-namespaces-in-general-availability/This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W24/More episodes and show notes https://shipitweekly.fm/
• 
  Kiro CLI Approval Bypass, Amazon Braket Pickle Risk, AWS Org Logging, KEDA Upgrades, and Automation’s Hidden Boundaries 05.06.2026 20m

  This episode of Ship It Weekly is about automation’s hidden boundaries. Brian covers Kiro CLI CVE-2026-9255, where piped stdin could act like user approval, Amazon Braket SDK CVE-2026-9291 and the very normal Python pickle risk hiding inside quantum job results, AWS Organizations finally emitting CloudTrail events when accounts join or leave an org, and KEDA updates that remind us autoscaling upgrades are production behavior changes.The bigger thread this week is that automation does not remove boundaries. It moves them. Approval paths, trusted data, account membership, scaling signals, platform access, and AI-generated output all need clear ownership and visibility.Brian also covers Kubernetes Dashboard being archived with Headlamp as the path forward, Google Cloud Remote MCP Server for AlloyDB, Apache Kafka 4.3.0, and Atlassian’s AI-native SDLC productivity claims.Sponsored by @Scale: Systems & Reliability, happening June 25 at the Meydenbauer Center in Bellevue, Washington. Register at https://bit.ly/4xd2FdGLinksKiro CLI CVE-2026-9255 https://aws.amazon.com/security/security-bulletins/2026-035-aws/Amazon Braket SDK CVE-2026-9291 https://aws.amazon.com/security/security-bulletins/2026-036-aws/AWS Organizations CloudTrail account events https://aws.amazon.com/about-aws/whats-new/2026/05/aws-organizations-cloudtrail/KEDA v2.20.0 release https://github.com/kedacore/keda/releases/tag/v2.20.0KEDA v2.19.0 release https://github.com/kedacore/keda/releases/tag/v2.19.0Kubernetes Dashboard archived / Headlamp path forward https://kubernetes.io/blog/2026/06/04/dashboard-archived-what-now/Google Cloud Remote MCP Server for AlloyDB https://cloud.google.com/blog/products/databases/alloydb-remote-mcp-server-now-gaApache Kafka 4.3.0 https://www.confluent.io/blog/apache-kafka-4-3-release-announcement/Atlassian AI-native SDLC productivity claims https://www.atlassian.com/blog/software-teams/ai-native-sdlcThis week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W23/More episodes and show notes https://shipitweekly.fm/
• 
  GitHub Supply Chain Attacks, Railway’s GCP Outage, Discord’s Voice Failure, AWS Retry Changes, and Trusted Tool Risk 29.05.2026 23m

  This episode of Ship It Weekly is about trusted tools becoming production dependencies. Brian covers a rough GitHub supply chain week, including the compromised Nx Console VS Code extension tied to exposed GitHub internal repositories and the Megalodon campaign abusing GitHub Actions workflows across thousands of public repos.The bigger thread this week is that the tools around production are increasingly part of production. Brian also covers Railway’s GCP account suspension outage, Discord’s voice outage during a Kubernetes migration, AWS changing SDK retry behavior, CVE-2026-9133 in the RabbitMQ AWS plugin, and a Reddit story about stolen AWS keys turning into a $14,000 Bedrock bill.Brian also touches on OpenTelemetry graduating from the CNCF, Claude Code security risk, GitLab Secrets Manager, Google Cloud AI spend caps, and a Redshift Python driver RCE.Full source list and extra links are available on this episode’s page at shipitweekly.fm.LinksNx Console compromise https://www.stepsecurity.io/blog/nx-console-vs-code-extension-compromisedMegalodon GitHub Actions attack https://www.stepsecurity.io/blog/megalodon-mass-github-actions-secret-exfiltration-across-5-500-public-repositoriesRailway GCP outage https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outageDiscord voice outage https://discord.com/blog/behind-the-scenes-of-the-3-25-26-voice-outageAWS SDK retry changes https://aws.amazon.com/blogs/developer/announcing-updated-retry-behavior-for-aws-sdks-and-tools/RabbitMQ AWS plugin CVE-2026-9133 https://aws.amazon.com/security/security-bulletins/2026-034-aws/AWS Bedrock cost spike Reddit thread https://www.reddit.com/r/aws/comments/1tm3ydo/aws_bedrock_cost_spike_14000_usd/This week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W22/More episodes and show notes https://shipitweekly.fm/
• 
  Ship It Conversations: Jake Warner on Cycle.io, Bare Metal’s Comeback, and Why Private Cloud Is Getting Interesting Again 26.05.2026 36m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.In this Ship It: Conversations episode, I talk with Jake Warner, founder and CEO of Cycle.io, about private cloud, bare metal, Kubernetes fatigue, and why some teams are rethinking how much infrastructure complexity they actually want to carry.We talk about why bare metal and private cloud are getting interesting again, especially around cost, performance, data sovereignty, compliance, and platform ownership. Jake explains how Cycle approaches infrastructure as a pool of resources, why he thinks in terms of “environments as code” instead of traditional infrastructure as code, and how teams can run containers and VMs together across bare metal, cloud, and hybrid environments.The bigger theme here is that this is not really a “cloud versus bare metal” conversation. It is about choosing the right level of abstraction. Sometimes Kubernetes is the right answer. Sometimes managed cloud services make sense. And sometimes teams just need a more opinionated platform that lets developers ship without requiring a large DevOps army to keep everything running.Highlights• Why some teams are moving back toward private cloud and bare metal• The role of cost, data sovereignty, compliance, and performance in infrastructure decisions• Why bare metal does not have to mean going back to old-school racking and stacking pain• How Cycle turns raw compute into a private cloud-style resource pool• Why Jake thinks about “environments as code” instead of only infrastructure as code• What “no DevOps army required” means in practice for engineering-heavy teams• Why some companies need VMs and containers running together on the same platform• Where Kubernetes still makes sense, especially for highly customized infrastructure needs• Why opinionated platforms can be valuable when teams want fewer knobs and better defaults• Active-active thinking, failover risk, and why application-level replication often matters more than platform-level storage magic• Why bandwidth, performance density, and predictable pricing can make bare metal attractive again• The weird continued gravity of AWS us-east-1, even for teams trying to move workloads elsewhere• How AI workloads, GPUs, and hype cycles fit into the private cloud and platform conversation• Jake’s advice for modernizing hybrid or on-prem infrastructure: containerize first, then look hard at your dependenciesJake’s links• Cycle.io: https://cycle.io/• Cycle Slack community: https://slack.cycle.io/• Jake Warner on LinkedIn: https://www.linkedin.com/in/jakewarner/Our linksMore episodes + show notes + links: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com
• 
  CISA’s GitHub Leak, AI Root Cause Analysis, Copilot Agents, Claude Code in CI/CD, and Kubernetes Seccomp Risk 22.05.2026 22m

  This episode of Ship It Weekly is about secrets, agents, risky defaults, and follow-up work that never gets done. Brian covers the CISA contractor GitHub leak involving AWS keys, internal docs, Terraform, Kubernetes, Argo CD, and CI/CD context, plus AWS DevOps Agent doing automated RCA across Datadog, Elasticsearch, CloudTrail, and EKS.Brian also covers MS Copilot Studio computer-using agents, Claude Code in Bitbucket Agentic Pipelines, CVE-2026-46333 and Kubernetes seccomp defaults, GitHub OIDC for Dependabot, Java pods getting OOMKilled, LLM-generated SQL that can be wrong but still run, and why postmortem action items die without ownership.Sponsored by Guardsquare https://hubs.ly/Q04fJgkJ0LinksCISA GitHub leak https://blog.gitguardian.com/how-we-got-a-cisa-github-leak-taken-down-in-26-hours/AWS DevOps Agent RCA https://aws.amazon.com/blogs/devops/automate-root-cause-analysis-across-datadog-and-elasticsearch-with-aws-devops-agent/Microsoft Copilot Studio computer-using agents https://techcommunity.microsoft.com/blog/copilot-studio-blog/computer-using-agents-in-microsoft-copilot-studio-are-now-generally-available/4519427Atlassian Agentic Pipelines with Claude Code https://support.atlassian.com/bitbucket-cloud/docs/agentic-pipelines/CVE-2026-46333 https://nvd.nist.gov/vuln/detail/CVE-2026-46333Kubernetes seccomp https://kubernetes.io/docs/reference/node/seccomp/GitHub OIDC for Dependabot and code scanning https://github.blog/changelog/2026-05-19-expanded-oidc-support-for-dependabot-and-code-scanning/Java pods OOMKilled in Kubernetes https://dzone.com/articles/java-pod-oomkill-kubernetesLLM-generated SQL risks https://readyset.io/blog/why-llms-write-incorrect-sql-and-what-that-means-for-your-databasePostmortem action items https://incident.io/blog/why-do-post-mortem-action-items-fail-how-to-make-incident-follow-ups-actually-get-doneOn Call Brief https://www.tellerstech.com/on-call-brief/2026-W21/More episodes + show notes https://shipitweekly.fm/
• 
  AI Agents Get API Access and Identity: GitHub Copilot Cloud Agents, MCP Auth, Ansible Automation, OpenAI Daybreak, and the New Production Risk 14.05.2026 23m

  This episode of Ship It Weekly is about AI agents moving from helpful coding assistants into real operational actors. Brian covers GitHub making Copilot cloud agent tasks available through a REST API, Auth0 bringing authentication and authorization to MCP servers, Red Hat positioning Ansible as a trusted execution layer for agentic IT operations, and OpenAI Daybreak pushing AI deeper into security research and remediation.The bigger thread this week is authority: what these agents can reach, what they can change, who approved the action, and who owns the outcome when something breaks.Brian also covers Discord’s ScyllaDB automation work, AWS GuardDuty crypto mining detection, queues and back pressure, and a Datadog PostgreSQL case where an index scan was still painfully slow.Sponsored by Guardsquare https://hubs.ly/Q04fJgkJ0LinksGitHub Copilot cloud agent tasks via REST API https://github.blog/changelog/2026-05-13-start-copilot-cloud-agent-tasks-via-the-rest-api/GitHub REST API endpoints for agent tasks https://docs.github.com/en/rest/agent-tasks/agent-tasksAuth0 Auth for MCP is now generally available https://auth0.com/blog/auth0-auth-for-mcp-servers-generally-available/Red Hat on Ansible as the execution layer for agentic IT https://www.redhat.com/en/about/press-releases/red-hat-establishes-ansible-automation-platform-trusted-execution-layer-it-operations-agentic-eraOpenAI Daybreak https://openai.com/daybreak/Discord automates ScyllaDB clusters at scale https://discord.com/blog/how-discord-automates-scylladb-clusters-at-scaleAWS GuardDuty crypto mining detection and prevention https://aws.amazon.com/blogs/security/detecting-and-preventing-crypto-mining-in-your-aws-environment/Queues do not absorb load, they delay failure https://dzone.com/articles/queues-dont-absorb-load-they-delay-bankruptcyDatadog on inefficient PostgreSQL index scans https://www.datadoghq.com/blog/detect-inefficient-index-scans-with-dbm/This week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W20/More episodes and show notes https://shipitweekly.fm/
• 
  Cursor Deletes PocketOS Prod DB, .de DNSSEC Outage, Bluesky Postmortem, Argo CD, and Copy Fail 08.05.2026 21m

  This episode of Ship It Weekly is about modern reliability getting squeezed from both directions. Old-school failures still hit hard, like broken DNSSEC, kernel privilege escalation bugs, and GitOps behavior changes. But newer automation layers add a second kind of risk, where AI agents, machine identity, and cloud control planes can do real damage fast when authority is too broad. Brian covers the Cursor and PocketOS production database wipe, the .de DNSSEC outage and Cloudflare’s response, Bluesky’s April outage postmortem, Argo CD v3.1.16 reaching end of life plus the v3.4.1 behavior change, Linux kernel CVE-2026-31431 under active exploitation, and why Google Cloud Agent Identity and AWS MCP Server GA both point to agents becoming first-class infrastructure actors.Sponsored by Guardsquare https://hubs.ly/Q04fJgkJ0LinksCursor / PocketOS production database wipe https://www.tellerstech.com/on-call-brief/2026-W19/Cloudflare on the .de DNSSEC outage https://blog.cloudflare.com/de-tld-outage-dnssec/Bluesky April 2026 outage postmortem https://pckt.blog/b/jcalabro/april-2026-outage-post-mortem-219ebg2Argo CD releases: v3.1.16 final release and v3.4.1 behavior change https://github.com/argoproj/argo-cd/releasesLinux kernel CVE-2026-31431 https://nvd.nist.gov/vuln/detail/CVE-2026-31431AWS bulletin for CVE-2026-31431 https://aws.amazon.com/security/security-bulletins/rss/2026-026-aws/Google Cloud Agent Identity https://cloud.google.com/blog/products/identity-security/whats-new-in-iam-security-governance-and-runtime-defenseAWS MCP Server is now generally available https://aws.amazon.com/blogs/aws/the-aws-mcp-server-is-now-generally-available/Cross-region disaster recovery for Amazon EKS using AWS Backup https://aws.amazon.com/blogs/containers/cross-region-disaster-recovery-for-amazon-eks-using-aws-backup/Google Ads new data retention policy starting June 1, 2026 https://ads-developers.googleblog.com/2026/05/new-data-retention-policy-for-google.htmlThis week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W19/More episodes and show notes https://shipitweekly.fm/
• 
  Ship It Conversations: Gareth Kersey on IaCConf 2026, AI, and Corey Quinn’s Terraform Keynote 05.05.2026 31m

  This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.This episode is not sponsored. I wanted to cover IaCConf because the theme lines up closely with what Ship It Weekly focuses on: infrastructure, platform engineering, DevOps, SRE, and how teams are adapting to AI-driven change.In this Ship It: Conversations episode, I talk with Gareth Kersey about IaCConf 2026, a free virtual conference focused on infrastructure as code, platform engineering, DevOps, SRE, and infrastructure operations. The conference is May 14th 2026.The main theme is “keeping pace.” Not just keeping pace with new tools, but keeping pace with the speed of software delivery now that AI is changing how quickly application teams can write, ship, and change code.We talk about what that means for the infrastructure teams underneath it all: the people responsible for Terraform, Kubernetes, GitOps, policies, secrets, cost, security, rollback paths, and making sure faster delivery does not turn into faster chaos.Gareth walks through the IaCConf 2026 agenda, including Corey Quinn’s keynote, AI and Terraform sessions, platform engineering panels, Kubernetes and Argo CD talks, AI agents managing infrastructure as code, governance challenges, and the risk of 10x code velocity becoming 10x operational risk.The bigger theme here is that AI is not just changing how code gets written. It is changing the pressure on the systems around delivery. Infrastructure as code, platform engineering, policy, and operational guardrails matter even more when the pace of change goes up.Highlights• What “keeping pace” means for infrastructure, DevOps, SRE, and platform teams• Why faster application development can create more downstream operational pressure• Corey Quinn’s keynote, “AI Speaks Terraform Like a Tourist”• How AI-generated infrastructure changes create new governance and review challenges• Why infrastructure as code still matters as AI agents and automation become more common• Sessions covering Terraform, Kubernetes, Argo CD, GitOps, platform engineering, and AI-driven workflows• The risk of 10x code velocity turning into 10x operational risk• How platform teams can support faster developers without giving up safety or governance• Why IaCConf includes panels, demos, technical talks, and practitioner stories instead of only tool-specific content• How IaCConf has grown from its first event in 2025 into a broader infrastructure community• Why the event is trying to stay community-focused instead of becoming just another vendor marketing conference• The role of feedback, future spotlight events, in-person meetups, and possible community spaces around IaCConf• Why registering still makes sense even if you cannot attend live, since sessions are available afterwardIaCConf links• IaCConf 2026 registration page - https://www.iacconf.com/iacconf-2026• IaCConf LinkedIn page - https://www.linkedin.com/showcase/iac-conf/• IaCConf: https://www.iacconf.com/• IaCConf is supported by Spacelift: https://spacelift.comOur linksMore episodes + show notes + links: https://shipitweekly.fmOn Call Brief: https://oncallbrief.com