AI Safety - Paper Digest

AI Safety - Paper Digest

Arian Abbasi, Alan Aqrawi
Country USA
Genres Technology
Language EN
Episodes 13
Latest 10.12.2025

This podcast breaks down the latest research and developments in AI safety, episode by episode. Each installment takes a deep dive into new cutting-edge papers, making complex ideas accessible to both experts and those who are just AI-curious. The show aims to keep listeners informed and ahead of the curve on AI security topics. Note that the podcast and its content are AI-generated, so listeners are encouraged to verify information independently.

Episodes

  • AI Agents: Adoption and Usage | Perplexity Comet
    AI Agents: Adoption and Usage | Perplexity Comet 10.12.2025 5m
    Dive into the key findings of the first large-scale field study on the adoption, usage intensity, and use cases of general-purpose AI agents, drawing on hundreds of millions of anonymized user interactions with Perplexity’s Comet Assistant. This paper tracks the frontier shift from conversational LLM chatbots to action-oriented AI agents, which are defined as AI assistants capable of autonomously pursuing user-defined goals by planning and executing multi-step actions.Key insights from the paper include:Who is Adopting: Adopters are concentrated in digital or knowledge-intensive sectors, with...
  • WEF & Accenture | Advancing Responsible AI Innovation: A Playbook
    WEF & Accenture | Advancing Responsible AI Innovation: A Playbook 25.09.2025 2m
    This episode of the AI Safety Paper Digest is about the World Economic Forum's new playbook on advancing responsible AI innovation. In cooperation with Accenture, the report provides a practical roadmap for turning responsible AI from an aspiration into a competitive advantage while building public trust.Link to the Report: https://www.weforum.org/publications/advancing-responsible-ai-innovation-a-playbook/ Disclaimer: This summary was generated with the assistance of Google’s NotebookLM AI. For full technical details and comprehensive findings, please consult the original report.
  • Okay Waymo, Crash My Car! 🗣️ Testing Autonomous Vehicle Safety with Adversarial Driving Scenarios | LD-Scene
    Okay Waymo, Crash My Car! 🗣️ Testing Autonomous Vehicle Safety with Adversarial Driving Scenarios | LD-Scene 20.08.2025 18m
    How can we make autonomous driving systems safer through generative AI? In this episode, we explore LD-Scene, a novel framework that combines Large Language Models (LLMs) with Latent Diffusion Models (LDMs) to create controllable, safety-critical driving scenarios. These adversarial scenarios are essential for evaluating and stress-testing autonomous vehicles, yet they’re extremely rare in real-world data.Sources referenced in this episode:Mingxing Peng, Yuting Xie, Xusen Guo, Ruoyu Yao, Hai Yang, Jun Ma: “LD-Scene: LLM-Guided Diffusion for Controllable Generation of Adversarial Safety-Critica...
  • The Full LLM Glossary and Foundations
    The Full LLM Glossary and Foundations 04.08.2025 1h 28m
    Ever wanted a clear, comprehensive explanation of all the key terms related to Large Language Models (LLMs)? This episode has you covered.In this >1-hour deep-dive, we'll guide you through the essential glossary of LLM-related terms and foundational concepts, perfect for listening while driving, working, or on the go. Whether you're new to LLMs or looking to reinforce your understanding, this episode is designed to make complex terms accessible.Sources referenced in this episode:Humza Naveed et al., "A Comprehensive Overview of Large Language Models"Tessa Gengnagel et al., "LLM Glossary (draft...
  • Anthropic's Best-of-N: Cracking Frontier AI Across Modalities
    Anthropic's Best-of-N: Cracking Frontier AI Across Modalities 25.12.2024 12m
    In this special christmas episode, we delve into "Best-of-N Jailbreaking," a powerful new black-box algorithm that demonstrates the vulnerabilities of cutting-edge AI systems. This approach works by sampling numerous augmented prompts - like shuffled or capitalized text - until a harmful response is elicited. Discover how Best-of-N (BoN) Jailbreaking achieves: 89% Attack Success Rates (ASR) on GPT-4o and 78% ASR on Claude 3.5 Sonnet with 10,000 prompts. Success in bypassing advanced defenses on both closed-source and open-source models. Cross-modality attacks on vision, audio, and multimodal A...
  • Auto-Rewards & Multi-Step RL for Diverse AI Attacks by OpenAI
    Auto-Rewards & Multi-Step RL for Diverse AI Attacks by OpenAI 30.11.2024 11m
    In this episode, we explore the latest advancements in automated red teaming from OpenAI, presented in the paper "Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learning." Automated red teaming has become essential for discovering rare failures and generating challenging test cases for large language models (LLMs). This paper tackles a core challenge: how to ensure attacks are both diverse and effective. We dive into their two-step approach: Generating Diverse Attack Goals using LLMs with tailored prompts and rule-based rewards (RBRs). Training an RL...
  • Battle of the Scanners: Top Red Teaming Frameworks for LLMs
    Battle of the Scanners: Top Red Teaming Frameworks for LLMs 04.11.2024 14m
    In this episode, we explore the findings from "Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysis." As large language models (LLMs) are integrated into more applications, so do the security risks they pose, including information leaks and jailbreak attacks. This study examines four major open-source vulnerability scanners - Garak, Giskard, PyRIT, and CyberSecEval - evaluating their effectiveness and reliability in detecting these risks. We’ll discuss the unique features of each tool, uncover key gaps in their reliability, and share strategic recommendat...
  • Watermarking LLM Output: SynthID by DeepMind
    Watermarking LLM Output: SynthID by DeepMind 24.10.2024 12m
    In this episode, we delve into the groundbreaking watermarking technology presented in the paper "Scalable Watermarking for Identifying Large Language Model Outputs," published in Nature. SynthID-Text, a new watermarking scheme developed for large-scale production systems, preserves text quality while enabling high detection accuracy for synthetic content. We explore how this technology tackles the challenges of text watermarking without affecting LLM performance or training, and how it’s being implemented across millions of AI-generated outputs. Join us as we discuss how SynthID-Text could re...
  • Open Source Red Teaming: PyRIT by Microsoft
    Open Source Red Teaming: PyRIT by Microsoft 08.10.2024 10m
    In this episode, we dive into PyRIT, the open-source toolkit developed by Microsoft for red teaming and security risk identification in generative AI systems. PyRIT offers a model-agnostic framework that enables red teamers to detect novel risks, harms, and jailbreaks in both single- and multi-modal AI models. We’ll explore how this cutting-edge tool is shaping the future of AI security and its practical applications in securing generative AI against emerging threats. Paper (preprint): Lopez Munoz, Gary D., et al. "PyRIT: A Framework for Security Risk Identification and Red Teaming in Generati...
  • Jailbreaking GPT o1: STCA Attack
    Jailbreaking GPT o1: STCA Attack 07.10.2024 8m
    This podcast, "Jailbreaking GPT o1, " explores how the GPT o1 series, known for its advanced "slow-thinking" abilities, can be manipulated into generating disallowed content like hate speech through a novel attack method, the Single-Turn Crescendo Attack (STCA), which effectively bypasses GPT o1's safety protocols by leveraging the AI's learned language patterns and its step-by-step reasoning process. Paper (⁠preprint): Aqrawi, Alan and Arian Abbasi. “Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA).” (2024). TechRxiv. Disclaimer: This podcast was generated using Google's...
  • The Attack Atlas by IBM Research
    The Attack Atlas by IBM Research 05.10.2024 11m
    This episode explores the intricate world of red-teaming generative AI models as discussed in the paper "Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI." We'll dive into the emerging vulnerabilities as LLMs are increasingly integrated into real-world applications and the evolving tactics of adversarial attacks. Our conversation will center around the "Attack Atlas" - a practical framework that helps practitioners analyze and secure against single-turn input attacks - and we'll examine the critical challenges in both red- and blue-teaming generative A...
  • The Single-Turn Crescendo Attack
    The Single-Turn Crescendo Attack 04.10.2024 6m
    In this episode, we examine the cutting-edge adversarial strategy presented in "Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA)." Building on the multi-turn crescendo attack method, STCA escalates context within a single, expertly crafted prompt, effectively breaching the safeguards of large language models (LLMs) like never before. We discuss how this method can bypass moderation filters in a single interaction, the implications of this for responsible AI (RAI), and what can be done to fortify defenses against such sophisticated exploits. Join us as we break down how a s...
  • Outsmarting ChatGPT: The Power of Crescendo Attacks
    Outsmarting ChatGPT: The Power of Crescendo Attacks 03.10.2024 9m
    This episode dives into how the Crescendo Multi-Turn Jailbreak Attack leverages seemingly benign prompts to escalate dialogues with large language models (LLMs) such as ChatGPT, Gemini, and Anthropic Chat, ultimately bypassing safety protocols to generate restricted content. The Crescendo attack begins with general questions and subtly manipulates the model’s responses, effectively bypassing traditional input filters, and shows a high success rate across popular LLMs. The discussion also covers the automated tool, Crescendomation, which surpasses other jailbreak methods, showcasing the vulnera...

Popular in

The podcast also appears in the podcast charts of these countries.