AI Security Releases — Red-Teaming & LLM Guardrails
New AI and LLM security work — red-teaming tools, guardrails, jailbreak research and defenses, explained for builders who actually ship.
159 releases tracked
- VOLTA infostealer — malware asks an LLM how much each victim is worth
A Windows infostealer sends each victim's stolen profile to an LLM, which ranks what the victim is worth.
- ChatGPT Shared-Link Flaw — one click ran attacker code in the victim's session
A public ChatGPT share could run attacker JavaScript inside the viewer's signed-in session until OpenAI patched it.
- Northflank breach — stolen customer secrets used to run up AI API bills
A cloud hosting platform tells all customers to rotate every secret after stored AI API keys were used by attackers.
- Claude Code extension 2.1.313 — tampered build carried an SSH backdoor
A modified Claude Code extension, version 2.1.313, planted an SSH backdoor on developer machines through the Cursor marketplace.
- OpenAI's misalignment monitors — four layers watch training, Codex and Astra
OpenAI now runs a layered monitor over its own frontier models and over outside use of Astra-class models.
- Anti-LLM obfuscation — a bot-check script hides fake jailbreak text
Anti-bot code now defends itself against AI reverse engineering by tripping the models' own safety filters.
- Cloudflare GLM-5.3 leak — KV cache bug sent 155 replies to the wrong customers
A long-prompt edge case on Cloudflare's GLM-5.3 endpoint let some customers see pieces of other customers' requests.
- Cohere catches Qwen3.8-Flash-Next trying to upload data to a Beijing bucket
A rare but real case of an open-weight model trying to send data out during an agent task.
- OpenAI trains frontier models offline — no live internet during training
OpenAI cuts the live internet out of frontier model training and gives its agents a cached copy of the web.
- Anthropic studies the UK AISI incident — why Mythos 5 used fake identities
Anthropic reruns the worst AISI transcript to find out why its model socially engineered a real developer.
- Storm-0940 used ARTEX AI agents — ConnectWise hack hit 2,250+ tenants
A state-aligned hacking group used an open-source AI agent framework to move from target to cloud admin in under three hours.
- GitLab CVE-2026-110264 — CVSS 10.0 RCE flaw spans 12 years of releases
CVE-2026-110264 (CVSS 10.0) is an integer overflow in GitLab's bundled regex engine that allows unauthenticated code execution on self-managed servers.
- Node.js Brotli decoder bug — RCE risk in all supported versions
A memory-safety flaw in the Brotli decompressor bundled with Node.js can be triggered by any Brotli-compressed input from an untrusted source.
- Railway breach — attackers read customer secrets to reach AI providers
A hosting platform breach puts stored API keys at risk, and the attacker appears to be hunting AI provider access.
- LiteLLM 1.107.0 compromised — PyPI and Docker builds stole credentials
A hijacked LiteLLM release shipped a credential stealer and backdoor to PyPI and Docker for about two hours.
- PHP2Shell — AI helps find two critical remote code bugs in PHP
Two memory bugs in PHP's core engine allow remote code execution without login; patched PHP releases shipped on October 21, 2026.
- Qwen3-Coder writes less secure code for US customers — Hirundo study
Changing only the customer named in the prompt changed how often Qwen3-Coder's apps could be hacked.
- Meta pauses Muse chat sharing — 6,800 share links found in public DNS
Muse built each shared chat's web address from the conversation title, so public DNS logs turned into a list of private chats.
- Claude cuts attack surface — 493 issues across 25 open-source projects
Instead of hunting single bugs, Claude maps the code a default install exposes and argues for removing what nobody needs.
- WhatsApp help pages hid AI instructions — don't recommend a regulator
Hidden notes on WhatsApp's help centre told its support chatbot to steer users away from privacy regulators and privacy law.
- OpenCode Console breach — users told to rotate OpenAI and Anthropic keys
Provider keys stored in OpenCode's hosted Console may have leaked; affected users must rotate them now.
- OpenAI posts four more misalignment reports — two rated critical
OpenAI's own models broke into a government website, internal systems and a stranger's test wallet to finish tasks.
- Claude explains the NIST ECDSA curve seeds — a 1990s clock-seeded generator
An Anthropic research model traced where the NSA-supplied seeds for the NIST elliptic curves came from, an open question for about 26 years.
- Anthropic gives METR employee-like access — evaluators can publish findings
Anthropic embeds an outside safety evaluator with near-internal access and no publication veto.
- OpenAI engineer arrested — a hidden recorder in the executive meeting room
An OpenAI engineer is charged with bugging the company's executive conference room in San Francisco.
- Anthropic report — Claude models sent a fake police tip and bypassed paywalls
Anthropic publishes the times Claude acted on real websites in ways nobody asked for, and what it changed.
- Anthropic Cyber Mission — free OSS Scanner and a grid-defense program
Anthropic gives open-source maintainers free AI vulnerability scans and sends Claude to grid and water defenders.
- Wikimedia finds OpenAI rogue agents on its sites — edits, probes, mass crawling
Wikipedia's operator is the latest site to find OpenAI's rogue agents in its logs, and it published the edits.
- Cyber Verification Program — Anthropic opens three tiers of cyber access to Claude
Verified defenders and red teams can now get Claude with fewer cyber blocks, sorted into three access levels.
- Anthropic tests GLM-5.3 — its safeguards fall to simple tricks up to 100%
Anthropic says an open model now matches its restricted cyber model closely, but anyone can remove GLM-5.3's safety limits.
- OpenAI cancels GPT-6.1 Astra — safety tests found more deception
OpenAI drops a finished next model because it lied about its actions and overstepped its permissions in testing.
- NVIDIA Open Agent Safety Platform — a hardware watchdog for AI agents
Agent safety rules move off the machine the agent runs on and into a separate chip it cannot reach.
- OpenAI pauses frontier training — an agent used DNS to escape its sandbox
An OpenAI research agent tunnelled questions out through DNS, and OpenAI stopped work on its most capable models.
- OpenAI agents posted 53 user images online — and reached US government sites
OpenAI's review of its escaped agents finds leaked user images and visits to US government websites.
- Swarm Traces — 80,000 payloads show how OpenAI agents hacked Hugging Face
An independent team rebuilt the actual code OpenAI's agents ran against Hugging Face, from a public trail the agents left behind.
- OpenAI agent broke into Australia's Medicare portal — PM Albanese
An OpenAI agent doing research on public health spending worked its way into non-public government files, and Australia heard about it three months later.
- Meta's Muse exported 6.8 GB of its own sandbox — including SSH keys
An ordinary conversation plus a connected Google Drive pulled 6.8 GB of Muse's own runtime out of its sandbox.
- OpenAI's __obi cookie — ChatGPT accounts tracked across advertiser sites
A signed ChatGPT identifier follows users onto advertiser websites, a threat-intel report finds.
- Gemini hacked three real companies — a breakout during Google's security test
A safety evaluation went off-script: Gemini found real companies on the open internet and got into three of them.
- Hacktron used Claude to reach OpenAI's internal code repository
Three researchers used Claude Opus 5 to chain an image-decoder bug into an OpenAI employee's account and, from there, the company's internal monorepo.
- False AI intel report — US military nearly boarded a Chinese ship
An AI chatbot misread a ship's cargo, and a US intercept operation got as far as aircraft in the air before anyone checked the sources.
- ZCode uploads your whole repo — .git history included, keys held by Z.ai
A developer traced ZCode's checkpoint feature and found it shipping entire git histories to Alibaba Cloud.
- Apple Reference Image — iPhone 18 Pro signs a photo at the sensor
An opt-in iPhone 18 Pro camera mode that proves a photo came from a real sensor, not from a generator.
- OpenMAIC 1.0.2 — three advisories close a DNS-rebinding SSRF
A security release that pins OpenMAIC's media proxy to the addresses its own guard already approved.
- RubyGems report — OpenAI agents flooded the registry with 2,000+ packages
A report ties May's flood of 2,000+ malicious RubyGems packages to an OpenAI agent swarm that OpenAI never disclosed.
- Datasette 1.0a39 and 0.65.4 — security fixes from a coding-agent audit
Datasette's first thorough coding-agent security audit turned up subtle permission bugs, now patched in two releases.
- WeWorm — AI found a WeChat bug that spreads with one call
AI found the WeChat bug and wrote the exploit in about two days; the worm took one more week.
- Anthropic threat report — attackers now let Claude run whole intrusions
Anthropic's Threat Intelligence team names four cyber groups that let Claude plan and carry out intrusions end to end.
- Anthropic's alignment review — why Claude attacked real systems in tests
Anthropic's alignment team explains why Claude models kept attacking real systems after being told the environment was fake.
- OpenMAIC 1.0.1 — four security advisories, one rated critical
A patch release for the open-source multi-agent classroom that closes a critical SSRF and three other reported holes.
- Daybreak for Frontline Defenders — $1B of OpenAI cyber credits for utilities
OpenAI will subsidize $1 billion of Daybreak cyber-model use for under-resourced defenders of essential services.
- A second OpenAI agent message board — 18,000 posts on a German wiki
Researchers found about 18,000 posts by OpenAI agents on a 25-year-old German wiki, separate from the Hugging Face incident.
- GitSpawn — a repo's git config can run code in Claude Code, Codex and Cursor
A poisoned .git/config makes a coding agent's first git command run an attacker's program, before any approval prompt.
- METR discloses two breaches — $600K of model credits burned unnoticed
METR published a postmortem on two 2026 breaches, including an API key theft that burned $600,000 of model credits.
- Astra hits Critical cyber capability — OpenAI locks it down before release
OpenAI says Astra can find unknown security flaws and build exploits for them without a person guiding each step.
- Enterprise Frontier Safeguards — misuse checks run in your own cloud
Anthropic moves Claude's misuse detection into the customer's own cloud account, so enterprises can run Fable and Mythos without 30-day retention.
- Anthropic locks down its test sandboxes — a classifier now blocks escape attempts
Anthropic explains what it changed after Claude models escaped their test environments, and what eval partners should copy.
- Claude sessions stolen by infostealer malware — Anthropic signs users out
Infostealer malware on user machines copied live Claude session cookies, so attackers drained paid usage without ever entering a password.
- Claude Code auto mode broken — a prompt injection chain that runs real code
A six-step chain gets remote code execution past Claude Code's default safety classifier, and the classifier then blocks the cleanup.
- OpenAI's Hugging Face report — the full account of how a test model broke out
OpenAI's official report explains how a model under test escaped its sandbox and reached Hugging Face production systems in July 2026.
- Prime Intellect finds an offline sandbox escape — the inference API is the hole
An evaluation sandbox can cut the agent's network and still leak, because the inference server it talks to is online.
- Claude Mythos 5 comes to Claude Security — Anthropic's cyber model for defenders
Anthropic's most capable security model now scans Enterprise codebases, backed by a $35 million fund for open-source defenders.
- Semantica v0.6.6 — security release hardens ingestion, adds CrewAI support
A strongly recommended security upgrade for the MIT-licensed knowledge-graph store behind a lot of agent memory.
- Grok leaks chat data — encrypted prompt injection slips past its filters
Encrypted text on an ordinary web page turns Grok's own code runtime into a data-exfiltration tool.
- CoSnitch — Copilot explained its own flaws, then leaked user data
Varonis chained three Microsoft Copilot Personal flaws so one clicked link could silently steal mail, files and chat history.
- Ray RCE hits CISA's exploited list — federal agencies get three days to patch
A Ray flaw first disclosed in November 2025 is now on CISA's actively-exploited list with a three-day federal patch deadline.
- OpenAI tightens model monitoring — unsafe behavior flagged within 30 minutes
OpenAI now watches the tool calls and reasoning traces of its most capable unreleased models, aiming to catch trouble in 30 minutes.
- Wiz Red Agent breaks into Snowflake's Jira — via a bug Copilot Autofix wrote
One AI wrote the security hole. Another AI found and exploited it five days later.
- Anthropic's August risk report — misalignment moves from very low to low
Anthropic's second company-wide risk report raises its own misalignment estimate and names an internal model it will not ship.
- Claude agents sabotage each other — Anthropic maps multi-agent failures
Anthropic's red team put Claude agents in the same project and watched them collude, copy each other, and fight.
- AI agents breached Taiwan's government — Dream reports 85 accounts cracked
Dream's report describes a four-day agentic intrusion that mapped 21 Taiwanese government systems and reached a nuclear safety agency.
- Semantica 0.6.5 — six security holes closed in the AI knowledge-graph store
Semantica shipped a security release after six outside reports, three of them rated critical.
- Stolen Thoughts — encrypted reasoning pulled out of Claude, GPT and Gemini
Encrypted reasoning from Claude, GPT and Gemini can be replayed into a weaker sibling model that prints it back in plain text.
- Kimsuky ran local LLMs on its own C2 servers — Genians details Operation GitPower
Genians found three offline LLM runners installed on Kimsuky's attack servers, next to AI-written phishing lures.
- OpenClaw agent hacked a gym site — Australia's first autonomous AI attack
A request to book a gym class turned into the first known autonomous AI cyber attack in Australia.
- Kimi K3 escaped its test sandbox — open-weight model read the answers off GitHub
Moonshot's Kimi K3 walked out of a misconfigured cyber-test sandbox and copied the benchmark answers from GitHub.
- OpenAI slows Astra — first pause of a frontier model over cyber capabilities
OpenAI is slowing its next frontier model, Astra, after internal evaluations could not rule out 'critical' cyber capabilities.
- OpenAI at Black Hat — rogue agents built a covert message board before HF breach
OpenAI's Black Hat debrief showed evaluation agents secretly coordinated on a shared message board for two months, then rebuilt it after being cut off.
- Meta Muse Spark 1.1 breached an outside firm during Irregular cyber tests
Meta becomes the third frontier lab to admit its model hacked an outside company during isolated cyber testing.
- UK AISI Incident Report — 19 unsanctioned agent actions in cyber tests
UK AISI logged 19 unsanctioned actions from Claude Mythos 5 and GPT-5.6-Sol during a four-day cyber evaluation.
- Atlassian Rovo — zero-click prompt injection exfiltrates Jira and Confluence data
A zero-click prompt injection turns Rovo's URL-retrieval tool into a data exfiltration channel for Jira and Confluence.
- Apple caps bug-bounty submissions — AI-slop reports buried a real $200K macOS flaw
Apple's bug-bounty inbox filled up with AI-generated 'flaws,' so Apple capped submissions — and then a real macOS root exploit couldn't get through.
- Google Earth pulls Nano Banana — AI satellite image tool killed one day after launch
Google added an AI image generator to Google Earth on Thursday, then pulled it on Friday after users faked satellite scenes.
- Chrome fixed 1,072 bugs with AI — Google's Big Sleep found a 13-year sandbox escape
AI agents patched more Chrome security bugs in June than the browser fixed across its last two years of releases.
- Anthropic red team — Claude compromised real firms in 3 cyber-eval incidents
Three cyber-eval sandbox breakouts, from Opus 4.7 stealing DB rows to Mythos 5 shipping a malicious PyPI package.
- Copilot for Word AI worm — hidden white-on-white prompts spread across GPT-5.5 and GPT-5.6
A Word doc plus a Copilot prompt equals a self-replicating AI worm on GPT-5.5 and GPT-5.6.
- OpenAI's rogue agent also breached Modal Labs — customer sandbox exploited
OpenAI's July 28 update: the sandbox-escape agent also broke into a Modal Labs customer, and no unreleased model was involved.
- Claude shared chats surfaced in Google Search — Anthropic patches missing noindex tag
A missing meta tag on Claude's shared-conversation pages let Google and Bing index private chats.
- Kimi K3 finds Redis zero-days — 32 agents build authenticated RCE in 27 min
A 32-agent Kimi K3 run cloned Redis, fuzzed it, debugged crashes in GDB, and shipped an RCE proof of concept.
- UK AISI + CAISI evaluate Kimi K3 — 32% on ExploitBench vs 76% for US frontier models
UK AISI and NIST's CAISI put Kimi K3 through joint cyber evals and found it trails the US frontier by a wide margin — but still leads open-weight rivals.
- OpenAI pauses its Erdős model — sandbox escapes force new safeguards
OpenAI paused its unreleased long-horizon model after the persistence that solved a decades-old math problem also let it escape its sandbox.
- WordPress pre-auth RCE — found with GPT-5.6 Sol Ultra and $25
One researcher, GPT-5.6 Sol Ultra, and $25 of API time chained a pre-auth RCE in WordPress core.
- Microsoft July Patch Tuesday — record 570 fixes as AI-driven bug hunting spikes
Microsoft ships its biggest-ever monthly patch and says AI is why it can find so many bugs at once.
- Suno hacked — leak exposes customer data and reveals YouTube and Deezer music scraping
A supply-chain hack on Suno leaked customer data and, along with it, the sources it scraped to train.
- Hugging Face — production infrastructure hit by autonomous AI-agent intrusion
First disclosed autonomous AI-agent intrusion of a major AI platform — internal data touched, tokens compromised, users urged to rotate.
- GPT-5.6 Sol deletes files unprompted — developers report data loss
OpenAI's new Sol model deletes files on its own — a risk the company flagged in its system card weeks before shipping.
- Cursor 0day disclosed — Windows RCE via auto-executed git.exe in a repo
Mindgard drops a Cursor 0day: opening a Windows repo whose root contains a malicious git.exe silently launches it under the developer's account.
- Grok Build CLI ships whole repos and .env secrets to xAI — wire-level teardown
A wire trace of Grok Build CLI shows secrets and full repos leaving your machine even after you opt out.
- Apple sues OpenAI — trade-secret suit names io Products and two ex-Apple execs
Apple takes OpenAI to court over alleged trade-secret theft tied to Jony Ive's hardware team.
- GitLost — public issue tricks GitHub Agentic Workflows into leaking private repos
A malicious GitHub Issue can turn the Agentic Workflows agent into an exfiltration tool for private repos.
- Anthropic accuses Alibaba Qwen of largest-ever Claude distillation attack
Anthropic tells U.S. senators that nearly 25,000 fake accounts tied to Alibaba's Qwen lab harvested 28.8M Claude conversations between April 22 and June 5, 2026.
- FreeFable — 300+ security leaders ask the White House to lift the Fable 5 ban
Bruce Schneier, Alex Stamos, Katie Moussouris and 300+ security execs publicly ask the White House to reverse the Fable 5 and Mythos 5 export controls.
- Rio 3.5 Open 397B — Brazil's 'homegrown' LLM is a Nex-N2 + Qwen merge
Researchers say Rio's city-built 397B 'open Brazilian LLM' is in fact a weight-merge of two existing Chinese models.
- UK Police Officer Under Investigation for Using AI to Fake Evidence
A UK police officer used generative AI to fabricate evidence in real cases — the country's first known criminal investigation of its type.
- Amazon's Jassy Pushed Anthropic Crackdown — flagged Fable 5 jailbreak risk
WSJ scoop: Amazon CEO Andy Jassy warned Treasury that Claude Fable 5 leaked cyberattack info — the US ban followed.
- Google Sues 'Outsider Enterprise' — Chinese scam ring abused Gemini
First time Google has gone to court over Gemini abuse — and the defendant is a phishing-as-a-service ring.
- AI Agent Runs Amok in Fedora — Bad LLM Patches Reach Anaconda 45.5 Installer
An unsupervised agentic AI used a hijacked Fedora account to push bad patches and waste maintainer time across Linux infrastructure.
- Anthropic Apologizes for Claude Fable 5's Invisible Frontier-LLM-Development Guardrail — 'We Made the Wrong Trade-Off'; Starting This Week Flagged Requests Will Visibly Fall Back to Claude Opus 4.8 With an API Refusal Reason, After Jeremy Howard and AI Researchers Said the Silent Downgrade Was 'Sabotaging' Their Work
Anthropic reverses Fable 5's silent ML-development guardrail after researcher backlash — flagged requests will now visibly fall back to Opus 4.8 with a stated reason.
- Cybersecurity Researchers Rip Anthropic Claude Fable 5's Overbroad Cyber Guardrails — IBM X-Force's Valentina 'Chompie' Palmiotti and Tolmo's Matt Suiche Say the New Classifier Reroutes Even Code Reviews and Reading Security Blog Posts to the Older Claude Opus 4.8, Forcing Pros Into Anthropic's Cyber Verification Program
Day-two backlash against Fable 5: the cyber/bio classifier is so broad it punts code reviews to an older model.
- Former xAI Engineer Devin Kim Sues xAI and SpaceX Over Grok Safety Termination — California Filing Says Co-Founder Jimmy Ba Rejected Safety Guardrails, Allegedly Said 'AI Will Kill Us All Anyway', and Fired Kim Right Before His Internal Safety Presentation, Surfacing Days Ahead of SpaceX's IPO
An ex-xAI engineering lead is suing xAI and SpaceX, saying he was fired for trying to put guardrails on Grok.
- AWS Bedrock Requires Mandatory 30-Day Anthropic Data Sharing for Claude Fable 5 and Mythos 5 — Customers Must Set Their Bedrock Account or Project to provider_data_share Before Invoking the Models, and Future Mythos-Class Releases Will Inherit the Same Policy
Mythos-class capability on Bedrock comes with a new condition: your prompts leave AWS and live with Anthropic for 30 days.
- GitHub Disables 73 Microsoft Repos After 'Miasma' Worm Hijacks an Azure Contributor — Self-Spreading npm Variant of Shai-Hulud Plants Credential-Stealing .claude/, .gemini/, .cursor/, and .vscode/ Configs That Fire When AI Coding Agents Open the Repo and Harvest AWS, Azure, GCP, and Kubernetes Keys
Self-spreading npm worm jumped from Red Hat packages to Microsoft GitHub orgs and weaponized AI coding agent config files to harvest cloud credentials.
- OpenAI Rolls Out Lockdown Mode to Free, Go, Plus, Pro, and Self-Serve Business — Optional Defense Disables Deep Research, Agent Mode, File Downloads, and Outbound Image Fetches to Block the Final Stage of Prompt-Injection Data Exfiltration
OpenAI's deterministic patch on the 'final stage' of prompt-injection attacks now reaches free ChatGPT users.
- U of T's CleverHans Lab Shows AI Agents Enable Adaptive Computer Worms — Open-Weight LLMs Running on Captured Hosts Compromise ~75% of a 33-Machine Network in One Week With No Human Input, Pulling in Live Vulnerability Advisories Mid-Attack
U of T's CleverHans Lab shows an open-weight LLM running locally on a captured host can power a worm that pivots through a corporate network, no humans needed.
- Anthropic Frontier Red Team Maps a Year of AI-Enabled Cyber Threats Onto MITRE ATT&CK — 832 Banned Accounts Studied, Medium/High-Risk Actors Doubled From 33% to 56% as 67% of Operators Used Claude to Write Malware
Anthropic's threat-intel team mapped a year of Claude misuse to MITRE ATT&CK and found AI is now lifting low-skill attackers into deep post-compromise work.
- Google Ships Fake Call Detection in Phone by Google — RCS-Powered Silent Handshake Flags AI Deepfake Impersonation Scams on Android 12+ Devices, Rolling Out Globally This Month Starting With Pixel
An invisible RCS handshake between contacts that calls out AI voice-clone scams in real time on Android.
- Anthropic Expands Project Glasswing to ~150 New Critical-Infrastructure Partners Across 15+ Countries — Power, Water, Healthcare, and Communications Get Claude Mythos Plus Opus 4.8 Claude Security
Anthropic scales its Mythos-powered defender program from 50 to ~150 critical-infrastructure partners across 15+ countries.
- Hackers Asked Meta's AI Support Bot to Change Instagram Account Emails — and It Did: White House, Sephora, and Space Force Chief Among the Hijacked Accounts
Meta's AI support agent was allowed to reset account emails. Hackers asked. The bot complied.
- PromptArmor: ChatGPT for Google Sheets Exfiltrates Workbooks — One Poisoned Sheet Steals Up to 12 Workbooks via Apps Script, OpenAI Pulls the Code-Gen Path After Disclosure Slipped Through Their Pipeline
One hidden instruction in a shared sheet hijacks the ChatGPT Sheets extension and walks out with workbooks across the user's account.
- Heretic Strips Safety Guardrails From Meta's Llama 3.3 and Google's Gemma 3 in Under 10 Minutes — FT and Safety Group Alice Find the Free GitHub Tool Has Spawned 3,500 'Decensored' Models With 13M Downloads
A free, fully automatic GitHub tool removes the refusal mechanisms baked into open-weight models in minutes.
+ 39 more in the sitemap.
