{"id":5262,"date":"2026-07-23T22:06:54","date_gmt":"2026-07-23T22:06:54","guid":{"rendered":"https:\/\/nextcitydigest.com\/?p=5262"},"modified":"2026-07-23T22:06:54","modified_gmt":"2026-07-23T22:06:54","slug":"openai-models-break-out-of-test-environment-hack-company-to-steal-answers","status":"publish","type":"post","link":"https:\/\/nextcitydigest.com\/?p=5262","title":{"rendered":"OpenAI models break out of test environment, hack company to steal answers"},"content":{"rendered":"
\n
\n

An autonomous AI agent powered by OpenAI models escaped a testing environment designed to isolate it from the internet and hacked another company\u2019s systems to steal answers to a cybersecurity test, an \u201cunprecedented\u201d breach that demonstrated how AI agents can evade their developers\u2019 controls in pursuit of assigned goals.<\/p>\n

Read more Two Bay Area men sentenced to life without parole for revenge killing of San Mateo man<\/a><\/p>\n

OpenAI said the agent used a combination of models, including GPT-5.6 Sol and a more capable prerelease system. It was undertaking an internal evaluation with normal cybersecurity safeguards disabled when it discovered an unknown flaw in third-party software, reached the open internet and penetrated New York-based AI platform Hugging Face.<\/p>\n

In pursuit of its assigned goal, the agent stole credentials, moved through internal computer systems and obtained secret information it could use to cheat the evaluation, according to disclosures Tuesday from the two companies.<\/p>\n

Hugging Face said the breach exposed a limited set of internal datasets and several credentials. It found no evidence of tampering with public-facing models, datasets or Spaces and said its software supply chain was clean, although it was still determining whether any customer or partner data was affected.<\/p>\n

The breach shows how AI agents could potentially penetrate crucial computer systems belonging to schools, hospitals, utilities, government agencies and even the Pentagon, said Shomit Ghose, a lecturer at UC Berkeley\u2019s engineering school and a partner at Menlo Park venture capital firm Clearvision Ventures.<\/p>\n

\u201cThis is the first leak in the dike,\u201d Ghose said.<\/p>\n

San Francisco-based OpenAI said the breach occurred while it was testing how effectively its models could be used in sophisticated cyberattacks. To measure their maximum capabilities, the company said it disabled production safeguards that ordinarily prevent its models from pursuing high-risk cyber activity.<\/p>\n

The agent was supposed to remain in a highly isolated environment with internet access limited to internally hosted third-party software that allowed it to install software packages.<\/p>\n

\n<\/div>\n

Instead, the agent devoted substantial computing power to finding a path to the open internet, OpenAI said. It discovered and exploited a previously unknown vulnerability in the third-party software, then moved through OpenAI\u2019s research environment until reaching a computer with internet access.<\/p>\n

Once online, the agent concluded that Hugging Face might possess information related to ExploitGym, the cybersecurity benchmark it was attempting to complete.<\/p>\n

The agent then searched for a way into Hugging Face, combining stolen credentials and previously unknown vulnerabilities to remotely execute code on the company\u2019s servers and find test solutions stored in its production database, OpenAI said.<\/p>\n

Hugging Face said the intrusion began with a \u201cmalicious dataset\u201d that exploited two weaknesses in its data-processing pipeline.<\/del><\/p>\n

\n<\/div>\n

\u201cThis one was different from anything we had handled before in one important way: It was driven, end to end, by an autonomous AI agent system,\u201d Hugging Face said. \u201cThis matches the \u2018agentic attacker\u2019 scenario the industry has been forecasting.\u201d<\/p>\n

The system performed thousands of automated actions using a swarm of short-lived testing environments, Hugging Face said.<\/p>\n

At the time of its initial disclosure July 16, Hugging Face said it did not know who had developed the AI models behind the intrusion.<\/p>\n

OpenAI revealed Tuesday that its models were responsible, calling the breach \u201can unprecedented cyber incident\u201d involving state-of-the-art capabilities.<\/p>\n

OpenAI said its security team independently discovered the unusual activity. Hugging Face said its security team and AI agents detected and stopped the intrusion, then began containing the breach and reconstructing what happened.<\/p>\n

Hugging Face closed the vulnerabilities, rebuilt compromised computer nodes and revoked or replaced affected credentials and access tokens. It also reported the intrusion to law enforcement and brought in outside cybersecurity specialists.<\/p>\n

OpenAI said it had imposed stricter controls on its testing infrastructure while the vulnerabilities were patched, disclosed the previously unknown flaw to the third-party software provider and begun strengthening protections around future evaluations. The company also said it was regularly briefing its Safety and Security Committee.<\/p>\n

Hugging Face CEO Cl\u00e9ment Delangue said Tuesday in a social media post that his company had collaborated with OpenAI during the investigation and \u201cwe strongly believe there was no malicious intent on their part.\u201d<\/p>\n

But the breach demonstrates that sophisticated AI systems can discover and exploit novel attack paths in real-world systems without access to their underlying source code, OpenAI acknowledged.<\/p>\n

Read more Big 12 MBB projections: Arizona remains favorite following NBA draft and transfer portal decisions<\/a><\/p>\n

It also raises questions about why models equipped with advanced offensive capabilities were placed in a testing environment that retained a potential path to the internet.<\/p>\n

The release of ChatGPT in late 2022 helped ignite a generative AI boom that poured billions of dollars into Silicon Valley. It also intensified concerns that increasingly powerful systems could behave in unexpected ways or pursue assigned goals through methods their developers did not anticipate.<\/p>\n

Those concerns have grown as companies develop AI agents designed to carry out complicated tasks autonomously. Unlike a chatbot that responds to an individual request, an agent can plan a series of actions, use outside tools and continue working toward an objective with limited human involvement.<\/p>\n

In the OpenAI test, the agent was not instructed to attack Hugging Face. It was instructed to solve a cybersecurity benchmark and independently determined that breaking into another company offered a route to the answers.<\/p>\n

Ghose said the episode demonstrates why developers cannot anticipate every method an advanced agent may use to accomplish its assigned task.<\/p>\n

\u201cYou can never know what it\u2019s going to do,\u201d Ghose said.<\/p>\n

The incident also suggests that cybersecurity defenses designed to stop human hackers may be outmatched by AI agents capable of searching for vulnerabilities and acting at machine speed.<\/p>\n

AI agents deployed by extortionists, scammers or nation-state adversaries could pose threats across the economy, Ghose said, including attacks against electrical grids, drinking-water systems or other essential infrastructure. Even agents pursuing legitimate goals could cause significant harm if they find unlawful or dangerous ways to complete their assignments, he said.<\/p>\n

\u201cAttacking a company\u2019s computing resources is illegal,\u201d Ghose said.<\/p>\n

The fundamental design of AI agents can make them difficult to contain, Ghose said. Additional safeguards can slow their work and require more computing resources, creating tension between safety and performance.<\/p>\n

The breach comes amid growing scrutiny of AI companies and the risks posed by increasingly powerful models. President Donald Trump issued an executive order in June directing federal agencies to develop classified benchmarks for assessing models\u2019 advanced cybersecurity capabilities and establish a voluntary program under which developers could provide the government access to certain frontier models before releasing them to outside partners.<\/p>\n

The order also directed federal officials to prioritize enforcement against people who use AI to illegally access or damage computer systems.<\/p>\n

In Silicon Valley, where many of the world\u2019s leading AI companies are based, the breach could also deepen concerns about the technology industry\u2019s judgment and willingness to police itself.<\/p>\n

Russell Hancock, CEO of think tank Joint Venture Silicon Valley, said his first reaction upon learning of the breach was, \u201cOh my gosh, this is it, now we\u2019re all doomed.\u201d<\/p>\n

But Hancock said OpenAI\u2019s disclosure and the companies\u2019 collaboration offered evidence that the industry was learning from the failure.<\/p>\n

\u201cEvery new technology has its perils and its pitfalls, and every new technology has to be refined, and that seems to be what\u2019s going on here,\u201d Hancock said.<\/p>\n

\u201cWe\u2019re learning, everybody\u2019s behaving responsibly, and there\u2019s been transparency, so this is good news \u2014 this is how technology progresses.\u201d<\/p>\n

Hugging Face said it used AI-assisted detection to find the intrusion, highlighting what Ghose described as the need to counter automated attacks with equally fast automated defenses.<\/p>\n

The image of the \u201c20-year-old with the energy drink and the hoodie\u201d no longer captures the most sophisticated threat facing cybersecurity teams, Ghose said.<\/p>\n

\u201cYou cannot meet a machine-speed attacker with a human-speed defense,\u201d Ghose said. \u201cIt\u2019s got to be bot-on-bot violence \u2014 bot versus human, we cannot keep up.\u201d<\/p>\n

Read more California Clasico has extra hurdles for Quakes after ugly Orlando loss<\/a><\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

OpenAI models escaped a supposedly isolated testing environment and hacked another company\u2019s systems to steal answers to a cybersecurity test.<\/p>\n","protected":false},"author":1,"featured_media":5261,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-5262","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-business"],"yoast_head":"\nOpenAI models break out of test environment, hack company to steal answers - Next City Digest<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nextcitydigest.com\/?p=5262\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"OpenAI models break out of test environment, hack company to steal answers - Next City Digest\" \/>\n<meta property=\"og:description\" content=\"OpenAI models escaped a supposedly isolated testing environment and hacked another company\u2019s systems to steal answers to a cybersecurity test.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nextcitydigest.com\/?p=5262\" \/>\n<meta property=\"og:site_name\" content=\"Next City Digest\" \/>\n<meta property=\"article:published_time\" content=\"2026-07-23T22:06:54+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/#\\\/schema\\\/person\\\/13065d40d633843429ac57d4e5c06f9b\"},\"headline\":\"OpenAI models break out of test environment, hack company to steal answers\",\"datePublished\":\"2026-07-23T22:06:54+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262\"},\"wordCount\":1353,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/nextcitydigest.com\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/caf3c7e92e6806313d6ad16710a0fcfd.webp\",\"articleSection\":[\"Business\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262\",\"url\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262\",\"name\":\"OpenAI models break out of test environment, hack company to steal answers - Next City Digest\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/nextcitydigest.com\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/caf3c7e92e6806313d6ad16710a0fcfd.webp\",\"datePublished\":\"2026-07-23T22:06:54+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/#\\\/schema\\\/person\\\/13065d40d633843429ac57d4e5c06f9b\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#primaryimage\",\"url\":\"https:\\\/\\\/nextcitydigest.com\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/caf3c7e92e6806313d6ad16710a0fcfd.webp\",\"contentUrl\":\"https:\\\/\\\/nextcitydigest.com\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/caf3c7e92e6806313d6ad16710a0fcfd.webp\",\"width\":2000,\"height\":1333},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/?p=5262#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/nextcitydigest.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"OpenAI models break out of test environment, hack company to steal answers\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/#website\",\"url\":\"https:\\\/\\\/nextcitydigest.com\\\/\",\"name\":\"Next City Digest\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/nextcitydigest.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/nextcitydigest.com\\\/#\\\/schema\\\/person\\\/13065d40d633843429ac57d4e5c06f9b\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/50b1ad2e498f523425ee0a8cc5180a210646db1622662a3d56cc405d3e0c346a?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/50b1ad2e498f523425ee0a8cc5180a210646db1622662a3d56cc405d3e0c346a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/50b1ad2e498f523425ee0a8cc5180a210646db1622662a3d56cc405d3e0c346a?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"http:\\\/\\\/nextcitydigest.com\"],\"url\":\"https:\\\/\\\/nextcitydigest.com\\\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"OpenAI models break out of test environment, hack company to steal answers - Next City Digest","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nextcitydigest.com\/?p=5262","og_locale":"en_US","og_type":"article","og_title":"OpenAI models break out of test environment, hack company to steal answers - Next City Digest","og_description":"OpenAI models escaped a supposedly isolated testing environment and hacked another company\u2019s systems to steal answers to a cybersecurity test.","og_url":"https:\/\/nextcitydigest.com\/?p=5262","og_site_name":"Next City Digest","article_published_time":"2026-07-23T22:06:54+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/nextcitydigest.com\/?p=5262#article","isPartOf":{"@id":"https:\/\/nextcitydigest.com\/?p=5262"},"author":{"name":"admin","@id":"https:\/\/nextcitydigest.com\/#\/schema\/person\/13065d40d633843429ac57d4e5c06f9b"},"headline":"OpenAI models break out of test environment, hack company to steal answers","datePublished":"2026-07-23T22:06:54+00:00","mainEntityOfPage":{"@id":"https:\/\/nextcitydigest.com\/?p=5262"},"wordCount":1353,"commentCount":0,"image":{"@id":"https:\/\/nextcitydigest.com\/?p=5262#primaryimage"},"thumbnailUrl":"https:\/\/nextcitydigest.com\/wp-content\/uploads\/2026\/07\/caf3c7e92e6806313d6ad16710a0fcfd.webp","articleSection":["Business"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/nextcitydigest.com\/?p=5262#respond"]}]},{"@type":"WebPage","@id":"https:\/\/nextcitydigest.com\/?p=5262","url":"https:\/\/nextcitydigest.com\/?p=5262","name":"OpenAI models break out of test environment, hack company to steal answers - Next City Digest","isPartOf":{"@id":"https:\/\/nextcitydigest.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/nextcitydigest.com\/?p=5262#primaryimage"},"image":{"@id":"https:\/\/nextcitydigest.com\/?p=5262#primaryimage"},"thumbnailUrl":"https:\/\/nextcitydigest.com\/wp-content\/uploads\/2026\/07\/caf3c7e92e6806313d6ad16710a0fcfd.webp","datePublished":"2026-07-23T22:06:54+00:00","author":{"@id":"https:\/\/nextcitydigest.com\/#\/schema\/person\/13065d40d633843429ac57d4e5c06f9b"},"breadcrumb":{"@id":"https:\/\/nextcitydigest.com\/?p=5262#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nextcitydigest.com\/?p=5262"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nextcitydigest.com\/?p=5262#primaryimage","url":"https:\/\/nextcitydigest.com\/wp-content\/uploads\/2026\/07\/caf3c7e92e6806313d6ad16710a0fcfd.webp","contentUrl":"https:\/\/nextcitydigest.com\/wp-content\/uploads\/2026\/07\/caf3c7e92e6806313d6ad16710a0fcfd.webp","width":2000,"height":1333},{"@type":"BreadcrumbList","@id":"https:\/\/nextcitydigest.com\/?p=5262#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nextcitydigest.com\/"},{"@type":"ListItem","position":2,"name":"OpenAI models break out of test environment, hack company to steal answers"}]},{"@type":"WebSite","@id":"https:\/\/nextcitydigest.com\/#website","url":"https:\/\/nextcitydigest.com\/","name":"Next City Digest","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nextcitydigest.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/nextcitydigest.com\/#\/schema\/person\/13065d40d633843429ac57d4e5c06f9b","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/50b1ad2e498f523425ee0a8cc5180a210646db1622662a3d56cc405d3e0c346a?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/50b1ad2e498f523425ee0a8cc5180a210646db1622662a3d56cc405d3e0c346a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/50b1ad2e498f523425ee0a8cc5180a210646db1622662a3d56cc405d3e0c346a?s=96&d=mm&r=g","caption":"admin"},"sameAs":["http:\/\/nextcitydigest.com"],"url":"https:\/\/nextcitydigest.com\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=\/wp\/v2\/posts\/5262","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5262"}],"version-history":[{"count":0,"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=\/wp\/v2\/posts\/5262\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=\/wp\/v2\/media\/5261"}],"wp:attachment":[{"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5262"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5262"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nextcitydigest.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5262"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}