Attackers breached Fireworks AI storage that held Notion AI chat transcripts. The inference vendor’s documentation says, “Fireworks has Zero Data Retention by default.”
Both statements can be true. By default, “prompt and generation data exist only in volatile memory for the duration of the request.” The Fireworks policy page adds, “A few features do store customer content, such as stored responses, batch jobs, and training.” The breach reached the stored side.
A breach at an AI provider can reach businesses that use its technology through someone else’s software. In this case, that means teams whose employees chatted with Notion AI. If they pasted in internal meeting notes, unreleased product plans, or details about a customer, that information could be in the chat transcripts. Notion says fewer than 1% of its workspaces may have been affected.
Any Security Operations Center (SOC) that accepts an AI vendor’s retention answer can check for the same gap.
Fireworks’ November 9 security update says, “We have not detected any impact to Inference, Training SDK and Nexus, which are all zero data retention (ZDR) by default.” CEO Lin Qiao’s post on X words it differently. “We have not detected any impact to Inference (excluding batch inference), Training SDK, and Nexus, which all have zero data retention (ZDR) by default.”
The Fireworks glossary defines batch inference as “Asynchronous large-scale inference via the Fireworks Batch API.” Qiao also wrote, “This allowed them to access a subset of storage systems, leading to additional customer data impact.”
Notion’s AI chats were in the breach data
Fireworks has more than 10,000 customers, according to The Record. Notion is one. “We used Fireworks for classification jobs to help customers understand their own AI usage through a feature in Notion called Insights,” its notice says.
Notion’s notice continues, “On November 8, Fireworks confirmed this new dataset for us.” Its security advisory is more precise. “Upon analysis of the new data, we discovered that it included Notion AI chat transcripts used for the Insights feature, which for the vast majority of impacted workspaces were unattributable.”
Notion says “<1% of Notion workspaces may have been affected by the Fireworks security incident.” It tells customers, “We recommend that you treat this data as if it were exposed.”
The later intrusion came through a third company. “A third-party cloud provider that hosted Fireworks R&D machines experienced a security incident on October 23. On October 24, the unauthorized third party used it to reach Fireworks development machines and then run an unauthorized workload that led to additional customer data impact.”
Fireworks says the evidence points to a group “financially motivated and aimed at selling tokens from unauthorized access to Large Language Models (LLMs) to illicit routing gateways,” and adds, “On November 2, Google Threat Intelligence Group and Mandiant published a blog post about the rise of LLMJacking,” the name for that practice.
The policy that blocks storage ships “Off by default”
The Zero Data Retention policy works by “rejecting every request and job that would persist or log customer content.” The policy page also says, “The Zero Data Retention policy is available on Enterprise accounts.” Its Default Policy setting reads, “Applies to Inference and Training. Off by default.” Three operations the policy rejects appear in the disclosures.
Fireworks operation | When the policy is on | What the disclosures say |
Chat completions and completions | Allowed | The November 9 update says “We have not detected any impact to Inference” |
Batch inference jobs | Rejected | Qiao’s post lists “Inference (excluding batch inference)” among products with no detected impact. Fireworks “disabled self-serve dataset uploads and downloads used by Batch Inference” during the investigation |
Evaluators and evaluation jobs | Rejected | The October 9 bulletin, on evaluation jobs, says “We believe that environment variables, including API keys and tokens stored as secrets, were captured in certain affected accounts.” |
Dataset uploads | Rejected | The November 9 update says that “for certain customers, a subset of incomplete model weights and datasets were accessed.” |
Neither company has said which Fireworks feature stored Notion’s transcripts, and nothing in the disclosures shows the policy would have stopped the intrusion.
Wiz saw stolen AI keys tested within 30 minutes
Wiz counted at least nine AI developer platforms disclosing incidents in October and traced two stolen enterprise keys to one of them. “They shared an identical first unauthorized activity, which happened within 30 minutes of the reported credential access within the vendor breach.”
The Cloud Security Alliance wrote on October 23, before Fireworks widened its disclosure, “Vendor questionnaires should ask how customer secrets are encrypted and who can decrypt them, whether internal staff or automated jobs can read plaintext customer secrets,” among other questions.
SOC action plan
Five checks follow from the disclosures.
Ask your product teams and each AI vendor which features store content. Fireworks’ list names “stored responses, batch jobs, and training.”
Check to see how policy settings are configured. Fireworks’ API reference says, “Any account member can read the policy settings.” It adds that “an absent zeroDataRetention means the policy is off.”
Hunt for the validation call. Wiz wrote, “It was a single InvokeModel from a residential IP address, in a one-minute sweep that tested stolen keys across multiple organizations.”
Rotate at the issuer. Fireworks warns, “Removing a stored secret alone does not invalidate the underlying credential.”
Preserve evidence, then decide what stays stored. “Turning on the policy doesn’t delete anything already stored on the account, such as stored responses, datasets, and uploaded models.”