DeepSeek's open inference stack makes cache economics the number to check

*A 96% cache hit rate says more about your AI bill than any per-token price, and now the code behind it is public.*

Opening read

DeepSeek has published the software it uses to serve its own models, and I think that release matters more than any model it has shipped this year. The figure buyers should pull from it is the 96% cache hit rate. For agent workloads that resend long contexts all day, how much of each request is already cached can drive most of the bill, and that changes which line of the invoice you negotiate.

Inference economics: DeepSeek opens the stack, and the cache is the story

The question running through this week's coverage has been what AI work a company can run on hardware it owns or chooses, and the honest answer has turned on the serving software as much as on the model. DeepSeek just added a large new input. At the PyTorch conference it released DeepInfer, the inference stack behind its own paid service (inference is the work of running a trained model to answer live requests), as open source, and said that service moves 155 trillion tokens (the word-sized chunks AI vendors bill by) a day through one cluster. Latent Space (https://www.latent.space/p/ainews-deepseek-deepinfer-open-source) The consensus read writes itself: DeepSeek does more with less, Chinese labs keep pushing on cost, and the per-token price war rolls on. A smart reader could stop there and be mostly right.

I think the useful number sits lower in the release. DeepSeek says 96% of the input its service handles is a cache hit, meaning the system already computed that stretch of text for an earlier request and reuses the stored result instead of redoing the work. That ratio is what makes the throughput claim (throughput being how much work the system handles in a day) believable, and it shows where the money goes. Fireworks saw the same shape when it ran DeepSeek's V4.1-Flash through DeepSWE, a coding test for agents: 99.6% of input tokens were cache hits, and charges for those hits still made up 59.9% of the total bill. Fireworks AI (https://fireworks.ai/blog/DeepSeek-V4.1-Flash-Astra) Put those two facts together. When agents (AI programs that carry out multi-step tasks) replay long contexts (the large blocks of text sent with each request) all day, charges for cached input were about three-fifths of Fireworks' bill, so everything else on the invoice combined was the smaller share. That is two workloads, and it says nothing about serving costs in general, but repetitive long-context agent traffic is exactly the kind more companies are running.

For an IT leader weighing a hosted API (a software interface the vendor runs so your programs can send it requests) against a self-run deployment, DeepInfer gives you a public baseline. Until now a vendor's quoted cache efficiency was a claim you took on trust; now there is open code that produces a published hit rate, and a team can test what rate its own traffic gets on infrastructure it controls. It does not expose any other vendor's code. A concrete check: pull a week of production usage, count cached input tokens as a share of all input tokens, then count cached charges as a share of spend. DeepSeek's 96% and Fireworks' 99.6% and 59.9% are reference points from two different workloads rather than pass marks, so measure your own two shares before drawing any conclusion about why they land where they do. If cached charges turn out to be most of your bill, your next negotiation is about the cached rate rather than the list price.

What the evidence cannot show is whether DeepSeek's economics hold anywhere but DeepSeek. The 155 trillion tokens and the 96% hit rate are DeepSeek's own numbers, and the newsletter headline that relayed them adds a margin and payback figure that no outside party has audited. I would treat the stack as real and the business numbers as marketing until a third party runs DeepInfer on infrastructure it controls and publishes the result.

Cyber defense: OpenAI's blueprint now comes with customer names

In June OpenAI published a governance blueprint calling for a new agency, or an existing civilian one called CAISI, to run mandatory tests on frontier models (the most capable systems), and it has since backed a provision in the FRONTIER Act that would require the top labs to let independent verifiers monitor model development. Don't Worry About the Vase (https://thezvi.substack.com/p/openai-offers-a-new-policy-blueprint) TechCrunch AI (https://techcrunch.com/2026/09/15/openai-anthropic-google-have-been-in-talks-on-ai-safety-for-weeks/) Part II arrived this week. It argues for arming cyber defenders with frontier models while tightening the safeguards around them, and it came paired with a Cloudflare case in which OpenAI's cyber model, Daybreak, hunts for flaws and drafts fixes, and Cloudflare's engineers review each proposed fix before deciding whether to apply it. OpenAI News (https://openai.com/index/empowering-cyber-defenders) OpenAI News (https://openai.com/index/cloudflare-cyber) That is a lobbying document and a marketing document arriving in the same week. The policy paper is advocacy, and the customer stories are OpenAI's to tell, but they now carry names: OpenAI's own case study says Sophos cut threat investigation time by 96%. OpenAI News (https://openai.com/index/sophos) Vendor-published wins are a reason to look rather than a reason to buy. For a security leader, two named buyers move this from curiosity to a scoped trial, with your own human sign-off on every proposed fix as the condition of entry.

Labor data: the measured AI effect lands on the entry-level pipeline

The best-measured AI effect on jobs so far is a thinner entry-level pipeline. Stanford's Digital Economy Lab launched real-time AI economic dashboards in June, and this week's report shows that jobs most open to AI, like coding and customer support, employ over 4% fewer people than a year ago. The sharper number is the age split: workers aged 22 to 25 in those fields have lost over 3% a year since ChatGPT arrived in November 2022. Stanford HAI (https://hai.stanford.edu/news/tracking-ais-real-time-footprint-on-the-labor-market) A Harvard study of 62 million workers points the same way, finding that companies adopting generative AI saw junior employment fall about 8% relative to non-adopters within 18 months, and Anthropic's own junior share of technical hires fell from 27% to 9% since early 2025. Nate Jones (https://www.youtube.com/watch?v=awV2kJzh8zk) TechCrunch AI (https://techcrunch.com/2026/10/17/anthropic-says-its-workforce-has-become-less-junior-and-is-spending-more-time-in-meetings/) Set that beside the AI as Normal Technology argument that AI is not causing mass layoffs among software engineers, where the number behind the argument is a Federal Reserve finding that software engineer employment is growing about 3 percentage points a year slower than it would without AI. AI as Normal Technology (https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers) The two reads do not conflict. The Normal Tech case rules out mass firing of engineers, and it says nothing about the entry-level decline the Stanford and Anthropic numbers show. The counter-case is the layoff list: Block cut 4,000 jobs this year and named AI tools and smaller teams as the reason, and TechCrunch keeps a running list of around twenty tech companies that have cited AI in layoffs. TechCrunch AI (https://techcrunch.com/2026/07/25/the-running-list-major-tech-layoffs-in-2026-where-employers-cited-ai/) Those are company statements, and they do not overturn the measured read, because a press release naming AI is a claim about motive while the dashboards count heads. None of these numbers cleanly separates slower hiring from layoffs across the wider market, so I hold the entry-level decline as established and the mechanism behind it as open. For a business leader the decision is the entry-level pipeline you stop filling and the senior bench you will lack in five years.

Independent review: METR agrees with Anthropic again, and that pattern has a shape

METR staff now sit inside Anthropic with roughly the access an employee has, so they can evaluate its systems from within, an arrangement the two sides began discussing in September as a self-funded pilot of embedded evaluation (outside reviewers working inside the lab). Anthropic News (https://www.anthropic.com/news/public-benefit-fund) Anthropic News (https://www.anthropic.com/news/accenture-embedded-evaluation) Twice now METR has signed off on Anthropic's verdict while faulting the proof behind it, and this week makes a third review. In March METR accepted that the catastrophic risk from Opus 4.6 was low but said the alignment tests (checks for misalignment, meaning a model pursuing goals its makers did not intend) were not sensitive enough, and in May it accepted the conclusion on R&D automation (AI doing its own lab's research work) while calling the evidence inadequate. Now it has reviewed the misalignment section of Anthropic's August 2026 Risk Report and agrees the catastrophic risk is low for the covered models. METR (https://metr.substack.com/p/2026-03-12-sabotage-risk-report-opus-4-6-review) METR (https://metr.substack.com/p/2026-05-08-rd-section-anthropic-risk-report-feb-2026-review) METR (https://metr.substack.com/p/2026-10-20-misalignment-risk-report-august-2026-review) The pattern is the point. I do not know from today's evidence whether this review carries the same methods complaint or drew on the new access. For an engineer deciding how much weight a vendor risk report deserves, the next METR review is worth reading for its argument about proof, because the verdict has been the easy part.

Augur calls

• Call: A named organization other than DeepSeek publishes measured throughput or cache-hit results from running DeepInfer on hardware it controls by 2027-01-31.
• Criteria: A public blog post, paper, or repository from an identifiable company, lab, or inference provider reporting its own DeepInfer measurements on or before the due date counts as a hit; absence of any such publication counts as a miss.
• Due: 2027-01-31
• Odds: 70%

• Call: Mistral and NVIDIA publicly release at least one benchmark or open dataset for physics models or AI agents in computer-aided engineering or chip design by 2027-03-31.
• Criteria: A public announcement, repository, or dataset page from Mistral or NVIDIA naming the benchmark or dataset and making it available counts as a hit; a repeated promise without a released artifact counts as a miss.
• Due: 2027-03-31
• Odds: 65%

What to watch

• DeepInfer is now runnable by anyone, so the first outside team to run it on its own traffic and publish a cache-hit figure will settle whether the economics travel. Latent Space (https://www.latent.space/p/ainews-deepseek-deepinfer-open-source)
• Mistral and NVIDIA have promised public benchmarks and open datasets for engineering AI, and the first release will show whether "AI-native engineering" can be measured rather than marketed. Mistral AI (https://mistral.ai/news/mistral-nvidia-ai-for-industrial-engineering/)
• Stanford's dashboards track the labor market in real time, so the next read on 22-to-25-year-old employment will show whether the decline is steepening. Stanford HAI (https://hai.stanford.edu/news/tracking-ais-real-time-footprint-on-the-labor-market)
• Policymakers' response to OpenAI's part II cyber blueprint, especially any move on the mandatory testing agency proposed in part I, is the signal that separates a lobbying document from a framework. OpenAI News (https://openai.com/index/empowering-cyber-defenders) Don't Worry About the Vase (https://thezvi.substack.com/p/openai-offers-a-new-policy-blueprint)
• Google's gestational-age ultrasound model, working from blind-sweep scans taken by novice operators, matched expert sonographers in a prospective multi-center study, and in a separate finding it held up in Nairobi with no local training data; the next evidence worth waiting for is a study where frontline workers act on the model's estimate in real care decisions. Google Research Blog (https://research.google/blog/empowering-frontline-maternal-care-with-ai-assisted-ultrasound/)

Verification notes

DeepSeek's 155 trillion tokens a day and 96% cache hit rate are the company's own numbers relayed through a newsletter summary, with no outside measurement yet, and the margin and payback figures belong to that newsletter's headline, which no outside party has audited. Latent Space (https://www.latent.space/p/ainews-deepseek-deepinfer-open-source) The Fireworks cache figures come from one vendor's single benchmark run on one model, so they illustrate the cost shape rather than establish a general rate. Fireworks AI (https://fireworks.ai/blog/DeepSeek-V4.1-Flash-Astra) The Sophos and Cloudflare results are published by OpenAI, not by the customers independently, and the labor evidence establishes an entry-level employment decline but cannot distinguish slower hiring from layoffs across the wider market.

Inspect the evidence behind this dispatch. (https://augurdispatch.ai/evidence/2026-10-22?utm_source=augur&utm_medium=web&utm_campaign=dispatch-2026-10-22)