Open Models Beat GPT-5.6 Sol on Retrieval: Neon
Neon says its Castform setup on open models beats GPT-5.6 Sol on retrieval at roughly 1/100th the cost. What the claim covers — and what it doesn't.
> **TL;DR:** Neon has published benchmarks claiming its Castform setup, running on open models, outperforms GPT-5.6 Sol on retrieval tasks at roughly one-hundredth the cost. The company frames the result as a pricing argument rather than a capability one: frontier rates, it says, are no longer justified for retrieval-heavy workloads. The post drew 234 points on Hacker News, where the underlying complaint about inference bills is widely shared.
Key Takeaways
- Neon says its Castform setup on open models beats GPT-5.6 Sol on retrieval tasks while costing about 100x less. - The argument is deliberately narrow — retrieval-heavy work, not general reasoning — which is what makes it hard to wave away. - A 100x gap is not a margin you optimise; it changes which workloads are economically viable at all. - It remains a vendor benchmark on vendor-chosen tasks, so the number to trust is the one you measure on your own corpus. - The thread hit 234 points on Hacker News, a signal that the pricing frustration is broadly felt, not niche.
Neon has published benchmark results claiming that its Castform setup, built on open models, outperforms GPT-5.6 Sol on retrieval tasks while costing roughly 100 times less to run. The company's [write-up](https://neon.com/blog/how-castform-neon-beats-frontier-models-on-price-and-efficiency) frames the finding as a pricing argument rather than a capability one: for retrieval-heavy workloads, it says, frontier-model rates are no longer justified.
That framing is the interesting part. Claims that open models have "caught up" arrive weekly and mostly evaporate on contact with a real workload. This one is narrower, and narrower claims are harder to dismiss.
The claim, stated narrowly
Neon is not arguing that open models are better than frontier models in general. It is arguing that on retrieval — the reading, ranking and extraction layer that sits underneath most production AI features — a well-assembled open-model stack matches or beats the frontier option, and does so at a cost ratio that isn't close.
The distinction matters because it changes who the claim is aimed at. It isn't aimed at people choosing a chat assistant. It's aimed at teams running the same retrieval step millions of times a month and watching the invoice compound.
Why retrieval is where this argument lands
Retrieval-heavy work has a shape that sets it apart from open-ended reasoning, and that shape is what makes cheaper models competitive.
The token mix skews hard toward input. You are feeding in large amounts of context and asking for comparatively little back — a ranking, a span, a short synthesis. Frontier pricing is built around the assumption that hard generation is happening; retrieval often isn't hard generation.
The task is also more bounded than it looks. Deciding which of forty chunks answers a question is a discrimination problem. It rewards a model that reads carefully and stays consistent, not one that can reason for thirty steps about something it has never seen. Capability headroom above the task threshold buys you nothing you can measure.
And the volume is relentless. A single user question can trigger many retrieval calls. Whatever the per-call cost is, you multiply it by a big number — which is exactly the regime where a 100x gap stops being an optimisation and starts being a product decision.
A 100x gap is a different kind of number
Engineers are used to arguing about cost differences of 20 or 30 percent. Those get absorbed into a roadmap. A hundredfold difference doesn't get absorbed; it reclassifies the workload.
At frontier rates, plenty of obviously useful ideas — re-ranking every document on ingest, re-embedding a corpus when the schema changes, running retrieval over a customer's entire history rather than a truncated slice — get cut in planning because the arithmetic doesn't survive contact with the finance team. Drop the unit cost by two orders of magnitude and those same ideas become unremarkable. That's the practical version of Neon's argument, and it's why the post travelled.
The reception, and what it signals
The post drew [234 points on Hacker News](https://news.ycombinator.com/item?id=49186762). Vendor benchmarks usually get a rough ride in that venue, and a company publishing numbers that flatter its own product is exactly the genre readers there are trained to distrust.
It landed anyway, which is the signal worth reading. The frustration Neon is naming — paying frontier prices for work that doesn't feel like frontier work — is widely felt among people shipping AI features rather than demoing them.
What the benchmark doesn't settle
Several things stay open, and honest coverage should say so.
It is a vendor benchmark on vendor-chosen tasks. That doesn't make it wrong, but it does mean the tasks were selected by someone with a preferred outcome. "Retrieval tasks" is also a wide label covering everything from tight lookup to messy multi-document synthesis, and the gap between an open stack and a frontier model is unlikely to be constant across that range.
The cost comparison also depends on where you draw the boundary. API pricing is a clean number; running open models involves capacity you provision whether or not it's saturated, plus the engineering time to keep the stack tuned. Whether 100x survives full accounting depends on your utilisation, not on the benchmark.
Frontier pricing is a moving target too. The ratio describes today, not a permanent structural fact.
How to use this if you're shipping
The actionable version is unglamorous: stop treating your pipeline as one model choice. Retrieval, reranking, extraction and final generation have different difficulty profiles, and routing each to the cheapest model that clears its bar is where the savings actually live. Then benchmark on your own corpus — the only number that binds you is the one your data produces.
The surrounding infrastructure is quietly following the same logic. Agent tooling is migrating toward environments the developer already controls, as with [Warp's coding agent arriving in any terminal](https://speka.info/blog/warp-agent-cli-warps-coding-agent-comes-to-any-terminal), and sandboxed execution is becoming a commodity layer rather than a differentiator, as with [Cloudflare's sandboxed machine for AI agents](https://speka.info/blog/cloudflare-computer-a-sandboxed-machine-for-ai-agents). Cheap, swappable inference is the same pattern applied to the model itself.
It also fits a broader industrialisation story, where the interesting advances are increasingly about throughput and unit cost rather than raw capability — the same dynamic visible in [China's push toward automated production](https://speka.info/blog/china-ai-automation-mind-controlled-robots-dark-factories). For more coverage of how model economics keep shifting, see our [LLM launches and updates](https://speka.info/llm-updates/) hub.
Neon's benchmark won't be the last word. But the question it forces — what am I actually paying frontier prices *for*? — is one every team running retrieval at volume should be able to answer.
Frequently Asked Questions
What did Neon claim about open models and GPT-5.6 Sol?
Neon published benchmarks claiming its Castform setup, running on open models, outperforms GPT-5.6 Sol on retrieval tasks while costing roughly 100 times less.
Does this mean open models are better than frontier models?
No. Neon's argument is specific to retrieval-heavy workloads, not general capability, and it is a pricing argument: that frontier rates aren't justified for this class of work.
Why would retrieval favour cheaper models?
Retrieval is dominated by reading and ranking large amounts of input rather than generating long novel reasoning, so capability headroom above the task threshold delivers little measurable benefit while multiplying cost across very high call volumes.
Should I trust a vendor's own benchmark?
Treat it as a hypothesis worth testing, not a result. The tasks were chosen by a party with a preferred outcome, so the number that should drive your decision is the one you measure on your own corpus.
Does the 100x cost figure include self-hosting overhead?
The published figure is a cost comparison from Neon's post; whether that ratio holds for you depends on your own utilisation, provisioning and engineering overhead, which the benchmark cannot account for.
How much attention did the post get?
It drew 234 points on Hacker News, an unusually warm reception for a vendor benchmark and a sign that frustration with inference pricing is widely shared.
Sources
- https://neon.com/blog/how-castform-neon-beats-frontier-models-on-price-and-efficiency - https://news.ycombinator.com/item?id=49186762
