RL-Tuned 9B Model Beats Frontier Models at 68x Lower Cost
A 9B open-source model fine-tuned with GRPO reinforcement learning (RL) achieved 87.3% of the maximum score on an e-commerce catalog review task, beating every frontier model tested. The best frontier configuration scored 76.9%. The fine-tuned model costs $0.50 per 1,000 listings, while the strongest frontier model costs $34—a 68x cost advantage.
This isn't a one-off. The same playbook—open-source model, proprietary task data, RL against a scored workflow—has been deployed at Bridgewater Associates, Harvey, and Intercom. Bridgewater's trained model makes ~30% fewer mistakes than the best frontier model on document relevance. Harvey's legal agent beats GPT-5.5 and Claude Opus 4.8 on its own rubrics. Intercom's Fin Apex resolves more support issues at lower cost.
The Playbook: Open-Source + RL
The winning approach follows a consistent pattern:
- Start with frontier models to establish a baseline and generate training data (inputs, decisions, corrections).
- Train an open-source model (e.g., a 9B parameter model) on proprietary task data.
- Apply reinforcement learning (specifically GRPO) against a scored copy of the workflow. The model interacts with your tools and data, a rubric scores outcomes, and the reward signal updates the model's weights.
This works because general-purpose frontier models lack the specific judgment your organization needs. They plateau within a tenth of a point of each other on narrow tasks. A specialist model, trained on your data, breaks through that ceiling.
Case Study: E-Commerce Catalog Review
E-commerce platforms face a massive catalog integrity problem. eBay carries ~2.5 billion live listings. Shopify absorbs 10+ million product updates daily. Walmart has said AI-assisted catalog work would require 100x the headcount if done by people alone. Errors cost revenue: 71% of shoppers have returned a product that didn't match its listing.
A mid-size marketplace generates ~10 million listing creates/edits per day. Using a frontier model for review costs roughly $500 million/year. The fine-tuned specialist does the same job for ~$10 million—a 50x gap.
The fine-tuned 9B model scored 87.3% of the maximum achievable score, compared to 76.9% for the best frontier configuration—a 13.5% relative improvement. The five frontier models, even with optimized prompts, plateaued within a tenth of a point of each other. The trained specialist cleared that ceiling.
Why This Matters
This isn't about replacing frontier models entirely. Most workflow automation still starts with them. But at scale, the economics flip. Frontier per-call pricing becomes prohibitive. A self-hosted specialist model, trained on your data, offers better performance at a fraction of the cost.
The same approach applies beyond catalog review. Bridgewater used it for document relevance. Harvey for legal document analysis. Intercom for customer support. The pattern repeats: RL pushes an open-source model past the frontier on a specific set of workflows.
Technical Details
The fine-tuning used GRPO (Group Relative Policy Optimization), a variant of PPO that groups responses to stabilize training. The base model was a 9B open-source transformer. Training data consisted of labeled catalog decisions from expert analysts. The rubric scored each decision on category correctness and attribute extraction accuracy.
At inference, the model runs on a single GPU. Self-hosting eliminates per-token API costs. The total cost per 1,000 listings: $0.50. Compare to $34 for the strongest frontier model—a 68x reduction.
What You Should Do Now
If you're running high-volume automated decisions, start by establishing a baseline with frontier models. Collect decision logs and human corrections. Then train a small open-source model on that data using RL. The cost savings alone justify the effort, and the performance gains are real.
Bridgewater, Harvey, and Intercom have already proven this works. Your workflow might be next.
