Claude Sonnet 5.5 Tops Android Coding Benchmark as Google Gemini 4 Argon Debuts in Fifth Place
Google has updated its Android Bench list to evaluate how effectively AI models assist with Android coding, with Google’s newly introduced Gemini 4 Argon ranking behind rival frontier models from Anthropic and OpenAI. 9to5google.com reported that the benchmark, which was launched earlier this year to help Android developers find suitable coding assistants based on tests reflecting Android-specific use cases and best practices, incorporated several new models and cost metrics in an October 26 update.
Performance and Cost Metrics for Frontier AI Models
The October 26 update introduced results for OpenAI’s GPT-6 Sol, GPT-6.1 Sol, Claude Sonnet 5, Claude Sonnet 5.5, and Google’s Gemini 4 Argon, alongside data detailing the average cost and reasoning level used for each model. Claude Sonnet 5.5 currently leads the rankings with a 40% pass rate and an 80.9% completion rate. Claude Opus 5.5, which joined the benchmark in early October, occupies second place.
OpenAI’s GPT-6.1 Sol achieved a 29.3% pass rate and an 82.8% completion rate, registering an average cost of $99.50. By comparison, Claude’s top-performing configuration recorded a significantly higher cost of $1,574.90.
Gemini 4 Argon Debut and Internal Reception
Google’s Gemini 4 Argon entered the rankings in fifth place, trailing behind other frontier models. The model recorded a 21.3% pass rate and an 81.8% completion rate, with an average cost of $267.70, placing it behind Claude Fable 5.1. Shortly after Google announced Gemini 4 Argon, internal sources claimed the model struggled with specific coding tasks, though Google maintained its support for the model’s capabilities.
Android Coding Benchmark Results
| Model | Pass Rate | Completion Rate | Average Cost | Benchmark Rank |
|---|---|---|---|---|
| GPT-6.1 Sol | 29.3% | 82.8% | $99.50 | Not specified |
| Claude Fable 5.1 | Not specified | Not specified | $267.70 | Ahead of Gemini 4 Argon |
| Gemini 4 Argon | 21.3% | 81.8% | $267.70 | 5th |
Frequently Asked Questions About Android Bench
When was Android Bench introduced by Google?
Android Bench was introduced early in the year to provide Android developers with a quick evaluation method for AI coding assistance based on Google’s tests.
What new data points were added in the October 26 update?
The October 26 update added the reasoning level used for each model, alongside new results for models from OpenAI, Anthropic, and Google, as well as average cost metrics.
How does the cost of GPT-6.1 Sol compare to Claude’s top finish?
GPT-6.1 Sol recorded an average cost of $99.50, whereas Claude’s top-ranked configuration cost $1,574.90.
Keep reading