Benchmarking Excel and Sheets AI
AutosheetBench evaluates real AI products on realistic spreadsheet tasks.
Latest run Code on GitHubDataset on Hugging Face
At a glance
Time per task
Lower is betterAI cost per task
Lower is betterSidebar add-ons, each on its best model and platform.
What is AI cost and how is it computed?| Product | Model | Effort | Score on 187 tasks | Time per task | AI cost per task | AI cost kind |
|---|---|---|---|---|---|---|
| Autosheet in Excel | Opus 5.5 | Medium | 97.3% | 27s | 12¢ | Tokens at API list prices |
| Claude in Excel | Opus 5.5 | High | 91.4% | 55s | 35¢ | Tokens at API list prices |
| ChatGPT in Excel | GPT-6 Astra | Medium | 86.1% | 59s | 76¢ | Tokens at API list prices |
| Copilot in Excel | Auto | Auto | 81.8% | 44s | 17¢ | Plan's month (no tokens recorded) |
| Gemini in Sheets | Auto | Auto | 74.9% | 2m 6s | 13¢ | Plan's month (no tokens recorded) |
Explore score, duration, cost in detail
Every add-on and MCP on each model it ran, with filters for platform, product type and model.
Overall score
- 97.3%Autosheetin Excel97.3%Opus 5.5
- 91.4%Claudein Excel91.4%Opus 5.5
- 86.1%ChatGPTin Excel86.1%GPT-6 Astra
- 81.8%Copilotin Excel81.8%Auto
- 92.0%Autosheetin Sheets92.0%Opus 5.5
- 89.8%Claudein Sheets89.8%Opus 5.5
- 74.9%Geminiin Sheets74.9%Auto
- 62.0%ChatGPTin Sheets62.0%GPT-6 Astra
- 89.8%Claude Code+ Sheets
MCP89.8%Opus 5.5 - 88.8%Claude Code+ Autosheet
MCP88.8%Opus 5.5 - 87.2%ChatGPT+ Autosheet
MCP87.2%GPT-6 - 81.8%ChatGPT+ Drive
MCP81.8%GPT-6
| Product | Platform | Type | Model | MCP model | Effort | Overall score (%) | Tasks scored |
|---|---|---|---|---|---|---|---|
| Autosheet in Excel | Excel | Sidebar add-on | Opus 5.5 | Medium | 97.3 | 187 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Astra | Low | 92.5 | 187 | |
| Autosheet in Excel | Excel | Sidebar add-on | Sonnet 5.5 | Medium | 92.0 | 187 | |
| Autosheet in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 91.4 | 187 | |
| Claude in Excel | Excel | Sidebar add-on | Opus 5.5 | High | 91.4 | 187 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6.1 Sol | Medium | 89.8 | 187 | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Astra | Medium | 86.1 | 187 | |
| Claude in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 84.5 | 187 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 83.4 | 187 | |
| Copilot in Excel | Excel | Sidebar add-on | Auto | Auto | 81.8 | 187 | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 76.5 | 187 | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 92.0 | 187 | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 89.8 | 187 | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 87.2 | 187 | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 84.5 | 187 | |
| Gemini in Sheets | Google Sheets | Sidebar add-on | Auto | Auto | 74.9 | 187 | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Astra | Medium | 62.0 | 187 | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Luna | High | 57.2 | 187 | |
| Claude Code + Sheets MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Medium | 89.8 | 187 | |
| Claude Code + Autosheet MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Opus 5.5 | Medium | 88.8 | 187 |
| ChatGPT + Autosheet MCP | Google Sheets | AI assistant + MCP | GPT-6 | Opus 5.5 | Medium | 87.2 | 187 |
| ChatGPT + Drive MCP | Google Sheets | AI assistant + MCP | GPT-6 | Medium | 81.8 | 187 |
Score vs duration
| Product | Platform | Type | Model | MCP model | Effort | Overall score (%) | Average duration per task (s) |
|---|---|---|---|---|---|---|---|
| Autosheet in Excel | Excel | Sidebar add-on | Opus 5.5 | Medium | 97.3 | 27 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Astra | Low | 92.5 | 29 | |
| Autosheet in Excel | Excel | Sidebar add-on | Sonnet 5.5 | Medium | 92.0 | 16 | |
| Autosheet in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 91.4 | 28 | |
| Claude in Excel | Excel | Sidebar add-on | Opus 5.5 | High | 91.4 | 55 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6.1 Sol | Medium | 89.8 | 32 | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Astra | Medium | 86.1 | 59 | |
| Claude in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 84.5 | 41 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 83.4 | 33 | |
| Copilot in Excel | Excel | Sidebar add-on | Auto | Auto | 81.8 | 44 | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 76.5 | 67 | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 92.0 | 39 | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 89.8 | 61 | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 87.2 | 50 | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 84.5 | 70 | |
| Gemini in Sheets | Google Sheets | Sidebar add-on | Auto | Auto | 74.9 | 126 | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Astra | Medium | 62.0 | 66 | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Luna | High | 57.2 | 72 | |
| Claude Code + Sheets MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Medium | 89.8 | 73 | |
| Claude Code + Autosheet MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Opus 5.5 | Medium | 88.8 | 75 |
| ChatGPT + Autosheet MCP | Google Sheets | AI assistant + MCP | GPT-6 | Opus 5.5 | Medium | 87.2 | 75 |
| ChatGPT + Drive MCP | Google Sheets | AI assistant + MCP | GPT-6 | Medium | 81.8 | 46 |
Score vs AI cost
| Product | Platform | Type | Model | MCP model | Effort | Overall score (%) | AI cost of one run of every task ($) | AI cost kind |
|---|---|---|---|---|---|---|---|---|
| Autosheet in Excel | Excel | Sidebar add-on | Opus 5.5 | Medium | 97.3 | 22.30 | Tokens at API list prices | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Astra | Low | 92.5 | 33.81 | Tokens at API list prices | |
| Autosheet in Excel | Excel | Sidebar add-on | Sonnet 5.5 | Medium | 92.0 | 12.39 | Tokens at API list prices | |
| Autosheet in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 91.4 | 10.35 | Tokens at API list prices | |
| Claude in Excel | Excel | Sidebar add-on | Opus 5.5 | High | 91.4 | 65.73 | Tokens at API list prices | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6.1 Sol | Medium | 89.8 | 19.52 | Tokens at API list prices | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Astra | Medium | 86.1 | 141.97 | Tokens at API list prices | |
| Claude in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 84.5 | 7.14 | Tokens at API list prices | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 83.4 | 11.59 | Tokens at API list prices | |
| Copilot in Excel | Excel | Sidebar add-on | Auto | Auto | 81.8 | 31.50 | Plan's month (no tokens recorded) | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 76.5 | 2.16 | Tokens at API list prices | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 92.0 | 25.83 | Tokens at API list prices | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 89.8 | 45.18 | Tokens at API list prices | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 87.2 | 12.37 | Tokens at API list prices | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 84.5 | 9.34 | Tokens at API list prices | |
| Gemini in Sheets | Google Sheets | Sidebar add-on | Auto | Auto | 74.9 | 24.00 | Plan's month (no tokens recorded) | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Astra | Medium | 62.0 | 149.88 | Tokens at API list prices | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Luna | High | 57.2 | 1.47 | Tokens at API list prices | |
| Claude Code + Sheets MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Medium | 89.8 | No data | No data | |
| Claude Code + Autosheet MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Opus 5.5 | Medium | 88.8 | No data | No data |
| ChatGPT + Autosheet MCP | Google Sheets | AI assistant + MCP | GPT-6 | Opus 5.5 | Medium | 87.2 | No data | No data |
| ChatGPT + Drive MCP | Google Sheets | AI assistant + MCP | GPT-6 | Medium | 81.8 | No data | No data |
Score by capability
| Overall | Formulas | Spreadsheet manipulation | Charts | Data operations | Formatting | Pivot tables | Bulk content generation | Bulk data processing | |
|---|---|---|---|---|---|---|---|---|---|
| Excel · Sidebar add-ons | |||||||||
| 97.3 | 100.0 | 100.0 | 96.7 | 100.0 | 96.2 | 87.5 | 100.0 | 88.2 | |
| 91.4 | 100.0 | 100.0 | 93.3 | 100.0 | 88.5 | 87.5 | 36.4 | 82.4 | |
| 86.1 | 100.0 | 94.4 | 80.0 | 94.1 | 76.9 | 87.5 | 36.4 | 82.4 | |
| 81.8 | 100.0 | 91.7 | 70.0 | 88.2 | 80.8 | 100.0 | 18.2 | 64.7 | |
| Google Sheets · Sidebar add-ons | |||||||||
| 92.0 | 97.6 | 97.2 | 83.3 | 94.1 | 92.3 | 87.5 | 81.8 | 88.2 | |
| 89.8 | 100.0 | 94.4 | 86.7 | 100.0 | 88.5 | 87.5 | 36.4 | 88.2 | |
| 74.9 | 83.3 | 86.1 | 60.0 | 88.2 | 80.8 | 87.5 | 9.1 | 70.6 | |
| 62.0 | 97.6 | 80.6 | 3.3 | 82.4 | 50.0 | 0.0 | 27.3 | 88.2 | |
| Google Sheets · AI assistant + MCP | |||||||||
| 89.8 | 100.0 | 97.2 | 73.3 | 100.0 | 88.5 | 87.5 | 54.5 | 94.1 | |
| 88.8 | 95.2 | 97.2 | 66.7 | 100.0 | 88.5 | 100.0 | 90.9 | 76.5 | |
| 87.2 | 92.9 | 91.7 | 76.7 | 94.1 | 84.6 | 87.5 | 81.8 | 82.4 | |
| 81.8 | 100.0 | 97.2 | 60.0 | 94.1 | 88.5 | 87.5 | 18.2 | 58.8 | |
Below 90%Above 90%
| Product | Platform | Type | Model | MCP model | Effort | Overall score (%) | Average duration per task (s) | Formulas (%) | Spreadsheet manipulation (%) | Charts (%) | Data operations (%) | Formatting (%) | Pivot tables (%) | Bulk content generation (%) | Bulk data processing (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Autosheet in Excel | Excel | Sidebar add-on | Opus 5.5 | Medium | 97.3 | 27 | 100.0 | 100.0 | 96.7 | 100.0 | 96.2 | 87.5 | 100.0 | 88.2 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Astra | Low | 92.5 | 29 | 97.6 | 94.4 | 90.0 | 88.2 | 92.3 | 87.5 | 81.8 | 94.1 | |
| Autosheet in Excel | Excel | Sidebar add-on | Sonnet 5.5 | Medium | 92.0 | 16 | 100.0 | 94.4 | 93.3 | 94.1 | 88.5 | 87.5 | 63.6 | 88.2 | |
| Autosheet in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 91.4 | 28 | 100.0 | 97.2 | 96.7 | 82.4 | 84.6 | 100.0 | 63.6 | 82.4 | |
| Claude in Excel | Excel | Sidebar add-on | Opus 5.5 | High | 91.4 | 55 | 100.0 | 100.0 | 93.3 | 100.0 | 88.5 | 87.5 | 36.4 | 82.4 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6.1 Sol | Medium | 89.8 | 32 | 100.0 | 94.4 | 80.0 | 94.1 | 84.6 | 87.5 | 63.6 | 94.1 | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Astra | Medium | 86.1 | 59 | 100.0 | 94.4 | 80.0 | 94.1 | 76.9 | 87.5 | 36.4 | 82.4 | |
| Claude in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 84.5 | 41 | 92.9 | 94.4 | 86.7 | 88.2 | 84.6 | 87.5 | 27.3 | 70.6 | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 83.4 | 33 | 100.0 | 86.1 | 76.7 | 82.4 | 76.9 | 87.5 | 45.5 | 82.4 | |
| Copilot in Excel | Excel | Sidebar add-on | Auto | Auto | 81.8 | 44 | 100.0 | 91.7 | 70.0 | 88.2 | 80.8 | 100.0 | 18.2 | 64.7 | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 76.5 | 67 | 100.0 | 86.1 | 56.7 | 94.1 | 76.9 | 62.5 | 0.0 | 70.6 | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 92.0 | 39 | 97.6 | 97.2 | 83.3 | 94.1 | 92.3 | 87.5 | 81.8 | 88.2 | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 89.8 | 61 | 100.0 | 94.4 | 86.7 | 100.0 | 88.5 | 87.5 | 36.4 | 88.2 | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 87.2 | 50 | 100.0 | 94.4 | 70.0 | 82.4 | 88.5 | 87.5 | 81.8 | 76.5 | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 84.5 | 70 | 100.0 | 88.9 | 73.3 | 88.2 | 88.5 | 75.0 | 45.5 | 76.5 | |
| Gemini in Sheets | Google Sheets | Sidebar add-on | Auto | Auto | 74.9 | 126 | 83.3 | 86.1 | 60.0 | 88.2 | 80.8 | 87.5 | 9.1 | 70.6 | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Astra | Medium | 62.0 | 66 | 97.6 | 80.6 | 3.3 | 82.4 | 50.0 | 0.0 | 27.3 | 88.2 | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Luna | High | 57.2 | 72 | 97.6 | 83.3 | 3.3 | 70.6 | 46.2 | 0.0 | 0.0 | 64.7 | |
| Claude Code + Sheets MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Medium | 89.8 | 73 | 100.0 | 97.2 | 73.3 | 100.0 | 88.5 | 87.5 | 54.5 | 94.1 | |
| Claude Code + Autosheet MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Opus 5.5 | Medium | 88.8 | 75 | 95.2 | 97.2 | 66.7 | 100.0 | 88.5 | 100.0 | 90.9 | 76.5 |
| ChatGPT + Autosheet MCP | Google Sheets | AI assistant + MCP | GPT-6 | Opus 5.5 | Medium | 87.2 | 75 | 92.9 | 91.7 | 76.7 | 94.1 | 84.6 | 87.5 | 81.8 | 82.4 |
| ChatGPT + Drive MCP | Google Sheets | AI assistant + MCP | GPT-6 | Medium | 81.8 | 46 | 100.0 | 97.2 | 60.0 | 94.1 | 88.5 | 87.5 | 18.2 | 58.8 |
Score by task
- Passed
- Failed
- Autosheet in Excel
- Autosheet in Sheets
- Claude in Excel
- Claude Code + Sheets MCP
- Claude in Sheets
- Claude Code + Autosheet MCP
- ChatGPT + Autosheet MCP
- ChatGPT in Excel
- ChatGPT + Drive MCP
- Copilot in Excel
- Gemini in Sheets
- ChatGPT in Sheets
| Product | Platform | Type | Model | MCP model | Effort | Overall score (%) | Tasks passed | Tasks failed | Titles of the tasks failed |
|---|---|---|---|---|---|---|---|---|---|
| Autosheet in Excel | Excel | Sidebar add-on | Opus 5.5 | Medium | 97.3 | 182 | 5 | Make a pie chart of revenue by year; Add thousands separators; Pivot revenue, cost and margin; Categorize support tickets; Normalize ticket priorities | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Astra | Low | 92.5 | 173 | 14 | Build a cover sheet for a project plan; Align zero on a dual-axis combo chart; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Add seniority columns in the table's format; Format amounts as US dollars; Pivot revenue, cost and margin; Normalize ticket priorities; Generate 1,000 sample CRM records; Sort by date, oldest first; Fix broken formulas and validate amounts; Look up and color letter grades; Fix numbers stored as text; Translate HTML descriptions | |
| Autosheet in Excel | Excel | Sidebar add-on | Sonnet 5.5 | Medium | 92.0 | 172 | 15 | Add a cover page to an annual report; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Add a matching table of failed tool calls; Add thousands separators; Make a table look professional; Pivot revenue, cost and margin; Normalize ticket priorities; Find and replace a year in text; Generate 1,000 sample CRM records; Group rows by category; Summarize support tickets; Tag the main theme of each review; Translate a catalog into English; Translate tickets into English | |
| Autosheet in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 91.4 | 171 | 16 | Show a chart's labels as whole percentages; Add seniority columns in the table's format; Format amounts as US dollars; Add thousands separators; Make a table look professional; Classify the sentiment of tickets; Normalize ticket priorities; Add checkboxes to a range; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Rate the quality of product listings; Fix broken formulas and validate amounts; Fix numbers stored as text; Translate HTML descriptions; Translate tickets into English | |
| Claude in Excel | Excel | Sidebar add-on | Opus 5.5 | High | 91.4 | 171 | 16 | Combine revenue bars with a rate line; Make a pie chart of revenue by year; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot revenue, cost and margin; Categorize support tickets; Classify the sentiment of tickets; Draft a reply to each review; Find each company's headquarters; Generate 1,000 sample CRM records; Rewrite product descriptions in place; Summarize support tickets; Translate HTML descriptions; Translate reviews into English; Translate tickets into English | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6.1 Sol | Medium | 89.8 | 168 | 19 | Add a mini trend chart per product; Align zero on a dual-axis combo chart; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Split two series onto their own axes; Build an e-commerce KPI dashboard; Add seniority columns in the table's format; Add a matching table of failed tool calls; Format amounts as US dollars; Add thousands separators; Pivot revenue, cost and margin; Normalize ticket priorities; Freeze the top row and three columns; Generate 1,000 sample CRM records; Sort by date, oldest first; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Fix broken formulas and validate amounts | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Astra | Medium | 86.1 | 161 | 26 | Combine revenue bars with a rate line; Create a Gantt chart; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Split two series onto their own axes; Chart traffic sources, then make it a pie; Add seniority columns in the table's format; Highlight duplicate email addresses; Highlight revenues above a threshold; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot revenue, cost and margin; Categorize support tickets; Classify the sentiment of tickets; Filter rows above a revenue; Filter to one region; Find each company's headquarters; Write SEO descriptions; Generate 1,000 sample CRM records; Rewrite product descriptions in place; Summarize reviews in one line; Summarize support tickets; Fix broken formulas and validate amounts; Translate HTML descriptions; Translate tickets into English | |
| Claude in Excel | Excel | Sidebar add-on | Haiku 5.5 | High | 84.5 | 158 | 29 | Combine revenue bars with a rate line; Use a log scale on data with zeros; Make a pie chart of revenue by year; Chart the top 10 products plus Other; Add a matching table of failed tool calls; Format amounts as US dollars; Add thousands separators; Make a table look professional; Combine first and last names; Count the days since each inquiry; Split full names into first and last; Pivot revenue, cost and margin; Categorize support tickets; Classify the sentiment of tickets; Add checkboxes to a range; Detect salary ranges in job posts; Draft a reply to each review; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Rate the quality of product listings; Sort a table with merged cells; Summarize job descriptions; Summarize support tickets; Fix broken formulas and validate amounts; Tag the main theme of each review; Translate HTML descriptions; Translate reviews into English; Translate tickets into English | |
| Autosheet in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 83.4 | 156 | 31 | Add a cover page to an annual report; Build a cover sheet for a project plan; Align zero on a dual-axis combo chart; Build a cash-flow waterfall chart; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Split two series onto their own axes; Chart traffic sources, then make it a pie; Build an e-commerce KPI dashboard; Add seniority columns in the table's format; Format amounts as US dollars; Add thousands separators; Turn a revenue table into a heatmap; Hide two columns; Make a table look professional; Pivot revenue, cost and margin; Normalize ticket priorities; Extract order IDs from tickets; Find and replace a year in text; Freeze the top row and three columns; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Sort a table with merged cells; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Fix broken formulas and validate amounts; Tag the main theme of each review; Fix numbers stored as text; Translate HTML descriptions | |
| Copilot in Excel | Excel | Sidebar add-on | Auto | Auto | 81.8 | 153 | 34 | Align zero on a dual-axis combo chart; Create a Gantt chart; Create a bar chart labeling only its maximum; Label only the tallest bar of a chart; Label only the latest point of a line; Show a chart's labels as whole percentages; Chart sorted values, source unchanged; Split two series onto their own axes; Chart traffic sources, then make it a pie; Highlight revenues above a threshold; Highlight whole rows above a threshold; Add thousands separators; Hide two columns; Make a table look professional; Categorize support tickets; Classify the sentiment of tickets; Detect salary ranges in job posts; Draft a reply to each review; Extract order IDs from tickets; Find each company's headquarters; Find the INDEX/MATCH formulas; Freeze the top row and three columns; Generate 1,000 sample CRM records; Group rows by category; Normalize company industries; Rewrite product descriptions in place; Sort a table with merged cells; Summarize job descriptions; Summarize support tickets; Fix broken formulas and validate amounts; Translate HTML descriptions; Translate a catalog into English; Translate reviews into English; Translate tickets into English | |
| ChatGPT in Excel | Excel | Sidebar add-on | GPT-6 Luna | High | 76.5 | 143 | 44 | Build a cover sheet for a project plan; Add a mini trend chart per product; Align zero on a dual-axis combo chart; Make a donut chart of spend; Plot price against units sold; Create a Gantt chart; Give a chart a proper title; Create a bar chart labeling only its maximum; Label only the latest point of a line; Use a log scale on data with zeros; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Reverse the order of a chart's bars; Split two series onto their own axes; Add seniority columns in the table's format; Highlight the top 10 values; Highlight revenues above a threshold; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot cumulative revenue over time; Pivot revenue by region with % of total; Pivot revenue by region, then filter it; Categorize support tickets; Classify the sentiment of tickets; Draft a reply to each review; Extract order IDs from tickets; Filter rows above a revenue; Filter to one region; Find each company's headquarters; Find and replace a year in text; Write SEO descriptions; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Sort a table with merged cells; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Tag the main theme of each review; Translate HTML descriptions; Translate a catalog into English; Translate reviews into English; Translate tickets into English | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 92.0 | 172 | 15 | Create a Gantt chart; Use a log scale on data with zeros; Make a pie chart of revenue by year; Split two series onto their own axes; Switch a chart's axis to a log scale; Format amounts as US dollars; Add thousands separators; Track what's left of a budget; Pivot cumulative revenue over time; Categorize job titles; Normalize ticket priorities; Set the height of data rows; Summarize support tickets; Fix broken formulas and validate amounts; Translate HTML descriptions | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Opus 5.5 | Medium | 89.8 | 168 | 19 | Label only the tallest bar of a chart; Use a log scale on data with zeros; Make a pie chart of revenue by year; Split two series onto their own axes; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot cumulative revenue over time; Categorize support tickets; Classify the sentiment of tickets; Draft a reply to each review; Write SEO descriptions; Generate 1,000 sample CRM records; Rewrite product descriptions in place; Set the height of data rows; Summarize job descriptions; Summarize support tickets; Translate HTML descriptions; Convert a range to a table | |
| Autosheet in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 87.2 | 163 | 24 | Align zero on a dual-axis combo chart; Make a donut chart of spend; Create a Gantt chart; Use a log scale on data with zeros; Make a pie chart of revenue by year; Reverse the order of a chart's bars; Chart sorted values, source unchanged; Split two series onto their own axes; Switch a chart's axis to a log scale; Add seniority columns in the table's format; Add in-cell bars to a column; Make a table look professional; Pivot cumulative revenue over time; Classify the sentiment of tickets; Standardize messy countries and revenues; Normalize ticket priorities; Find the cells holding a value; Find and replace a year in text; Generate 1,000 sample CRM records; Group rows by category; Rate the quality of product listings; Set the height of data rows; Fix broken formulas and validate amounts; Translate HTML descriptions | |
| Claude in Sheets | Google Sheets | Sidebar add-on | Haiku 5.5 | High | 84.5 | 158 | 29 | Chart sales stacked by region; Combine revenue bars with a rate line; Create a Gantt chart; Label only the tallest bar of a chart; Label only the latest point of a line; Make a pie chart of revenue by year; Show the value on each bar; Chart the top 10 products plus Other; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot revenue, cost and margin; Pivot sales by region and product, ranked; Categorize support tickets; Classify the sentiment of tickets; Draft a reply to each review; Filter rows above a revenue; Find the cells holding a value; Find each company's headquarters; Group rows by category; Rewrite product descriptions in place; Set the height of data rows; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Fix broken formulas and validate amounts; Tag the main theme of each review; Translate HTML descriptions; Convert a range to a table | |
| Gemini in Sheets | Google Sheets | Sidebar add-on | Auto | Auto | 74.9 | 140 | 47 | Find hardcoded numbers among formulas; Model team capacity and hiring needs; Compute SMA, RSI and trading signals; Add a cover page to an annual report; Trace the cells that depend on a cell; Build a cash-flow waterfall chart; Combine revenue bars with a rate line; Create a Gantt chart; Label only the tallest bar of a chart; Use a log scale on data with zeros; Make a pie chart of revenue by year; Reverse the order of a chart's bars; Chart sorted values, source unchanged; Split two series onto their own axes; Switch a chart's axis to a log scale; Chart and polish a sales report; Build an e-commerce KPI dashboard; Band the rows that share a case name; Add in-cell bars to a column; Show numbers as K and M, negatives in red; Make a table look professional; Make a financial report presentation-ready; Count the days since each inquiry; Pivot sales by region and product, ranked; Categorize support tickets; Classify the sentiment of tickets; Draft a reply to each review; Find the INDEX/MATCH formulas; Assign clean product categories; Write SEO descriptions; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Score companies for customer fit; Set the height of data rows; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Break event budgets into line items; Fix broken formulas and validate amounts; Look up and color letter grades; Look up supplier rates and totals; Merge media mentions into one sheet; Tag the main theme of each review; Translate HTML descriptions; Translate a catalog into English; Translate reviews into English | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Astra | Medium | 62.0 | 116 | 71 | Add a cover page to an annual report; Add a missing legend to a chart; Align zero on a dual-axis combo chart; Turn a chart into a horizontal bar chart; Make a donut chart of spend; Chart the distribution of order sizes; Plot price against units sold; Chart sales stacked by region; Build a cash-flow waterfall chart; Combine revenue bars with a rate line; Create a Gantt chart; Give a chart a proper title; Create a bar chart labeling only its maximum; Label only the tallest bar of a chart; Label a chart's y-axis; Label only the latest point of a line; Use a log scale on data with zeros; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Reverse the order of a chart's bars; Show the value on each bar; Chart sorted values, source unchanged; Split two series onto their own axes; Switch a chart's axis to a log scale; Chart enrollment, then split it by level; Chart and polish a sales report; Chart revenue by quarter; Chart traffic sources, then make it a pie; Build an e-commerce KPI dashboard; Chart the top 10 products plus Other; Add seniority columns in the table's format; Highlight duplicate email addresses; Highlight the top 10 values; Highlight revenues above a threshold; Highlight whole rows above a threshold; Fit a wide report on an A4 page; Format amounts as US dollars; Add thousands separators; Add a centered title across a table; Make a table look professional; Protect a sheet from edits; Widen and wrap a description column; Wrap overflowing notes; Create and use a named range; Group transactions by month and quarter; Pivot cumulative revenue over time; Pivot revenue, cost and margin; Pivot revenue by region with % of total; Pivot hotel rates and nights by room; Pivot revenue by region, then filter it; Pivot the top 5 products by revenue; Pivot sales by region and product, ranked; Categorize support tickets; Classify the sentiment of tickets; Add checkboxes to a range; Add a dropdown of existing values; Draft a reply to each review; Filter rows above a revenue; Filter to one region; Write SEO descriptions; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Set the height of data rows; Sort a table with merged cells; Summarize job descriptions; Fix broken formulas and validate amounts; Translate HTML descriptions; Translate reviews into English; Translate tickets into English; Convert a range to a table | |
| ChatGPT in Sheets | Google Sheets | Sidebar add-on | GPT-6 Luna | High | 57.2 | 107 | 80 | Add a missing legend to a chart; Align zero on a dual-axis combo chart; Turn a chart into a horizontal bar chart; Make a donut chart of spend; Chart the distribution of order sizes; Plot price against units sold; Chart sales stacked by region; Build a cash-flow waterfall chart; Combine revenue bars with a rate line; Create a Gantt chart; Give a chart a proper title; Create a bar chart labeling only its maximum; Label only the tallest bar of a chart; Label a chart's y-axis; Label only the latest point of a line; Use a log scale on data with zeros; Show a chart's labels as whole percentages; Make a pie chart of revenue by year; Reverse the order of a chart's bars; Show the value on each bar; Chart sorted values, source unchanged; Split two series onto their own axes; Switch a chart's axis to a log scale; Chart enrollment, then split it by level; Chart and polish a sales report; Chart revenue by quarter; Chart traffic sources, then make it a pie; Build an e-commerce KPI dashboard; Chart the top 10 products plus Other; Add seniority columns in the table's format; Add in-cell bars to a column; Highlight the top 10 values; Clear all formatting, keep the data; Highlight revenues above a threshold; Highlight whole rows above a threshold; Fit a wide report on an A4 page; Format amounts as US dollars; Add thousands separators; Add a centered title across a table; Make a table look professional; Protect a sheet from edits; Widen and wrap a description column; Wrap overflowing notes; Create and use a named range; Group transactions by month and quarter; Pivot cumulative revenue over time; Pivot revenue, cost and margin; Pivot revenue by region with % of total; Pivot hotel rates and nights by room; Pivot revenue by region, then filter it; Pivot the top 5 products by revenue; Pivot sales by region and product, ranked; Categorize support tickets; Classify the sentiment of tickets; Add checkboxes to a range; Add a dropdown of existing values; Detect salary ranges in job posts; Draft a reply to each review; Extract order IDs from tickets; Filter rows above a revenue; Filter to one region; Find each company's headquarters; Find and replace a year in text; Write SEO descriptions; Generate 1,000 sample CRM records; Group rows by category; Rewrite product descriptions in place; Set the height of data rows; Sort a table with merged cells; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Fix broken formulas and validate amounts; Tag the main theme of each review; Fix numbers stored as text; Translate HTML descriptions; Translate a catalog into English; Translate reviews into English; Translate tickets into English; Convert a range to a table | |
| Claude Code + Sheets MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Medium | 89.8 | 168 | 19 | Make a donut chart of spend; Build a cash-flow waterfall chart; Create a Gantt chart; Label only the latest point of a line; Use a log scale on data with zeros; Make a pie chart of revenue by year; Split two series onto their own axes; Switch a chart's axis to a log scale; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot cumulative revenue over time; Filter rows above a revenue; Write SEO descriptions; Generate 1,000 sample CRM records; Rewrite product descriptions in place; Score companies for customer fit; Summarize job descriptions; Translate HTML descriptions | |
| Claude Code + Autosheet MCP | Google Sheets | AI assistant + MCP | Opus 5.5 | Opus 5.5 | Medium | 88.8 | 166 | 21 | Align zero on a dual-axis combo chart; Make a donut chart of spend; Build a cash-flow waterfall chart; Create a Gantt chart; Create a bar chart labeling only its maximum; Label only the latest point of a line; Use a log scale on data with zeros; Make a pie chart of revenue by year; Split two series onto their own axes; Switch a chart's axis to a log scale; Format amounts as US dollars; Add thousands separators; Make a table look professional; Count the days since each inquiry; Categorize job titles; Categorize support tickets; Normalize ticket priorities; Filter rows above a revenue; Generate 1,000 sample CRM records; Rate the quality of product listings; Flag abnormal patient vitals |
| ChatGPT + Autosheet MCP | Google Sheets | AI assistant + MCP | GPT-6 | Opus 5.5 | Medium | 87.2 | 163 | 24 | Compute SMA, RSI and trading signals; Create a Gantt chart; Create a bar chart labeling only its maximum; Label only the tallest bar of a chart; Use a log scale on data with zeros; Make a pie chart of revenue by year; Split two series onto their own axes; Switch a chart's axis to a log scale; Add seniority columns in the table's format; Format amounts as US dollars; Add thousands separators; Make a table look professional; Count the days since each inquiry; Pivot cumulative revenue over time; Categorize support tickets; Filter rows above a revenue; Find each company's headquarters; Generate 1,000 sample CRM records; Rewrite product descriptions in place; Set the height of data rows; Fix broken formulas and validate amounts; Flag abnormal patient vitals; Tag the main theme of each review; Convert a range to a table |
| ChatGPT + Drive MCP | Google Sheets | AI assistant + MCP | GPT-6 | Medium | 81.8 | 153 | 34 | Add a cover page to an annual report; Align zero on a dual-axis combo chart; Turn a chart into a horizontal bar chart; Make a donut chart of spend; Create a Gantt chart; Label only the tallest bar of a chart; Label only the latest point of a line; Use a log scale on data with zeros; Make a pie chart of revenue by year; Reverse the order of a chart's bars; Split two series onto their own axes; Switch a chart's axis to a log scale; Build an e-commerce KPI dashboard; Format amounts as US dollars; Add thousands separators; Make a table look professional; Pivot cumulative revenue over time; Categorize job titles; Categorize support tickets; Classify the sentiment of tickets; Detect salary ranges in job posts; Draft a reply to each review; Extract order IDs from tickets; Write SEO descriptions; Generate 1,000 sample CRM records; Rewrite product descriptions in place; Rate the quality of product listings; Summarize job descriptions; Summarize reviews in one line; Summarize support tickets; Fix broken formulas and validate amounts; Tag the main theme of each review; Translate HTML descriptions; Translate a catalog into English |
Strengths and weaknesses
Where each product shines, where it slips, and what the scores don't show, on its best model.
| Product / model | Strong | Watch out for | Good to know |
|---|---|---|---|
| Excel · Sidebar add-ons | |||
Autosheet in ExcelOpus 5.5 (Medium) | The most consistent: first or tied first in six of eight capabilities, and far ahead on bulk content generation. | Bulk results might depend on the model chosen for the bulk run. | — |
Claude in ExcelOpus 5.5 (High) | Near perfect on formulas, structural edits and data operations, and reliable charts. | Lower bulk content generation quality. It writes every row itself: long jobs are slow, can stop partway and use templates. | Often asks permission to keep a long script running: about one task in five. |
ChatGPT in ExcelGPT-6 Astra (Medium) | Strong on formulas and data operations. | Fixed formats and edits instead of conditional rules or filters. Slow, templated content generation. | — |
Copilot in ExcelAuto | Passes every pivot table task, and is faster than Claude and ChatGPT in Excel. | Several weaknesses: templated content generation, quotas on translations. It misses fine chart details. | Copilot picks the model; there is no choice. |
| Google Sheets · Sidebar add-ons | |||
Autosheet in SheetsOpus 5.5 (Medium) | Best in Google Sheets. The best add-on at bulk content generation, and at the top on native spreadsheet work. | — | Only Claude in Sheets passed one of the two log-scale chart tasks in Google Sheets, while every product passed both in Excel. |
Claude in SheetsOpus 5.5 (Medium) | The same strong native spreadsheet work as in Excel. | Slow and low quality bulk content generation, as in Excel. Charts are less reliable than in Excel. | Asks before running some scripts. A request left unanswered stalls the task. |
Gemini in SheetsAuto | Data operations and simple structural edits. | The slowest, at over two minutes per task. It passes one bulk content generation task in eleven. | Low monthly AI credits: finishing the benchmark took a second month (they renew on the 1st) and an upgrade. |
ChatGPT in SheetsGPT-6 Astra (Medium) | Formulas. | No native charts or pivot tables: it draws text bars and formula summaries instead. Passes half the formatting tasks. | — |
| Google Sheets · AI assistant + MCP | |||
Claude Code + Sheets MCPOpus 5.5 (Medium) | Good native spreadsheet work through a general-purpose coding agent. | Slow, often over a minute per task, and middling on bulk content generation. | Better than the Claude add-ons at bulk content generation, by spawning subagents that work in parallel. |
Claude Code + Autosheet MCPOpus 5.5 (Medium) | Handing bulk work to Autosheet does better on bulk content than the Sheets MCP. | Slower than Autosheet in Sheets, often over a minute per task, due to the back-and-forth between the two assistants. | Claude Code asks you to approve the Autosheet tool on many tasks. |
ChatGPT + Autosheet MCPGPT-6 (Medium) | Strong on data operations. Far ahead of ChatGPT + Drive MCP on bulk work, and ahead on charts. | Bulk tasks still depend on the bulk model. Risk of a “telephone game” effect between ChatGPT and the Autosheet agent. | — |
ChatGPT + Drive MCPGPT-6 (Medium) | Strong on formulas and structural edits. | Weak on bulk work on long datasets, both on data processing and content generation. | Rebuilds charts rather than editing them, without carrying over every setting: they can lose their colors. |
Head to head
Pick two products to see where each one wins, capability by capability and task by task.
Compare the outputs
Or select a capability on the chart.
| Capability | Tasks compared | Autosheet in Excel (Opus 5.5, Medium) score (%) | Copilot in Excel (Auto) score (%) | Autosheet in Excel (Opus 5.5, Medium) wins | Copilot in Excel (Auto) wins | Both passed | Both failed |
|---|---|---|---|---|---|---|---|
| Formulas | 42 | 100.0 | 100.0 | 0 | 0 | 42 | 0 |
| Spreadsheet manipulation | 36 | 100.0 | 91.7 | 3 | 0 | 33 | 0 |
| Charts | 30 | 96.7 | 70.0 | 9 | 1 | 20 | 0 |
| Data operations | 17 | 100.0 | 88.2 | 2 | 0 | 15 | 0 |
| Formatting | 26 | 96.2 | 80.8 | 4 | 0 | 21 | 1 |
| Pivot tables | 8 | 87.5 | 100.0 | 0 | 1 | 7 | 0 |
| Bulk content generation | 11 | 100.0 | 18.2 | 9 | 0 | 2 | 0 |
| Bulk data processing | 17 | 88.2 | 64.7 | 5 | 1 | 10 | 1 |
| All capabilities | 187 | 97.3 | 81.8 | 32 | 3 | 150 | 2 |
Selected examples
Pick an example to see the prompt, the starting spreadsheet and what each product made of it.
Create a Gantt chart.
| A | B | C | |
|---|---|---|---|
| 1 | Task | Start Date | Duration (days) |
| 2 | Plan | 2025-04-01 | 3 |
| 3 | Design | 2025-04-03 | 5 |
| 4 | Build | 2025-04-08 | 8 |
| 5 | QA | 2025-04-14 | 4 |
| 6 | Launch | 2025-04-19 | 2 |
| 7 | Training | 2025-04-20 | 3 |
| 8 | Review | 2025-04-24 | 2 |
| 9 | Wrap-up | 2025-04-26 | 1 |
- A horizontal bar chart
- Only the task blocks are colored
- Readable dates in order
- A clear title; any legend at the top
Passed 3
Autosheet in Excel Opus 5.5 · Medium
Claude in Excel Opus 5.5 · High
Claude in Sheets Opus 5.5 · Medium
Failed 9
Autosheet in Sheets Opus 5.5 · Medium
Why it failed: The horizontal axis shows day numbers ('0', '10', '20', '30') labeled 'Days from project start' instead of readable calendar dates.
Claude Code + Sheets MCP Opus 5.5 · Medium
Why it failed: The horizontal axis shows numeric increments (0, 10, 20, 30) rather than dates.
Claude Code + Autosheet MCP Opus 5.5 · Medium
Why it failed: The horizontal axis shows numeric day offsets rather than readable dates.
ChatGPT + Autosheet MCP GPT-6 · Medium
Why it failed: No chart object was created (chartCount is 0).
ChatGPT in Excel GPT-6 Astra · Medium
Why it failed: The horizontal axis counts days from the start, 0 to 28, instead of showing dates.
Copilot in Excel Auto
Why it failed: Floating Gantt blocks were not formed; standard solid bars originate from the vertical axis.
ChatGPT + Drive MCP GPT-6 · Medium
Why it failed: The horizontal axis displays numeric offsets (0, 10, 20, 30) rather than readable dates.
Gemini in Sheets Auto
Why it failed: The horizontal axis plots numeric day offsets (=B2-$B$2) rather than readable calendar dates.
ChatGPT in Sheets GPT-6 Astra · Medium
Why it failed: No chart was created.
Browse the full dataset
Check out all 187 tasks in detail. Click on any task to zoom in.
187 tasks
- EasyFormulasFind hardcoded numbers among formulas
- EasyFormulasModel team capacity and hiring needs
- MediumFormulasCompute SMA, RSI and trading signals
MediumSpreadsheet manipulationAdd a cover page to an annual report
EasySpreadsheet manipulationBuild a cover sheet for a project plan- EasyFormulasTrace the cells that depend on a cell
EasyChartsAdd a missing legend to a chart- EasyChartsAdd a mini trend chart per product
HardChartsAlign zero on a dual-axis combo chart
MediumChartsTurn a chart into a horizontal bar chart
HardChartsMake a donut chart of spend
EasyChartsChart the distribution of order sizes
MediumChartsPlot price against units sold
MediumChartsChart sales stacked by region
MediumChartsBuild a cash-flow waterfall chart
MediumChartsCombine revenue bars with a rate line
HardChartsCreate a Gantt chart
MediumChartsGive a chart a proper title
MediumChartsCreate a bar chart labeling only its maximum
HardChartsLabel only the tallest bar of a chart
EasyChartsLabel a chart's y-axis
HardChartsLabel only the latest point of a line
HardChartsUse a log scale on data with zeros
MediumChartsShow a chart's labels as whole percentages
HardChartsMake a pie chart of revenue by year
MediumChartsReverse the order of a chart's bars
MediumChartsShow the value on each bar
MediumChartsChart sorted values, source unchanged
HardChartsSplit two series onto their own axes
HardChartsSwitch a chart's axis to a log scale
EasyChartsChart enrollment, then split it by level
MediumChartsChart and polish a sales report
EasyChartsChart revenue by quarter
MediumChartsChart traffic sources, then make it a pie
MediumChartsBuild an e-commerce KPI dashboard
MediumChartsChart the top 10 products plus Other
EasyFormattingBand the rows that share a case name- MediumFormattingAdd seniority columns in the table's format
EasyFormattingAdd a matching table of failed tool calls- EasyFormattingFit columns to their text
MediumFormattingAdd in-cell bars to a column
EasyFormattingHighlight duplicate email addresses
MediumFormattingHighlight the top 10 values
EasyFormattingClear all formatting, keep the data
MediumFormattingHighlight revenues above a threshold
MediumFormattingHighlight whole rows above a threshold
EasyFormattingShow numbers as K and M, negatives in red- EasyFormattingFit a wide report on an A4 page
- HardFormattingFormat amounts as US dollars
EasyFormattingFormat dates as dd/mm/yy
EasyFormattingShow a column as percentages- HardFormattingAdd thousands separators
EasyFormattingShow two decimals without changing values
EasyFormattingTurn a revenue table into a heatmap- EasyFormattingHide two columns
EasyFormattingAdd a centered title across a table
HardFormattingMake a table look professional- EasyFormattingProtect a sheet from edits
- EasyFormattingColor-code grades and band the rows
- EasyFormattingMake a financial report presentation-ready
- EasyFormattingWiden and wrap a description column
EasyFormattingWrap overflowing notes- EasyFormulasAdd 14 days to due dates
- EasyFormulasCombine first and last names
- EasyFormulasCount the orders above an amount
EasyFormulasSort months in calendar order with a formula- EasyFormulasCreate and use a named range
- EasyFormulasShow a top N set by a cell
- MediumFormulasCount the days since each inquiry
- EasyFormulasExtract domains from email addresses
- EasyFormulasExtract the year and month from dates
- EasyFormulasFlag orders over $1,000
- EasyFormulasKeep each product's latest record
- EasyFormulasFilter rows on region, revenue and date
- EasyFormulasTurn scores into letter grades
- EasyFormulasCount weekdays between two dates
- EasyFormulasPad IDs with leading zeros
- EasyFormulasCompute each row's share of the total
- EasyFormulasCompute a running total
- EasyFormulasSplit full names into first and last
- EasyFormulasTotal only the visible rows
- EasyFormulasSum a metric across monthly tabs
- EasyFormulasTotal a column
- EasyFormulasSum revenue for one region and quarter
- EasyFormulasCompute budget variances and totals
- EasyFormulasCompute pay with overtime
- EasyFormulasTrack what's left of a budget
- EasyFormulasReturn ISO week numbers
EasyPivot tablesGroup transactions by month and quarter
HardPivot tablesPivot cumulative revenue over time
MediumPivot tablesPivot revenue, cost and margin
MediumPivot tablesPivot revenue by region with % of total
EasyPivot tablesPivot hotel rates and nights by room
MediumPivot tablesPivot revenue by region, then filter it
EasyPivot tablesPivot the top 5 products by revenue
MediumPivot tablesPivot sales by region and product, ranked- EasySpreadsheet manipulationAdd a column to every tab
- MediumBulk data processingCategorize job titles
- EasyBulk data processingAssign product types to a catalog
- HardBulk data processingCategorize support tickets
- EasyBulk data processingClassify the sentiment of reviews
- HardBulk data processingClassify the sentiment of tickets
- EasyBulk data processingStandardize messy countries and revenues
- EasyData operationsClean codes into 5-digit text
- EasyData operationsStrip a currency from amounts
- MediumBulk data processingNormalize ticket priorities
- EasySpreadsheet manipulationRemove a filter
- EasySpreadsheet manipulationClear a row
- EasyFormulasCount the customers from one country
MediumData operationsAdd checkboxes to a range- EasyData operationsAdd a dropdown of existing values
- EasyData operationsMerge duplicate customer rows
- EasySpreadsheet manipulationDelete a column by name
- EasySpreadsheet manipulationDelete blank rows
- EasySpreadsheet manipulationDelete a sheet
- MediumBulk data processingDetect salary ranges in job posts
- HardBulk content generationDraft a reply to each review
- EasySpreadsheet manipulationDuplicate and rename a tab
- MediumBulk data processingExtract order IDs from tickets
- EasyBulk data processingExtract product weights in grams
- EasyData operationsFill empty discounts with zero
- EasyBulk data processingFill in missing ticket priorities
- HardSpreadsheet manipulationFilter rows above a revenue
- MediumSpreadsheet manipulationFilter to one region
- EasyData operationsFind the cells holding a value
- MediumBulk data processingFind each company's headquarters
- MediumData operationsFind the INDEX/MATCH formulas
- EasyFormulasFind the top-selling product
MediumData operationsFind and replace a year in text- EasyBulk data processingAssign clean product categories
- EasyData operationsConvert formulas to values
- EasyData operationsFill blank group labels down
- EasySpreadsheet manipulationFreeze the first column
- EasySpreadsheet manipulationFreeze the header row
- MediumSpreadsheet manipulationFreeze the top row and three columns
- HardBulk content generationWrite SEO descriptions
- HardBulk content generationGenerate 1,000 sample CRM records
- EasySpreadsheet manipulationCopy matching rows into a second table
- HardSpreadsheet manipulationGroup rows by category
- EasyFormulasLook up a value to the left
- EasySpreadsheet manipulationInsert a column between two others
- EasySpreadsheet manipulationInsert a title row above a table
- EasyData operationsList the #DIV/0! errors
- EasySpreadsheet manipulationMove a column to the front
- EasyFormulasReturn the first non-empty value
- EasySpreadsheet manipulationAdd a new tab
- EasyBulk data processingNormalize company industries
- EasyData operationsRemove duplicate rows
- EasySpreadsheet manipulationRename a sheet
- EasyData operationsReplace a phrase in one column
- HardBulk content generationRewrite product descriptions in place
- MediumBulk data processingScore companies for customer fit
- MediumBulk data processingRate the quality of product listings
- HardSpreadsheet manipulationSet the height of data rows
- MediumSpreadsheet manipulationSort a table with merged cells
- EasySpreadsheet manipulationSort by two keys
- EasySpreadsheet manipulationSort by date, oldest first
- EasySpreadsheet manipulationSort by revenue, highest first
- EasyData operationsSplit city and state into two columns
- EasySpreadsheet manipulationMove booked rows to their own tab
- HardBulk content generationSummarize job descriptions
- MediumBulk content generationSummarize reviews in one line
- HardBulk content generationSummarize support tickets
- EasyFormulasSummarize spend by department
- EasySpreadsheet manipulationBreak event budgets into line items
- EasySpreadsheet manipulationAdd a weekly timesheet
- EasySpreadsheet manipulationCompute reorder quantities
- EasySpreadsheet manipulationFill in room capacities and durations
- HardData operationsFix broken formulas and validate amounts
- MediumFormulasFlag abnormal patient vitals
- EasyFormulasLook up and color letter grades
- EasyFormulasLook up supplier rates and totals
- EasyFormulasCompute ad metrics and flag profitable ones
- EasySpreadsheet manipulationMerge media mentions into one sheet
- EasySpreadsheet manipulationConsolidate plant data onto one sheet
- EasyFormulasLook up prices and compute totals
- HardBulk data processingTag the main theme of each review
- EasyData operationsFix numbers stored as text
- EasyFormulasTotal the committed capital
- HardBulk content generationTranslate HTML descriptions
- MediumBulk content generationTranslate a catalog into English
- MediumBulk content generationTranslate reviews into English
- MediumBulk content generationTranslate tickets into English
- EasySpreadsheet manipulationTranspose a table in place
- EasyFormulasLook up a value by row and column
- EasySpreadsheet manipulationUnfreeze rows and columns
MediumSpreadsheet manipulationConvert a range to a table
Every task and run is also in the AutosheetBench dataset on Hugging Face.
How we built the tasks
AutosheetBench runs mostly on synthetic data, modeled on the Autosheet usage of thousands of users: 166 of its 187 tasks. The other 21 start from real data, public or anonymized, with names and identifiers replaced.
Real use cases
The benchmark covers what people ask Autosheet to do in Excel and Google Sheets. We analyzed tens of thousands of anonymized Autosheet conversations and broke their requests into unit tasks.
Realistic human prompts
We wrote every prompt for the benchmark, short and direct like our users' own. They don't spell out every requirement the way an AI would. What the person asking would expect is in the pass criteria, not the prompt.
Synthetic data, real variety
The workbooks are synthetic: generated from anonymized conversations, then adjusted by hand. Some start from real data, public or anonymized, with names and identifiers replaced.
Pass criteria edited by hand
An LLM judge grades the tasks whose result can take several forms. Every checkpoint of its checklist was edited by hand to match what users expect.
How each task is scored
Every task is pass or fail, with no partial credit. We use deterministic checks, LLM judges and, for bulk tasks, whole-set checks.
Deterministic checks
96 native tasksWe compare the workbook with the expected final state (values, formulas, rules and more). A single miss fails the task. When a task can be done more than one right way, each way is accepted.
LLM judge with a checklist
63 native tasksWhen the result could take several forms (a chart, a cover sheet, an answer in the chat), an LLM judge grades the final workbook, screenshots of it and the transcript against a checklist. One missed checkpoint fails the task.
We used Gemini 3.8 Flash as judge.Whole-set checks
28 bulk tasksIn bulk tasks, every row must be filled and follow the prompt's rules (e.g., a maximum length or a fixed list of values): at least 98% of rows must match when each row has one right answer, and three tasks require every row. When answers can vary (translation, SEO description, etc.), an LLM judge also grades 50 sampled rows, and 45 must pass. Generated text must also vary: a task fails when 10 or fewer templates (sentence patterns reused from row to row) cover 80% of the rows, and a generated dataset fails when the judge finds its entries repeated or combined from a few reused words.
Criteria calibrated to real-life expectations
Numbers must be readable. Presentation must be intuitive. Generated text must be high quality.
Numbers must be readable
Right values aren't enough. We also check formats, rules, charts and how the sheet renders. A number hidden behind ###### or cut off by its column fails, even when the value is right.
How it's checked: LLM judge with a checklist
Add thousands separators to the big numbers.
The separators are there, but column B is too narrow: ###### hides two of the numbers.
The column fits, so every number reads in full.
Presentation must be intuitive
Prompts are short, the way people write them. We grade what the person asking would expect, not only what the prompt spells out. Every checklist, accepted answer and quality criterion was calibrated that way.
How it's checked: LLM judge with a checklist
Create a Gantt chart.
The bars are right, but the axis counts days from the start: 0, 2, 4, 6.
The axis shows dates, which is what a project plan reads by.
High quality generated text
Generated text, such as a summary, a translation or a product description, can follow every rule and still be wrong. An LLM judge reads a sample of rows against their source. A row fails if a careful person could easily make it more accurate, complete or useful.
How it's checked: Whole-set checks
Summarize each job description in column C (job_description). Add a new column with the headerjob_summaryin column F. Write a clear one-line summary of at most 160 characters describing the role's main purpose and responsibilities. Do not copy the full description and fill every row.
…extracting actionable insights to help empower businesses to better manage their fleets.
Lead teams and projects, build machine-learning models, and analyze complex data to improve healthcare decisions.
Nothing in the job is about healthcare. Copilot reused one boilerplate: 845 of its 1,000 summaries start with “Lead teams and projects”.
Develop machine learning models and statistical tools from connected-vehicle data to improve fleet insights and transportation decisions.
Specific to the job: vehicle data, used to serve fleets.
FAQ
How we ran AutosheetBench's 187 tasks, judged them, timed them and measured cost.
What are the tasks?
Each task is one request a user would make to a spreadsheet assistant: a workbook, a prompt and the final state we check. We built them by breaking real, anonymized requests into single steps, each on a workbook: synthetic for most tasks, real data with identifiers replaced for 21 of them (see How we built the tasks). 22 tasks add a scripted follow-up prompt, sent once the assistant finishes the first.
| Category | Tasks | For example |
|---|---|---|
| Formulas | 42 | Writing formulas, tracing dependents |
| Spreadsheet manipulation | 36 | Hiding, deleting, or inserting rows and columns |
| Charts | 30 | Creating and updating charts |
| Data operations | 17 | Trimming, find and replace, filling blanks, finding specific cells |
| Formatting | 26 | Conditional formatting, number formats |
| Pivot tables | 8 | Creating pivot tables |
| Bulk content generation | 11 | Translating text, writing SEO descriptions, summarizing reviews |
| Bulk data processing | 17 | Extracting a salary from a job description, cleaning or standardizing data, classifying the sentiment of reviews |
Each task starts from a fresh copy of its workbook and must be finished within 5 minutes. The 26 long bulk tasks, which fill up to 1,000 rows, get 10 minutes.
Every task passes or fails. Of the 159 native tasks, 96 are checked deterministically against the expected final state, and 63 by an LLM judge with a checklist, 6 of them on an exact answer. The 28 bulk tasks get whole-set checks, deterministic checks that every row is filled and follows the prompt's rules: 14 by those checks alone, and the 14 whose answers can vary, such as translations, by those checks and a quality judge that grades 50 sampled rows (see How each task is scored). 8 of the 4,114 results were scored by hand, mostly where a Google Sheets export loses what the assistant did (a sheet's protection, its checkboxes); the dataset marks each one with its reason.
How do the LLM judges work?
The checklist judge
The checklist judge scores 63 of the native tasks: those where a script can't tell whether the result is right, such as a chart or a pivot table, or where the answer is in the assistant's reply rather than in the sheet. The task passes only if every checkpoint passes.
It runs Gemini 3.8 Flash through OpenRouter, at medium effort, and judges each output once. Judging all 1,386 outputs cost $12.44, about 1¢ each. Each task's page shows a short version of its checklist.
Its system prompt for a task without charts:
You evaluate whether a spreadsheet agent fulfilled a user's spreadsheet request. You will receive: - the authored user request - a structured scoring rubric with weighted control points - a full transcript - actual spreadsheet images - expected images when available - workbook summaries Use the authored user request and rubric to define success. Final artifacts are the primary source of truth for the final result. Do not treat the transcript as proof that a step succeeded unless the final artifacts support it. Prefer image/workbook evidence over transcript when judging the final state. Be strict, concise, and evidence-based. Respond with JSON only.
For a task with charts, the same prompt gets chart and document details instead of workbook summaries:
You evaluate whether a spreadsheet agent fulfilled a user's spreadsheet request. You will receive: - the authored user request - a structured scoring rubric with weighted control points - a full transcript - actual spreadsheet images - expected images when available - chart/document payload metadata Use the authored user request and rubric to define success. Final artifacts are the primary source of truth for the final result. Do not treat the transcript as proof that a step succeeded unless the final artifacts support it. Prefer image/workbook evidence over transcript when judging the final state. Be strict, concise, and evidence-based. Respond with JSON only.
For a task where every checkpoint is about the assistant's reply, the judge gets only the prompt, the checklist and the transcript, with this system prompt:
You evaluate whether a spreadsheet agent correctly answered a user's spreadsheet question. Use only the authored user request, rubric, and transcript supplied to you. Do not inspect, infer from, or score any workbook summary, spreadsheet image, or output workbook state. Be strict, concise, and evidence-based. Respond with JSON only.
The quality judge
The quality judge scores the 14 bulk tasks whose answers can vary, such as translations, summaries or product descriptions. The quality judge reads a sample of 50 rows of the bulk output, and passes or fails each row against the task's quality bar. It sees the prompt, the 50 rows, the quality bar and the rule for passing. The task passes if at least 45 of the 50 rows pass and the deterministic checks pass too. For the one task that generates a dataset, the judge also grades the dataset as a whole, which fails when its entries repeat or are combined from a few reused words. Three tasks that generate text also fail if 10 or fewer templates (sentence patterns reused from row to row) write 80% of the rows: a script checks this, not the judge.
It runs Gemini 3.8 Flash through OpenRouter, at high effort and temperature 0, and judges each output once. It needs high effort because it has no checklist to follow: at medium, it let real errors through. Judging all 308 outputs cost $26.20, about 9¢ each. Each task's page shows its quality bar and its rule for passing.
Its prompt, below, is a single message, with each task's own parts in brackets.
You are a strict independent quality-control reviewer. Judge one anonymous spreadsheet submission against the task. Review every row independently and return a binary PASS or FAIL for each row. Do not assign numeric quality scores and do not compare the output to any other product. TASK: [the task's prompt] QUALITY BAR: - [one line per criterion of the task's quality bar] [the task's decision rule] For every row FAIL, name the concrete correction. For PASS rows, return an empty reason and an empty issues array. Do not spend output tokens explaining acceptable rows. Return JSON only in exactly this shape: {"caseId":"[the task's id]","rows":[{"sampleId":"row-2","pass":true,"reason":"brief, specific evidence","issues":["short_issue_code"]}],"batchRationale":"brief overall pattern"} Return exactly one decision for every supplied sample ID. Keep each FAIL reason under 25 words. SAMPLES: [the 50 sampled rows, each with its source columns and the output]
Why are we comfortable using Gemini 3.8 Flash to judge stronger models?
- Calibrated on human expectations. We tested the judge and its effort level, and checked that its verdicts agree with human judgement.
- Checking is easier than doing. The judge doesn't solve the task: it compares the output with what is expected, validating it against a strict checklist.
- It is independent of most products. It comes from neither Anthropic nor OpenAI, whose models run most of the products here, so it has no reason to prefer the style of the text they write.
- It keeps judging costs reasonable. Judging every output on this page once cost ~$39 on OpenRouter. Opus 5.5 would have cost about five times as much for the same tokens, some $210.
Which products, models and effort levels did we test?
| Product | Model | Effort |
|---|---|---|
| Excel · Sidebar add-ons | ||
| Autosheet in Excel | Opus 5.5 | Medium |
| Sonnet 5.5 | Medium | |
| Haiku 5.5 | High | |
| GPT-6 Astra | Low | |
| GPT-6.1 Sol | Medium | |
| GPT-6 Luna | High | |
| Claude in Excel | Opus 5.5 | High |
| Haiku 5.5 | High | |
| ChatGPT in Excel | GPT-6 Astra | Medium |
| GPT-6 Luna | High | |
| Copilot in Excel | Auto | Auto |
| Google Sheets · Sidebar add-ons | ||
| Autosheet in Sheets | Opus 5.5 | Medium |
| Haiku 5.5 | High | |
| Claude in Sheets | Opus 5.5 | Medium |
| Haiku 5.5 | High | |
| Gemini in Sheets | Auto | Auto |
| ChatGPT in Sheets | GPT-6 Astra | Medium |
| GPT-6 Luna | High | |
| Google Sheets · AI assistant + MCP | ||
| Claude Code (web) + Sheets MCP | Opus 5.5 | Medium |
| Claude Code (web) + Autosheet MCP | Opus 5.5 · MCP runs Opus 5.5 | Medium · MCP Medium |
| ChatGPT (web) + Autosheet MCP | GPT-6 · MCP runs Opus 5.5 | Medium · MCP Medium |
| ChatGPT (web) + Drive MCP | GPT-6 | Medium |
How each model and its effort level were set:
- Autosheet: for each model, the level with the best quality-to-cost ratio on the internal benchmark we use to recommend settings.
- Copilot and Gemini choose their model and effort level themselves and don't show them: the page reads Auto.
- ChatGPT on the web, in the two ChatGPT + MCP configurations, doesn't say which model answers: the page reads GPT-6.
- The other products: the model closest to an Autosheet configuration (Opus 5.5 in Claude, GPT-6 Astra in ChatGPT in Excel), at the product's default or recommended effort level. In Excel and Google Sheets, Claude also ran Haiku 5.5 and ChatGPT ran GPT-6 Luna, matching two more Autosheet configurations.
What is AI cost and how is it computed?
AI cost is what a run's model calls cost at the AI providers' API list prices: the tokens each call used, and the web searches.
What you really pay depends greatly on the plan you're on, as each product and plan takes a different margin on the API prices. That's why we compare the API cost: it's the only objective cost we can track. Claude's and ChatGPT's subscriptions in particular end up far cheaper than their API cost: a $25 Standard seat of either holds a whole run of all 187 tasks, 13¢ a task, while their heaviest runs cost 35¢ and 80¢ a task at API list prices (see How much does it cost to run the full benchmark?).
How the AI cost is computed
From the tokens recorded for each model call, at the providers' API list prices of November 2026.
| Model, per million tokens | Input | Cache write5m on Claude30m on OpenAI | Cache read | Output | Web searchper 1,000 |
|---|---|---|---|---|---|
| Claude Opus 5.5 | $4 | $5 | $0.20 | $20 | $10 |
| Claude Sonnet 5.5 | $2 | $2.50 | $0.10 | $10 | $10 |
| Claude Haiku 5.55× on a prompt over 100,000 tokens | $0.10 | $0.125 | $0.01 | $0.50 | $10 |
| GPT-6 Astra | $10 | $12.50 | $1 | $50 | $10 |
| GPT-6.1 Sol | $2 | $2.50 | $0.10 | $10 | $10 |
| GPT-6 Luna | $0.10 | $0.125 | $0.01 | $0.50 | $10 |
| Perplexity FastBulk only · Agent API (GPT-5.6 Luna underneath) | $0.20 | $0.25 | $0.02 | $1.20 | $2.50 |
Autosheet runs every OpenAI model but GPT-6 Astra on the priority tier (prices ×2). ChatGPT's and Claude's add-ons run on the standard tier.
Autosheet's bulk rows run on GPT-6 Luna. A few tasks in seven configurations called the bulk model before the switch; their tokens are priced at GPT-6 Luna's rates (under 7 cents per configuration).
We couldn't record Copilot's and Gemini's token usage. The charts show their plan's month instead. We couldn't compute the API usage of the assistants on an MCP either, so those configurations have no AI cost.
How much does it cost to run the full benchmark?
The cheapest plan that holds one run of all 187 tasks within a month, at its US monthly price (billed monthly). Prices don't include the Microsoft 365 or Google Workspace subscription for Excel or Sheets itself.
For Autosheet: it's the Standard plan ($25, $25 of usage included), plus the cheapest extra-usage packs ($29, $79, $139, $299, $999) if needed.
| Configuration | Plan | Month |
|---|---|---|
| Gemini in Sheets · Auto | Google Workspace AI Expanded Access | $24 |
| Claude in Excel · Opus 5.5 | Claude Team – Standard seat | $25 |
| Autosheet in Excel · Haiku 5.5 | Autosheet Standard | $25 |
| Claude Code + Sheets MCP · Opus 5.5 | Claude Team – Standard seat | $25 |
| Claude in Sheets · Opus 5.5 | Claude Team – Standard seat | $25 |
| ChatGPT in Excel · GPT-6 Astra | ChatGPT Business – Standard seat | $25 |
| Claude in Excel · Haiku 5.5 | Claude Team – Standard seat | $25 |
| ChatGPT + Drive MCP · GPT-6 | ChatGPT Business – Standard seat | $25 |
| ChatGPT in Excel · GPT-6 Luna | ChatGPT Business – Standard seat | $25 |
| ChatGPT in Sheets · GPT-6 Astra | ChatGPT Business – Standard seat | $25 |
| Copilot in Excel · Auto | Microsoft 365 Copilot (Premium) | $31.50 |
| Autosheet in Excel · Opus 5.5 | Autosheet Standard + $29 usage | $54 |
| Autosheet in Excel · GPT-6 Astra | Autosheet Standard + $29 usage | $54 |
| Autosheet in Excel · Sonnet 5.5 | Autosheet Standard + $29 usage | $54 |
| Autosheet in Excel · GPT-6.1 Sol | Autosheet Standard + $29 usage | $54 |
| Autosheet in Sheets · Haiku 5.5 | Autosheet Standard + $29 usage | $54 |
| Autosheet in Excel · GPT-6 Luna | Autosheet Standard + $29 usage | $54 |
| Autosheet in Sheets · Opus 5.5 | Autosheet Standard + $58 usage | $83 |
| Claude Code + Autosheet MCP · Opus 5.5 | Claude Team – Standard seat + Autosheet Standard + $79 usage | $129 |
| ChatGPT + Autosheet MCP · GPT-6 | ChatGPT Business – Standard seat + Autosheet Standard + $79 usage | $129 |
A Claude Team or ChatGPT Business Standard seat holds every run, by the usage history of the seats the runs used: a Claude Standard month holds at least $226 of API-equivalent usage and a ChatGPT one about $602, while the heaviest runs need $66 and $150.
How did we run and record the tasks?
How we run the tasks
A script, the runner, runs every task the same way:
- Open a fresh copy of the workbook
- Start the screen recording
- Send the prompt
- Grant any permission the assistant asks for, then wait for it to finish
- Send the follow-up, if any
- Save the workbook and the transcript
In Excel, the runner drives the add-on through macOS accessibility. In Google Sheets and for the assistants connected through an MCP, it drives Chrome through computer use, an AI agent that operates the screen and follows the same steps, with the chat beside the sheet so the recording shows both. A few Excel runs used computer use too.
When a run doesn't go to plan
- We discard runs that fail for a reason outside the product and run them again from a fresh copy. That covers a network error, credits or a usage cap running out, and a model at capacity. We also run a task again when the task itself changed: its input or its prompt. We score the first run that ends cleanly, never the best of several.
- When an assistant stops to ask a question, we send one fixed reply: keep going and complete the original task, using your best judgment for any remaining choices. We never add facts or hints. The time it waits counts toward the time limit.
- We stop a run at its time limit and score the workbook as it is. The limit is 5 minutes, or 10 for the long bulk tasks. 44 of the 4,114 runs reached it, and 5 of them still pass.
What we keep from each run
From one run, Claude in Excel with Opus 5.5 on the “Chart sales stacked by region” task:
- Screen recordingThe whole run, the sheet beside the chat
- TranscriptThe product's own export where it has one
- chart_stacked_bar_output.xlsxOutput workbookSaved as the run left it
- Output screenshotTaken after the run, cropped to the result
- Timings, approvals and questionsClaude's session time, checked on the recording
- Duration
- 9.2 s
- Turns
- 1
- Approvals
- 0
- Questions
- 0
How duration is computed
We time each run from the moment the prompt is sent to the end of the answer, added up over its turns, so the runner's time to send a follow-up doesn't count. Most runs are timed on their screen recording, frame by frame. Autosheet's durations come from its own logs, which we checked against its recordings: they agree within 2 seconds. Its runs without a log are timed on their recording like the others. Claude in Excel's with Opus 5.5 come from Claude's usage dashboard, its session time, which agrees with its recordings within a second on 181 of its 187 tasks; the other 6 are cut off at their time limit. Claude in Excel's with Haiku 5.5, and a few runs of ChatGPT and Claude in Sheets, come from the product's own timing. When a Claude add-in asks permission to run a script, the time its card waits for our click is measured on the recording and taken out.
Where no recording shows the end of a turn, the runner's clock times the run: 138 Copilot runs, and 63 runs of the other assistants, most of them connected through the Autosheet MCP. It notices the end of a turn late, so those durations are upper bounds.
What setup did we use?
- MachineMacBook Pro 14-inchApple M5 Pro chip, 48 GB of memorymacOS Tahoe (26)
- ExcelExcel for MacThe desktop app, kept up to dateIn French (fr-FR)
- Google SheetsGoogle ChromeOn the same Mac
- Claude CodeClaude Code on the webclaude.ai/code, in ChromeFor the MCP runs
- ChatGPTChatGPT on the webchatgpt.com, in ChromeFor the MCP runs
We always use the latest version of each product, which changes frequently. At the time of writing, the Mac runs macOS 26.7.2 and Excel is version 16.114.1.
How can I verify these results?
Every run is on Hugging Face, in the AutosheetBench dataset: each task's starting workbook, prompts, expected result and pass criteria, and each configuration's output workbook and transcript. Most runs also have their screen recording, which is what lets us time a run precisely, and which shows exactly what the assistant did when you want to check a verdict yourself.
Anyone can score the outputs again with the scoring script in the AutosheetBench repo: give it a task and an output workbook, and it says whether the output passes.
How can I run the benchmark myself?
Everything you need is on GitHub, in the AutosheetBench repo: the 187 tasks, the instructions for running them, and the scripts that run and score them. You can run a few tasks or all of them.
The easiest way to start is to point an AI agent at the repo and ask it to run one task.
Cite AutosheetBench
If AutosheetBench is useful for your research or writing, please cite it. Every configuration's scores, task results and each run's AI cost and duration are also available as CSV files. The tasks, the scorer and the runner are on GitHub, and every result with its evidence is on Hugging Face. The tasks (prompts, workbooks and checks) and the scores are licensed under CC BY 4.0, except the job postings, the product listings, the rewritten app reviews and two SpreadsheetBench-v2 workbooks (MIT). The products' answers, workbooks, transcripts, screenshots and recordings are not covered.
Earlier runs are described in Best Excel AI models: Introducing AutosheetBench (October 6, 2026) and Best Excel AI add-ins: Autosheet vs Claude vs ChatGPT vs Copilot (October 26, 2026).
AutosheetBench: Benchmarking Excel and Sheets AINicolas Augusti and Stanislas Marion · November 2026Contact: nicolas.augusti@talarian.io · stan@talarian.io
@misc{autosheetbench2026,
title = {{AutosheetBench}: Benchmarking Excel and Sheets {AI}},
author = {Augusti, Nicolas and Marion, Stanislas},
organization = {Talarian},
year = {2026},
month = nov,
url = {https://autosheet.com/benchmark},
note = {Latest run November 17, 2026},
}