All AutosheetBench tasks

Fill in missing ticket priorities

Bulk data processingEasyWhole-set scoringLatest run

Passed by all 12 products on their best model.

Prompt and starting spreadsheet

Task files

Prompt

Fill in the missing priority values in column E from each ticket message respecting existing priority values.

Starting spreadsheet

Sheet1 · rows 1–21 of 38
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link expired twice and the user cannot access payroll before the noon deadline.Account Access
3T-2002ContosoUser wants to change the company logo shown on PDF quotes.Settings
4T-2003FabrikamAPI latency is causing production checkout requests to time out intermittently.API
5T-2004Adventure WorksCustomer asks whether the dashboard can show weekly sales instead of monthly.Reporting
6T-2005Blue YonderQuestion about whether archived projects count against the storage limit.Billing
7T-2006Coho WineryPayment failed for the annual renewal and every seat is now locked for the whole company.Billing
8T-2007Datum LabsNew teammate did not receive an invite email after being added by the admin.Account Access
9T-2008Wide World ImportersMobile app crashes every time the user opens the order history screen after the latest update.BugHigh
10T-2009Lucerne PublishingCustomer asks how to export last quarter's invoices for their accountant.Billing
11T-2010Tailspin ToysUploaded product photos appear blurry in the catalog preview.Media
12T-2011ProsewareWebhook deliveries have failed for every event since yesterday evening.API
13T-2012Alpine Ski HouseUser needs help finding the CSV import template for contacts.Onboarding
14T-2013City PowerA compliance export is missing audit entries for two administrators.ReportingLow
15T-2014Humongous InsuranceCustomer asks for best practices to organize teams by region.Settings
16T-2015Graphic Design InstituteSeveral users see a blank page after SSO login from the identity provider.Account Access
17T-2016RelecloudAdmin wants to rename a workspace before next month's rollout.Settings
18T-2017Fourth CoffeeCheckout returns a 500 error for all EU customers since the last deploy.Bug
19T-2018Trey ResearchHow do I add a second billing contact to the account?Billing
20T-2019Woodgrove BankTwo-factor codes are arriving 10+ minutes late for several admins.Account AccessUrgent
21T-2020Margie's TravelCustomer requests a walkthrough of the reporting filters.Onboarding

A passing output

Autosheet in Excel · Opus 5.5
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

How each AI configuration did

22 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked automatically.
  • E2:E37: filled with "Low", "Medium", "High" or "Urgent"
  • E9, E14, E20 and 3 more ranges hold the expected values

Passed 22

Autosheet in Excel Opus 5.5 · Medium

Passed32s · 10¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Autosheet in Excel GPT-6 Astra · Low

Passed30s · 10¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIUrgent
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessHigh
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Autosheet in Sheets Opus 5.5 · Medium

Passed40s · 10¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

Autosheet in Excel Sonnet 5.5 · Medium

Passed18s · 4¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIUrgent
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Claude in Excel Opus 5.5 · High

Passed23s · 18¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Autosheet in Excel Haiku 5.5 · High

Passed29s · 0.8¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessMedium
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Claude in Sheets Opus 5.5 · Medium

Passed30s · 12¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed34s · N/A
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

Autosheet in Excel GPT-6.1 Sol · Medium

Passed24s · 4¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed59s · N/A
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingMedium
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed1m 4s · N/A
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIUrgent
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

Autosheet in Sheets Haiku 5.5 · High

Passed31s · 0.8¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingMedium
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessHigh
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

ChatGPT in Excel GPT-6 Astra · Medium

Passed21s · 48¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Claude in Excel Haiku 5.5 · High

Passed16s · 0.6¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Claude in Sheets Haiku 5.5 · High

Passed33s · 1¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

Autosheet in Excel GPT-6 Luna · High

Passed30s · 0.9¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIUrgent
5T-2004Adventure WorksCustomer asks whether t…ReportingMedium
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingMedium

10 of 37 rows

Copilot in Excel Auto

Passed22s · N/A
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

ChatGPT + Drive MCP GPT-6 · Medium

Passed40s · N/A
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

ChatGPT in Excel GPT-6 Luna · High

Passed24s · 0.5¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingMedium
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 37 rows

Gemini in Sheets Auto

Passed1m 43s · N/A
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessUrgent
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

ChatGPT in Sheets GPT-6 Astra · Medium

Passed25s · 38¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessMedium
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows

ChatGPT in Sheets GPT-6 Luna · High

Passed52s · 0.4¢
Sheet130 cells changed
ABCDE
1TicketIDCustomerMessageCategoryPriority
2T-2001NorthwindPassword reset link exp…Account AccessHigh
3T-2002ContosoUser wants to change th…SettingsLow
4T-2003FabrikamAPI latency is causing …APIHigh
5T-2004Adventure WorksCustomer asks whether t…ReportingLow
6T-2005Blue YonderQuestion about whether …BillingLow
7T-2006Coho WineryPayment failed for the …BillingUrgent
8T-2007Datum LabsNew teammate did not re…Account AccessLow
9T-2008Wide World ImportersMobile app crashes ever…BugHigh
10T-2009Lucerne PublishingCustomer asks how to ex…BillingLow

10 of 1,000 rows