All AutosheetBench tasks

Count the customers from one country

FormulasEasyLLM judge scoringLatest run

Passed by all 12 products on their best model.

Prompt and starting spreadsheet

Task files

Prompt

How many customers are from Germany?

Starting spreadsheet

Analysis
ABCDEFGHI
1ProductSalesCalculationTotalCountryCommitted CapitalLookup ProductLookup Value
2Alpha10Germany1250000Alpha100
3Bravo48199638France2500000Bravo200
4Charlie250Germany1750000Charlie300
5Delta900#DIV/0!5000Germany500000Delta400
6Echo1200Spain900000Echo500
7Foxtrot775#DIV/0!Italy650000

A passing output

Autosheet in Excel · Opus 5.5
Transcript excerptExpected answer
Next, I'll add a live count formula so the number stays up to date. 3 entries are from Germany: Alpha, Charlie and Delta. The other countries are France, Spain and Italy, with one each. One thing to know: the sheet lists products, not customers. So this counts products whose Country is Germany. There's no separate customer list to count. I put a live count in K2, labeled in K1, so the number updates if the data changes: =COUNTIF(E2:E100,"Germany")

How each AI configuration did

22 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • Answers 3 customers

Passed 22

Autosheet in Excel Opus 5.5 · Medium

Passed10s · 3¢
Transcript excerptExpected answer
Next, I'll add a live count formula so the number stays up to date. 3 entries are from Germany: Alpha, Charlie and Delta. The other countries are France, Spain and Italy, with one each. One thing to know: the sheet lists products, not customers. So this counts products whose Country is Germany. There's no separate customer list to count. I put a live count in K2, labeled in K1, so the number updates if the data changes: =COUNTIF(E2:E100,"Germany")

Autosheet in Excel GPT-6 Astra · Low

Passed8s · 6¢
Transcript excerptExpected answer
I’ll count the Germany entries in the Country field and place the result beside your data. Format and verify customer count 3 customers are from Germany.

Autosheet in Sheets Opus 5.5 · Medium

Passed12s · 2¢
Transcript excerptExpected answer
3 rows have Germany as the country: Alpha, Charlie and Delta, out of 6 in total. There's no "Customer" column in the sheet, so I've assumed each row (named in the Product column) is one customer. I counted the matches in E2:E7 and didn't change the sheet. To get a count that updates automatically, use: Formula =COUNTIF(E2:E7, "Germany")

Autosheet in Excel Sonnet 5.5 · Medium

Passed4s · 1¢
Transcript excerptExpected answer
3 rows have Germany as the country (Alpha, Charlie and Delta). The data has no customer column, so I counted each row as one customer. I put the count in K2 with a COUNTIF formula, labeled "Germany Count" in K1.

Claude in Excel Opus 5.5 · High

Passed7s · 4¢
Transcript excerptExpected answer
3: rows 2, 4 and 5 (E2, E4, E5) list Germany in the Country column, E2:E7. The other three rows are France, Spain and Italy, one each. The sheet names each row by product (Alpha, Charlie, Delta), not by customer. So this counts German rows, assuming each row is one customer. Those three rows add up to £3.5m of committed capital (1,250,000 + 1,750,000 + 500,000). I didn't change anything in the sheet. If you want the count to update itself, =COUNTIF(E2:E7,"Germany") will do it.

Autosheet in Excel Haiku 5.5 · High

Passed6s · 0.1¢
Transcript excerptExpected answer
I'll check the Country column to count the Germany entries. 3 customers are from Germany (rows 2, 4, and 5 in the Analysis sheet's Country column). =COUNTIF(E2:E8,"Germany")

Claude in Sheets Opus 5.5 · Medium

Passed10s · 3¢
Transcript excerptExpected answer
3 rows have Germany in the Country column (E2:E7): Alpha, Charlie and Delta. The sheet doesn't have a customer column. Each row is a product, so I counted rows. If you want this as a live count in the sheet, you could use =COUNTIF(E2:E7,"Germany"). I only looked at the sheet and didn't change anything.

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed13s · N/A
Transcript excerptExpected answer
3. The only tab, "Analysis", has 6 data rows, and 3 of them list Germany in the Country column (column E): Product Country Committed Capital Alpha Germany 1,250,000 Charlie Germany 1,750,000 Delta Germany 500,000 The other 3 rows are France (Bravo), Spain (Echo) and Italy (Foxtrot). The sheet has no column labelled "customer", so I counted each row as one customer, named in the Product column. If customers are defined some other way, the count could differ. I only read the data and didn't change anything in the spreadsheet.

Autosheet in Excel GPT-6.1 Sol · Medium

Passed11s · 2¢
Transcript excerptExpected answer
I’ll count the Germany entries in the Country field and add the result beside the existing data. Verify and format Germany customer count 3 customers are from Germany. The count is also in Analysis!K2.

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed21s · N/A
Transcript excerptExpected answer
3 rows are from Germany. The spreadsheet has one tab, Analysis. Its table is at A1:F7 with 6 data rows, and the country is in column E. Cells E2, E4 and E5 (the Alpha, Charlie and Delta rows) say exactly "Germany". No other spelling of Germany appears, such as "DE" or "Deutschland". The other rows are France, Spain and Italy. The table has no customer name or ID column. Each row is labeled by product (Alpha to Foxtrot), so 3 is a count of rows, not of distinct customers. If one customer can appear on more than one row, the number of distinct German customers could be lower. Separately, cells C5, C7 and D7 show formula errors. They don't affect the count. I didn't change anything in the spreadsheet.

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed23s · N/A
Transcript excerptExpected answer
3 customers are from Germany out of 6 total customers. According to the spreadsheet's Country column (E), they are: Alpha (row 2) Charlie (row 4) Delta (row 5) That's 50% of all customers. The spreadsheet was not modified.

Autosheet in Sheets Haiku 5.5 · High

Passed23s · 0.2¢
Transcript excerptExpected answer
3 customers are from Germany. This counts the Germany entries in Analysis!E2:E7 (rows 2, 4 and 5). I used a temporary COUNTIF formula to get the count and then cleared it.

ChatGPT in Excel GPT-6 Astra · Medium

Passed16s · 36¢
Transcript excerptExpected answer
3 customers are from Germany. (Analysis!E2:E7)

Claude in Excel Haiku 5.5 · High

Passed7s · 0.2¢
Transcript excerptExpected answer
3 of the records in the Country column (E2:E7) are from Germany. Those are rows 2, 4, and 5. The sheet has no separate customer column, so I counted each row as one customer. I didn't change the workbook.

Claude in Sheets Haiku 5.5 · High

Passed11s · 0.8¢
Transcript excerptExpected answer
3 customers are from Germany, in rows 2, 4, and 5 of the Country column (E2, E4, E5). I treated each row as one customer. The sheet labels the rows by product (Alpha through Foxtrot), so tell me if a row is not a customer. I haven't changed the sheet. If you want the count kept live, I can add a =COUNTIF(E2:E, "Germany") cell. I re-read A1:I7 to confirm the count.

Autosheet in Excel GPT-6 Luna · High

Passed8s · 0.1¢
Transcript excerptExpected answer
I’ll check the country entries for all listed customers and count the Germany matches. 3 customers are from Germany.

Copilot in Excel Auto

Passed15s · N/A
Transcript excerptExpected answer
There are 3 customers from Germany in Analysis!E2:E7. There are 3 customers from Germany in Analysis!E2:E7.

ChatGPT + Drive MCP GPT-6 · Medium

Passed17s · N/A
Transcript excerptExpected answer
There are 3 customers from Germany in the Google spreadsheet, based on the Country column (E) in the Analysis tab. Source: Google Spreadsheet.

ChatGPT in Excel GPT-6 Luna · High

Passed7s · 0.3¢
Transcript excerptExpected answer
There are 3 customers from Germany. (Analysis!E2:E7)

Gemini in Sheets Auto

Passed16s · N/A
Transcript excerptExpected answer
Based on the provided data, there are 3 records associated with Germany: Alpha (Sales: 10, Committed Capital: 1,250,000) Charlie (Sales: 250, Committed Capital: 1,750,000) Delta (Sales: 900, Committed Capital: 500,000) Note: Since the dataset does not contain an explicit "Customer" column, each product entry in the table was assumed to represent an individual account/customer record.

ChatGPT in Sheets GPT-6 Astra · Medium

Passed11s · 29¢
Transcript excerptExpected answer
3 customers are from Germany. AnalysisE2:E7

ChatGPT in Sheets GPT-6 Luna · High

Passed10s · 0.3¢
Transcript excerptExpected answer
There are 3 customers from Germany. AnalysisE2:E7