Add a matching table of failed tool calls
FormattingEasyLLM judge scoringLatest run
Passed by all 12 products on their best model.
Prompt
Add a similar table to "# turns with tool call" ones below it, but only with the ones were the tool failed at least once (use dataset col J)
Dataset
Starting sheet · rows 1–21 of 95,215 and columns A–J of 14| A | B | C | D | E | F | G | H | I | J | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | agent_run_id | agent_id | product | first_pack_type | start_event_timestamp | Date | nb_loops | distinct_tools_used | ok_tools | failed_tools |
| 2 | 3c4d6ccb-5366-4267-84b2-342b574b2e6d | 663bb40f-ea62-4899-8862-e6c72c880f39 | excel | 2026-06-08 14:00:58.574 UTC | Jun 8, 2026 | |||||
| 3 | f61a43a5-ffd8-4267-8e15-e783b8169f81 | e6725675-7a0c-4e09-80e3-1bd707ea8070 | excel | FREE_TRIAL | 2026-06-08 14:00:48.485 UTC | Jun 8, 2026 | 1 | write_range | write_range | |
| 4 | ef030721-83f3-4080-87e9-535c36921822 | 45500c7e-a84e-4d3b-8bcc-9ed1e73e22a3 | excel | PAID_PACK | 2026-06-08 14:00:45.646 UTC | Jun 8, 2026 | 2 | get_sheet_overview | get_sheet_overview | |
| 5 | 11a7dd6a-f619-4132-877b-791a6f5b6251 | 1cff89f2-0e27-400e-80c9-156ef9e4b5e1 | excel | FREE_TRIAL | 2026-06-08 14:00:43.926 UTC | Jun 8, 2026 | 3 | get_sheet_overview, read_range | get_sheet_overview, read_range | |
| 6 | dce3f6d4-aa4b-4ee7-8fa3-24cf06eb7332 | 618d474d-df15-4804-8fd0-e14816bb124a | excel | FREE_TRIAL | 2026-06-08 14:00:40.372 UTC | Jun 8, 2026 | 1 | |||
| 7 | 53026ebf-abb7-4b57-8b71-1fc37d974a24 | 74c40571-26d6-4b5b-8bb3-00b7b7c9688f | excel | FREE_TRIAL | 2026-06-08 14:00:37.32 UTC | Jun 8, 2026 | 2 | write_range | write_range | |
| 8 | 27e8df97-3d67-4cc4-882b-1c33b8c6c644 | ad185af1-ae1e-4b69-8d68-c433ee609ee5 | excel | FREE_TRIAL | 2026-06-08 14:00:37.046 UTC | Jun 8, 2026 | 2 | get_sheet_overview | get_sheet_overview | |
| 9 | 18c9f6d6-96dd-4c20-817a-0230af92c505 | 33b5af13-5ec4-4f41-8011-e34463d85d47 | excel | FREE_TRIAL | 2026-06-08 14:00:36.807 UTC | Jun 8, 2026 | 2 | write_range | write_range | |
| 10 | 3421c317-dcc2-4212-887d-e1c822f7ff14 | 03f46d2a-cf5c-49c7-8bcf-88f4a0c5e424 | excel | FREE_TRIAL | 2026-06-08 14:00:35.732 UTC | Jun 8, 2026 | 2 | execute_officejs_script | execute_officejs_script | |
| 11 | c3f939f7-6431-4c5e-8104-f1d005c2af2f | 46013fa2-66a1-4ee6-85a2-01e1033a7641 | excel | FREE_TRIAL | 2026-06-08 14:00:34.792 UTC | Jun 8, 2026 | 1 | |||
| 12 | 2862f8a6-1813-4db7-8776-451cf4001a3f | 5b7fd5b9-05ab-4800-8163-92544482b53d | excel | FREE_TRIAL | 2026-06-08 14:00:34.011 UTC | Jun 8, 2026 | 1 | |||
| 13 | 452ac514-73d1-48e4-8076-2b98a42fca08 | 8b1e03b1-893c-4bbf-8f5a-96a46c7d488c | excel | FREE_TRIAL | 2026-06-08 14:00:31.659 UTC | Jun 8, 2026 | 3 | execute_officejs_script, write_range | execute_officejs_script, write_range | |
| 14 | 139e1032-f4e5-4ebf-802f-8697d61a9ec1 | 47819f2d-f5b7-4ed2-8e73-4165e10524bd | excel | FREE_TRIAL | 2026-06-08 14:00:25.992 UTC | Jun 8, 2026 | 2 | copy_paste_range | copy_paste_range | |
| 15 | 9b0b3c72-59e2-4243-8195-f116cb878db9 | 08c0a702-e756-4540-8075-4db55a3101f8 | excel | FREE_TRIAL | 2026-06-08 14:00:23.718 UTC | Jun 8, 2026 | 2 | write_range | write_range | |
| 16 | 155a398f-f11c-4bcc-8992-b9b627501f68 | 414044b1-aee9-4101-8b34-c244d2e4dbcf | excel | FREE_TRIAL | 2026-06-08 14:00:21.593 UTC | Jun 8, 2026 | 3 | copy_paste_range, write_range | copy_paste_range, write_range | |
| 17 | 6876a140-9e3e-452f-8d37-b59335d06d32 | 2aee2b37-cf6c-4a9e-8e57-754521cd0af7 | excel | FREE_TRIAL | 2026-06-08 14:00:21.159 UTC | Jun 8, 2026 | 3 | execute_officejs_script, write_range | execute_officejs_script, write_range | |
| 18 | 1c5c2f4f-72e2-4bdb-825a-cb618c4d750b | cbfc142a-a975-4bfe-8a46-218c0b9aa848 | sheets | PAID_PACK | 2026-06-08 14:00:20.569 UTC | Jun 8, 2026 | 2 | ai_bulk_process_rows | ai_bulk_process_rows | |
| 19 | a9459dc5-f8c0-455e-8305-8fd946f8a71b | 128e14a1-207a-4471-81ba-3f1b581c0195 | excel | FREE_TRIAL | 2026-06-08 14:00:19.823 UTC | Jun 8, 2026 | 3 | execute_officejs_script, get_sheet_overview | execute_officejs_script, get_sheet_overview | |
| 20 | f1f09115-309c-4933-86e0-ddd7c04a0ecf | a9a8ea07-82f3-4489-8199-08c3509d97ab | excel | PAID_PACK | 2026-06-08 14:00:14.355 UTC | Jun 8, 2026 | 2 | get_sheet_overview, read_range | get_sheet_overview, read_range | |
| 21 | a71b6f3f-7a31-493c-832a-523f1031ce7a | 56296678-05c0-44c3-8e31-4be77f891066 | excel | FREE_TRIAL | 2026-06-08 14:00:08.165 UTC | Jun 8, 2026 | 4 | execute_officejs_script, write_range | execute_officejs_script, write_range |
Analysis
Starting sheet| B | C | D | E | F | G | H | I | J | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | |||||||||
| 2 | Pack | FREE_TRIAL | |||||||
| 3 | |||||||||
| 4 | |||||||||
| 5 | # turns with tool call | ||||||||
| 6 | Date | Jun 2, 2026 | Jun 3, 2026 | Jun 4, 2026 | Jun 5, 2026 | Jun 6, 2026 | Jun 7, 2026 | Jun 8, 2026 | |
| 7 | read_range | 2,867 | 3,007 | 2,594 | 1,934 | 1,204 | 1,142 | 1,452 | |
| 8 | write_range | 4,588 | 4,788 | 4,057 | 3,487 | 2,874 | 2,806 | 3,182 | |
| 9 | copy_paste_range | 1,982 | 2,109 | 1,820 | 1,444 | 1,121 | 1,054 | 1,270 | |
| 10 | ai_generate_list | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| 11 | ai_bulk_process_rows | 501 | 646 | 514 | 394 | 242 | 304 | 373 | |
| 12 | get_sheet_overview | 1,419 | 1,398 | 1,268 | 1,075 | 700 | 666 | 823 | |
| 13 | spreadsheet_batch_update | 159 | 152 | 167 | 126 | 79 | 76 | 79 | |
| 14 | execute_officejs_script | 6,464 | 6,715 | 5,868 | 4,872 | 3,787 | 3,597 | 4,292 | |
| 15 | read_spreadsheet_fields | 147 | 161 | 146 | 124 | 72 | 64 | 68 | |
| 16 | load_skill | 886 | 922 | 853 | 697 | 578 | 538 | 568 |
Expected result
Excel and Sheets · All types · All models · All outputs
Pass criteria
All must be met, checked by an LLM judge.- A new table of failed tool calls by tool and date
- Formatted like the original table
- The original data is kept
Passed 19
Autosheet in Excel Opus 5.5 · Medium
Passed24s · 12¢
Autosheet in Excel GPT-6 Astra · Low
Passed20s · 17¢
Autosheet in Sheets Opus 5.5 · Medium
Passed44s · 14¢
D20=D6
| A | B | C | D | E | F | G | H | I | J | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||
| 11 | ai_bulk_process_rows | 501 | 646 | 514 | 394 | 242 | 304 | 373 | ||
| 12 | get_sheet_overview | 1,419 | 1,398 | 1,268 | 1,075 | 700 | 666 | 823 | ||
| 13 | spreadsheet_batch_update | 159 | 152 | 167 | 126 | 79 | 76 | 79 | ||
| 14 | execute_officejs_script | 6,464 | 6,715 | 5,868 | 4,872 | 3,787 | 3,597 | 4,292 | ||
| 15 | read_spreadsheet_fields | 147 | 161 | 146 | 124 | 72 | 64 | 68 | ||
| 16 | load_skill | 886 | 922 | 853 | 697 | 578 | 538 | 568 | ||
| 17 | ||||||||||
| 18 | ||||||||||
| 19 | # turns with failed tool call | |||||||||
| 20 | Date | Jun 2, 2026 | Jun 3, 2026 | Jun 4, 2026 | Jun 5, 2026 | Jun 6, 2026 | Jun 7, 2026 | Jun 8, 2026 | ||
| 21 | read_range | 45 | 8 | 14 | 8 | 1 | 3 | 3 | ||
| 22 | write_range | 95 | 107 | 79 | 79 | 54 | 56 | 60 | ||
| 23 | copy_paste_range | 29 | 21 | 27 | 22 | 24 | 29 | 31 | ||
| 24 | ai_generate_list | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
| 25 | ai_bulk_process_rows | 232 | 275 | 221 | 173 | 102 | 128 | 157 | ||
| 26 | get_sheet_overview | 3 | 2 | 2 | 3 | 1 | 2 | 5 | ||
| 27 | spreadsheet_batch_update | 5 | 1 | 2 | 2 | 1 | 3 | 0 | ||
| 28 | execute_officejs_script | 98 | 107 | 123 | 83 | 66 | 69 | 75 | ||
| 29 | read_spreadsheet_fields | 1 | 0 | 0 | 1 | 0 | 0 | 0 | ||
| 30 | load_skill | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Claude in Excel Opus 5.5 · High
Passed22s · 20¢
Autosheet in Excel Haiku 5.5 · High
Passed40s · 0.7¢
Claude in Sheets Opus 5.5 · Medium
Passed50s · 14¢
Claude Code + Sheets MCP Opus 5.5 · Medium
Passed1m 28s · N/A
Claude Code + Autosheet MCP Opus 5.5 · Medium
Passed2m 16s · N/A
ChatGPT + Autosheet MCP GPT-6 · Medium
Passed1m 37s · N/A
Autosheet in Sheets Haiku 5.5 · High
Passed2m 34s · 2¢
ChatGPT in Excel GPT-6 Astra · Medium
Passed30s · 62¢
Claude in Sheets Haiku 5.5 · High
Passed47s · 1¢
Autosheet in Excel GPT-6 Luna · High
Passed59s · 1¢
Copilot in Excel Auto
Passed56s · N/A
ChatGPT + Drive MCP GPT-6 · Medium
Passed51s · N/A
ChatGPT in Excel GPT-6 Luna · High
Passed1m 45s · 2¢
Gemini in Sheets Auto
Passed2m 58s · N/A
ChatGPT in Sheets GPT-6 Astra · Medium
Passed55s · 60¢
ChatGPT in Sheets GPT-6 Luna · High
Passed1m 54s · 1¢
Failed 3
Autosheet in Excel Sonnet 5.5 · Medium
Failed21s · 6¢
Why it failed: No new table was created below the existing table on the Analysis sheet.
Autosheet in Excel GPT-6.1 Sol · Medium
Failed39s · 12¢
Why it failed: The formatting does not match the original table: the title header lacks the yellow background (it is styled with a pale blue fill), and the header background is pale blue rather than the…
Claude in Excel Haiku 5.5 · High
Failed39s · 2¢
Why it failed: The table title in C18 is '# turns with tool call (tools that failed at least once)' rather than the required '# turns with failed tool call'.