Agent runs
Every episode a real provider produced, with its grade, cost and failure labels. Open a run for the full observable trajectory: tool calls, code, outputs, errors, the final answer and the grader's verdict per check.
| Run | Task | Agent | Model | Outcome | Score | Calls | Tools | Errors | Tokens | Cost | Wall | Repro | Labels |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 09-06 17:18 · r0 | t1-missing-audittrap | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.150 | 1 | 2 | 1 | 1,996 | $0.00059 | 17 s | no | no_outputwrong_datasetduplicated_eventswrong_variablemissing_valuessentinel_valuenon_reproducibleexecution_failed |
| 09-06 17:18 · r0 | t1-schema-summary | Single-shot | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 1 | 2 | 0 | 1,464 | $0.00038 | 12 s | exact | |
| 09-06 17:18 · r0 | t2-mass-histogram | Single-shot | Qwen3-8B (gariyuu gateway) | core only | 0.475 | 1 | 2 | 1 | 2,147 | $0.00068 | 20 s | no | no_outputplotting_errorwrong_variablenon_reproducibleexecution_failed |
| 09-06 17:18 · r0 | t2-selection-counttrap | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.117 | 1 | 2 | 1 | 2,174 | $0.00069 | 21 s | no | no_outputwrong_datasetinvalid_cutfabricated_resultempty_selectionwrong_variableunjustified_interpretationnon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t3-dimuon-kinematics | Single-shot | Qwen3-8B (gariyuu gateway) | core only | 0.433 | 1 | 2 | 1 | 2,314 | $0.00074 | 31 s | no | no_outputplotting_errorwrong_variablenon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t3-invariant-mass | Single-shot | Qwen3-8B (gariyuu gateway) | core only | 0.625 | 1 | 2 | 1 | 2,536 | $0.00086 | 26 s | no | no_outputplotting_errornon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t3-unit-mismatchtrap | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.480 | 1 | 2 | 0 | 2,186 | $0.00068 | 22 s | exact | duplicated_eventsmissing_valuesunit_error |
| 09-06 17:19 · r0 | t4-cut-optimization | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.083 | 1 | 2 | 1 | 2,253 | $0.00067 | 23 s | no | no_outputplotting_errorspec_noncomplianceincorrect_normalizationinvalid_cutnon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t4-feature-separationtrap | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.058 | 1 | 2 | 1 | 2,272 | $0.00068 | 21 s | no | no_outputplotting_errorwrong_variablestatistical_misuseleakagenon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t5-bump-fit | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.484 | 1 | 2 | 0 | 2,587 | $0.00085 | 32 s | exact | statistical_misuseincorrect_normalization |
| 09-06 17:19 · r0 | t5-bump-significance | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.313 | 1 | 2 | 1 | 2,917 | $0.00097 | 33 s | no | no_outputplotting_errorstatistical_misuseincorrect_normalizationnon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t5-low-statstrap | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.055 | 1 | 2 | 1 | 3,106 | $0.00105 | 32 s | no | no_outputplotting_errorinvalid_cutstatistical_misuseincorrect_normalizationunjustified_interpretationinsufficient_statisticsnon_reproducibleexecution_failed |
| 09-06 17:19 · r0 | t6-broken-pipelinetrap | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.640 | 1 | 2 | 0 | 2,275 | $0.00050 | 12 s | runs, differs | silent_exceptionplotting_errornon_reproducible |
| 09-06 17:20 · r0 | t6-full-analysis | Single-shot | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.427 | 1 | 2 | 1 | 4,354 | $0.00159 | 57 s | no | no_outputplotting_errorstatistical_misuseincorrect_normalizationspec_noncompliancenon_reproducibleexecution_failed |
| 09-06 17:20 · r0 | t1-missing-audittrap | ReAct | Qwen3-8B (gariyuu gateway) | core only | 0.920 | 5 | 5 | 2 (2 fixed) | 19,607 | $0.00341 | 1.1 min | exact | missing_valuessentinel_value |
| 09-06 17:20 · r0 | t1-schema-summary | ReAct | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 3 | 3 | 0 | 5,672 | $0.00099 | 23 s | exact | unsupported_claim |
| 09-06 17:20 · r0 | t2-mass-histogram | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.000 | 5 | 5 | 1 | 12,629 | $0.00194 | 33 s | no | no_outputplotting_errorinvalid_cutwrong_variablenon_reproduciblefabricated_resultexecution_failedsilent_exception |
| 09-06 17:20 · r0 | t2-selection-counttrap | ReAct | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 4 | 4 | 0 | 8,355 | $0.00143 | 28 s | exact | |
| 09-06 17:20 · r0 | t3-dimuon-kinematics | ReAct | Qwen3-8B (gariyuu gateway) | core only | 0.800 | 6 | 6 | 1 (1 fixed) | 20,365 | $0.00340 | 1.2 min | no | non_reproducibleunsupported_claim |
| 09-06 17:21 · r0 | t3-invariant-mass | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.000 | 22 | 22 | 20 | 140,735 | $0.028 | 12.3 min | no | no_outputplotting_errorspec_noncomplianceinvalid_cutwrong_variableunit_errornon_reproduciblefabricated_resultexecution_failedstep_budget_exhausted |
| 09-06 17:21 · r0 | t3-unit-mismatchtrap | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.480 | 14 | 14 | 9 (9 fixed) | 82,605 | $0.014 | 6.5 min | exact | duplicated_eventsmissing_valuesunit_error |
| 09-06 17:21 · r0 | t4-cut-optimization | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.450 | 22 | 22 | 20 | 140,564 | $0.027 | 11.9 min | exact | spec_noncomplianceincorrect_normalizationinvalid_cutexecution_failedstep_budget_exhausted |
| 09-06 17:22 · r0 | t4-feature-separationtrap | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.351 | 6 | 6 | 2 (2 fixed) | 21,789 | $0.00361 | 1.0 min | no | no_outputplotting_errorstatistical_misusenon_reproduciblefabricated_result |
| 09-06 17:23 · r0 | t5-bump-fit | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.484 | 19 | 19 | 17 | 126,013 | $0.025 | 12.0 min | exact | statistical_misuseincorrect_normalizationexecution_failedstep_budget_exhausted |
| 09-06 17:27 · r0 | t5-bump-significance | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.356 | 10 | 10 | 4 (4 fixed) | 53,713 | $0.011 | 5.7 min | no | statistical_misuseincorrect_normalizationunjustified_interpretationnon_reproducible |
| 09-06 17:33 · r0 | t5-low-statstrap | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.258 | 6 | 6 | 1 | 24,461 | $0.00377 | 1.3 min | no | no_outputplotting_errorstatistical_misuseincorrect_normalizationnon_reproduciblefabricated_resultexecution_failedsilent_exceptionunsupported_claim |
| 09-06 17:33 · r0 | t6-broken-pipelinetrap | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.332 | 21 | 21 | 3 (3 fixed) | 107,010 | $0.014 | 3.2 min | no | no_outputwrong_variableunit_errorinvalid_cutnon_reproduciblefabricated_result |
| 09-06 17:33 · r0 | t6-full-analysis | ReAct | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.000 | 13 | 13 | 10 | 90,348 | $0.022 | 11.7 min | no | no_outputplotting_errorwrong_datasetinvalid_cutstatistical_misuseincorrect_normalizationspec_noncompliancenon_reproduciblefabricated_resultexecution_failedstep_budget_exhausted |
| 09-06 17:34 · r0 | t1-missing-audittrap | Planner / executor | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 6 | 5 | 2 (2 fixed) | 23,009 | $0.00425 | 1.7 min | exact | |
| 09-06 17:35 · r0 | t1-schema-summary | Planner / executor | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 3 | 2 | 0 | 6,334 | $0.00123 | 24 s | exact | |
| 09-06 17:35 · r0 | t2-mass-histogram | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.625 | 6 | 5 | 1 (1 fixed) | 23,915 | $0.00397 | 1.3 min | exact | invalid_cutwrong_variable |
| 09-06 17:36 · r0 | t2-selection-counttrap | Planner / executor | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 4 | 3 | 1 (1 fixed) | 14,627 | $0.00280 | 1.1 min | exact | |
| 09-06 17:36 · r0 | t3-dimuon-kinematics | Planner / executor | Qwen3-8B (gariyuu gateway) | core only | 0.800 | 6 | 5 | 3 (3 fixed) | 28,613 | $0.00536 | 2.1 min | no | non_reproducible |
| 09-06 17:36 · r0 | t3-invariant-mass | Planner / executor | Qwen3-8B (gariyuu gateway) | core only | 0.800 | 26 | 25 | 14 (10 fixed) | 159,350 | $0.028 | 9.9 min | no | non_reproducibleexecution_failedstep_budget_exhausted |
| 09-06 17:37 · r0 | t3-unit-mismatchtrap | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.440 | 5 | 4 | 0 | 11,533 | $0.00193 | 37 s | exact | duplicated_eventsmissing_valuesunit_error |
| 09-06 17:38 · r0 | t4-cut-optimization | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.400 | 17 | 16 | 11 (11 fixed) | 105,997 | $0.020 | 8.7 min | no | incorrect_normalizationnon_reproducible |
| 09-06 17:38 · r0 | t4-feature-separationtrap | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.326 | 8 | 7 | 3 (3 fixed) | 36,696 | $0.00631 | 2.2 min | no | no_outputplotting_errorstatistical_misuseleakagenon_reproduciblefabricated_result |
| 09-06 17:40 · r0 | t5-bump-fit | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.000 | 19 | 18 | 16 | 130,628 | $0.026 | 11.8 min | no | no_outputplotting_errorstatistical_misuseincorrect_normalizationnon_reproduciblefabricated_resultexecution_failedstep_budget_exhausted |
| 09-06 17:45 · r0 | t5-bump-significance | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.446 | 10 | 9 | 4 (4 fixed) | 53,436 | $0.00887 | 3.2 min | no | statistical_misuseincorrect_normalizationnon_reproducible |
| 09-06 17:46 · r0 | t5-low-statstrap | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.607 | 7 | 6 | 1 (1 fixed) | 33,891 | $0.00580 | 2.1 min | no | statistical_misuseincorrect_normalizationnon_reproducibleunsupported_claim |
| 09-06 17:46 · r0 | t6-broken-pipelinetrap | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.033 | 9 | 8 | 2 | 30,427 | $0.00473 | 1.2 min | no | no_outputsilent_exceptionplotting_errorspec_noncompliancewrong_variableunit_errorinvalid_cutunjustified_interpretationnon_reproducibleexecution_failed |
| 09-06 17:48 · r0 | t1-missing-audittrap | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.350 | 2 | 4 | 1 (1 fixed) | 5,427 | $0.00136 | 36 s | runs, differs | no_outputwrong_datasetduplicated_eventswrong_variablemissing_valuessentinel_valuenon_reproducible |
| 09-06 17:48 · r0 | t1-schema-summary | Self-debugging | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 1 | 2 | 0 | 1,506 | $0.00040 | 11 s | exact | |
| 09-06 17:49 · r0 | t2-mass-histogram | Self-debugging | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 2 | 4 | 1 (1 fixed) | 5,920 | $0.00146 | 40 s | exact | |
| 09-06 17:49 · r0 | t2-selection-counttrap | Self-debugging | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 2 | 4 | 1 (1 fixed) | 6,417 | $0.00162 | 46 s | exact | |
| 09-06 17:49 · r0 | t3-dimuon-kinematics | Self-debugging | Qwen3-8B (gariyuu gateway) | core only | 0.808 | 3 | 6 | 2 (2 fixed) | 11,534 | $0.00226 | 1.3 min | runs, differs | no_outputwrong_variablenon_reproducible |
| 09-06 17:49 · r0 | t3-invariant-mass | Self-debugging | Qwen3-8B (gariyuu gateway) | ✓ strict | 1.000 | 4 | 8 | 3 (3 fixed) | 17,101 | $0.00366 | 1.5 min | exact | |
| 09-06 17:51 · r0 | t3-unit-mismatchtrap | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.440 | 2 | 4 | 1 (1 fixed) | 5,124 | $0.00123 | 33 s | exact | duplicated_eventsmissing_valuesunit_error |
| 09-06 17:51 · r0 | t4-cut-optimization | Self-debugging | Qwen3-8B (gariyuu gateway) | core only | 0.950 | 4 | 8 | 3 (3 fixed) | 17,241 | $0.00351 | 1.6 min | exact | incorrect_normalization |
| 09-06 17:51 · r0 | t4-feature-separationtrap | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.743 | 4 | 8 | 3 (3 fixed) | 17,104 | $0.00393 | 1.8 min | exact | statistical_misuse |
| 09-06 17:52 · r0 | t5-bump-fit | Self-debugging | Qwen3-8B (gariyuu gateway) | core only | 0.863 | 1 | 2 | 0 | 2,968 | $0.00102 | 32 s | exact | statistical_misuse |
| 09-06 17:53 · r0 | t5-bump-significance | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.646 | 2 | 4 | 1 (1 fixed) | 8,636 | $0.00230 | 1.1 min | exact | statistical_misuseincorrect_normalization |
| 09-06 17:53 · r0 | t5-low-statstrap | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.679 | 4 | 8 | 3 (3 fixed) | 21,468 | $0.00500 | 2.5 min | exact | statistical_misuseincorrect_normalization |
| 09-06 17:53 · r0 | t6-broken-pipelinetrap | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.640 | 4 | 8 | 0 | 14,756 | $0.00267 | 58 s | runs, differs | silent_exceptionplotting_errornon_reproducible |
| 09-06 17:54 · r0 | t6-full-analysis | Self-debugging | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.844 | 3 | 6 | 2 (2 fixed) | 16,613 | $0.00406 | 2.4 min | exact | statistical_misuseincorrect_normalization |
| 09-06 18:00 · r0 | t6-full-analysis | Planner / executor | Qwen3-8B (gariyuu gateway) | ✗ fail | 0.000 | 7 | 6 | 4 | 37,734 | $0.00915 | 9.1 min | no | no_outputplotting_errorwrong_datasetinvalid_cutstatistical_misuseincorrect_normalizationspec_noncompliancenon_reproduciblefabricated_resultexecution_failedstep_budget_exhausted |