AgentHEPGitHub ↗
Tier 3 · Derived variables · v1.0.0

Per-event HT and leading-jet energy with a MeV column

Aggregate a long-format jet table to per-event quantities; requires de-duplication, NaN handling and a MeV-to-GeV conversion documented in the data card.

t3-unit-mismatchderived_variablessynth_jetstrap: unit_mismatchtrap: duplicated_eventstrap: missing_valuesgroupby_aggregationunit_conversiondeduplicationmissing_values

Task prompt (what the agent sees)

From `data/synth_jets.csv` (one row per jet; read the data card in README.md carefully) build a
per-event table and summary. All energies and momenta in your outputs must be in **GeV**.

Definitions:
- `ht_gev`: scalar sum of jet transverse momenta over the event's valid jets (jets with a measured pt)
- `lead_jet_e_gev`: energy of the jet with `jet_index == 0`, in GeV
- `n_jets_valid`: number of distinct jets in the event with a measured pt

Deliverables:
- `events.csv` with columns `event_id`, `ht_gev`, `lead_jet_e_gev`, `n_jets_valid`, one row per event, sorted by `event_id`.
- `result.json` with `n_events` (int), `mean_ht_gev` (float), `mean_lead_jet_e_gev` (float), `max_lead_jet_e_gev` (float),
  `frac_events_ht_gt_200` (float, fraction of events with ht_gev > 200).
- `solution.py`: the complete analysis; `python solution.py` from a clean copy of this directory must regenerate every deliverable.

Data card (README.md in the workdir)

Data card: data/synth_jets.csv (SYNTHETIC)

Simulated jet table for 20,000 events, one row per jet, produced by a toy generator with a known seed. This is synthetic data; it is not a measurement.

column meaning unit
event_id event number –
jet_index index of the jet within its event, 0 = leading (highest-pt) jet –
jet_pt transverse momentum. NaN when jet reconstruction failed. GeV
jet_eta pseudorapidity. The value -999.0 is a sentinel meaning "not measured". –
jet_phi azimuthal angle rad
jet_e_mev jet energy. Note the unit: MeV, not GeV. MeV
btag_score b-tagging discriminant in [0, 1] –

Known issues

  • A jet is uniquely identified by the pair (event_id, jet_index).
  • Because of a bug in the ntuple merger, some rows are exact duplicates. Duplicates must be removed (keep the first occurrence) before any physics quantity is computed.
  • Jets with missing jet_pt failed reconstruction and must be excluded from any sum over jet pt. Jets with the eta sentinel are otherwise valid (their pt and energy are measured).

Expected artifacts

  • events.csv csv
  • result.json json
  • solution.py script

Deterministic checks and tolerances

rubric: artifact 0.1 · numeric 0.6 · plot 0 · compliance 0.1 · reproducibility 0.2
CheckTypeTargetToleranceWeightCategoryCriticalFailure implies
events_existsfile_existsevents.csv1artifactcriticalno_output
result_existsfile_existsresult.json1artifactno_output
solution_existsfile_existssolution.py1artifactno_output
columnscsv_columnsevents.csv ["event_id","ht_gev","lead_jet_e_gev","n_jets_valid"]1numericspec_noncompliance
rowscsv_row_countevents.csvexact1numericduplicated_events
htcsv_column_matchevents.csv › event_id › ht_gevatol 0.0013numericcriticalduplicated_eventsmissing_values
lead_ecsv_column_matchevents.csv › event_id › lead_jet_e_gevrtol 0.0000013numericcriticalunit_error
n_validcsv_column_matchevents.csv › event_id › n_jets_validexact1numericduplicated_eventsmissing_values
mean_htjson_valueresult.json › mean_ht_gevrtol 0.000012numericduplicated_eventsmissing_values
mean_lead_ejson_valueresult.json › mean_lead_jet_e_gevrtol 0.000012numericcriticalunit_error
max_lead_ejson_valueresult.json › max_lead_jet_e_gevrtol 0.0000011numericunit_error
frac_htjson_valueresult.json › frac_events_ht_gt_200atol 0.0000011numericduplicated_events
rerunsscript_runssolution.py1reproducibilitynon_reproducible
not_hardcodedno_hardcoded_resultsolution.py1compliancefabricated_result

Tolerance rationale. A missed MeV->GeV conversion is off by 1000x; a missed de-duplication changes ~3% of HT values by more than 1e-3 GeV.

Ground truth (produced by the reference in the sandbox)

built 2026-09-06 · numpy 1.26.4 · scipy 1.13.1 · 0.758s
result.json
{
 "n_events": 20000,
 "mean_ht_gev": 147.36127126000002,
 "mean_lead_jet_e_gev": 170.5915307,
 "max_lead_jet_e_gev": 2021.512,
 "frac_events_ht_gt_200": 0.20245
}
reference.py
import json
import numpy as np
import pandas as pd

raw = pd.read_csv("data/synth_jets.csv")
df = raw.drop_duplicates(keep="first").drop_duplicates(["event_id", "jet_index"], keep="first")
valid = df[df["jet_pt"].notna()]
ht = valid.groupby("event_id")["jet_pt"].sum()
nval = valid.groupby("event_id")["jet_index"].nunique()
lead = df[df["jet_index"] == 0].set_index("event_id")["jet_e_mev"] / 1000.0
events = pd.DataFrame({"event_id": sorted(df["event_id"].unique())}).set_index("event_id")
events["ht_gev"] = ht.reindex(events.index).fillna(0.0)
events["lead_jet_e_gev"] = lead.reindex(events.index)
events["n_jets_valid"] = nval.reindex(events.index).fillna(0).astype(int)
events = events.reset_index().sort_values("event_id")
events.to_csv("events.csv", index=False)
res = {"n_events": int(len(events)), "mean_ht_gev": float(events.ht_gev.mean()), "mean_lead_jet_e_gev": float(events.lead_jet_e_gev.mean()),
       "max_lead_jet_e_gev": float(events.lead_jet_e_gev.max()), "frac_events_ht_gt_200": float((events.ht_gev > 200).mean())}
json.dump(res, open("result.json", "w"), indent=2)
print(res)

Results on this task

4 runs
AgentModelRunsStrict successMean score
Self-debuggingQwen3-8B (gariyuu gateway)10%0.44
Planner / executorQwen3-8B (gariyuu gateway)10%0.44
ReActQwen3-8B (gariyuu gateway)10%0.48
Single-shotQwen3-8B (gariyuu gateway)10%0.48
RunAgentModelResultScoreLabels
20260906T171913…r0Single-shotQwen3-8B (gariyuu gateway)✗ fail0.48duplicated_eventsmissing_valuesunit_error
20260906T172109…r0ReActQwen3-8B (gariyuu gateway)✗ fail0.48duplicated_eventsmissing_valuesunit_error
20260906T173724…r0Planner / executorQwen3-8B (gariyuu gateway)✗ fail0.44duplicated_eventsmissing_valuesunit_error
20260906T175112…r0Self-debuggingQwen3-8B (gariyuu gateway)✗ fail0.44duplicated_eventsmissing_valuesunit_error