AgentHEPGitHub ↗
Tier 3 · Derived variables · v1.0.0

Derived dimuon kinematics on real CMS data

Compute pair transverse momentum, rapidity, wrapped azimuthal separation and Delta-R per event from CMS four-vectors.

t3-dimuon-kinematicsderived_variablescms_dimuon_2011four_vectorsangular_variablesderived_variablecsv_output

Task prompt (what the agent sees)

Using `data/cms_dimuon_2011.csv` (data card in README.md), compute for every event:

- `pt_ll`: transverse momentum of the dimuon system, |(px1+px2, py1+py2)| in GeV
- `y_ll`: rapidity of the dimuon system, 0.5 * ln((E + pz) / (E - pz)) with E = E1+E2, pz = pz1+pz2
- `delta_phi`: azimuthal separation of the two muons wrapped into [0, pi]
- `delta_r`: sqrt(delta_eta^2 + delta_phi^2) with delta_eta = eta1 - eta2

Deliverables:
- `kinematics.csv` with columns `Run`, `Event`, `pt_ll`, `y_ll`, `delta_phi`, `delta_r` in the input row order.
- `result.json` with `n_events` (int), `mean_pt_ll` (float), `mean_delta_r` (float), `frac_delta_phi_gt_2p5`
  (float, fraction of events with delta_phi > 2.5), `max_abs_y_ll` (float).
- `figures/pt_ll.svg`: histogram of pt_ll from 0 to 200 GeV, x-axis label including "GeV".
- `solution.py`: the complete analysis; `python solution.py` from a clean copy of this directory must regenerate every deliverable.

Data card (README.md in the workdir)

Data card: data/cms_dimuon_2011.csv (REAL DATA)

Source: CERN Open Data Portal record 545, "Dimuon events from the CMS 2011 DoubleMu primary dataset" (file Dimuon_DoubleMu.csv). Licence CC0. 100,000 events, one per row.

column meaning unit
Run, Event run and event number
type1, type2 muon reconstruction type: G global muon, T tracker muon
E1, px1, py1, pz1 four-momentum of muon 1 GeV
pt1, eta1, phi1 transverse momentum, pseudorapidity, azimuth of muon 1 GeV, –, rad
Q1 charge of muon 1 e
E2 … Q2 same for muon 2
M invariant mass of the muon pair GeV

All energies and momenta are in GeV. There are no missing values.

Expected artifacts

  • kinematics.csv csv
  • result.json json
  • figures/pt_ll.svg svg
  • solution.py script

Deterministic checks and tolerances

rubric: artifact 0.1 · numeric 0.5 · plot 0.15 · compliance 0.05 · reproducibility 0.2
CheckTypeTargetToleranceWeightCategoryCriticalFailure implies
kin_existsfile_existskinematics.csv1artifactcriticalno_output
result_existsfile_existsresult.json1artifactno_output
figure_existsfile_existsfigures/pt_ll.svg1artifactno_outputplotting_error
solution_existsfile_existssolution.py1artifactno_output
kin_columnscsv_columnskinematics.csv ["Run","Event","pt_ll","y_ll","delta_phi","delta_r"]1numericspec_noncompliance
pt_llcsv_column_matchkinematics.csv › pt_llatol 0.0012numericcriticalwrong_variable
y_llcsv_column_matchkinematics.csv › y_llatol 0.0012numericwrong_variable
delta_phicsv_column_matchkinematics.csv › delta_phiatol 0.00013numericcriticalwrong_variable
delta_rcsv_column_matchkinematics.csv › delta_ratol 0.00012numericwrong_variable
mean_ptjson_valueresult.json › mean_pt_llrtol 0.000011numericwrong_variable
mean_drjson_valueresult.json › mean_delta_rrtol 0.000011numericwrong_variable
frac_dphijson_valueresult.json › frac_delta_phi_gt_2p5atol 0.0000012numericwrong_variable
max_yjson_valueresult.json › max_abs_y_llrtol 0.000011numericwrong_variable
svg_unitsvg_textfigures/pt_ll.svg ["GeV"]1plotplotting_error
svg_drawnsvg_has_marksfigures/pt_ll.svg≥ 51plotplotting_error
rerunsscript_runssolution.py1reproducibilitynon_reproducible
not_hardcodedno_hardcoded_resultsolution.py1compliancefabricated_result

Tolerance rationale. Per-event columns atol 1e-3 GeV / 1e-4 rad; the input has four decimals. Unwrapped delta_phi fails the delta_phi and frac checks by construction.

Ground truth (produced by the reference in the sandbox)

built 2026-09-06 · numpy 1.26.4 · scipy 1.13.1 · 2.222s
result.json
{
 "n_events": 100000,
 "mean_pt_ll": 9.713848678583945,
 "mean_delta_r": 2.3290955580440396,
 "frac_delta_phi_gt_2p5": 0.58301,
 "max_abs_y_ll": 2.3912620543767447
}
reference.py
import json
import os
import numpy as np
import pandas as pd
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

df = pd.read_csv("data/cms_dimuon_2011.csv")
px, py, pz, E = df.px1 + df.px2, df.py1 + df.py2, df.pz1 + df.pz2, df.E1 + df.E2
pt_ll = np.hypot(px, py)
y_ll = 0.5 * np.log((E + pz) / (E - pz))
dphi = np.abs(df.phi1 - df.phi2)
dphi = np.where(dphi > np.pi, 2 * np.pi - dphi, dphi)
deta = df.eta1 - df.eta2
dr = np.sqrt(deta**2 + dphi**2)
out = pd.DataFrame({"Run": df.Run, "Event": df.Event, "pt_ll": pt_ll, "y_ll": y_ll, "delta_phi": dphi, "delta_r": dr})
out.to_csv("kinematics.csv", index=False)
res = {"n_events": int(len(df)), "mean_pt_ll": float(pt_ll.mean()), "mean_delta_r": float(dr.mean()),
       "frac_delta_phi_gt_2p5": float((dphi > 2.5).mean()), "max_abs_y_ll": float(np.abs(y_ll).max())}
json.dump(res, open("result.json", "w"), indent=2)
os.makedirs("figures", exist_ok=True)
fig, ax = plt.subplots(figsize=(7, 4.5))
ax.hist(pt_ll, bins=100, range=(0, 200), histtype="stepfilled", alpha=0.7)
ax.set_xlabel("Dimuon transverse momentum pT(ll) [GeV]")
ax.set_ylabel("Events / 2 GeV")
ax.set_yscale("log")
fig.savefig("figures/pt_ll.svg")
print(res)

Results on this task

4 runs
AgentModelRunsStrict successMean score
Self-debuggingQwen3-8B (gariyuu gateway)10%0.8083
Planner / executorQwen3-8B (gariyuu gateway)10%0.8
ReActQwen3-8B (gariyuu gateway)10%0.8
Single-shotQwen3-8B (gariyuu gateway)10%0.4333
RunAgentModelResultScoreLabels
20260906T171905…r0Single-shotQwen3-8B (gariyuu gateway)✗ core only0.4333no_outputplotting_errorwrong_variablenon_reproducibleexecution_failed
20260906T172053…r0ReActQwen3-8B (gariyuu gateway)✗ core only0.8non_reproducibleunsupported_claim
20260906T173629…r0Planner / executorQwen3-8B (gariyuu gateway)✗ core only0.8non_reproducible
20260906T174945…r0Self-debuggingQwen3-8B (gariyuu gateway)✗ core only0.8083no_outputwrong_variablenon_reproducible