AgentHEPGitHub ↗
Tier 2 · Histograms & selections · v1.0.0

Dimuon mass histogram around the Z peak

Histogram the opposite-charge dimuon mass in a fixed binning, save the bin contents as JSON and a labelled SVG plot.

t2-mass-histogramhistogramscms_dimuon_2011selectionhistogramplottingjson_output

Task prompt (what the agent sees)

Using `data/cms_dimuon_2011.csv` (data card in README.md), select events with opposite-sign muons
(Q1*Q2 < 0) and histogram the dimuon invariant mass `M` in the range [60, 120] GeV with 60 uniform
bins (use numpy's convention: bins are half-open except the last one, i.e. `np.histogram(x, bins=60, range=(60, 120))`).

Deliverables:
- `hist.json` with keys `bin_edges` (61 floats), `counts` (60 ints), `n_in_range` (int, events with 60 <= M <= 120
  after the charge selection) and `n_opposite_charge` (int, events passing the charge selection before the mass range cut).
- `result.json` with `peak_bin_center` (float, centre of the most populated bin) and `peak_count` (int).
- `figures/mass_hist.svg`: the histogram with an x-axis label that includes the unit "GeV" and a y-axis label.
- `solution.py`: the complete analysis; `python solution.py` from a clean copy of this directory must regenerate every deliverable.

Data card (README.md in the workdir)

Data card: data/cms_dimuon_2011.csv (REAL DATA)

Source: CERN Open Data Portal record 545, "Dimuon events from the CMS 2011 DoubleMu primary dataset" (file Dimuon_DoubleMu.csv). Licence CC0. 100,000 events, one per row.

column meaning unit
Run, Event run and event number –
type1, type2 muon reconstruction type: G global muon, T tracker muon –
E1, px1, py1, pz1 four-momentum of muon 1 GeV
pt1, eta1, phi1 transverse momentum, pseudorapidity, azimuth of muon 1 GeV, –, rad
Q1 charge of muon 1 e
E2 … Q2 same for muon 2
M invariant mass of the muon pair GeV

All energies and momenta are in GeV. There are no missing values.

Expected artifacts

  • hist.json json
  • result.json json
  • figures/mass_hist.svg svg
  • solution.py script

Deterministic checks and tolerances

rubric: artifact 0.1 · numeric 0.5 · plot 0.15 · compliance 0.05 · reproducibility 0.2
CheckTypeTargetToleranceWeightCategoryCriticalFailure implies
hist_existsfile_existshist.json1artifactcriticalno_output
result_existsfile_existsresult.json1artifactno_output
figure_existsfile_existsfigures/mass_hist.svg1artifactno_outputplotting_error
solution_existsfile_existssolution.py1artifactno_output
edgesjson_list_equalhist.json › bin_edgesatol 1e-91numericplotting_error
countsjson_list_equalhist.json › countsexact3numericcriticalinvalid_cut
n_in_rangejson_valuehist.json › n_in_rangeexact1numericinvalid_cut
n_osjson_valuehist.json › n_opposite_chargeexact1numericinvalid_cut
peak_centerjson_valueresult.json › peak_bin_centeratol 0.0000011numericwrong_variable
peak_countjson_valueresult.json › peak_countexact1numericwrong_variable
svg_unitsvg_textfigures/mass_hist.svg ["GeV"]1plotplotting_error
svg_drawnsvg_has_marksfigures/mass_hist.svg≥ 51plotplotting_error
rerunsscript_runssolution.py1reproducibilitynon_reproducible
not_hardcodedno_hardcoded_resultsolution.py = hist.json1compliancefabricated_result

Tolerance rationale. Bin counts are exact; edges atol 1e-9.

Ground truth (produced by the reference in the sandbox)

built 2026-09-06 · numpy 1.26.4 · scipy 1.13.1 · 1.56s
hist.json
{
 "bin_edges": [
  60,
  61,
  62,
  63,
  64,
  65,
  66,
  67,
  68,
  69,
  70,
  71,
  72,
  73,
  74,
  75,
  76,
  77,
  78,
  79,
  80,
  81,
  82,
  83,
  84,
  85,
  86,
  87,
  88,
  89,
  90,
  91,
  92,
  93,
  94,
  95,
  96,
  97,
  98,
  99,
  100,
  101,
  102,
  103,
  104,
  105,
  106,
  107,
  108,
  109,
  110,
  111,
  112,
  113,
  114,
  115,
  116,
  117,
  118,
  119,
  120
 ],
 "counts": [
  37,
  44,
  41,
  27,
  37,
  35,
  33,
  45,
  31,
  34,
  34,
  30,
  42,
  30,
  35,
  53,
  42,
  43,
  42,
  47,
  62,
  59,
  70,
  94,
  96,
  128,
  171,
  263,
  422,
  675,
  806,
  873,
  595,
  314,
  228,
  107,
  80,
  54,
  46,
  31,
  27,
  21,
  12,
  14,
  13,
  19,
  11,
  7,
  13,
  7,
  5,
  8,
  8,
  1,
  3,
  6,
  3,
  4,
  3,
  3
 ],
 "n_in_range": 6124,
 "n_opposite_charge": 100000
}
result.json
{
 "peak_bin_center": 91.5,
 "peak_count": 873
}
reference.py
import json
import os
import numpy as np
import pandas as pd
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

df = pd.read_csv("data/cms_dimuon_2011.csv")
os_sel = df[df["Q1"] * df["Q2"] < 0]
m = os_sel["M"].to_numpy()
counts, edges = np.histogram(m, bins=60, range=(60, 120))
centers = 0.5 * (edges[:-1] + edges[1:])
i = int(np.argmax(counts))
json.dump({"bin_edges": edges.tolist(), "counts": [int(c) for c in counts], "n_in_range": int(((m >= 60) & (m <= 120)).sum()),
           "n_opposite_charge": int(len(os_sel))}, open("hist.json", "w"), indent=2)
json.dump({"peak_bin_center": float(centers[i]), "peak_count": int(counts[i])}, open("result.json", "w"), indent=2)
os.makedirs("figures", exist_ok=True)
fig, ax = plt.subplots(figsize=(7, 4.5))
ax.stairs(counts, edges, fill=True, alpha=0.7)
ax.set_xlabel("Dimuon invariant mass M [GeV]")
ax.set_ylabel("Events / 1 GeV")
ax.set_title("CMS 2011 opposite-sign dimuons")
fig.savefig("figures/mass_hist.svg")
print(counts.sum(), centers[i], counts[i])

Results on this task

4 runs
AgentModelRunsStrict successMean score
Self-debuggingQwen3-8B (gariyuu gateway)1100%1
Planner / executorQwen3-8B (gariyuu gateway)10%0.625
ReActQwen3-8B (gariyuu gateway)10%0
Single-shotQwen3-8B (gariyuu gateway)10%0.475
RunAgentModelResultScoreLabels
20260906T171852…r0Single-shotQwen3-8B (gariyuu gateway)✗ core only0.475no_outputplotting_errorwrong_variablenon_reproducibleexecution_failed
20260906T172020…r0ReActQwen3-8B (gariyuu gateway)✗ fail0no_outputplotting_errorinvalid_cutwrong_variablenon_reproduciblefabricated_resultexecution_failedsilent_exception
20260906T173535…r0Planner / executorQwen3-8B (gariyuu gateway)✗ fail0.625invalid_cutwrong_variable
20260906T174903…r0Self-debuggingQwen3-8B (gariyuu gateway)✓ strict1