DrugTargetBench

An Environment for Therapeutic Target Discovery

Run the benchmark →Read the paper →GitHub →9 models · 20 worlds · 540 episodes

v1.0rubric v0.9 · run 1

Mean benchmark score and component scores for nine models over 540 episodes. Sortable by any numeric column.
#	Model							Serving
1	Opus 5api-alias@2026-09-07	39.98	14.80	7.19	13.05	86.64	$4.14	API
2	GPT-5.6 Solapi-alias@2026-09-05	35.38	15.81	3.16	13.00	83.19	$2.02	API
3	Sonnet 5api-alias@2026-09-04	21.33	10.24	1.16	6.96	83.88	$1.47	API
4	Haiku 4.5api-alias@2026-09-04	12.92	5.27	0.21	5.91	75.00	$0.26	API
5	gpt-oss-20b6cee5e81	7.41	2.94	0.68	3.43	75.00	$0.07	GPU
6	Qwen3-Coder-30Bb2cff646	5.90	3.26	0.14	1.94	75.00	$0.05	GPU
7	GLM-4-32B077b5c2f	1.46	1.12	0.08	0.25	25.00	$0.20	GPU
8	Qwen3-8Bb968826d	1.31	0.70	0.39	0.22	30.71	$0.22	GPU
9	Devstral-Smallbd165ab2	0.81	0.75	0.06	0.00	15.00	$0.20	GPU

Mean score across 20 worlds × 3 budget conditions, one replicate each. Scores are out of 100; measured ceiling = 85.5.

Qwen3-8B and GLM-4-32B ran a reduced 8,000-token output budget and are not a like-for-like comparison.

20
WORLDS: 54K
PARTICIPANTS/WORLD: 2,941
PROTEINS: 8,192
VARIANTS: 3
BUDGETS

02 Disease states

Each synthetic world expresses hidden disease biology through participant-level medical data.

world w_201988

ControlCardiac MRI · 25 frames · 128×128

HFpEFCardiac MRI · 25 frames · 128×128

Preserved ejection fraction with a thickened wall and impaired filling.

03 Environment

How it works

Biobank
Genetics, omics, imaging, ECG, EHR.
Phenotype
Segment the myocardium from raw arrays, engineer features, fit a model against a proxy outcome.
Causal targets
Screen the proteome, instrument it genetically, separate drivers from decoys.
Experiments
Spend a finite research budget.
Submission
Nominate targets and therapeutic direction.

Release

world_07/
├─ genotypes.vcf.gz
├─ proteomics.parquet
├─ transcriptomics.parquet
├─ metabolomics.parquet
├─ covariates.parquet
├─ ecg_features.parquet
├─ coronary_ct.parquet
├─ ehr_diagnoses.parquet
├─ mortality.parquet
├─ mace_events.parquet
├─ targetability.parquet
├─ imaging/
└─ imaging_visit2/

Episode

>>> d = np.load(f); cine, t1 = d['cine'], d['t1map']

(3, 25, 128, 128) uint8 (3, 128, 128) float32, 98.2% NaN

>>> mc = ndi.binary_closing(~np.isnan(t1[s])); filled = ndi.binary_fill_holes(mc)

>>> cav = filled & ~mc; thr = (np.median(cm[:, mc]) + np.median(cm[fED][cav])) / 2

>>> A[s, f] = np.bincount(ndi.label((cm > thr) & reg)[0].ravel())[1:].max()

SUBJ_00001 [494 387 277 212 174 140 132 ... 467 474 492] EF 0.775

>>> for fp in files: rows.append(feat(fp)) # 10,800 subjects

159.9 s elapsed # turn limit 300 s

>>> y = hf_icd | mace | cardiac_death # proxy label, prev 0.307

>>> lr = LogisticRegression(max_iter=2000).fit(Z[feats], y)

edvi 0.111 esvi 0.424 ef 0.427 myoi -0.502 wt 0.217 gfr -0.442

corr(score, y) 0.260 hand-built composite 0.241

>>> pd.DataFrame({"subject_id": ids, "phenotype": z}).to_csv(OUT_CSV)

>>> G = load_genotypes(); B = (Gs.T @ Yc) / M.sum(0) # 8,192 x 2,941

>>> b_iv, z = wald(g[:, j], y, mask) # cis Wald ratio

PROT_1606 rs104994 bx 0.436 MR -0.396 z -17.4

PROT_1858 rs102089 bx 0.153 MR 0.005 z 0.1 obs r 0.36

>>> request_experiment("knockdown", "PROT_1850")

delta {cavity_r: 0.00, wall_t: 0.00, ef: 0.000, surv5y: 0.00}

The agent receives the files, writes Python, and chooses analyses and experiments over 30 turns.

04 Method

Hidden causal worlds

Each world is procedurally generated from a sealed structural causal model linking genetics, molecular measurements, hidden disease state, observable phenotypes, and intervention outcomes.

Disease state

L_i = z\left[\sum_{j\in\mathcal D} w_j P_{ij} + \gamma^\top C_i + \delta G_{i,\mathrm{direct}} + \epsilon_i\right]

The identity and number of causal molecular drivers are hidden from the agent.

Research policy

a_t \sim \pi(a \mid s_t, B_t)

B_{t+1} = B_t - c(a_t)

Analyses and experiments are chosen against accumulated evidence and the remaining budget.

T1 Confounding · T2 Reverse causation · T3 Selection bias · T4 Causal non-identifiability · T5 Imaging batch effects · T6 Benign remodeling · T7 Instrument pleiotropy · T8 Surrogate-outcome discordance · T9 Assay unit mixing · A9 Slow effect

DrugTargetBench

How it works

Biobank

Phenotype

Causal targets

Experiments

Submission

Hidden causal worlds