Leave It

A morning filter for a backlog you have decided not to read.

Every item below is a real issue from a real repository. For each one the model answers a single question: does this need a maintainer today? Items it defers are held by a durable Temporal fuse, so a decision to ignore something cannot be lost by a crash, a reboot or a closed laptop lid.

What the filter buys you

11%

of the held-out stream never reached the human, while wrongly deferring 11 of the 235 items that genuinely needed someone within 24 hours. A perfect ranker could defer 68%; deferring everything would defer 100% and miss everything.

arm	deferred	of pool	wrong	AUC	what it is
tfidf + hand features	78	10.7%	11	0.663	word counts plus keyword rules
fine-tuned 4B (Tinker)	73	10.0%	11	0.665	LoRA, 2,924 items, held-out window
tfidf + logistic reg	57	7.8%	11	0.634	word counts, no semantics
base 4B, prompted	49	6.7%	11	0.516	the same weights, no fine-tune
length_only	28	3.8%	11	0.426	longer report = more serious
has_label	23	3.1%	11	0.393	does it already carry a maintainer label

Held-out window: 2026-10-16 to 2026-10-21, n=731. Operating point: at most 5% of the items that needed a maintainer today may be deferred. Grouped by the same policy for every arm.

The guarantee, executed

A worker was killed with kill -9 with three fuses pending, and one attention signal was delivered while no worker process existed at all.

t+0sthree seals opened: A fuse 24s, C fuse 16s, B fuse 40s (B is the rescue case)
sealed A vercel/next.js#100381 p=0.15 fuse=24s wf=seal-A-1792773234
sealed B vercel/next.js#100518 p=0.18 fuse=40s wf=seal-B-1792773234
sealed C vercel/next.js#100398 p=0.20 fuse=16s wf=seal-C-1792773234
t+9skill -9 the worker (pid 1453470)
t+9sworker is dead: no process exists that could run an activity
t+9s3 workflows still Running on the server (they were not living in the worker)
t+11sdeliver 'on_attention' to seal B while NO worker process exists
signal delivered to seal-B-1792773234 (worker alive: no)
t+16srestart the worker
t+40sfuses fired; ledger folded from the append-only log:
LEDGER (append-only, folded):
vercel/next.js#100398 LAPSED p=0.20 fuse expired after 16s; nobody ever engaged
vercel/next.js#100381 LAPSED p=0.15 fuse expired after 24s; nobody ever engaged
vercel/next.js#100518 RESCUED p=0.18 a maintainer commented while the worker was dead
MORNING CARD:
One thing survived the fuse: vercel/next.js#100381 (sealed at P=0.15, nobody touched it for 0s past the fuse)
t+40sserver-side run history: 3/3 seals Completed
t+40sdone: 3 seals, 1 kill -9, 1 signal delivered to nobody, 0 fuses lost

Ledger

Append-only. Verdicts are recorded and folded on read, so any decision can be re-read and argued with later.

vercel/next.js#100381 SEALED P(handled)=0.15 — below the line; sealed with a fuse
vercel/next.js#100518 SEALED P(handled)=0.18 — below the line; sealed with a fuse
vercel/next.js#100398 SEALED P(handled)=0.20 — below the line; sealed with a fuse
vercel/next.js#100518 RESCUED a maintainer commented while the worker was dead
vercel/next.js#100398 LAPSED fuse expired after 16s; nobody ever engaged
vercel/next.js#100381 LAPSED fuse expired after 24s; nobody ever engaged

Reproduce

git clone https://github.com/memeshee/leaveit
cd leaveit
python3 data/fetch_issues.py 10      # real issues + real outcomes
python3 data/build_dataset.py        # labels + temporal split
python3 eval/baselines.py            # the arms to beat
python3 train/lm.py train tuned 400  # LoRA on Tinker
python3 train/lm.py score tuned
python3 eval/policy.py eval/scores_*.json
cd app && ./run_demo.sh              # kill -9 proof

Leave It · Hacktoberfest Open-Source AI Challenge Week 3 · every number on this page is regenerated from data/ and eval/ by app/build_site.py.