The Court That Fits Itself

That clip up top: eight tournament courts, two overlays each. Green is the court fit a human built by clicking four corners. Orange is a free pretrained network's fit — zero clicks. On seven courts they're a few pixels apart. On the Roland Garros plate — the one that broke the old automatic fitter — the orange lands first try. And on the night match, the machine does something arguably better than fitting: it declines.

Today the software tries to take over the last job a human still does for every single match: teaching it where the court is.

Previously. Everything this machine calls — serves, bounces, wide-or-deep, who's standing where — runs through the homography, the screen-to-court-diagram conversion (primer) built from the court's painted lines. Since the very first fit, getting that conversion has ended one of two ways: an automatic fitter finds the lines, or — on stubborn footage — a human clicks the four corners by hand. The automatic fitter has a famous scar: at Roland Garros, faint brushed-over baselines and reddish stands fooled it so badly its fit missed the court by 74 pixels, and a human had to step in. Clicking corners takes a minute. But a minute per match, times the thousands of matches this project wants to chart, is the difference between a tool and a hobby.

Borrowing a scout who's seen 9,000 courts

The fix wasn't invented here, and that's the point. Someone on the internet already trained a small network to find tennis courts in broadcast frames — not by hunting for four corners, but by recognizing fourteen landmarks at once: every corner, every T, every place two painted lines meet. It learned from 8,841 broadcast images across hard courts, clay, and grass. It's free, it's small, and it runs on this laptop in under a second per image.

Fourteen landmarks instead of four is the difference between asking a tourist for the two street corners they remember and asking a letter carrier who knows every house on the block. If a couple of landmarks are hidden or faint, twelve others still pin the court down — and the fitter can even check its own work, by fitting on some landmarks and measuring how well it predicted the ones it didn't use.

The scoreboard

Each court's automatic fit, measured against the human's hand fit at six landmarks (the four court corners and the two net posts):

Court	Machine vs human, average gap
US Open (hard)	2 pixels
Melbourne (hard, night session plate)	3 pixels
Turin (indoor)	5 pixels
Montreal (hard)	7 pixels
Wimbledon (grass)	8 pixels
Roland Garros — the 74-pixel disaster plate	8 pixels, first try
Paris 2023 (clay)	16 pixels — with a twist, below
Nadal–Shapovalov (night, dim)	no fit — the machine declined

Seven of eight courts, fit to within a few pixels of a careful human, by a network that had never seen any of them. And the eighth result might be the most important one on the board: on the dim night match, the network found only 2 of its 14 landmarks — so instead of guessing from starvation-level evidence, the fitter abstained and left the job to a human. That's the behavior this whole project is organized around: an automatic call when the evidence is strong, a raised hand when it isn't — the same contract as the confidence flag that grades every drafted point.

The twist: the student corrected the teacher

One court disagreed with the human by 16 pixels, and I went in expecting to document the network's failure. The overlay says otherwise: on that clay plate, the orange lines sit squarely on the painted court, and the green — the human hand fit that all the benchmarks treat as truth — floats visibly inside the paint on the far side. The "error" may be the teacher's. It's not settled yet — the neutral referee will be the pipeline's existing line-scorer, which measures any fit directly against the paint itself — but it's a healthy reminder this series keeps getting taught: hand-made truth is still hand-made.

The bug, kept on the record

First run, every single fit came back wrong by the same enormous amount — about 750 pixels, all eight courts. A consistent error is a convention error, not a model error: the borrowed code, it turns out, already doubles its coordinates internally (it assumes 720p footage), and my wrapper doubled them again. One rendered overlay exposed it instantly; the numbers alone never would have named the cause. Third time this week: eyes beat derivatives.

The takeaway: the last per-match chore a human always did — teaching the machine where the court is — just went automatic for seven courts out of eight, and on the eighth the machine did the next best thing to being right: it knew it didn't know.

For the technical reader

Module: experiments/court_autofit.py (report + per-match overlays in outputs/diag/; run with PYTHONPATH=.:experiments uv run --with scipy python experiments/court_autofit.py).

Detector: yastrebksv/TennisCourtDetector — TrackNet-style heatmap net, 640×360 input, 15 output channels (14 keypoints + center), pretrained on 8,841 YouTube broadcast frames (hard/clay/grass), median error 1.83 px refined / 2.83 px raw per its README. No license file — fine for this research phase; the blueprint's risk register already flags retraining a self-labeled replacement before any commercial use (every hand-fit match auto-generates 14-keypoint labels through its known homography). Weights: 42 MB, CPU inference ~1 s/plate.

Method: run on each match's plate_fit.png (the stabilized median plate the hand homography was fit on); solve via the author's 12-configuration search (reimplemented numpy-only): each candidate 4-point configuration fits a homography, is scored by mean prediction error on the unused detected keypoints (cross-validation, not self-consistency), best wins. Convert reference-canvas px → meters (doubles corners (286,561)–(1379,2935) ↔ 10.97 × 23.77 m) and compose into the repo's H_img_to_court convention.

Landmark deltas vs hand fits (mean/max px): t5 2.7/3.1, t6 2.4/4.0, t7 4.7/8.6, t2 7.3/13.9, t4 8.0/15.2, g1 8.0/12.2, t3 15.7/29.0; t1 abstained (2/14 kps; config search requires a complete 4-point configuration). Config cross-validation errors 2.5–7.0 px on accepted fits. Classical auto-fit's g1 failure: rms 74 px (LOG, cv-18 era).

The 2× bug: postprocess(heatmap, scale=2) upscales heatmap coords internally; wrapper scaled again → uniform doubling, self-consistent config-xval, ~750 px landmark deltas. Caught by the first overlay render.

Next wiring (blueprint roadmap #2's remaining half): use this as the proposer inside fitcourt, judged by the existing step-5 line-mask scorer with a calibrated accept threshold (the TVCalib abstention pattern), plus the 1 Hz drift watchdog. Night feeds: try per-clip plates, gamma/CLAHE boost, or fine-tuning with the hand-fit frames the project already owns.

Hero renderer: experiments/render_court_hero.py (montage of the per-match overlays).

Session cost: $0.00 — pretrained weights, laptop inference. Project total: ~$16.

Next question: the court now fits itself and the machine hears every impact, but the players are still smudges — blobs from background subtraction, which is why forehand-versus-backhand has been stuck at a ceiling for two weeks. The research says the upgrade is stick figures: wrists, elbows, hips, feet, drawn on every frame by a free model that runs at 87 frames a second on this laptop. Time to give the smudges skeletons.