The 33rd Charter
That clip up top: the actual file, playing. All 134 points of the Sabalenka–Pegula US Open final draft scroll past, and 39 light up green — "trust this enough to correct instead of typing from scratch," a claim that held up 94% of the time on matches the model had never seen. Two green rows then play against their video clips, draft string and jump-to timestamp on screen. It ends with what the file admits it can't do yet, and a question for anyone who charts tennis for real.
For sixteen posts, this project has graded its own homework — checking drafts against charts it already had the answers to. Today, for the first time, a draft goes to a real judge: someone who charts tennis for a living, whose only job is to say whether it helped.
Previously: cv-01 set the destination — the Match Charting Project (charters call it the MCP), run by Jeff Sackmann of Tennis Abstract, is volunteers watching tennis and writing down every shot in a compact code; nearly 18,000 matches charted so far. cv-14 gave the project a way to grade a machine-written line against a human one: how many corrections an editor would need, the token edit distance (more here). cv-15 turned that grade into a stoplight — HIGH ("start from this draft") or LOW ("re-chart it") — tuned so the flag never grades its own homework (more on confidence). cv-16 grew the test to seven matches across five broadcast feeds, 491 human-charted points, fixed a blind spot on the Wimbledon match, and landed the number this post leans on: HIGH is right about 94% of the time.
None of it has been true for long. In 2025 the MCP ran on 32 active charters, covering about a quarter of tour matches — and Sackmann published a plea for more. That's the door this post walks through.
The human loop
Being graded correctly isn't the same as being useful. Groove a stroke against a ball machine for months and you prove your technique reproduces — never that it survives a live point, because a machine can't return your shot and tell you how it felt. Every number this project has published so far graded the pipeline's aim, not its use: does the drafted string match a human's, was a flag right when checked later. Nobody who charts a match for a living has ever opened one of these files and tried to work from it.
That's the gap this post closes. The step where a tool's output leaves the lab, lands on a real person's desk, and the only score that counts is what they report back, is the human loop: grading against a fixed answer stops, and getting judged by whether you saved someone real time starts. Nothing in the pipeline changed this session — who's on the other side of the net did.
In tennis terms: the ask is what a doubles player makes of a prospective partner before ever stepping on court — play one set with me, then tell me if I made you faster or slower.
What ships today
Seven files went public today, one per benchmark match, in the exact column layout the MCP already uses for its own point-by-point files. The familiar columns are all there — match, point number, set score, game score, point score, server — a real charter could read those blind. Then come the columns Court Vision added:
| Column | What's in it | What it means |
|---|---|---|
1st | e.g. s5b2f1f2f2x@ | the machine's guess at the point, serve to last shot, in a charter's own shorthand |
2nd | always blank | can't yet tell a first serve from a second, so second-serve rallies are simply missing |
confidence | HIGH or LOW | HIGH: trust it enough to correct. LOW: rough hint, expect a rewrite |
conf_p | e.g. 0.902 | the raw score behind the label — closer to 1 is more confident |
clip | a filename | which extracted broadcast clip the row came from |
serve_s | seconds | where the serve happens in that clip — the jump-to |
n_shots | a small number | shots in the point, serve included |
A real row, from the US Open final, flagged HIGH at 0.902:
s5b2f1f2f2x@. Read left to right: s5 — serve into the body. b2 —
backhand return, up the middle. f1, f2, f2 — three forehands,
one into an outer third, then twice down the middle. x@ — unforced
error, wide and deep. Six tokens, one point.
Every match's file, with how many points it drafted and how many got the green flag:
| Match | Points drafted | Flagged trustworthy |
|---|---|---|
| Nadal vs. Shapovalov, Montreal 2017 | 24 | 10 |
| Federer vs. Haase, Montreal 2017 | 6 | 4 |
| Djokovic vs. Ruud, French Open final 2023 | 59 | 12 |
| Krejcikova vs. Paolini, Wimbledon final 2024 | 49 | 18 |
| Sinner vs. Zverev, Australian Open final 2025 | 71 | 3 |
| Sabalenka vs. Pegula, US Open final 2024 | 134 | 39 |
| Djokovic vs. Sinner, ATP Finals 2023 | 165 | 13 |
| Total | 508 | 99 |
99 of 508 — about one point in five — got the green flag; cv-16 found it right about 94% of the time on matches the grading model never trained on. Start with the US Open final: best hit rate in the batch, and the model called who served right 95% of the time there too — a camera that holds still pays off.
A drill doesn't prove anything until it's played against a real opponent. That's the whole reason this post exists.
The honest label
A draft that hides its weak spots wastes an hour, so the weak spots come first.
The drafts are drafts. Across all 491 human-charted points behind
the whole benchmark, only 5.7% — roughly 1 in 18 — match the human's
line closely enough to need just one fix. The real value sits in the
green rows: HIGH means within five corrections, holding up 94% of the
time on matches the grading model never trained on (96% on matches it
did study — the friendlier, less trustworthy number, included
anyway). LOW, 409 of 508 rows, means exactly what it sounds like:
re-chart from the video, draft as hint, serve_s as the jump-to.
Every missed serve is invisible. The pipeline can't yet see a fault, so every point is written as if won on the first serve, and the second-serve column is empty on all 508 rows. On a real scoresheet, roughly a third of points start with a miss. Fixing that isn't a footnote of the charter's pass — it's most of it.
The shorthand isn't official yet. The s before a serve number is
the pipeline's own invention, not real MCP notation, and a ? shows
up when the machine can't tell which way a shot went. The draft also
skips most of what a full MCP chart captures — no depth, no court
position, no volley vs. lob vs. drop shot; endings and rough
directions only. Whether to write strict, official strings with
blanks instead, or keep the uncertainty visible, is undecided —
question three, below.
Seventeen rows have no point number. The machine caught the point on video but couldn't match it to a spot on the scoreboard, so that column is blank. The point happened; a person still has to say where it belongs.
One answer key has problems of its own. cv-16's autopsy caught the Wimbledon final's human chart running at half speed on some rallies — strokes written down that the video contradicts. That file still ships, 18 of its 49 rows green. Open that one first if you're hunting for the model being too sure of itself.
The ask
If you chart for the MCP — or ever have — three questions matter here, in the order they'd change what gets built next.
-
The stopwatch. Chart ten points cold, straight off a clip. Then correct ten different green-flagged drafts against their clips instead. Which pass is faster, and by how much? The whole project is a bet on that ratio, never yet measured by anyone who actually holds the pencil.
-
The triage. Is a draft right 94% of the time, within five corrections, the right bar for "start from this"? And with only about one point in five earning that flag, does a pre-filled file even help mid-session — or does switching between "check this" and "chart from nothing" cost more than it saves?
-
The schema. What would this file need before you'd reach for it mid-session — official MCP strings with blanks, a real fault column, a charting-sheet layout instead of a spreadsheet, something else entirely?
The channel is an issue on court-vision — a one-line "this wastes my time because X" is signal, not rudeness. It would be the single most valuable sentence anyone has contributed to this project.
Lineage and license
Worth knowing before opening these files: the score and server columns exist because these drafts were lined up against the MCP's own published charts — the same charts the 94% number is graded against. So these files carry the MCP's license, CC BY-NC-SA 4.0: credit the source, don't sell it, share what's built from it the same way. The draft strings and confidence columns are this project's own addition, released under that license to keep the files one whole thing.
For the technical reader
Exact numbers, for anyone doing their own math:
- File key (
t1–t7, internal match IDs, same order as the table above): t1 Nadal–Shapovalov, t2 Federer–Haase, t3 Djokovic–Ruud (clay), t4 Krejcikova–Paolini (grass), t5 Sinner–Zverev (night), t6 Sabalenka–Pegula, t7 Djokovic–Sinner (indoor). Points/HIGH per match: table above. match_id, Pt, Set1/Set2, Gm1/Gm2, Pts, Svr, 1st/2nd— MCP's own points-file columns, preserved as-is.- HIGH tier LOMO figure: 90/96 at 19.6% coverage.
- court-vision is public; run cost ~$16 across seventeen posts, marginal match $0.
Session cost: $0.00 — nothing ran but a video renderer. Project total: ~$16. The next number that matters isn't the pipeline's to produce: it's one charter's minutes-per-point, with and without the draft.