Engineering
Scrape, rate, forecast, publish. Every figure below is out-of-sample. Code: github.com/quinnbrighton/datadrivensailing.
Who built it, and at what scale
Built and operated solo since spring 2025: one person wrote every line, runs the pipeline and answers the email. The codebase is about 72K lines of Python in 294 files plus 19 shell scripts, with 325 pytest tests that run on a clean checkout. The nightly job (a systemd timer at 03:00) scrapes new results, rebuilds two SQLite databases, retrains the weekly Plackett–Luce model on an RTX 3060, refits the discipline, crew and team race layers, regenerates about 120,000 static pages and deploys only the pages whose content changed; a full run takes 20 to 40 minutes on a 6-core Ryzen with 32 GB. Checks gate the deploy: if a stage fails or a sanity check on the output trips, the previous build stays live.
The outage. In September 2026 the site started returning 503s. The Flask app behind the fantasy API was mounted at the site root with eight LiteSpeed workers, so every directory URL fell through to Python, and crawlers (about 65% of traffic) saturated the workers; orphaned workers then held slots after recycles, and a few API workers hung at 0% CPU for up to an hour. The fix was three things: rewrite rules so LiteSpeed serves the static pages itself and only /api/ reaches Python, worker environment limits (parent-process check on, 120 s max process time), and removing an import-time background thread that could deadlock a forked worker. Static pages have served from the web server since, and the API has not run out of workers.
Pipeline
Code
The public repository holds the evaluator, the trainer, the team race framework, the scraper, the page generators and the tests. Data files, credentials and deploy scripts stay out of it. The pieces a reader will want first:
| Stage | File | What it does |
|---|---|---|
| Rate | analysis/plackett_luce/train_pl_weekly.py | weekly Plackett-Luce trainer: log-strength per sailor-week, drift penalty, team pooling, GPU |
| Evaluate | analysis/plackett_luce/evaluate_models.py | season-ahead and championship tests against Elo, static PL and average finish; regatta bootstrap |
| Newcomers | analysis/plackett_luce/newcomer_prior.py | the one forward-legal rule every model uses for sailors with no history |
| Simulate | analysis/plackett_luce/regatta_form.py | fitted regatta, drift and same-boat effects that make the simulations honest |
| Team race | analysis/team_race/model.py | three-boat side strength, school term, home edge; evaluate.py refits before every regatta date |
| Collect | scraper.py | Techscore results, rosters and entries into CSV, resumable |
| Publish | generate_site.py | every ICSA page from SQLite; writes only pages whose content changed |
| Tests | tests/ | 138 of the 325 tests ship with the public subset; the rest need the local databases |
The evaluator's forward-only split, trimmed from evaluate_models.py. For each target season the training set ends the week before the season's first race, the baselines' settings are tuned on the season before that, and newcomers are filled by one forward-legal rule for every model:
def run_forecast_fold(races, ids, id_to_idx, target, prev, device, skip_train):
N = len(ids)
cut = min(r.wn for r in races if r.season == target) # ISO week of the target season's first race
vcut = min(r.wn for r in races if r.season == prev) # ... and of the tuning season before it
train = [r for r in races if r.wn < cut] # everything before the cutoff, both leagues
test = [r for r in races if r.season == target] # the whole target season, ratings frozen
val_train = [r for r in races if r.wn < vcut] # baselines are tuned one season back ...
val = [r for r in races if r.season == prev and r.wn < cut] # ... on data before the cutoff
# Elo: tune K (and the logistic scale) on the previous season only
def elo_eval(K):
snaps = elo_ratings(races, N, K, [vcut, cut]) # ratings as of each cutoff, past weeks only
d = pair_diffs(val, snaps[vcut]); k = fit_scale(d)
return mean_logloss(d, k), (k, snaps[cut])
elo_K, (elo_scale, elo_S), elo_edge, _ = tune_scalar(ELO_K_GRID, elo_eval, 1.0, 4096.0)
# (average finish and static PL tuned the same way; omitted)
# Weekly PL: the site model, retrained with the same cutoff
out = run_trainer(EVAL_DIR / f"forecast_{target}", week_label(cut), skip_train)
weekly_S = load_latest(out, id_to_idx, N)
# Who has a history before the cutoff, and where they debuted
seen = np.zeros(N, dtype=bool); first_season = np.full(N, "", dtype=object)
for r in train:
fresh = ~seen[r.idx]
first_season[r.idx[fresh]] = r.season
seen[r.idx] = True
frozen = {"uninformed": (np.zeros(N), 1.0), "avg_finish": (avg_S, avg_scale),
"elo": (elo_S, elo_scale), "static_pl": (static_S, 1.0), "weekly_pl": (weekly_S, 1.0)}
scores = {}
for name, (S, scale) in frozen.items():
# newcomers: school debut median, else league median, in that model's own units
S_f = npri.fill(S, seen, first_season, first_league, SCHOOL_KEYS, prev2) if name != "uninformed" else S
scores[name] = ([S_f[r.idx].copy() for r in test], scale)
accs, calib, per_reg = evaluate(test, scores, seen) # pairwise accuracy, log-loss, calibration, per-regatta rows
return fold, accs, calib, per_reg
Accuracy
Squad model error (rating points, lower is better)
Fitted before each season, tested on it. Exact driver named: 63% / 46%. Ratings are today's, so slightly optimistic.
Decisions
| Problem | Change | Evidence |
|---|---|---|
| Trainer stopped before converging | rating increments + first-race prior, retuned on week-ahead forecasts | 69.6 → 70.4% pairwise |
| Newcomers have no history | start at school debut median, not league mean | +3.5 pts on newcomer pairs |
| Races in a regatta are correlated | simulate with regatta effect, drift, same-boat effect | team totals checked at every championship |
| Most likely squad is a poor rating forecast | price a calibrated, probability-weighted rating | error 0.43 vs 0.55 (SH), 0.44 vs 0.55 (MR) |
| Strongest-event-first sent specialists to fleet | staff singlehanded / match race first | 0.9% of such drivers also sail fleet that week |
| Availability is uncertain | Monte Carlo weekend, one sailor per boat, 2,000 draws | 80% interval shown per entry |
| Collected data vs source | re-pull, compare boat by boat with Techscore | 0 dropped boats, all numeric fields match |
| Page and server must quote the same price | one snapping rule in Python and JS | unit test compares both |
Operations
Limits
- Match-race results on Techscore are one final ranking: no round-robin data.
- ~5,400 singlehanded rows lack a class year, ~1,300 keelboat boats lack a skipper; neither feeds ratings.
- Discipline accuracy uses today's ratings. The fantasy market is pretend: Mollicoins are not real money, and it is not gambling.