Engineering

Scrape, rate, forecast, publish. Every figure below is out-of-sample. Code: github.com/quinnbrighton/datadrivensailing.

1.58Mboat-race rows
7,693regattas, 37 seasons
23,824sailors rated
325tests, CI on clean checkout
72Klines of Python, 294 files
1person, since 2025

Who built it, and at what scale

Built and operated solo since spring 2025: one person wrote every line, runs the pipeline and answers the email. The codebase is about 72K lines of Python in 294 files plus 19 shell scripts, with 325 pytest tests that run on a clean checkout. The nightly job (a systemd timer at 03:00) scrapes new results, rebuilds two SQLite databases, retrains the weekly Plackett–Luce model on an RTX 3060, refits the discipline, crew and team race layers, regenerates about 120,000 static pages and deploys only the pages whose content changed; a full run takes 20 to 40 minutes on a 6-core Ryzen with 32 GB. Checks gate the deploy: if a stage fails or a sanity check on the output trips, the previous build stays live.

The outage. In September 2026 the site started returning 503s. The Flask app behind the fantasy API was mounted at the site root with eight LiteSpeed workers, so every directory URL fell through to Python, and crawlers (about 65% of traffic) saturated the workers; orphaned workers then held slots after recycles, and a few API workers hung at 0% CPU for up to an hour. The fix was three things: rewrite rules so LiteSpeed serves the static pages itself and only /api/ reaches Python, worker environment limits (parent-process check on, 120 s max process time), and removing an import-time background thread that could deadlock a forked worker. Static pages have served from the web server since, and the API has not run out of workers.

Pipeline

1 Collectscraper.pyTechscore: results,rosters, entries2 Storebuild_db.pySQLite x2, women'stags, HS-to-collegelinks3 Ratetrain_pl_weekly.pyPlackett-Luce, teampooling; discipline +crew layers4 Forecastpredict_upcoming_lineups.pylineups, squads, MonteCarlo odds5 Publishgenerate_site.py120K+ static pages,Flask API6 Shiprefresh_f26.shchecks, rsync,restart, health checks

Code

The public repository holds the evaluator, the trainer, the team race framework, the scraper, the page generators and the tests. Data files, credentials and deploy scripts stay out of it. The pieces a reader will want first:

StageFileWhat it does
Rateanalysis/plackett_luce/train_pl_weekly.pyweekly Plackett-Luce trainer: log-strength per sailor-week, drift penalty, team pooling, GPU
Evaluateanalysis/plackett_luce/evaluate_models.pyseason-ahead and championship tests against Elo, static PL and average finish; regatta bootstrap
Newcomersanalysis/plackett_luce/newcomer_prior.pythe one forward-legal rule every model uses for sailors with no history
Simulateanalysis/plackett_luce/regatta_form.pyfitted regatta, drift and same-boat effects that make the simulations honest
Team raceanalysis/team_race/model.pythree-boat side strength, school term, home edge; evaluate.py refits before every regatta date
Collectscraper.pyTechscore results, rosters and entries into CSV, resumable
Publishgenerate_site.pyevery ICSA page from SQLite; writes only pages whose content changed
Teststests/138 of the 325 tests ship with the public subset; the rest need the local databases

The evaluator's forward-only split, trimmed from evaluate_models.py. For each target season the training set ends the week before the season's first race, the baselines' settings are tuned on the season before that, and newcomers are filled by one forward-legal rule for every model:

def run_forecast_fold(races, ids, id_to_idx, target, prev, device, skip_train):
    N = len(ids)
    cut  = min(r.wn for r in races if r.season == target)   # ISO week of the target season's first race
    vcut = min(r.wn for r in races if r.season == prev)     # ... and of the tuning season before it
    train     = [r for r in races if r.wn < cut]            # everything before the cutoff, both leagues
    test      = [r for r in races if r.season == target]    # the whole target season, ratings frozen
    val_train = [r for r in races if r.wn < vcut]           # baselines are tuned one season back ...
    val       = [r for r in races if r.season == prev and r.wn < cut]   # ... on data before the cutoff

    # Elo: tune K (and the logistic scale) on the previous season only
    def elo_eval(K):
        snaps = elo_ratings(races, N, K, [vcut, cut])       # ratings as of each cutoff, past weeks only
        d = pair_diffs(val, snaps[vcut]); k = fit_scale(d)
        return mean_logloss(d, k), (k, snaps[cut])
    elo_K, (elo_scale, elo_S), elo_edge, _ = tune_scalar(ELO_K_GRID, elo_eval, 1.0, 4096.0)
    # (average finish and static PL tuned the same way; omitted)

    # Weekly PL: the site model, retrained with the same cutoff
    out = run_trainer(EVAL_DIR / f"forecast_{target}", week_label(cut), skip_train)
    weekly_S = load_latest(out, id_to_idx, N)

    # Who has a history before the cutoff, and where they debuted
    seen = np.zeros(N, dtype=bool); first_season = np.full(N, "", dtype=object)
    for r in train:
        fresh = ~seen[r.idx]
        first_season[r.idx[fresh]] = r.season
        seen[r.idx] = True

    frozen = {"uninformed": (np.zeros(N), 1.0), "avg_finish": (avg_S, avg_scale),
              "elo": (elo_S, elo_scale), "static_pl": (static_S, 1.0), "weekly_pl": (weekly_S, 1.0)}
    scores = {}
    for name, (S, scale) in frozen.items():
        # newcomers: school debut median, else league median, in that model's own units
        S_f = npri.fill(S, seen, first_season, first_league, SCHOOL_KEYS, prev2) if name != "uninformed" else S
        scores[name] = ([S_f[r.idx].copy() for r in test], scale)
    accs, calib, per_reg = evaluate(test, scores, seen)   # pairwise accuracy, log-loss, calibration, per-regatta rows
    return fold, accs, calib, per_reg

Accuracy

50%56%61%66%72%Coin flip50.0%Avg finish %65.3%Elo68.7%PL static68.8%PL weekly, pre-fix69.6%PL weekly (site)70.4%
Pairs of boats ordered correctly, 4 held-out seasons (~16,000 races). Championships: 72.4% (95% CI 71.6–73.3), 166 regattas.
50506060707080809090100100said 53%, happened 52%said 58%, happened 57%said 62%, happened 61%said 68%, happened 66%said 72%, happened 70%said 77%, happened 75%said 82%, happened 80%said 87%, happened 86%said 92%, happened 90%said 97%, happened 94%
Calibration. x: model said, y: happened (%). Under the dashed line = overconfident; honest to ~85%.

Squad model error (rating points, lower is better)

Singlehanded
0.00.20.40.60.8Calibrated (site)0.43Best-rated sails0.49Most likely squad0.55Pool average0.60
Match race
0.00.20.40.60.8Calibrated (site)0.44Best-rated sails0.62Most likely squad0.55Pool average0.57

Fitted before each season, tested on it. Exact driver named: 63% / 46%. Ratings are today's, so slightly optimistic.

Decisions

ProblemChangeEvidence
Trainer stopped before convergingrating increments + first-race prior, retuned on week-ahead forecasts69.6 → 70.4% pairwise
Newcomers have no historystart at school debut median, not league mean+3.5 pts on newcomer pairs
Races in a regatta are correlatedsimulate with regatta effect, drift, same-boat effectteam totals checked at every championship
Most likely squad is a poor rating forecastprice a calibrated, probability-weighted ratingerror 0.43 vs 0.55 (SH), 0.44 vs 0.55 (MR)
Strongest-event-first sent specialists to fleetstaff singlehanded / match race first0.9% of such drivers also sail fleet that week
Availability is uncertainMonte Carlo weekend, one sailor per boat, 2,000 draws80% interval shown per entry
Collected data vs sourcere-pull, compare boat by boat with Techscore0 dropped boats, all numeric fields match
Page and server must quote the same priceone snapping rule in Python and JSunit test compares both

Operations

Nightly pipelinegated deploy
Optional stagesfail soft
Page generatorsidempotent
Worker restartpkill lswsgi
Import-time threadsnone
Prices (Py = JS)tested
Static pagesLiteSpeed, not Python
Full nightly run20–40 min
Training hardwareRTX 3060, Ryzen 5 5600X
Deployrsync changed pages

Limits