betting_combat.rounds.backtest.metrics

How a backtest configuration is judged: totals, drawdown by card, card-resampled percentiles and ROI intervals, and the probability of backtest overfitting (research ufc/modeling/execution.py::summarize / pbo and the card bootstrap of ufc/modeling/flat_bets.py::report).

Cards are the unit of resampling: bets on one card share the evening’s news and the same fighters’ rounds, so they are not independent.

Functions

card_boot_idx

card_boot_idx(n_cards: int, n_boot: int = 2000, seed: int = 0) -> np.ndarray

Resamples of the cards (with replacement), as positions into the card order: n_boot rows of n_cards (research: default_rng(0).integers(0, n, (2000, n))).

card_order

card_order(events: pd.DataFrame) -> list[Any]

The cards (event_id) in (event_date, event_id) order: two cards on one date never depend on how a sort breaks ties (research cpcv.card_dates, fixed 2026-09-30).

card_roi_ci

card_roi_ci(bets: pd.DataFrame, n_boot: int = 2000, seed: int | np.random.Generator = 0, q: tuple[float, float] = (5.0, 95.0)) -> tuple[float, float, float]

ROI (total P&L / total staked) of bets (event_id, pnl, cost) and its interval over n_boot resamples of the cards (research flat_bets.report: each resample draws as many cards as there are, with replacement, and pools their P&L and stakes).

seed may be a Generator to continue one random stream across several calls, as the research did across its (model, market) groups.

pbo

pbo(profits: pd.DataFrame, n_blocks: int = 8) -> float

Probability of backtest overfitting by combinatorially symmetric cross-validation (Bailey, Borwein, Lopez de Prado and Zhu; AFML ch. 11). profits: cards (rows, in time order) x configurations, per-card profit.

The cards are cut into n_blocks contiguous blocks; for every choice of half the blocks as in-sample, the configuration with the largest in-sample total is ranked among all configurations on the other half: rank = share strictly below + half the share tied (itself included), clipped to (1e-6, 1 - 1e-6). PBO = the share of splits whose rank’s log-odds is <= 0 (the in-sample winner at or below the out-of-sample median).

summarize

summarize(bets: pd.DataFrame, cards: Sequence[Any], boot_idx: np.ndarray) -> tuple[dict[str, Any], pd.Series]

One configuration’s bets (event_id, cost, pnl) -> (totals, P&L per card in card order). Totals: bets, staked, profit, roi, max_dd (the deepest fall of cumulative card P&L below its running high, the high floored at 0), losing_cards, and p05 / p50 / p95 of total profit over the card resamples in boot_idx.