Skip to content

pmlab 1.1.3stable API · MIT · Python 3.12+

Prediction markets, without lookahead.

pmlab is a plugin-based ML framework for Polymarket. Write four methods for a market family; pmlab does the rest: edge and cost arithmetic, walk-forward backtests, a hard GO / NO_GO gate before anything is promoted, paper trading, and settlement that waits until the truth is final.

pip install pmlab
Start the tutorial API reference GitHub
rolling origin · bundled sample 59 folds
  1. fold 1 0.092
  2. fold 13 0.099
  3. fold 24 0.085
  4. fold 36 0.075
  5. fold 47 0.089
  6. fold 59 0.088

traintest daynot seen

brier by fold, lower is better (0.075 to 0.120)

folds
59
trades
289
brier
0.0957
gate
NO_GO

Synthetic sample: 750 markets, 5 cities, 150 days, no network. It says nothing about real markets.

signals
  • Athens 07-31 <=24°C mkt 0.109 model 0.239 +0.126 YES
  • Istanbul 07-31 30°C mkt 0.113 model 0.231 +0.115 YES
  • Lisbon 07-31 31°C mkt 0.208 model 0.311 not traded: Lisbon failed the gate
  • Madrid 07-31 >=37°C mkt 0.086 model 0.220 +0.131 YES
  • Rome 07-31 <=19°C mkt 0.065 model 0.292 +0.224 YES
  • Athens 08-01 <=26°C mkt 0.112 model 0.011 +0.094 NO
  • Lisbon 08-01 29°C mkt 0.179 model 0.286 not traded: Lisbon failed the gate
  • Madrid 08-01 32°C mkt 0.170 model 0.011 +0.152 NO
  • Rome 08-01 >=31°C mkt 0.078 model 0.009 +0.062 NO
  • Athens 08-02 28°C mkt 0.131 model 0.019 +0.105 NO
  • Istanbul 08-02 25°C mkt 0.107 model 0.246 +0.135 YES
  • Lisbon 08-02 29°C mkt 0.244 model 0.340 not traded: Lisbon failed the gate

It will not look ahead, or ship a loser.

Enforced in code and checked by the test suite, not left to convention. Each claim below links to the page that shows how.

  1. No lookahead.

    rolling_origin_eval trains only on rows whose label was already known when the fold decided. build_panel refuses a price seen at or after the decision it informs, and check_no_leakage raises when a feature gives the outcome away.

    fold  1  train 2025-04-01 → 2025-05-30  test 2025-06-01
fold 59  train 2025-04-01 → 2025-07-27  test 2025-07-29  (5310 rows)
    How the checks work

  2. A failed gate cannot ship.

    ChampionManifest.publish raises unless the gate says GO, and that is not configurable. On the sample the gate reads every city, one of them loses money, and publishing is refused. Scope the champion to the cities that pass and it publishes, with that scope recorded in champion.json.

    • Athens +17.38 ok
    • Istanbul +22.99 ok
    • Lisbon -16.28 negative_pnl
    • Madrid +51.31 ok
    • Rome +59.83 ok 
    ValueError: Cannot publish champion with NO_GO gate. Gate decision: NO_GO
    The champion registry

  3. Reproducible.

    Every backtest records its seeds, library versions, resolved config and a fingerprint of the data. pmlab reproduce re-runs it and verifies that the metrics come back.

    reproduce → reproduced  data match: yes  metrics compared: 5
    Reproducibility

  4. Frozen API.

    From 1.0 pmlab follows semantic versioning. What is public is compared with a checked-in snapshot on every change, and nothing public is removed before 2.0 without a deprecation warning that names the replacement.

    145 public names in 8 modules · 11 CLI commands · tests/api/public_api.json
    What is public

  5. Five rules, in code.

    A champion is never published with a NO_GO gate. Canonical training data is never overwritten without an explicit flag. Workspaces are never mixed: live forward evidence and training data stay apart. The champion's segment gate is always read from champion.json, the record made at promotion. A trade is never settled before MarketPlugin.is_truth_final says the truth is final.

    Architecture

Run it in a minute.

Offline, on a small synthetic sample that ships in the package: no network, no account, no order placed. The code is the README's, and CI runs it.

Quickstart

This runs offline on a small synthetic sample bundled in the package: 750 weather-temperature markets, no network and no account. The numbers say nothing about real markets.

from pmlab import (ChampionManifest, HoldoutGateResult, LGBMForecaster, TradePolicy,
                   build_panel, rolling_origin_eval)
from pmlab.datasets import load_sample

sample = load_sample()                       # 750 synthetic weather markets, checked against a checksum
resolver = sample.plugin()                   # the real WeatherTmaxPlugin, wired to the bundled file
found = [(day, spec) for day in sample.days for spec in sample.plugin(day).discover_markets()]

# One row per (market, bin), each at its own price. build_panel refuses a price seen at or after
# the decision, and check_leakage=True raises if a feature gives the outcome away.
panel = build_panel(
    [resolver.build_training_rows(s, "previous_evening", decision_time=f"{d}T12:00:00+00:00")
     for d, s in found],
    check_leakage=True,
)
# The forecast alone cannot tell a market's bins apart: add how far each bin sits from it.
bounds = {(s.market_id, b.label): [x for x in (b.lower, b.upper) if x is not None]
          for _, s in found for b in s.outcome_bins}
centre = [sum(bounds[k]) / len(bounds[k]) for k in zip(panel["market_id"], panel["outcome_label"])]
panel["feature_bin_dist"] = centre - panel["feature_fcst_tmax"]

# Walk-forward backtest: every fit sees only rows from before the day it predicts.
model = LGBMForecaster(random_state=7, n_jobs=1)
result = rolling_origin_eval(
    panel, model, stride=10, min_train_rows=2700,
    policy=TradePolicy.live(min_edge=0.05),  # what `pmlab record-trades` trades: best bin and side
)
gate = HoldoutGateResult.evaluate(result.trades)
print(len(result.trades), "trades | PnL", round(result.trades["realized_pnl"].sum(), 2), "|", gate.decision)
panel.to_parquet("panel.parquet")            # the command line below reads this file

# A gate that fails cannot publish a champion.
failed = HoldoutGateResult.evaluate(result.trades, required_segments=["Atlantis"])
try:
    ChampionManifest.publish(model, failed, "champion", plugin_family="weather_tmax")
except ValueError as err:
    print("refused:", err)

It prints the number of trades, the total PnL and the gate's decision, then the refusal. The same panel runs from the command line, and the tutorial continues from here: tuning, the gate per city, paper trading and the report.

pip install pmlab            # Python 3.12+; or: uv add pmlab
pip install "pmlab[tune]"    # + Optuna, for TunedForecaster
pip install "pmlab[track]"   # + MLflow, for MLflowTracker

Where to go.

Source and issues on GitHub · PyPI · Changelog · Roadmap · Deprecation policy. pmlab is research software and not financial advice: trading prediction markets can lose all the money you put in, and no pmlab command places an order.