Prediction Market

QA mode is active — seeded test data may appear alongside real markets. Manage in Admin → QA

Sign in

Acting as:

All Markets

My Account

Open Positions

Loading…

Closed Positions

Loading…

Transaction History

Loading…

Submit a Question

Structured submission (gate redesign): the title is what bidders see in market lists; the deadline is the date by which the outcome must be resolvable. Source and invalid conditions are optional — the gate can complete them where the rules allow, or you can propose your own.

Question Voting

Question voting UI lives here once fe_display_questions_voting is built.

🔨 Admin Resolution

Pick a market from the dropdown above (or click a market on the Markets tab).

QA Harness

QA mode: …

Primitives:
Scenarios:
Click any button to see the response here.

Recent Events

Loading…

Back Office — Users

Loading…

Glossary — Notes

Working notes about the glossary — terms to add, structural ideas, things to verify. NOT sent to the LLM. The actual glossary lives in the pane below.

Glossary — Body

Terms, jargon, sources, and in-scope/out-of-scope conventions the LLM needs to interpret submitted questions. Sent to the LLM at eval time — either as a sibling cached block, or substituted in-place via the {INJECT_GLOSSARY_BODY} placeholder in the system prompt.

System Prompt — Notes

Working notes about the prompt — ideas, things to test, edge cases. NOT sent to the LLM. The actual prompt body lives in the pane below.

System Prompt — Body

The actual system prompt sent to the LLM when you click Submit Evals. Keep it cache-friendly (no per-call dates or other variable values — those go in the user message at run time).

Failure dimensions

One per line. Closed list — the LLM is forced to pick from these. The aim is to refine prompt + glossary so other is used as rarely as possible.

OpenRouter models

One vendor/model id per line, # for comments. Ids must match openrouter.ai/models exactly. Costs aren't estimated up front — OpenRouter reports the actual cost per call and it accumulates on the run. Requires OPENROUTER_API_KEY on the server; without it these models run in stub mode.

Submit Evals

Send every eval question through the selected LLM and grade the response against the expected verdict + failure dimension + injection_flag. Manual scoring of the returned redirect happens in the results table.

No run yet. Pick a model and click Submit Evals.

Eval questions

Manually-curated test cases for the LLM Check Questions gate. Verdict is binary (pass / fail). On FAIL: Failure dimension picks from the editable list above; Expected redirect is what the LLM should suggest. Injection? marks questions that probe the gate itself — scored separately from the dimension. Save all commits every row's current values.

Loading…

Eval runs

Past Submit-Evals runs. Click a row to drill into the per-question results. Scores are auto-graded (verdict + dimension + injection_flag) plus the human-rated redirect score (0–2 per question).

Loading…

Project Status

Modules from the flowchart. Orange = backend, green = frontend, blue = platform/infra (added since the flowchart). Status reflects the code as built, not the flowchart's (stale) annotations. Only major features noted. Test counts are from pytest (backend only — there's no frontend test harness yet); 400 backend tests total, covering every meaningful backend module, run by CI on every push.

Backend Modules
a. Check Questions50 tests (harness)Basic + full eval harness
Done: Production still auto-approves, but the LLM machinery around it is built: full eval harness (Admin → Evals question editor with structured-submission fields, Submit Evals against Anthropic-direct or any OpenRouter model for minimum-model-strength sweeps, Scoring tab with auto-grades + cost per run, harness_version comparability boundary), prompt + glossary version-controlled in Admin sub-tabs, and the gate-redesign Phase 1 schema (structured submissions, §9 {applicable, value} contract objects, completion-action scoring).
Missing: The production gate itself — gate redesign Phases 2–5: deterministic pre-LLM layer, LLM gate + confirm flow behind a runtime toggle, 3× self-consistency + review queue, override flywheel + submission fees. Plan: .claude/plans/plan_gate_redesign_implementation.md.
c. Manage Market0 testsNot built
Done: Nothing — no file exists.
Missing: Whole module. Intended to drive market lifecycle (open → closed → admin review) and vote-weighted resolution. Today markets only resolve by direct admin action.
d. Bid Engine56 testsComplete
Done: Contracts-based matching (2026-05-27 refactor — remaining_contracts is the source of truth, no stake↔contract drift). Price-time priority, partial fills, self-match prevention. Price-improvement residual released back to user at bid close via release_funds; non-BALANCE_004 release failures surface at WARNING (no silent stake retention). Sub-MIN_CONTRACT remainders close as closed_dust with residual refunded. Displayed market price is the order-book midpoint with a €1 volume floor; decoupled from the trade-tape fields. Money-conservation invariant (sum(user_totals) + sum(unresolved_match_pool)) covered by dedicated tests that bypass the per-test drift-repair fixture. Interface kept narrow (single process_bids entry point) so a future batch-auction engine can swap in. Price priority symmetric on both sides (NO-side inversion fixed 2026-07-06). Cancellation race hardened: consumes execute at match time and a failed consume VOIDS the match (counterparty compensated + re-reserved); status updates carry a WHERE guard so a concurrent cancel is never clobbered.
Missing: Platform fees on matches. The price-over-time chart's interpolation through quiet periods is a separate frontend concern noted in the bid-engine handover.
e. Bid Queue69 testsComplete
Done: Input validation, synchronous fund reservation, idempotency. Cancel-first fail-safe ordering (bid dies before funds release); pending-bid snapshot filtered to OPEN markets so bids queued just before a resolution can't match on the resolved market. The in-memory same-cycle guard only works in-process — cross-process safety is DB-level in the bid engine.
Missing: Complete for current scope.
f. Resolve Engine26 testsComplete
Done: Settles YES/NO/INVALID — winner payouts, loser & unmatched refunds, reservation release. Emits market_resolved events. Idempotent: re-running on a market that's already been processed is a no-op (matches query filters by linked bid status). closed_dust bids settle as 'settled' (economically matched; nothing refunded at settlement). Writes per-user financial results + delegates predictive scoring to the pluggable scoring package (Brier by limit price today).
Missing: Complete for current scope.
g. Main Loop43 testsComplete
Done: Orchestrates the full cycle; configurable interval, cycle-health warnings, daily summary, graceful shutdown. Commit-failure retry buffer + op-level dead-letter (a poison op no longer destroys the cycle's queued facts). Runtime money-invariant sweep (1/60s, deduped alerts) — caught a real stranded-dust bug on its first day live. Trading-close sweep flips markets past their deadline to 'closed' before each cycle's bid snapshot.
Missing: Complete for current scope.
h. Write to DB12 testsComplete
Done: In-memory operation queue batching every module's writes per cycle; supports $increment. Atomic drain() (take-and-clear) is the only safe committer pattern for the shared endpoint queue — closed a lost-write race between concurrent requests.
Missing: Complete.
j. Convert Question to Market9 testsComplete
Done: Turns approved questions into markets (new UUID, insert, question status → converted). Copies the structured contract onto the market: title → question, proposed source / invalid condition, resolution deadline (end-of-day) + trading_closes_at defaulting to it. Legacy prose questions still convert open-ended.
Missing: Gate-proposed earlier trading close (Phase 3 copies it from the completed contract).
k. Commit to DB8 testsComplete
Done: Executes the write queue inside a transaction (asyncpg), rolls back on failure. Insert ops accept on_conflict/returning knobs; update ops accept an optional where guard (deferred updates can refuse to clobber a state that changed between queueing and commit).
Missing: Complete.
l. Notification Queue0 testsPlaceholder
Done: Accepts queued notifications and logs them each cycle.
Missing: Persistence + dispatch. Intended to store site notifications (read/unread) and hand off to Send Notification.
m. Send Notification0 testsNot built
Done: Nothing — no file exists.
Missing: Whole module. Intended to deliver queued notifications (MVP: DB-backed site notifications; later email / push).
n. Balance Queue15 testsComplete
Done: Queues reserve/release/consume/payout/refund ops with idempotency + duplicate detection; FIFO processing. 15 tests.
Missing: Complete.
p. Balance Management38 testsComplete
Done: Atomic balance ops holding the available + reserved = total invariant, with full audit trail. Audit-grade ledger: market_id + market_title denormalised onto every transaction row at write time. Consume dust (≤0.001 tolerance) returns to available instead of stranding in reserved — the bug the runtime invariant sweep caught live. All money writes pass Decimal, never float.
Missing: Play-money only — no deposit / withdrawal path (intentional for now).
q. Service Module0 testsNot built
Done: Nothing — no file exists.
Missing: Whole module. Intended as a caching layer (local + Redis + DB) to cut DB load. Not needed at current scale — everything reads Postgres directly.
r. Log Queue3 testsBasic
Done: In-memory FIFO queue at backend/app/modules/log_queue.py. event_log() enqueues; send_log drains once per main_loop cycle. Symmetric to Write to DB → Commit to DB but for log events. Local DB write stays synchronous inline in event_log() so the in-page panel sees events promptly.
Missing: Persistence (DB-table queue) for cross-process durability — currently lossy on shutdown. Acceptable for current scale.
s. Send Log5 testsBasic
Done: backend/app/modules/send_log.py drains log_queue and POSTs each item to AWKanban with a 3-second timeout. Silent no-op if AWKANBAN_URL/KEY unset. Failures aggregated to a single WARNING.
Missing: Retry with backoff, fallback channels, dead-letter table for unrecoverable failures. Best-effort delivery for now.
t. PAM (Player Account Management)21 testsComplete (lean+modular-scoring)
Done: pam.py with 5 read functions (account summary, open/closed positions, transactions, per-market leaderboard). Five endpoints under /api/users/{id}/... + /api/markets/{id}/leaderboard wired into the "My Account" top-tab and the specific-market detail page. ResolveEngine writes user_market_results on settlement, now including stake-weighted Brier loss + delta. Lifetime Brier averaged across resolved markets; per-market Brier shown in Closed Positions and on the resolved-market leaderboard.
Missing: Deposits / withdrawals, per-user limits, KYC. Real auth (separate track — Auth is "who you are", PAM is "what's true about your account"). The current "Acting as" picker treats users as a dev convenience. Pseudonymity (currently surfaces account names directly).
u. Back Office0 testsBasic
Done: Admin → Back Office tab — table of all users with creation date and balances, backed by existing /api/users. No dedicated backend file yet.
Missing: Admin actions (edit / adjust balances / ban). Read-only today. Will grow into the ops surface (audit-log views, market controls, system state).
v. Tooling (QA Harness)12 testsBuilt
Done: backend/app/qa.py + Admin → QA tab — seed primitives (incl. price-center selector), two scenarios, runtime loop-interval control, tag-by-prefix wipe. QA mode is runtime-toggleable from the UI (overrides the QA_MODE_ENABLED env default). 12 tests focused on the dangerous + foundational paths (wipe safety, seed_users, SeededRng determinism).
Missing: More scenarios (cancellation storm, partial fills) as the features they test mature.
w. Question Queue13 testsComplete
Done: Structured submission (gate redesign Phase 1): title + submitter + resolution deadline + optional source/invalid condition, with parent_question_id lineage for future resubmit-with-fix. Endpoint is dual-shape (legacy raw_text still accepted — QA harness). Full state-machine statuses in place (proposed / accepted / rejected / review / expired). (Letter w rather than m to disambiguate from Send Notification.)
Missing: Complete for current scope. (Validation lives in the gate — Phases 2–3.)
Frontend Modules
1. Input Questionno harnessBuilt
Done: Submit Question tab — structured 4-field submission (title, resolution deadline, optional source + invalid conditions) with a persistent "last submitted" status (queued → converted, with a link to the market).
Missing: Submissions list with proposal cards + Accept/Edit (gate redesign Phase 3), deterministic-rejection feedback (Phase 2), per-user rate limiting (Phase 5).
2. Display Questions for Votingno harnessNot built
Done: Empty placeholder tab.
Missing: The whole feature — browse pending questions, vote approve/reject/duplicate, threshold tracking. Depends on the voting backend.
3. Display Current Marketsno harnessBuilt
Done: Markets tab — four biggest-open market cards with price-over-time charts, plus the full market list.
Missing: Filter/sort controls (all/open/resolved, by volume/date). Deferred until the list grows.
4. Display Specific Marketno harnessBuilt
Done: Dedicated page at #markets/<id> (shareable/bookmarkable) — price-over-time chart, order book, Market terms block (resolution source / criteria / invalid conditions / both close dates), bid placement, resolution-state badge, post-resolution leaderboard, QA-only bid/match lists. Auto-refreshes.
Missing: Resolution-voting UI (gated on Manage Market); spread display (per-card note).
6. Admin Resolveno harnessBuilt
Done: Admin → Resolve sub-tab — select a market, resolve YES/NO/INVALID, settlement is queued.
Missing: A "pending resolution" queue (markets that hit a vote threshold or end date). Today admin resolves any open market manually.
7. My Accountno harnessBuilt (basic)
Done: Header card (balance + lifetime stats), open + closed positions, paged transaction history with market-title links, resolved-market leaderboard, currency formatting helper. Reads from PAM (t.).
Missing: Notifications hooks (waiting on Notification Queue), real auth (waiting on Auth module), deposit/withdrawal UX (real-money era).
Platform & Infrastructure (cross-cutting, not on the module flowchart)
Deploymentno harnessLive + CI
Done: Running at prediction.anw.ee — Docker-compose stack on Hetzner, fronted by Caddy with Let's Encrypt TLS. CI: GitHub Actions runs all 400 tests against a from-scratch database on every push (which also proves setup_db.sql works on fresh installs). Deploy ritual: git pull → run any new deploy/migrations/*.sql → docker compose up -d --build. Boot-time schema guard emits a red schema_behind_code event naming any missing columns if a migration was skipped.
Missing: CD (auto-deploy on green), backups, log rotation.
Authenticationno harnessBuilt
Done: Site-wide shared API key (X-API-Key); frontend prompt + localStorage; 401 re-prompt.
Missing: Real per-user logins. Currently identity is a fixture dropdown ("Acting as"); PAM (t.) provides the account half, this provides the credentials half.