Admin console & console API

Citadel ships a self-contained admin console at /dashboard — a single HTML document served by the node itself (no build step, no CDN, works offline). Every sidebar section is live: the SPA signs in against /console/v1/login, drives the operator-only console API under /console/v1 for accounts, groups, chat, notifications, storage, leaderboards, matches, purchases, API keys, configuration, runtime, audit logs, script logs, match records, the error journal, and the read-only Database Explorer, and its Status section renders live node gauges from the public /status endpoint (no login required for that data source).

The console API is an operator surface, separate from the game-client API: different credentials, different roles, and no participation in the client SDK contract.

Login and roles

Operators authenticate with the static credentials from the [console] config section:

POST /console/v1/login
{ "username": "admin", "password": "password" }

200 OK
{ "token": "<opaque hex>", "role": "admin", "expires_in_sec": 3600 }

Send the token on every other console route:

GET /console/v1/me
Authorization: Bearer <token>

200 OK
{ "username": "admin", "role": "admin" }

Two roles exist:

Role	Granted by	Access
admin	console.password	Read everything, perform mutations.
viewer	console.viewer_password (optional)	Read-only; mutations return 403 forbidden.

These human sessions are short-lived opaque bearer tokens, not JWTs. Durable machine credentials are separate principals with explicit read scopes; see API keys and service accounts. Only a human admin session can manage them.

Failure behavior is deliberately uniform: a wrong username, wrong password, missing header, malformed header, unknown token, and expired token all return the same 401 authentication_failed — the boundary is not a credential oracle. Tokens live in process memory and expire after console.token_expiry_sec (default one hour); a node restart logs every operator out.

Change the default credentials

The out-of-the-box credentials are admin / password so the drop-and-run flow works immediately. The startup banner warns until you change them. Set [console] password in citadel.toml (or CITADEL_CONSOLE_PASSWORD) before exposing a node beyond localhost.

Route map

Route	Method	Auth	Purpose
/dashboard	GET	none	The console single-page app (public shell; data requires login).
/status	GET	none	Public machine-readable node status (Status section data source).
/console/v1/login	POST	none	Exchange credentials for a bearer token.
/console/v1/me	GET	bearer	The authenticated operator identity.
/console/v1/errors	GET	bearer	Redacted server failures and process panics retained locally (live).
/console/v1/config	GET	bearer	Effective node configuration, secrets redacted (live — see below).
/console/v1/audit	GET	bearer	Console action audit trail (live — see below).
/console/v1/storage	GET	bearer	Storage object browser (live — see below).
/console/v1/database	GET	bearer	Read-only database explorer (live — see Database Explorer).
/console/v1/matches	GET	bearer	Live rooms/matches introspection (live — see below).
/console/v1/runtime	GET	bearer	Runtime introspection + RPC caller (live — see below).
/console/v1/accounts	GET	bearer	Account administration (live — see below).
/console/v1/groups	GET	bearer	Groups administration (live — see below).
/console/v1/chat	GET	bearer	Chat channel moderation (live — see below).
/console/v1/notifications	GET	bearer	Notification review/composer (live — see below).
/console/v1/leaderboards	GET	bearer	Leaderboard administration (live — see below).
/console/v1/purchases	GET	bearer	Validated purchase records (live — see below).
/console/v1/subscriptions	GET	bearer	Subscription records (live — see below).
/console/v1/api-keys	GET, POST	human admin bearer	List metadata or create a machine credential; see API keys.
/console/v1/api-keys/{id}	GET	human admin bearer	Read API-key metadata (never a secret).
/console/v1/api-keys/{id}/rotate	POST	human admin bearer	Rotate and reveal a replacement secret once.
/console/v1/api-keys/{id}/revoke	POST	human admin bearer	Revoke the current generation.
/console/v1/lag/reports	GET	bearer	Redacted, keyset-paged derived lag reports.
/console/v1/lag/reports/{report_id}	GET	bearer	One redacted derived report.
/console/v1/lag/reports/{report_id}/windows	GET	bearer	Bounded aggregate timeline windows, never packet rows.
/console/v1/lag/captures	GET	bearer	Redacted, keyset-paged capture overview.
/console/v1/lag/captures/{capture_id}/raw	GET	admin	Paged opaque raw-artifact handles.
/console/v1/lag/captures/{capture_id}/raw/{handle}	GET, DELETE	admin	Download a private attachment or delete it while retaining reports.
/console/v1/lag/captures/{capture_id}/regenerate	POST	admin	Regenerate from a currently retained opaque raw handle.
/console/v1/logs	GET	bearer	Durable game-script log stream, keyset-paged (live — see below).
/console/v1/logs/{log_id}	GET	bearer	One stored log line including its author-supplied payload.
/console/v1/matchlogs	GET	bearer	Durable match records, keyset-paged — history, not live rooms.
/console/v1/matchlogs/{match_id}	GET	bearer	One match record plus exact per-domain counts.
/console/v1/matchlogs/{match_id}/entries	GET	bearer	The match drill-down: logs, telemetry slices, lag reports, audit.

A section whose backend has not landed yet answers 501 { "code": "not_implemented" } — authenticated and routed, never a 404 — so the SPA treats every section uniformly. Section documentation is added to this page as each backend ships.

Lag diagnostics

Logs & Diagnostics > Lag Diagnostics shows compact derived reports and capture overviews. Both viewer and admin can read report lists, a report detail, bounded histogram/timeline windows, and redacted capture summaries. The report surface has no raw paths, artifact digest, token, MIME grant, packet body, packet row, player/session identity, or raw availability state for viewers. Pages use opaque keyset cursors: pass after from next_after with a bounded limit, rather than treating cursors as filesystem paths or database ids.

The report surface requires SQLite, PostgreSQL, or CockroachDB durable report storage. MongoDB deployments can expose the admin raw-artifact lifecycle but reject analyze = true; they never substitute a process-local report cache for an empty or durable report list.

Only admin can list raw handles, download an artifact, delete retained raw evidence, or request regeneration. A raw download is an application/octet-stream attachment with Cache-Control: no-store, not a JSON endpoint and not a static URL. Handles are opaque (lc1-...) and do not reveal a disk path. viewer gets 403 forbidden for every raw or regeneration operation, including an attempted request.

Deleting raw bytes preserves existing derived reports and marks their raw availability unavailable; regeneration then returns a conflict until a new capture is collected. Raw lifecycle attempts, including failed or denied operations, are audited without recording paths, raw bytes, upload grants, or tokens. A report’s raw_available field is admin-only and means only that retained material is currently usable; the API does not expose whether it was expired or manually deleted.

Interpret charts as diagnostic correlation, not a latency verdict. The UI shows samples, exclusions, overflows/truncation, percentiles, decoder/analyzer versions, and UTC-clock uncertainty. no_analysis, pending, no_data, partial, complete, and failed are explicit report outcomes. Raw unavailability is a separate admin-only retention projection, deliberately without an expiry-versus-deletion reason. None should silently turn into a healthy-network result. Lag reports do not claim RTT, one-way latency, asymmetric latency, or packet loss.

For the private-root, quota, and cluster restrictions, see Lag diagnostics operations.

Logs (game script)

Logs & Diagnostics > Script Logs is the durable stream your game script writes through citadel.log.write. Nothing else writes to it: the server contributes no line of its own, and it attaches no credential, bearer token, session id, participant id, or transport identifier to any field. Server-side failures live in the Error Journal instead, and operator actions in the Audit Logs.

GET /console/v1/logs?level=warn&tag=combat&limit=50
Authorization: Bearer <token>

200 OK
{
  "items": [
    { "log_id": "ml1-0198f2c1a40b33c7d000000000007",
      "match_id": "mt1-0198f2c19e2a33c7d000000000000",
      "node_id": "citadel-1", "created_at_unix_ms": 1751791000000,
      "level": "warn", "tag": "combat.round",
      "message": "round ended early", "has_payload": true }
  ],
  "next_after": "ml1-0198f2c1a40b33c7d000000000007",
  "durable": true,
  "dropped_total": 0
}

Query param	Meaning
match_id	Exact durable match id (mt1-…). Absent lists every line, including the ones written outside any match.
level	Exact severity: trace, debug, info, warn, or error. An unrecognized name is a 400, never a silent widening.
tag	Tag prefix match — combat matches combat.round. Max 64 bytes.
after	Opaque keyset cursor: the previous page’s next_after.
limit	Page size, newest-first. Default 50, capped at 200.

GET /console/v1/logs/{log_id} returns one line with the payload_json field the listing omits — a page of 200 author-supplied payloads is a large response for rows the operator has not opened yet. A malformed or unknown log_id is a 404; a malformed cursor, match id, level, or tag is a 400. Unknown query parameters are rejected.

Two response fields describe the pipeline rather than the rows:

• durable — whether these rows came from a table. There is no in-process ring behind this surface: script logs are either persisted or discarded, so false means this backend keeps no script-log history at all and the empty page is the whole truth. The in-memory and MongoDB backends answer 200 { "items": [], "durable": false }, never a 503, because durable history must stay readable in exactly the node states where operators need it.
• dropped_total — lines the bounded write-behind queue dropped since boot. A non-zero value means the node produced logs faster than it could flush them.

Payloads are stored verbatim

message and payload_json are author-supplied. The server does not inspect, reformat, or redact them, and re-serializing would reorder keys and renormalize numbers, so what you read is exactly what the script wrote. A script that writes a secret into a payload has published it to every console operator.

match_id is nullable by design and is not a gate. A game with no match concept at all — an MMORPG world tick, a global scheduled job — still writes logs; the row simply carries no match. A log outside a match is stored, never rejected.

Machine credentials need the logs:read scope (see API keys). Reads are recorded in the audit trail as logs.list and logs.detail with counts and ids only — never a log message or payload text.

Match records (durable history)

Logs & Diagnostics > Match Records answers “what happened”, from the matches table, after the fact. Matches (live rooms) answers “what is happening right now”, from the realtime gateway. They are separate routes because they are separate questions, and because a durable record outlives the process that produced it.

The namespace is /console/v1/matchlogs, deliberately not /console/v1/matches/history: axum routes through matchit, which rejects a static segment sharing a position with a path parameter, and /console/v1/matches/{id} already claims that position.

A match record is server-owned. Its id, membership shape, clock, and termination reason are selected by the gateway; game code can neither open nor close a match. The one author-supplied column is result_json, stamped through citadel.match.set_result and returned verbatim. The record stores no participant identity, account id, session id, or transport identifier — that detail lives in the child domains the drill-down joins.

GET /console/v1/matchlogs?open=false&limit=50
Authorization: Bearer <token>

200 OK
{
  "items": [
    { "match_id": "mt1-0198f2c19e2a33c7d000000000000",
      "node_id": "citadel-1", "boot_id": "bt1-…", "room_id": 7,
      "name": "arena", "map": "corneria", "mode": "versus",
      "max_players": 8, "clock_epoch": 1751790000000,
      "opened_at_unix_ms": 1751791000000,
      "closed_at_unix_ms": 1751791900000,
      "termination_reason": "final_departure",
      "peak_participants": 6, "join_total": 11, "open": false }
  ],
  "next_after": "mt1-0198f2c19e2a33c7d000000000000",
  "durable": true,
  "dropped_total": 0
}

Query param	Meaning
open	true restricts the page to matches that have not been closed yet.
after	Opaque keyset cursor: the previous page’s next_after.
limit	Page size, newest-first. Default 50, capped at 200.

termination_reason is one of final_departure, server_closed, or formation_abandoned, and is absent while a match is still open. boot_id names the run of the node that opened the match: room_id is a per-process counter, so the pair is what makes a room identifiable across restarts. durable and dropped_total mean exactly what they mean for script logs above.

GET /console/v1/matchlogs/{match_id} adds result_json and an exact counts object — logs, telemetry_slices, lag_reports, audit.

The drill-down

GET /console/v1/matchlogs/{match_id}/entries?after&limit returns one match with its children:

{
  "record": { "match_id": "mt1-…", "result_json": "{\"winner\":\"kitsune\"}", ... },
  "logs": [...],
  "logs_next_after": "ml1-…",
  "telemetry_slices": [...],
  "lag_reports": [],
  "audit": [],
  "counts": { "logs": 1284, "telemetry_slices": 3, "lag_reports": 0, "audit": 0 }
}

logs is the only paged child — it is the only one that grows without a natural bound — and it keysets on log_id through after/logs_next_after. The other domains are inlined up to 50 rows each beside their exact count, so a truncated view never hides the real total.

Three of those fields are routinely empty, each for its own reason:

• audit is empty by design. Operator actions are not match-scoped, and this layer deliberately never forces one into a match.
• telemetry_slices are aggregate-only projections of closed authoritative slices: report id, context, close reason, duration, marker count, and accepted/rejected/corrected totals. Marker text is validated, used, and discarded — there is no column for it and there never will be.
• lag_reports is a bounded scan filtered by match scope. The durable column and its index shipped ahead of their write path, so this list is always empty today: a capture is node-scoped, and nothing yet constructs a capture participant carrying a match. It becomes correct with no API change once that write path lands. Each entry, when populated, carries only report_id, capture_id, status, and created_at_unix_ms — open it on Lag diagnostics, which owns its own admin gate and its own redaction.

A malformed or unknown match_id is a 404; a malformed cursor is a 400. Machine credentials need the existing matches:read scope. Reads are recorded as matchlog.list, matchlog.detail, and matchlog.entries, carrying counts and ids only.

Configuration browser

GET /console/v1/config returns the node’s effective resolved configuration (defaults + file + env + flags) as grouped, dotted key/value pairs, plus node_id, version, and the selected backend:

{
  "node_id": "dev-1", "version": "0.5.1", "backend": "sqlite",
  "groups": [
    { "name": "transport",
      "entries": [ { "key": "quic.bind", "value": "127.0.0.1:7353" } ] }
  ]
}

Redaction is explicit and tested: console.password and console.viewer_password render as <redacted>, and database.url keeps its host/database shape with any user:password@ credentials stripped. New config fields appear in the browser automatically; new secret fields must be added to the redaction list in the same change.

Storage browser

Operator-scope administration of the collection/key/user storage engine, against the node’s real persistence backend (in-memory, SQLite, or Postgres). Console operations run with runtime authority — object permissions do not apply; the bearer token and role are the gate. Ownership is addressed with an optional user_id query parameter (absent = the system owner).

Route	Method	Role	Purpose
/console/v1/storage	GET	any	Every collection with its total object count.
/console/v1/storage/{collection}?user_id&limit&cursor	GET	any	Paged object summaries (key, owner, version, permission codes — no values). limit defaults to 50, capped at 200; cursor resumes a page.
/console/v1/storage/{collection}/{key}?user_id	GET	any	One full object: value, version, permission codes. 404 when absent.
/console/v1/storage/{collection}/{key}?user_id	PUT	admin	Create/overwrite. Body: { "value": {…}, "read_permission": 0-2, "write_permission": 0-1, "version": "<token>" } — permissions default to owner-private; version makes the write conditional (409 on mismatch). Audited as storage.write.
/console/v1/storage/{collection}/{key}?user_id&version	DELETE	admin	Delete (idempotent; optional version precondition). Audited as storage.delete.

Values are JSON objects (up to 512 KiB through the console). Permission codes mirror Nakama’s numbering: read 0 none / 1 owner / 2 public; write 0 none / 1 owner.

Accounts

Operator-scope account administration over the node’s real identity repositories (in-memory, SQLite, or Postgres) — the same accounts device/custom auth creates:

Route	Method	Role	Purpose
/console/v1/accounts?filter&limit&offset	GET	any	Paged listing, username-ordered. filter is a substring match over id and username. Includes disabled and tombstoned accounts.
/console/v1/accounts	POST	admin	Create an account: { "username", "display_name"?, "metadata"? }. Audited as accounts.create.
/console/v1/accounts/{id}	GET	any	Detail: profile, state, timestamps, metadata, and every linked credential (device/custom + external id).
/console/v1/accounts/{id}	PUT	admin	Edit username / display_name ("" clears) / metadata. Audited as accounts.update.
/console/v1/accounts/{id}/ban	POST	admin	Disable the account and revoke its sessions — live tokens stop working and re-login is rejected with the uniform 401. Audited as accounts.ban.
/console/v1/accounts/{id}/unban	POST	admin	Re-enable a banned account. Audited as accounts.unban.
/console/v1/accounts/{id}	DELETE	admin	Logical delete: tombstone (never authenticatable), unlink every credential, revoke sessions. Audited as accounts.delete.
/console/v1/accounts/{id}/export	GET	any	The full account as JSON (profile + metadata + identities).

Bans use the account lifecycle (active → disabled) already enforced by the authentication service, so a banned player is rejected with the same generic 401 as an unknown credential — the boundary stays oracle-free.

Wallet & friends panels

Each account carries a virtual-currency wallet and a friends list (in-process stores — a restart clears them; recorded technical debt):

Route	Method	Role	Purpose
/console/v1/accounts/{id}/wallet	GET	any	Currency-ordered balances + newest-first ledger (last 100 entries).
/console/v1/accounts/{id}/wallet	POST	admin	Credit/debit: { "currency", "delta", "reason"? }. Balances never go negative — an overdraft is a 409 and appends nothing. Audited as accounts.wallet.adjust.
/console/v1/accounts/{id}/friends	GET	any	Relations: invited_sent, invited_received, friend, blocked.
/console/v1/accounts/{id}/friends	POST	admin	Invite/accept for the account: { "user_id" } — a matching add from the other side completes a mutual friend. Audited as accounts.friends.add.
/console/v1/accounts/{id}/friends/{other}	DELETE	admin	Remove the relation (both sides; also unblocks). Audited as accounts.friends.remove.

Both panels 404 for accounts that do not exist.

Matches (live rooms)

Point-in-time snapshots of the realtime gateway’s room registry — the same rooms clients create with join_or_create:

• GET /console/v1/matches?filter&limit — every live room, id-ordered: { id, name, map, mode, players, max_players, open, script_revision, script_generation }, plus an optional matchmaker telemetry object when realtime is attached. It contains only aggregate queued_tickets, evaluation count/duration, formed-match/ticket, cancelled-ticket, and expired-ticket totals — never ticket properties, IDs, or player identities. filter is a substring match over name/map/mode; limit defaults to 100 (cap 500). The response carries realtime_attached: false with zero rooms when no realtime transport is running.
• GET /console/v1/matches/{id} — one room plus its member roll: members: [{ participant, user_id }], where user_id is present for participants that authenticated their socket and null for guests. 404 for an unknown id.

script_revision/script_generation name the GameScript load a room was born bound to on a runtime.require_script node (null on ungated nodes). On a gated node whose script is not ready, both endpoints fail closed with 503 { "code": "runtime_unavailable", "message": "game script unavailable" } — the same stable message game clients see — and the Runtime section below explains why.

Snapshots are read-only copies; polling the console never blocks the realtime hot path.

API Explorer / Runtime

GET /console/v1/runtime reports the embedded Lua runtime’s state and what the loaded script registered:

{
  "enabled": true,
  "configured_language": "lua",
  "selected_language": "lua",
  "selection_source": "explicit",
  "entrypoint": "./game/main.lua",
  "adapter": "embedded",
  "tier": "trusted",
  "attached": true,
  "tick_hz": 20,
  "require_script": true,
  "readiness": {
    "state": "ready",
    "revision_id": "sha256:2c26b46b…",
    "generation": 3,
    "since_unix_millis": 1754500000000,
    "recovery": { "circuit_open": false, "consecutive_failures": 0, "restart_limit": 3 }
  },
  "script": {
    "source": "./game/main.lua", "reloadable": true, "deadline_ms": 100,
    "rpcs": ["ping"], "message_kinds": [1], "hooks": ["on_join", "on_tick"]
  }
}

attached: false (with no script) means no script is loaded — the node is running the built-in relay — or the realtime transports have not started.

require_script reports whether this node gates matches on GameScript readiness. When it does, readiness mirrors the gate authority once the transports start: state is one of no_script, validating, ready, activating, degraded, or unavailable; revision_id/generation identify the loaded script (matches are born bound to exactly this pair); and recovery reflects the supervised worker’s restart posture when the external-worker adapter is in use. Only ready opens the gate — every other state refuses match listing, creation, and admission with the stable game script unavailable error.

The language fields show whether [runtime] language was explicit or autodetected, the selected entrypoint path, and the adapter/tier currently in use (embedded / trusted today).

RPC caller (admin, audited as runtime.rpc):

POST /console/v1/runtime/rpc/ping
Authorization: Bearer <token>
{ "payload": "" }

200 OK
{ "ok": true, "reply": "pong" }

The call runs through the exact isolated, deadline-bounded path game traffic uses, with no participant bound (ctx.sender = 0, ctx.user_id = nil). Failures (unknown method, handler error, timeout) return ok: false with the same short generic message a game client would see. The reply is rendered as UTF-8 text (lossy for binary replies).

Groups

Operator administration of player groups (clans/guilds): unique name, description, an open/closed flag, an optional member cap, and a three-tier role ladder — member -> admin -> superadmin. A group always keeps at least one superadmin; demoting or kicking the last one is rejected with 409. Any role may read; mutations require admin and are audited.

Route	Method	Role	Purpose
/console/v1/groups?filter&limit&offset	GET	any	Paged group summaries: { id, name, description, open, max_size, member_count, created_at_unix_ms }. filter is a substring match over the name; limit defaults to 50 (cap 200). Returns { items, total }.
/console/v1/groups	POST	admin	Create a group. Body: { "name", "description"?, "open"?, "max_size"?, "creator_user_id"? } — open defaults true, max_size defaults 0 (unlimited), creator_user_id defaults to the operator’s username and becomes the founding superadmin. 409 on a duplicate name. Audited as groups.create.
/console/v1/groups/{id}	GET	any	One group plus its member roll: members: [{ user_id, role, joined_at_unix_ms }]. 404 for an unknown id.
/console/v1/groups/{id}	PUT	admin	Patch description/open/max_size (each optional; absent fields are unchanged). Audited as groups.update.
/console/v1/groups/{id}	DELETE	admin	Delete the group and its membership. Audited as groups.delete.
/console/v1/groups/{id}/members	POST	admin	Add a member. Body: { "user_id" }. 409 if already a member or the group is at max_size. Audited as groups.member.add.
/console/v1/groups/{id}/members/{user_id}/promote	POST	admin	member → admin → superadmin. 409 if already superadmin. Audited as groups.member.promote.
/console/v1/groups/{id}/members/{user_id}/demote	POST	admin	superadmin → admin → member. 409 if already member, or if the target is the group’s last superadmin. Audited as groups.member.demote.
/console/v1/groups/{id}/members/{user_id}/kick	POST	admin	Remove a member outright. 409 if the target is the group’s last superadmin. Audited as groups.member.kick.

Groups are an in-process, in-memory store: like the audit trail, there is no persistence yet, so a node restart clears every group. This is a recorded shortcut tracked in the technical-debt log alongside the other domain services introduced for the admin console (chat, notifications, leaderboards, wallet/friends, purchases).

Chat

Scope note: this section is a channel/history and moderation model only. It does not deliver messages in real time over the socket — realtime chat wire delivery is future work. Until then, the console’s POST route is the producer, so the channel/history model is exercisable end to end before wire delivery lands. History is kept in an in-process, per-channel bounded ring (default 1000 messages) — a node restart clears it, same as Groups/Notifications/Leaderboards/Purchases (see the technical-debt log).

A channel is created implicitly by its first appended message, with a type of room, group, or direct. The type is fixed at creation and later appends to the same channel ignore any channel_type in the body.

Route	Method	Role	Purpose
/console/v1/chat?filter&limit	GET	any	Every channel, most-recently-active first: { channel, channel_type, messages, last_activity_unix_ms }. filter is a case-sensitive substring match on the channel id; limit defaults to 100 (cap 500). Answers 200 with no query params.
/console/v1/chat/{channel}/messages	POST	admin	Console-side message producer. Body: { "sender": "…", "content": "…", "channel_type": "room" } (channel_type optional, defaults to room, ignored if the channel already exists). Creates the channel on first use. Audited as chat.message.append.
/console/v1/chat/{channel}/messages?limit&before	GET	any	Paged history, newest first. limit defaults to 50 (cap 200); before resumes a page (pass the previous page’s oldest returned id).
/console/v1/chat/{channel}/messages/{id}	DELETE	admin	Tombstone a message: content is blanked and deleted becomes true, but the row (and its id) stays in history. Idempotent; unknown channel or message id is 404. Audited as chat.message.delete.

GET /console/v1/chat/lobby/messages
{
  "channel": "lobby",
  "items": [
    { "id": 2, "sender": "bob", "content": "hi alice",
      "created_at_unix_ms": 1751791000123, "deleted": false },
    { "id": 1, "sender": "alice", "content": "hello world",
      "created_at_unix_ms": 1751791000000, "deleted": false }
  ]
}

Notifications

A targeting-or-broadcast composer over an in-process notification store: send a message to one user by id, or broadcast it to everyone, and browse/delete what has been sent. Both roles may list; sending and deleting are admin-only and audited.

GET /console/v1/notifications?user_id=u-1&limit=50&before=42
Authorization: Bearer <token>

200 OK
{
  "items": [
    { "id": 43, "user_id": "u-1", "subject": "welcome",
      "content": { "level": 1 }, "code": 0,
      "created_at_unix_ms": 1751791000000, "read": false }
  ],
  "total": 1
}

Query param	Meaning
user_id	Restrict to that user’s own targeted notifications plus every broadcast. Absent lists everything (the operator-wide view).
limit	Page size, newest-first. Default 50, capped at 200.
before	Resume cursor: only notifications strictly older than this id.

Send (admin, audited as notifications.send):

POST /console/v1/notifications
Authorization: Bearer <token>
{ "user_id": "u-1", "subject": "welcome", "content": { "level": 1 }, "code": 0 }

201 Created
{ "id": 43, "user_id": "u-1", "subject": "welcome",
  "content": { "level": 1 }, "code": 0,
  "created_at_unix_ms": 1751791000000, "read": false }

Omit user_id to broadcast to every account. subject must be non-empty (400 otherwise); content defaults to {} and must be a JSON object; code defaults to 0.

Delete (admin, audited as notifications.delete):

DELETE /console/v1/notifications/{id}
Authorization: Bearer <token>

204 No Content

404 for an unknown id.

Known limitations: no realtime push yet — a recipient only sees a notification by polling this route, there is no delivery over the realtime socket when they are online. The store is in-memory: a node restart clears it. Both are recorded technical debt pending the notifications persistence and delivery follow-up.

Leaderboards

An in-process leaderboards store: definitions (id, sort order, operator, optional reset schedule) plus one record per user per board. The console is also the record producer today — there is no player-facing leaderboard API yet — so ranking is exercisable end to end from the console alone.

Route	Method	Role	Purpose
/console/v1/leaderboards	GET	any	Every leaderboard, id-ordered: { id, sort, operator, reset_schedule, records } (records is the current record count).
/console/v1/leaderboards	POST	admin	Create a board. Body: { "id": "...", "sort": "asc"|"desc", "operator": "best"|"set"|"incr", "reset_schedule": "..." } — sort defaults to desc, operator defaults to best. 409 on a duplicate id. Audited as leaderboards.create.
/console/v1/leaderboards/{id}	DELETE	admin	Delete a board and every one of its records. 404 if unknown. Audited as leaderboards.delete.
/console/v1/leaderboards/{id}/records	POST	admin	Submit a score. Body: { "user_id": "...", "score": 0, "subscore": 0, "metadata": {} } — subscore defaults to 0, metadata must be a JSON object when present. Audited as leaderboards.record.submit.
/console/v1/leaderboards/{id}/records?limit&offset	GET	any	A ranked page: { board, items: [{ rank, user_id, score, subscore, metadata, updated_at_unix_ms, submissions }], total }. limit defaults to 50 (cap 500); offset is a rank offset (0 starts at rank 1).
/console/v1/leaderboards/{id}/records/{user_id}	DELETE	admin	Delete one user’s record. 404 if unknown. Audited as leaderboards.record.delete.

Ranking. Records are ordered by (score, subscore) in the direction sort prefers — ascending for asc, descending for desc — then by user_id ascending as a final, deterministic tie-break. Rank 1 is always the best record.

Operators decide how a new submission combines with a user’s existing record:

• set unconditionally overwrites score, subscore, and metadata.
• incr adds the submitted score and subscore to the existing totals (initializing at zero on a user’s first submission) and replaces metadata.
• best keeps whichever of the existing and submitted (score, subscore) pair is better for the board’s sort: lower is better for asc (a tied score prefers the lower subscore), higher is better for desc (a tied score prefers the higher subscore) — the same direction that governs ranking, so the record that would rank highest wins either way. A losing submission still counts toward submissions but leaves the stored score and metadata untouched.

POST /console/v1/leaderboards
{ "id": "points" }

201 Created
{ "id": "points", "sort": "desc", "operator": "best", "reset_schedule": null, "records": 0 }

POST /console/v1/leaderboards/points/records
{ "user_id": "u-1", "score": 90 }

200 OK
{ "user_id": "u-1", "score": 90, "subscore": 0, "metadata": null,
  "updated_at_unix_ms": 1751792000000, "submissions": 1 }

GET /console/v1/leaderboards/points/records

200 OK
{ "board": "points", "total": 1,
  "items": [ { "rank": 1, "user_id": "u-1", "score": 90, "subscore": 0,
               "metadata": null, "updated_at_unix_ms": 1751792000000,
               "submissions": 1 } ] }

Known limitations

• In-memory only. Boards and records live in process memory; a node restart clears every leaderboard. Durable persistence is tracked as known technical debt.
• reset_schedule is stored, not executed. The string round-trips through create/list responses but Citadel does not parse or run it — no board ever resets automatically yet.

Purchases & Subscriptions

Validated in-app purchase records behind a pluggable receipt-validator seam. Today the node ships the deterministic dev validator: the “receipt” is a JSON document { "transaction_id", "product_id", "subscription_expiry_unix_ms"? }, validated with no network calls — honest for prototyping and tests. Real App Store / Google Play validators are a recorded follow-up (they need outbound HTTPS and store credentials).

Route	Method	Role	Purpose
/console/v1/purchases?user_id&limit	GET	any	Newest-first validated purchases: transaction id, user, product, store, SHA-256 receipt fingerprint, validation time.
/console/v1/purchases	POST	admin	Validate + record a receipt: { "user_id", "store": "apple"|"google"|"huawei"|"custom", "receipt": "…" }. A replayed transaction id is a 409; a malformed receipt is a 400. Audited as purchases.validate.
/console/v1/purchases/{transaction_id}	GET	any	One purchase; 404 when unknown.
/console/v1/subscriptions?user_id&limit	GET	any	Subscription rows with active/expired derived against the read-time clock.

The raw receipt is never stored — only its SHA-256 digest — and records are in-process (a restart clears them; recorded technical debt).

Error Journal

GET /console/v1/errors?offset=0&limit=100 returns retained local incident summaries newest first. Both admin and viewer roles can inspect it. The backing citadel-errors.jsonl file lives beside the server executable; the configured [errors] retention limits prune its oldest records.

Each entry contains fingerprint, kind (error or panic), category when applicable, component, a generic redacted message, first/last-seen timestamps, and count. Raw panic payloads, internal error detail, request data, credentials, and the Sentry telemetry DSN are never returned. Matching incidents are aggregated by fingerprint so recurring failures remain readable.

The response is { "entries": [...], "total": <number>, "next_offset": <number|null> }. The console’s Error Journal page presents this data. See the configuration reference for retention and optional Sentry telemetry.

Audit Logs

Every console mutation — and every login attempt — is recorded in an audit trail: time, actor, role, action, target, and a sanitized detail line (never passwords, tokens, or raw payloads). Read it newest-first:

GET /console/v1/audit?limit=100&actor=admin&action=storage
Authorization: Bearer <token>

200 OK
{
  "entries": [
    { "time_unix_ms": 1751791000000, "actor": "admin", "role": "admin",
      "action": "console.login", "target": "console",
      "details": "login succeeded (admin)" }
  ],
  "retained": 1,
  "capacity": 1024
}

Query param	Meaning
limit	Page size, newest-first. Default 100, capped at 500.
actor	Exact actor (username) match.
action	Action prefix match — storage matches storage.write.

Actions are dotted verbs: console.login, console.login_failed, and one verb per mutation as sections land (e.g. storage.write, accounts.ban). Both roles may read the trail; unknown query parameters are rejected with 400.

A bounded in-process ring (1024 entries) always fronts the trail, so the newest history is readable with no database at all. On the SQLite, PostgreSQL, and CockroachDB backends the ring is also written behind to a console_audit_entries table and survives a restart; on the in-memory and MongoDB backends it does not, and the ring is the whole history. See Audit log for retention, the durability flag, and the full parameter set.

Errors

Console errors use the shared JSON error shape:

{ "code": "authentication_failed", "message": "authentication failed" }

Status	Code	Meaning
400	invalid_request	Malformed body or parameters.
401	authentication_failed	Missing/invalid credentials or token (uniform).
403	forbidden	A viewer attempted a mutation.
501	not_implemented	Section backend not landed yet.

Test coverage

The login flow, role assignment, bearer guard, uniform 401 behavior, stub routing, and the public reachability of /status, /health, and /dashboard are covered by tests/console_api_auth.rs; token issue/expiry/ revocation and the constant-time credential comparison are unit-tested in src/services/console.rs. The Groups section’s full membership lifecycle (create, list/filter, detail, add/promote/demote/kick, the last-superadmin guard, update, delete, the viewer-403 boundary, and the audit trail) is covered by tests/console_groups.rs; role-ladder and store invariants are unit-tested in src/services/groups.rs. Chat channel/history bounds, tombstone semantics, paging, and the moderation endpoints are covered by unit tests in src/services/chat.rs and src/http/console_api/chat.rs, plus the end-to-end integration test tests/console_chat.rs. Notifications (send/list/delete, targeting vs broadcast visibility, role gating, audit trail) are covered end-to-end by tests/console_notifications.rs, with store semantics (ordering, eviction, validation) unit-tested in src/services/notifications.rs. Leaderboard operator semantics (best/set/ incr under both sort orders), ranking with ties, and duplicate-id/not-found errors are unit-tested in src/services/leaderboards.rs; the full console lifecycle (create, submit, rank, delete, viewer 403s, audit) is covered by tests/console_leaderboards.rs.

Known limitations

• Console tokens are in-process: restarts log operators out (by design), and in a future multi-node cluster a token is only valid on the node that issued it.
• There is no lockout/rate-limit on login attempts yet; run nodes behind a trusted network boundary until that lands.