hi, i'm rob

giteablackbeard/observer

Rust 64.7%TypeScript 30.6%CSS 2.4%Shell 1.4%Go Template 0.6%HTML 0.1%Python 0.1%JavaScript 0.0%

blackbeard.observer

Live miner leaderboard and network hashrate for the Quantus blockchain and its Planck testnet, at https://blackbeard.observer (blackbeard.internal from the mesh).

The point of it: a miner can find their own row, see what share of the network they are actually winning, and watch it move — against everyone else, in real time. Mining is a competition that normally gives its participants no scoreboard. This is the scoreboard.

How it knows who mined what

Every block header carries its author's 32-byte wormhole reward preimage in a PreRuntime digest stamped with the engine id pow_. The node writes it there so the mining-rewards pallet can read it back and derive the payout address on-chain — and as a side effect, authorship for every miner on the network is derivable from headers alone.

That is the whole mechanism, and its consequences are worth stating plainly:

  • No registration, no opt-in, no way to be left out. Every miner appears the moment they win a block. Nothing here depends on anyone's cooperation.
  • The preimage is public, not secret. It is published from a miner's first authored block onward. Spending the rewards needs a plonky2 proof of knowledge of the underlying secret, which the preimage does not reveal — but it does make mining income permanently attributable. The privacy in this design is at the exit, not at receipt. lair/quantus's doc/wormhole-rewards.md has the full analysis.
  • A leaderboard row is a preimage, not a person. One operator may run several; the count of distinct preimages is a lower bound on miner count.

Difficulty comes from the QPoWApi_get_difficulty runtime call. Difficulty is the expected number of hashes to win a block, so network hashrate is difficulty divided by the block interval — and a miner's implied hashrate is their share of recent blocks times that.

The mechanics were first worked out in lair/quantus's Prometheus arena exporter (asset/arena/quantus-arena-exporter.py), which remains the reference for anyone checking this implementation against a second one.

Names

The chain gives us preimages; a board of 64-character hex strings is honest and unreadable. substrate-telemetry closes the gap indirectly.

The feed reports block imports with a propagation time, stamping the first reporter of a hash with 0. A node imports its own block before announcing it, so the author — if it is on telemetry at all — is nearly always first. One block proves nothing (measured on Planck, an author's lead over the second reporter is 50–620 ms, which a well-peered bystander can produce), so the observer votes over a rolling window and only shows a name once enough blocks agree.

An attributed name is therefore an inference, never a claim the miner made and never an identity check. The UI marks it as such, carries the confidence in a tooltip, and falls back to the abbreviated preimage rather than guessing. Where two miners resolve to the same node name — one operator with several preimages, or two people who never renamed their node — every member of the colliding group gets a preimage suffix, so no row is silently privileged as "the real one".

Layout

Cargo workspace plus a Vite frontend, per ~/git/architecture/generic.md §1.

crates/
  blackbeard-entities/   domain types + the browser wire protocol; ts-rs exports
                         the TypeScript the frontend consumes. No I/O.
  blackbeard-core/       header/SCALE decoding, hashrate maths, the rolling
                         window, telemetry attribution voting. Pure — no I/O,
                         no clock, no sockets, exercised entirely by unit tests.
  blackbeard-data/       chain JSON-RPC, the telemetry feed, Postgres.
  blackbeard-api/        the daemon: ingest tasks, REST, WebSocket fanout.
  blackbeard-cli/        operator tools: probe, backfill, standings.
web/                     Vite + React + SWC + TS. Static build, served by nginx.
asset/                   systemd, firewalld, nginx, config template, bootstrap SQL.
script/infra-setup.sh    one-time host provisioning, operator-run.

The frontend's types are generated from the Rust. cargo test -p blackbeard-entities writes web/src/api/generated/; CI fails the build if the committed output is stale. Edit the Rust DTO, never the TypeScript — otherwise the two compile cleanly and disagree at runtime.

Real time, without observables

One WebSocket per browser tab. On subscribe the client gets a full snapshot; after that only deltas — a block on every block, a summary when the headline numbers move, a leaderboard when the standings change. The page never polls and never refetches.

The head stream is itself a push: chain_subscribeNewHeads over the node's WebSocket, so the observer learns of a block the moment the node imports it. The only polling anywhere is difficulty and sync state, on a four-second timer.

RxJS was considered and not used. This needs one stream, one reducer and one subscriber list; useSyncExternalStore is React's own contract for exactly that, gets concurrent rendering right, and costs no dependency and no idiom in every component. The store is web/src/api/socket.ts — about 200 lines including the reconnect and resubscribe logic.

Server-side, messages are serialised once per broadcast and every socket writes the same Arc<str>; leaderboards are recomputed only for windows that actually have a subscriber.

Windows are block counts, not durations

Every window on the site is a block count — 600, 3 600, 14 400, 100 800 — and since those are what is windowed on, they are also what the selector says and what the URL carries: /quantus/3600, not /quantus/six_hours.

The duration names came off because they were a promise the site could not keep. A window is a fixed number of blocks, so how far back it reaches is whatever the chain's rate makes it. When Planck's miners left for mainnet its rate fell ninefold, and /planck/six_hours went on saying six hours over a window that spanned twenty-nine — with a miner that had not touched the chain in a day sitting in the standings looking current. The name in the address bar was the reason a reader believed it.

The duration is still shown, next to the count, but it is now measured: the authored-time span of exactly the blocks tallied, which is the same figure that is already the denominator of every per-miner hashrate. It carries no "", because nothing is being estimated. Only the windows that are not being displayed still show an estimate from the current interval, and those keep the "". The old URLs still resolve — the router rewrites them to the block count — so links already shared keep working.

This is not pedantry. A duration is meaningless while a node is catching up: it imports historical blocks at disk speed, so "the last hour" can contain half a million blocks. The same reasoning is why a measured block interval is only used when the node is at the tip; while syncing, the configured target is used and every hashrate on the page is labelled nominal.

The sparklines under the headline tiles hold the same line. They are bucketed by height, not by time: every point is the same number of blocks, so every point carries the same statistical weight and a stretch where nothing was recorded stays the same width on the axis as one where everything was. A bucket is the window divided by fifty, which is why every window is a multiple of it — a full window is then exactly fifty buckets, and the rolling miner count at the last point is the same number printed in the tile above it.

Their block interval comes from the author's own timestamp inherent rather than from when this observer saw the block, for the same reason the measured interval only counts tip observations: a batch of blocks fetched from history all carry very nearly the same observation time, and dividing by that reads as a chain producing hundreds of blocks a second. Each point is a moving average over three buckets, because the mean of one bucket's gaps still carries enough Poisson noise to draw a chain that looks like it is changing size every few minutes.

A block has two addresses, and only one of them is stable

/quantus/block/1069799 is a position. Because block is keyed on (chain, height) and a reorg overwrites, a link to a height quietly comes to mean a different block the moment the chain forks there. So opening one resolves it and rewrites the address to /quantus/block/0x…, which names one block for good — including after it has lost, which is the only way an orphan is linkable at all.

The page is assembled from both sources because neither is sufficient. The node holds the block and forgets it: blocks-pruning defaults to archive-canonical, so a losing fork's body goes once finality passes it, and difficulty is a state read behind a 256-block window. The observer holds what the node never had — when the block was seen, and whether that sighting was at the tip — and what it has since forgotten, the difficulty recorded while the state still existed. Anything neither can answer says so in words rather than going blank, because a gap on a block page reads as a zero.

A runtime describes itself

/quantus/runtime/152 is every pallet, call, event, error, storage entry and constant the runtime declares — with the constants' values, decoded against their own declared types. Nothing on that page is transcribed from a source tree, which is the distinction worth having: a source tree says what the chain should be running, and metadata says what it ran, at a height, possibly months after it was upgraded past.

The set of runtimes is found rather than waited for. Decoding caches a runtime the moment it needs one, so left alone the cache is "whatever the backfill has walked past" — days, on a chain a million blocks deep. spec_version only increases, so the upgrade boundaries are a sorted sequence and bisection finds each in log₂(height) probes. On Heisenberg that turned up six runtimes and the block each took over at: v126 from 1, v128 from 132, v131 from 342,813, v136 from 669,129, v144 from 812,055, v148 from 977,079.

It runs as its own task, not in the poll loop, and the reason is worth recording: state_getRuntimeVersion at an old block makes the node load and instantiate the runtime WASM out of that block's state, which measured four seconds against Heisenberg's public endpoint against a quarter of a second at the tip. A hundred of those inside a four-second poll loop would stall difficulty, the summary broadcast and the telemetry attach for minutes on every start.

Reading the two ends of Heisenberg's history side by side is the case for the page existing. Between v126 and v148 the chain dropped Referenda, ConvictionVoting, Recovery, Assets and AssetsHolder, added Vesting and Origins, gained a WeightReclaim signed extension and moved ChargeTransactionPayment after the two Quantus-specific ones. Every one of those is a change that breaks a decoder written against the other version, and none of them is discoverable from a block.

Two renderings are not what the registry literally says, both for the same reason and both identified by registry path rather than by shape: AccountId32 becomes 0x hex rather than thirty-two numbers, and U256/U512 become one decimal rather than four or eight little-endian limbs. Difficulty as [1189189, 0, 0, 0, 0, 0, 0, 0] is a correct description of the bytes and tells a reader nothing.

Money in flight

/quantus/reversible shows scheduled transfers that have not yet landed, counting down to the block they execute at. ReversibleTransfers is the most distinctive thing this chain does — a transfer waits out a delay, default 7,200 blocks, during which its sender or a nominated guardian can call it back — and no other explorer can show it, because no other chain has the pallet.

It is the only view here built from both halves of the observer, each authoritative about a different thing:

  • State — PendingTransfers — says what is still pending. A cancelled or executed transfer is simply gone from the map, and that absence is more trustworthy than replaying every event since genesis and reconstructing the set, which fails silently when it fails.
  • The event index says when each is due, because TransactionScheduled carries execute_at and the stored struct does not. A transfer scheduled before the index reaches shows as pending with an unknown deadline rather than being dropped for lacking half its story.

execute_at is a DispatchTime<BlockNumber, Moment> — an enum, so it names a block on one arm and a timestamp on another, and reading the wrong arm would present a millisecond timestamp as a height. Time remaining is an estimate from a block interval that moves; the block count is the fact, and the page says which is which.

Enumerating a map needs state_getKeysPaged on the entry's prefix — there is no list, only keys derived from the things in it — and recovering each key from its storage key needs the hasher to have kept it. Blake2_128Concat and Twox64Concat do; Twox128 and the rest do not, so StorageMap::key_offset is None there rather than a guess, and such a map can be counted but not attributed.

A block is what was asked and what happened

The block page lists its extrinsics with each one's own events beneath it, and the block's own events in their phases. An event's phase says whether it belongs to an extrinsic or to the block, and extrinsic_index says which — that join is why both columns are stored, and it turns a block from two lists into one account of what it did.

Block 1 on this chain is the argument for it. Showing only extrinsics, it is a single Timestamp::set and nothing else. Its events are the whole opening distribution:

EXTRINSICS (1)
  0  Timestamp::set                    1788943917807
    42  Vesting::LaunchMomentSet       1788943917807

INITIALIZATION (21)
   1  Wormhole::NativeTransferred      3 QTC          → qzmtKfCX…z6HW
   …
  35  Wormhole::NativeTransferred      5,669,940 QTC  → qzmviwoP…nxW7
   …

FINALIZATION (6)
  45  System::NewAccount · Balances::Endowed · Balances::Minted
  49  Wormhole::NativeTransferred      0.3 QTC
  50  MiningRewards::MinerRewarded     0.3 QTC
  51  ZkTree::TreeGrew                 3

Two things only visible this way. The chain's opening distribution is twenty-one wormhole transfers in Initialization, owned by no extrinsic at all and sent from the minting account — not block zero, and not anything a transaction did. And Vesting::LaunchMomentSet fires here, so vesting has been exercised even though the call index shows zero Vesting::* dispatched: a pallet the runtime uses looks unused from the call side alone.

Accounts the chain names, and block zero

Some accounts are special and nothing about the address says so. The treasury looks like any other empty account; a minting sentinel looks like a wallet; the accounts endowed at genesis look like they earned it.

Nothing here is curated. The chain names them itself, in three places that are three different claims and are deliberately not flattened into one badge:

  • A runtime constant — compiled in, immutable for that spec_version. MiningRewards::MintingAccount and Wormhole::MintingAccount, both 0x0101…0101.
  • A storage value — assigned, and changeable by whatever call the pallet provides. TreasuryPallet::TreasuryAccount.
  • A balance in block zero — permanent history, and not a role. An account can be endowed and have no job.

Every label is derived from the chain's own name — TreasuryPallet:: TreasuryAccount becomes Treasury — so a pallet added next year gets its accounts labelled without an edit. The chip is the claim and the citation beneath it is the evidence, because "named by a constant" and "named by state" mean different things about how permanent the arrangement is.

Two rules hold this together. An account is told from a hash by registry path, never by shape: after normalise both are 0x and sixty-four hex characters, and filtering by shape claims System::ParentHash names an account. And an entry must be an account, not merely contain one — System::Events is full of them, none named by it, and treating "contains" as a role labelled half the chain's active addresses Events on the first attempt.

A chip states what the chain says and does not endorse it. MintingAccount has no System::Account entry at all; labelling it as what the runtime calls it is a fact, and implying it holds funds would be the site vouching for an address.

Block zero

Endowments have no extrinsic and no event. They are balances the chain was born holding, invisible to anyone who has not read the chainspec — so /:chain/block/0 carries them, and a Genesis link sits in the section nav to give somebody who would never think to look an unmissable way to.

Mainnet started with 21 accounts and 5,670,000 QTC, of which one account holds 5,669,940 and the other twenty got 3 each. The page shows the endowment beside what each holds now, so what a founding account did with its stake is one row.

Genesis state lives only on an archive node. When it cannot be read the page says so rather than showing an empty table, because missing data and a chain that endowed nobody are not the same claim.

State: what is true now

Everything else here is history — headers, events, extrinsics, a record of what happened. /quantus/state is the other half, and it exists because of a question history could not answer.

Which address is this chain's treasury? TreasuryPallet::set_treasury_account reads never on the call index and TreasuryAccountUpdated reads never on the event index. Both are true: the address was set at genesis, so no extrinsic ever carried it and no event ever announced it. It exists only in state, and until the observer could read state the honest answer was that we did not know. It is qzjsuLN7Nhu4bjvmUbjSTr2ZTeZ7oRxXpQP9fdv6PcHUCRrVR, and it has never been funded — no System::Account entry at all.

A storage key is twox128(pallet prefix) ++ twox128(item) ++ hashed keys, and which hash each key uses is declared per entry: System::Account is Blake2_128Concat, another map is Twox64Concat. Picking wrong yields a key that reads as absent rather than as an error, which is why Runtime::storage_key computes every part of it from the runtime's own description — including the pallet's storage prefix, which is usually the pallet name and is not required to be — and why its test pins two keys against ones read off the live chain by hand.

Absent is not zero, and the metadata is what knows the difference. An optional entry holding nothing means nothing; a default entry holding nothing means the value the runtime supplies, which the page marks as default rather than passing off as something the chain wrote. The same distinction is why an unfunded account shows — and "no account on chain" rather than a balance of zero: Substrate reaps an empty account, so a missing entry means it does not exist, and a zero would say it had been funded and spent.

Read live on every request. State is what is true now, and a cached copy of now is a copy of some earlier now wearing the same face.

What the chain can do, and what it has done

/quantus/event is its other half, and the one a chain analyst reads for narrative: 19 of 107 event kinds have ever fired. A dispatchable says what somebody can ask for; an event says what the runtime does, including the parts nobody dispatches directly — a vesting schedule ending, an account reaped, a proof verified. Vesting::VestingCompleted at zero and System::CodeUpdated at zero are two different statements about a chain's life, and neither is visible on an explorer that lists only what happened.

Which is why nothing is excluded from the index for being unused. The event interest list is a denylist, so a pallet the chain grows next year is indexed the day it ships rather than the day somebody remembers to add a line. The rule for exclusion is narrower than volume and narrower than usefulness: an event carrying no account can never answer "everything involving this account", which is what the index is for, so the three that both name no account and fire every block are skipped — ZkTree::LeafInserted, QPoW::DifficultyAdjusted, System::ExtrinsicSuccess. They show as not indexed rather than as a zero, because a zero would read as disuse.

Balances::Minted looked like a fourth candidate — it fired exactly as often as MiningRewards::MinerRewarded across a 120-block sample, and for the same reason. It is kept: it names an account, minting is not only for miners, and "who received newly issued tokens" is worth being able to ask even in a month where the answer is only the miners.

That widening took the index from ~1.2 to ~3.4 rows per block measured on live mainnet, and bought 739 accounts' worth of wormhole transfers that were previously invisible.

/quantus/call is the one page here that does not exist elsewhere. Every explorer shows activity; this shows activity against the declared surface, because the runtime describes all 58 of its dispatchables in the same metadata the decoder already reads. The join is free and the gap is the interesting half:

quantus  v152    6 of 58 calls used
heisenberg v148  6 of 58 — and a different six

ReversibleTransfers, TechReferenda, Vesting and Preimage are shipped, documented, and untouched. "This chain has governance nobody has used" is a different statement from "this chain has no governance", and only one of them is visible on an explorer that lists what happened. The unused rows keep their signature and their documentation, both straight from the chain rather than from a source tree.

Two chains diverging on the same runtime family is the other thing it shows: mainnet has exercised the wormhole batch verifiers and nothing else, while Heisenberg has a multisig lifecycle — create, propose, approve, execute — and no wormhole traffic at all.

Nothing on the page is written per call. /quantus/call/Utility/batch_all feeds from the same generic index and renders its arguments through the same Payload, so a pallet added next year has a working page the moment somebody uses it — and a row on the index the moment the runtime declares it.

Extrinsics: what was asked, not only what happened

An event is the chain's account of what happened. An extrinsic is the request that caused it, and it carries what no event does: who signed, what they actually called, the nonce and tip they set, and whether it worked. A history built from events alone silently omits every failed attempt and never names a submitter.

The same oracle does both. The metadata's extrinsic type carries four parameters and the registry names all of them — Address is a MultiAddress, Call is RuntimeCall, Extra is the twelve signed extensions, and Signature is qp_dilithium_crypto::types::DilithiumSignatureScheme. That last one is the part that would otherwise need hand-written post-quantum knowledge, and it does not: the chain describes its own signature scheme, so decode_extrinsic reads a Quantus transaction without a line of code that knows Quantus exists.

Two things about this chain's extrinsics are worth knowing before touching them. The first byte is not the version — the top two bits are a type tag, and mainnet carries 0x84 (signed, v4) and 0x05 (bare, v5) in the same block while the metadata declares version 4. And a signature is 5.3 KiB, two orders of magnitude larger than the call it authorises; it is decoded to find where the call begins and then thrown away, with only the scheme's name and the byte count kept.

Both halves of a block are indexed in one pass, because both come from calls already being made: the events blob for chain_event, the body for chain_extrinsic and the block's own timestamp. Whether a dispatch succeeded comes from the System::ExtrinsicSuccess and ExtrinsicFailed events in that same decoded list — read, used, and not stored, because two rows per block forever for one boolean each is not a trade worth making.

An account's page merges the two into one history ordered by block, with the extrinsic above the events it caused. source is 1 for an extrinsic and 0 for an event precisely so every part of the sort key descends, which is what lets the paging cursor be a plain three-part row comparison rather than a mixed-direction one Postgres cannot express.

Nested calls are the reason this is worth doing generically. A Utility::batch_all carries whole calls in its arguments, so a transfer's recipient can be three levels down — and because the decoder collects account ids wherever it finds them, that batched transfer appears on the recipient's page tagged received even though they signed nothing.

One account, from our own index

/quantus/account/qzp2AxZw… answers what the official explorer does not: what an address has actually been paid, every reward, back as far as the index has read. The address is converted to a raw account id at the edge, so a typo is rejected as malformed rather than answered with an empty history — which reads exactly like a real account that has never been paid — and a Polkadot address is rejected too: same thirty-two bytes, different network, and answering for it would show someone activity that is not theirs.

The activity table has no case per event type. Every chain_event row carries the accounts it mentions in a column the decoder filled — it knows which values are AccountId32 by their registry path, which is also how it avoids rendering a block hash as an address — so accounts @> array[…] finds everything, including from pallets that do not exist yet. The frontend labels and formats what it recognises and shows anything else as its own key and value, so a new event type appears as itself rather than not at all.

The preimage and the address are the two ends of one identity, and each page links to the other: the miner page counts blocks, the account page counts tokens, and the derivation joins them. Going the other way — address to preimage — is a lookup among preimages already seen, not a calculation, because the Poseidon2 derivation runs one way only.

Sections, so the addresses are findable

Every route here was reachable only by already knowing an address: a block hash, a preimage, an SS58 string, a spec_version. That is fine for a link somebody sent you and useless for finding out the pages exist. A kind with nothing after it is now the index of that kind — /quantus/block, /quantus/account, /quantus/runtime — and a section nav under the chain chips names all of them.

There is deliberately no /quantus/miner. The standings are the index of miners, and a second page listing the same rows under a different address would be two answers to one question; that path redirects there, as does anything else that resolves to the front page but is not spelled like it.

The three indexes are not the same page three times:

  • Blocks is the record behind the live ticker — the same rows from Postgres rather than memory, paged by height. Its gaps come from authored_at, the chain's own clock, not from observed_at: a gap fill writes a whole batch within the same second, and the first cut of this page reported three milliseconds between blocks on a chain targeting twelve seconds.
  • Accounts ranks by tokens earned over the whole indexed record, which is a different question from the standings' blocks-per-window and on this chain a different answer — difficulty rose 346-fold inside mainnet's first day, so an early block cost a three-hundredth of a current one and paid accordingly. It shows the address beside any node name, because one node legitimately reports for several payout addresses and the first version had two rows reading pearl-prover with nothing to tell them apart.
  • Runtimes is the only list with no other home at all: a block does not name its runtime and the standings do not either.

Every view has an address

Paths, not a hash: /quantus/day, /quantus/block/13160, /quantus/miner/0x…. A fragment is never sent to a server, so a hash URL cannot be given a title, cannot be crawled, and cannot be resolved by anything but the page itself — which was a fine trade while the whole surface was a chain and a window, and stopped being one once a block and a miner each became a page worth sending someone. Both vhosts already answer any path with index.html, so nothing outside the client changed. lib/routes holds the grammar, and fromLegacyHash rewrites the old form on load, so every #/… link already published still lands where it meant to.

Route segments are named — /quantus/block/13160 rather than /quantus/13160 — rather than told apart by shape. Shape works right up until two kinds of thing can look alike, and an SS58 address is arbitrary base58 with no rule keeping it clear of a window name forever.

A miner's page carries the window too: /quantus/miner/0x…/day. Its rank and hashrate are of a window, so a link without one means something different from what the sender was looking at — and until it was in the path, changing the window on a miner's page could only navigate back to the standings, because there was no URL for "this miner, over a day". The window control appears on exactly the two pages where it changes something, the standings and a miner; a block happened once and a balance has no span, and a control that does nothing where it sits teaches a reader to distrust it everywhere else.

Everything navigable is an <a>, including the chain chips and the window control. That is what "real URLs" has to mean in practice: middle-click opens the 24-hour view of another chain in a tab, right-click copies a link to a miner, and the back button walks the panels the way it walks pages. The one deliberate exception is a height resolving to its hash, which replaces rather than pushes — it is the same block under a better name, not a second place the reader has been.

No single node can stop a chain

A [[chains]] entry takes rpc_urls and ws_urls, tried in order. rpc_url and ws_url still work as a one-element list, because a config naming one endpoint is still a valid config.

They are stuck to, not balanced across, and that is the whole design. A storage read at an old block hash needs a node that still holds that block's state; nodes prune on their own schedules, so alternating between them would return a mixture of answers and absences that reads as sparse data rather than as a configuration problem. Requests stay on one endpoint until it fails, and the cursor is shared across every task on the chain so a failover one of them discovers is not rediscovered by the rest.

Failing over on the wrong thing is the trap. A JSON-RPC error is the node answering — count exceeds maximum value is a caller's mistake, and moving on it would hide that behind a second node making the same complaint. A pruned block is a success: state_getStorage returns {"result": null}, and treating that as a failure would walk every endpoint asking a question none of them can answer. Only transport failures and malformed responses move.

The WebSocket rotates at reconnect, where the loop already was. Holding subscriptions to every endpoint and deduplicating heads buys nothing: heads are a liveness signal and ingest fills gaps against chain_getBlockHash anyway, so a few seconds on the next endpoint costs a reconnect rather than data.

Verified against a chain configured with a dead endpoint first: Planck stayed full at its live height, and the failover logged once — from, to, and how many endpoints the chain has, because "the chain is unreachable" and "one of three endpoints is unreachable" are different operational facts.

Backfill is cheap, and bound by somebody else's node

Three chains walking their history at once — mainnet, Heisenberg and Planck's million blocks — costs about 5% of one core, holds five of eight Postgres connections with one active, and does not move API latency: summary at 7 ms p95, block at 13 ms. The walk is bound by RPC round trips, not by anything here, which is why it needs no deployable of its own.

It is also separable if it ever does. backfill only reads in-memory state — the tip height and the current runtime — and writes exclusively to Postgres, so it contends with nothing the API serves. Live ingest is the half that could not move: it feeds the window, the ticker and the attributions that the API answers from.

Progress survives a restart because event_scan is a table rather than a field. That is load-bearing rather than tidy: blackbeard-api-cert.path restarts the service several times a day when the host certificate rotates, and a cursor in memory would mean Planck never finishing.

Data

Postgres on magrathea.kosherinata.internal, mTLS and passwordless — the host's own certificate is the credential and pg_ident.conf maps its CN to the role. There is no password anywhere in this repo and nowhere in the config to put one.

Blocks are keyed on (chain, height), not on the block hash, deliberately: this is a proof-of-work chain and it reorgs. An upsert on that key means a block replacing another at the same height overwrites it, so the standings reflect the canonical chain rather than the union of every fork the observer witnessed.

What that overwrites is copied into block_displacement on the way past, so a contested height leaves a trace of who lost it. The node will not answer for the losing block for long — it keeps only canonical bodies once finality passes them, and difficulty is a state read behind a 256-block window — so the row written at the moment of the reorg is the only record there will be. It is not an orphan store: the block itself is gone, and only what the observer had already seen of it survives.

The database is what lets the site keep promises a rolling window cannot: "how did I do last week", a hashrate line older than the process, and a leaderboard that survives a deploy. On start the daemon replays the window and the held attribution names out of it, so a restart does not serve an empty site.

The runtime is the oracle

A header tells you who mined a block. It does not tell you what they were paid: the amount is remaining supply over an emission divisor plus collected fees, quantized with the dust carried into the next block, so it is only knowable from the MiningRewards::MinerRewarded event the runtime emits. Reading that means decoding SCALE against the type registry of the runtime that produced the block — and this chain's encodings differ from vanilla Substrate, because the post-quantum signature scheme changes the shapes.

Rather than transcribe those shapes into Rust and re-transcribe them after every upgrade, the observer asks the chain to describe itself. state_getMetadata at a block hash makes the node execute Metadata_metadata against the runtime WASM in that block's state, which returns metadata v14: every pallet, every event, every field, as a portable scale-info registry. blackbeard-core::runtime decodes against that registry and knows nothing about mining, rewards or transfers — INDEXED_EVENTS in ingest.rs is the only place a name appears, and adding transfers or multisig or whatever the chain grows next is one line there and a query, not a decoder.

Metadata is cached per spec_version in runtime_metadata, because a node that starts pruning would otherwise make every historical block undecodable. Blocks are decoded against the tip's runtime first — which costs no extra RPC call and is right for all but a handful of blocks — and decode_events refuses a partial read, so a block from before an upgrade fails rather than decoding into plausible nonsense. That failure is what triggers asking which runtime actually produced it. It is the property the whole approach rests on.

Events land in one table, chain_event, in the runtime's own vocabulary: pallet, variant, and the fields as jsonb. Account ids are normalised to 0x hex on the way in, and integers past 2^53 to strings — balances here are u128 and a JSON number would be silently rounded by the browser. A jsonb_path_ops GIN index over the fields is what makes "everything involving this account" a query rather than a table per event type.

ingest::backfill walks the chain outward from the tip: to the tip first, then toward genesis, one contiguous interval recorded in event_scan. Live indexing alone is not enough for an account page — a restart, a brief RPC failure or a gap fill that ran before the metadata was cached each leave a hole that nothing would return to. Progress is a cursor rather than max(height) from chain_event, because a block with no indexed event and a block never looked at are otherwise indistinguishable, and the first is the common case.

A miner's pow_ preimage and their reward event agree without a lookup table: wormhole::reward_address runs the same Poseidon2 derivation the chain does, and its output is exactly the miner field of MinerRewarded. Verified on mainnet block 13160 — preimage 0x02dacf59…, address qzowWAgb…, account 0xc6801725…, which is the event's own field.

Build and run

# A throwaway Postgres for the dev loop and for the sqlx query cache.
podman run -d --rm --name bb-pg -e POSTGRES_PASSWORD=dev \
  -e POSTGRES_DB=blackbeard -p 55432:5432 docker.io/library/postgres:18-alpine

export DATABASE_URL='postgres://postgres:dev@127.0.0.1:55432/blackbeard'
export TEST_DATABASE_URL="$DATABASE_URL"

cargo fmt --all && cargo clippy --all-targets -- -D warnings && cargo test --all

# The daemon, against a real node. BLACKBEARD_DEV_DATABASE_URL is the
# development-only path that skips mTLS; the systemd unit never sets it.
BLACKBEARD_DEV_DATABASE_URL="$DATABASE_URL" \
  cargo run -p blackbeard-api -- --config dev-config.toml

cd web && pnpm install && pnpm dev     # proxies /v1 to 127.0.0.1:25864

cargo test skips the database integration tests when TEST_DATABASE_URL is unset, so it stays useful on a machine with no Postgres.

Before adding a chain, probe it. Every failure the probe reports is one the deployed daemon would hit silently — an RPC without the QPoW runtime API gives a site with no hashrate, and headers without a pow_ digest give an empty leaderboard, both of which look like a working deployment:

cargo run -p blackbeard-cli -- probe --rpc-url http://bob.hanzalova.internal:9944

Deploy

CI-driven (architecture/deployment-gitea-actions.md): push to main, or run the workflow from the Actions UI. Hosts, ports and paths live in .gitea/workflows/deploy.yaml and nowhere else.

One-time host provisioning is operator-run, from a workstation with full sudo:

./script/infra-setup.sh --pubkey ~/.ssh/id_gitea_ci.pub

Re-run it whenever the deploy gains a new file to ship — each deploy job preflights the target's sudoers against the grants in that script and fails up front naming what is missing, rather than dying partway through an rsync.

Two things it deliberately does not automate, both because they touch shared infrastructure: the public Let's Encrypt certificate (external-tls.md) and the split-horizon blackbeard.internal record, which must be added to both site routers' Unbound or it NXDOMAINs everywhere but one site.

PortWhatWhere
25864blackbeard-api REST + WebSocketbob (hanzalova), mesh address only, plain HTTP behind nginx

The edge proxy is oolon (kosherinata) and the API is on bob (hanzalova), so /v1 is a cross-site hop over the mesh. That is deliberate — the API has to sit beside the node whose loopback RPC it reads — and it is the one hop a loopback health probe cannot see, so the deploy checks it explicitly.

Registered in architecture/port-allocations.md §5; derived from the service name per §3.

Mainnet

Quantus mainnet is not yet live. It is a commented-out [[chains]] block in asset/config/config.toml.tmpl, waiting on a published chain spec and an endpoint.

A chain may be configured before it launches: the observer reports it as awaiting, the UI says so plainly, and it comes alive on its own the moment the node starts answering. No redeploy, no restart. Adding it is a config change and a probe run.

Deviations from house convention

  • runs-on: infra for the deploy jobs, not fedora-43. The targets are mesh-only .internal names and the fedora runners have no route to the mesh. Same reasoning as lair/quantus and lair/mail.
  • Dark theme only. generic.md says nothing about themes, but the dataviz conventions expect a selected dark mode alongside light. This site commits to one look: it is a scoreboard for a proof-of-work chain, and a light variant would be a second palette to validate for a context this content does not have.
  • BLACKBEARD_DEV_DATABASE_URL bypasses the mandated mTLS Postgres connection. It exists so cargo run works on a workstation with no pg_ident mapping on the fleet cluster. It is an environment variable rather than a config key precisely so it cannot be reached by editing a deployed config, and taking it logs a warning loud enough to spot in the journal.

What this cannot tell you

  • Hashrate figures are estimates, not measurements. A miner's implied hashrate is their share of blocks won over a finite window times the network estimate. Winning blocks is a Poisson process: a miner holding 1% of the network will quite ordinarily show anywhere from 2 to 11 blocks in a 600-block window. Short windows flatter the lucky and libel the unlucky.
  • Distinct miners is a lower bound. One operator, several preimages.
  • Orphan and stale-work loss is not measured. Answering "do I win the share of blocks my hashrate predicts?" honestly needs a hashrate the observer cannot see. lair/quantus issue #1 tracks the same gap from the exporter side.
  • A telemetry name is an inference. See Names above.
  • History has holes, and shows them. Difficulty is a state read, so a pruning node cannot say what it was at a height fetched long after the fact, and a stretch the observer was not running for was never recorded at all. Both are gaps in the sparkline rather than a line drawn across them.

Related

  • lair/quantus — the node and miner deployment, the arena exporter these mechanics come from, and the measured hashrate numbers behind the fleet.
  • ~/git/architecture — the house conventions this project follows.

263 activities