/releases/wikimedia-adjacency-serving

Wikimedia observation release → durable serving

The public record of the Wikimedia observation corpus now served in production: what the immutable release carries, the storage-locality repair that took one entity page from roughly eight thousand origin reads to seven, and the measured serving result — including the one target that was missed.

Read the top for a 30-second explanation; the sections below preserve the exact technical record, the failures worth keeping, and the final disposition. This page follows release record #1.

In 30 seconds

What this is

Counterpedia serves 285,381 Wikidata entity pages over 1,129,159 observed nodes and 1,566,412 observed relations, derived from an 848,281-identifier acquisition and published as one immutable, content-addressed release. Every one of those pages is a provider observation, not an admitted Counterpedia record.

A remote measurement showed the reader was not viable over a network: one entity page issued about eight thousand bounded reads against the release and did not finish in over two minutes. The repair changed only storage locality — each subject's bounded working set was projected into one row of a new artifact, added to the release without touching any inherited byte — and the same page now costs seven reads. Surfaced output was proven identical, then the release was placed in durable custody and bound to production.

The release

What it carries

entity pages: 285,381; wikidata_entity_observed subjects
graph nodes: 1,129,159; observed
graph edges: 1,566,412; observed relations
search terms: 1,175,220
acquired identifiers: 848,281; the WIDEN0 target universe
artifacts: 10; 8 inherited byte-identical from release #1, plus 2 added

Posture on every page: Provider observation · Not admitted. Counts on this page are the release manifest's artifact row counts plus the WIDEN0 acquisition target, not estimates. The two added artifacts are a per-subject adjacency projection and its byte-offset index; they carry no new observation.

release: sha256:103e2391bb4bcfc06f941f535daa28a8f6954d694cc93b2a8b48d67118092fd0
source release (inherited, unchanged): sha256:5a62831ba7204a01ad18f95a42cb1f1df9e27401930ff91b232123cd782aac15
adjacency: sha256:9e459e3e17aa2f90f5bab367d26abbe50a97f1d9a775566ff425beab9f3170d0
adjacency index: sha256:4e689fcb717705a1fc09cc2effb389fd6411dac2ce8fcb8def711b6fe18cb885

The chain

Measured failure → locality projection → exact equivalence → immutable recut → durable serving

A real deployment reading release #1 over a network was measured, not assumed: one entity page issued ~7,800+ bounded reads and did not finish in over 130 seconds. Origin bandwidth was never the problem; the number of sequential reads was.
A deterministic producer materialised each subject's bounded working set — its edge buckets, the nodes the page resolves, its neighbourhood fetches, and the real resolver outcome for each navigation target — into one row. 285,381 rows, reconciled exactly to the subjects present in the release.
An equivalence oracle rebuilt every surfaced output from the row alone and compared it to the original readers: 5 / 5 representative subjects, 1,000 / 1,000 sampled subjects, and later 1,005 / 1,005 through the integrated readers — exact.
The release was recut immutably from release #1: all 8 inherited artifacts entry- and byte-identical, exactly two artifacts added, a new release identity, the source release never written.
The readers were switched to the projection as a read-source substitution only; the original path is kept as the independent oracle and can be forced with a switch.
The release was uploaded to durable custody, every artifact hashed back from the public origin, bounded reads proven, then bound to production.

Serving result

The same route, before and after

entity page, old path: 7,965 reads; 80.9 s · same build, same release, locality projection disabled
entity page, new path: 7 reads; 6 index blocks + 1 row · budget ≤ 32
graph page, new path: 7 reads; budget ≤ 24 · was 417
production Q42, cold: 2.64 s; was 61.9 s on the previous production deployment
production Q42, warm: 0.52 s
visible output: preserved; visible text and links identical; raw RSC chunk ordering may differ

Read counts were measured on a production-mode reader against the durable origin, one fresh process per cold run, counting only reader-issued bounded reads. Wall times are whole-route measurements of the real deployment; responses carried x-vercel-cache: MISS and no-store throughout, so no cache produced these numbers.

Target missed, recorded: the frozen acceptance contract set a ≤ 2 s target for a cold entity page. The durable-origin preview measured 2.56 s true-cold, and production measured 2.64 s — above the target, below the 5 s failure line. The read budgets, which were the mandatory gates, passed. Six sequential index reads are the recorded baseline for a later latency lane; they were deliberately not optimised inside this one.

Custody

Durable, verified from the outside

pre-upload audit: 10 / 10; hashes and lengths exact; no unlisted bytes
remote readback: 10 / 10; SHA-256 + length, streamed from the public origin
manifest: byte-exact; remote release identity equal to local
bounded reads: 206 · 4 / 4; exact Content-Range, exact bytes
total bytes: 12.66 GB
binding: index → adjacency; digest-bound, checked at read time

Uploading the release created no new release: the durable origin is custody for already-accepted immutable bytes, and the deployment receives URLs, not authority over them. The reader refuses a projection whose declared source digests do not match the bound release.

Boundaries

What these pages do not mean

provider observation ≠ admitted Counterpedia record
observed relation ≠ support or evidence
a resolved navigation target ≠ identity equivalence
page count ≠ corpus completeness
a locality projection ≠ new observation
faster serving ≠ verification, standing, or truth
a target miss recorded ≠ a gate passed
a release identity ≠ a claim about the world

Failures worth preserving · historical

What went wrong on the way, and what was kept

A field said less than it carried. An independent recheck of the 266 navigation targets the resolver could not find showed all 235 distinct targets were property and lexeme identifiers, correctly refused by a Q-item-only grammar. The field naming them qid was therefore a false machine-readable claim. It was renamed before the first release was canonicalised.
Deterministic content, nondeterministic bytes. The producer populated one map from concurrent callbacks, so key order followed I/O timing; two runs over identical input produced different bytes. The producer now sorts before constructing, the validator refuses unsorted keys, and a committed test runs the producer twice and requires identical artifacts. An equivalence oracle on surfaced output cannot catch this; only hashing two runs can.
Two uploads that reported success and stored nothing. The 3.6 GB adjacency artifact was uploaded twice by the standard command line tool with exit code 0 and, both times, no object existed. The trace showed one part's connection closing, the transfer aborting, and the tool still exiting clean. The third attempt was an explicit transaction — numbered parts, each verified against its local checksum, completed only when all 35 matched — and existence with the exact length was made the commit point. An exit code is diagnostic, not acceptance.
Manifest before objects. The manifest reached the new prefix before the largest object, so the prefix was briefly readable in an incomplete state. Nothing was bound to it and the bytes were exact, so nothing was rewritten to cosmetically fix the order; the lesson — all objects first, manifest last — is recorded instead.
Bindings stored where they could not be read back. The two production binding values were first stored as unreadable secrets by tooling default, which defeated the pull-back check that guards a production build. They were re-stored as readable configuration and verified before any build ran.

Disposition

Closed / green

WIKIMEDIA-ADJACENCY-PROJECTION0   = CLOSED
COUNTERPEDIA-PUBLIC-DEPLOY0       = CLOSED
PRODUCTION SERVING                = GREEN

See a live entity page (Q42) →Its bounded neighbourhood →Release record #1 →Machine-readable release manifest →