NYC Concordance: registered claims and their outcomes
This collection holds the claim register of team Hypergraph (Jeff Nickerson and Ida Benedetto) for the System Public Challenge of November 2026, Build Challenge 4. The challenge asks what published research, as held in the System Public Graph, and New York City's own data can confirm, question and connect. Each claim is a prediction about city data that was written down and committed before the data were read. The collection belongs to the Underlay organization system-community (System Inc. Community), where the challenge's teams publish their work.
| Type | Records | Id |
|---|---|---|
| Claim | 94 | the claim id, for example heat-shootings |
| Outcome | 93 | the claim id |
| Revision | 88 | claim id, /r and a sequence number, for example heat-shootings/r1 |
One claim, construction-2022-rush-units-2027, has no outcome yet: it is a prospective claim about completions by 30 June 2027. The current verdicts are 33 confirmed, 19 contested, 36 unsettled, 1 bridged and 4 no_test.
Registration before data
A registration states the hypothesis, the research evidence, the predicted direction and size, what result would support it (support_if), what result would refute it (refute_if), the rival explanations and the registrant's prior. It was written before any city data were queried for the claim. The registering code wrote the time and a hash into the file, and the file was committed at once. A registration is never edited. A changed plan is a new claim that names the old one in supersedes.
A Claim record gives four ways to check the order. registration_text is the exact text of the file and registration_file_sha256 is the SHA-256 of that text. registration_sha256 is the hash the code wrote into the file: the SHA-256 of Python's json.dumps(d, sort_keys=True), where d is the parsed file without its sha256 and registered_at fields. Every Outcome repeats the hash of the registration it was scored against. registration_commit is the git commit that added the file. prereg_branch is a branch named prereg/<id> that was pushed to the remote at registration, with its commit; 64 of the 94 claims have one, and the others were registered before the project pushed such branches.
Verdicts
An Outcome's verdict is confirmed, contested, unsettled, bridged or no_test. The rules are the text PRECEDENCE in concord/claims.py, at commit 3b42e042eff03417536304e47440db057288f14d of the working repository. Rule 1: the registration's own support_if and refute_if decide the verdict when they are met, even when the city interval includes zero. Rule 2: when neither is met, the verdict is unsettled. Rule 3: when the registered test could not be run as planned, the verdict is no_test. Rule 4: a measure chosen after the data were seen never decides a verdict. Later rules cover bridges (the stance of a chain is that of its weakest link), the cases where no stance is defined, and placebos (when a registered placebo shows an effect as large as the estimate, the verdict is unsettled). Rules added after the first verdicts say so in the text and name the verdicts they changed.
city_effect is the estimate that was scored, with its interval, scale and no-effect value. stance is a number between -1 and 1 for the agreement of the city evidence with the research direction. sources lists the data each outcome rests on. A New York City Open Data source gives the dataset id, rows_updated_at, the exact query and the fetch time. The city rewrites many tables daily, so rows_updated_at names a table state that the city no longer publishes.
Referees and revisions
Outcomes were given to an independent referee, an agent that had not done the analysis and that reran the deciding number where it could. referee_report gives the report's path and SHA-256, critique gives the answer, and notes_referee gives what the referee found, where it was recorded. 87 of the 93 outcomes have a referee report. placebo_estimate (14 outcomes) and referee_value (22 outcomes) are in the units of city_effect; placebo_estimate.registered says whether the placebo was named in the registration.
A Revision record is one change to an outcome after criticism. before holds the verdict, city effect, stance and scoring as they stood, and reason and critique say why the outcome changed and which report led to it. Of the 88 revisions, 12 changed the verdict. The outcome file keeps the reason of its latest revision only; 7 earlier reasons were read from the git history of the file.
Lineage audit
An audit on 13 November 2026 compared the fetch time of every city-data source with the time of registration (docs/lineage_audit.md). In 46 claims at least one source was fetched before the claim was registered. lineage.class gives the most serious class among a claim's early fetches, and lineage.early_sources gives each one with the evidence for its class.
| Class | Meaning | Claims |
|---|---|---|
| d | the outcome data were read for this claim before registration | 2 |
| e | cannot tell | 1 |
| c | a shared cache: the query was fetched for an earlier claim and holds this claim's outcome | 10 |
| a | exposure or treatment only, not the outcome | 20 |
| b | a design, power or calibration read, pre-period data, or metadata | 10 |
| f | no early read of city data (a date-only or misparsed fetch time, or a published benchmark) | 3 |
| none | the audit found no early fetch | 48 |
Two claims read their outcome data first. For afford-gentrification-prevalence (confirmed), a faulty simulation multiplied the real later-year values by noise and the registrant read its output; the registration discloses this. For heat-extremes-downturn (contested), the four deciding series were built before registration; the registrant states that only yearly totals and mean daily counts were read. A fetch time shows that the data were on disk. It does not show whether anyone looked at them.
Replication packages
12 claims have a replication package, named in the Claim's replication field. A package copies the inputs of the deciding number, pins each by SHA-256, and reruns the number from the copies. status is the result of that rerun: MATCH means the rerun gave the recorded number within the stated tolerance. All 12 packages gave MATCH when checked on 13 November 2026 (docs/replication.md).
Code and provenance
The code, the register, the referee reports and the replication packages are in the public pull request https://github.com/SystemPublic/system-public-challenge-2026/pull/3, under projects/hypergraph/. Paths in the records are relative to that folder. The working repository, nyc-concordance on GitHub, is private for now; the commit hashes and prereg/ branches in the records refer to it and can be checked when it is opened. This version was built by tools/underlay_export.py at commit 7a86412643eb2cc4f4a4cdceda00ef6a3ea9065f of the working repository.
Licence
The registrations, outcomes and reports are the team's own work and are offered under CC BY 4.0. The city data they describe are public records of the City of New York and are not included here. The System Public Graph, which the registrations quote, is licensed CC BY-NC 4.0 by System Inc.