Claim at v1.2.0
restaurants-score-complaints
51a728d526d809076cda3a9a05d22531d5b3a2a46371d1e08b1cf3b8c25eb694
Data
{
"has_outcome": true,
"kind": "confirmation",
"lineage": {
"audit": "docs/lineage_audit.md; data/derived/lineage_audit.json",
"class": "a",
"class_meaning": "exposure or treatment only, not the outcome",
"classes": [
"a"
],
"early_sources": [
{
"class": "a",
"dataset": "43nn-pn8j",
"evidence": "Inspection scores are the exposure. Referee: \"The inspection extract that defines the exposure cohort ... was fetched at 14:08:53 and 14:09:07\"; 311 reads before registration were aggregates for the power check.",
"fetched": "2026-11-13T14:09:07-05:00",
"name": "DOHMH New York City Restaurant Inspection Results"
}
]
},
"outcome": "restaurants-score-complaints",
"prereg_branch": {
"name": "prereg/restaurants-score-complaints",
"sha": "a70335a4283e34f0e80c3048f3bcdee40cf7e81e"
},
"prior": 0.3,
"registered_at": "2026-11-13T14:12:00-0500",
"registrant": "claude (restaurants agent, 13 November 2026)",
"registration": {
"alternatives": [
"Volume: busy restaurants have more diners, so more complaints, and more chances for violations; the noise placebo and cuisine and chain controls address part of this.",
"Reporting propensity: neighbourhoods and clienteles that complain more to 311 also draw stricter inspections or are in areas inspected differently.",
"Persistent quality: a restaurant's hygiene is stable, so the association reflects the restaurant, not the inspection; the pre-window secondary shows this and does not make the result causal.",
"Survivorship: the live data keep only restaurants open now; restaurants that closed after a bad score and many complaints are missing, biasing the IRR towards 1.",
"Linkage: single-CAMIS BBLs are smaller buildings and differ from food halls and towers."
],
"graph_evidence": [
{
"association_id": "RzNvMLQc9u (variable) protection from contamination violation -> outbreak level",
"direction": "+",
"n_findings": 1,
"n_studies": 1,
"pubmed_ids": [
"31928421"
],
"summary": "Minnesota chain outbreak (Firestone 2020): a protection-from-contamination violation at the last routine inspection, OR 4.92 (1.57-15.39) for outbreak level; 54 restaurants."
},
{
"association_id": "FCDuRbXqiy (variable) evidence of vermin -> food-borne outbreaks",
"direction": "+",
"n_findings": 2,
"n_studies": 1,
"pubmed_ids": [
"11344897"
],
"summary": "Miami-Dade 1995 case-control (Cruz 2001): overall inspection outcome and number of critical violations did not differ between outbreak and control restaurants; only vermin, OR 3.3 (1.1-13.1)."
},
{
"association_id": "z1sSUpT_EZ (variable) complaint rates -- outbreak rates",
"direction": "+",
"n_findings": 1,
"n_studies": 1,
"pubmed_ids": [
"21375880"
],
"summary": "US local health departments (Li 2011): complaint and outbreak rates correlate, r = 0.38; complaints detect a median 69% of outbreaks."
},
{
"association_id": "CCatfE2F4k (variable) chain restaurants -- violations per inspection",
"direction": "-",
"n_findings": 1,
"n_studies": 1,
"pubmed_ids": [
"28060568"
],
"summary": "Philadelphia 2013-2014 (Leinwand 2017): chains 6.5 against 9.6 violations per inspection; used here as a covariate."
}
],
"id": "restaurants-score-complaints",
"kind": "confirmation",
"literature": [
{
"citation": "Irwin K et al. Am J Public Health 1989;79:586-590 (PMID 2705592)",
"doi": "10.2105/ajph.79.5.586",
"finding": "Seattle-King County, 28 outbreak restaurants and 56 matched controls: poor routine scores 5 times, temperature-control violations 10 times more likely in outbreak restaurants."
},
{
"citation": "Jones TF et al. Emerg Infect Dis 2004;10:688-692 (PMID 15200861)",
"doi": "10.3201/eid1004.030343",
"finding": "Tennessee 1993-2000, 167,574 inspections: mean scores of outbreak restaurants did not differ from others; inspector means ranged 69-92."
},
{
"citation": "Petran RL, White BW, Hedberg CW. J Food Prot 2012;75:2016-2027 (PMID 23127711)",
"doi": "10.4315/0362-028X.JFP-12-147",
"finding": "No overall association between routine inspection scores and outbreak inspection scores at outbreak restaurants; some specific violations more likely."
},
{
"citation": "Harrison C et al. MMWR 2014;63:441-445 (PMID 24848215)",
"doi": null,
"finding": "NYC Yelp pilot: of 468 illness reviews only 3% had been reported to DOHMH by phone or online, so 311 complaints are a small, selected share of illness."
},
{
"citation": "Wong MR et al. Am J Public Health 2015;105:e81-e87 (PMID 25602861)",
"doi": "10.2105/AJPH.2014.302404",
"finding": "NYC letter grading: probability of an A-range score on unannounced inspection +35% (31-40%) three years after launch."
},
{
"citation": "Firestone MJ, Hedberg CW. Emerg Infect Dis 2018;24:2164-2168 (PMID 30457518)",
"doi": "10.3201/eid2412.180544",
"finding": "Salmonella rates in NYC fell 5.3% a year relative to the rest of New York State after letter grading (2011-2015 against 2006-2010)."
},
{
"citation": "Jin GZ, Leslie P. Q J Econ 2003;118:409-451",
"doi": "10.1162/003355303321675428",
"finding": "Los Angeles grade cards raised hygiene scores and were followed by a fall in foodborne-illness hospitalisations (about 20%). Not in PubMed; abstract not read here, cited from the paper's published summary."
}
],
"plan": "Script analyses/restaurants/complaints_test.py at commit 0c147b3 (no resampling, no seed). Cohort: first Cycle Inspection / Initial Inspection with a score per CAMIS, 2023-01-01 to 2025-06-30, citywide live 43nn-pn8j, restricted to CAMIS whose 10-digit BBL holds exactly one CAMIS in the live data from 2022 on. Exposure: score >= 14 against 0-13. Outcome: count of erm2-nwe9 complaints, type Food Poisoning, location type Restaurant/Bar/Deli/Bakery, at the BBL, created on days 1-365 after the inspection. Model: Poisson GLM, HC1 robust SE, covariates borough, cuisine group (top 20), chain (dba with >= 10 CAMIS), inspection quarter. Placebo: same model with Noise - Commercial complaints at the BBL. Secondary, deciding nothing: C-range (>= 28) against A; per 10 points; NTA fixed effects; Manhattan only; complaints in the 365 days before the inspection.",
"prediction": {
"direction": "+",
"magnitude": "IRR (not-A against A-range) 1.10 to 1.60, midpoint 1.33. The literature is split: Irwin 1989 found poor scores predict outbreaks (about 5 times), Cruz 2001, Jones 2004 and Petran 2012 found no overall association. 311 complaints are a noisy, self-selected signal of illness (Harrison 2014), which dilutes any effect.",
"power": "Computed before the join from marginal counts (data/derived/restaurants/power_complaints.json): about 417.5 complaints at not-A and 562.6 at A-range restaurants expected; se(log IRR) 0.079 to 0.102 for dispersion 1.5 to 2.5; if the midpoint is true, z = 2.77 to 3.57 and D against no effect = 7.66 to 12.76 (>= 4, so the test discriminates the midpoint from no effect); at IRR 1.10 it does not (z about 1).",
"refute_if": "The primary IRR has a 95% upper bound below 1.10 (the low end of the band is excluded).",
"rule11": "The placebo is the Noise - Commercial model. If its IRR has a 95% lower bound above 1.00 (it meets the first half of support_if), or D_placebo = z_placebo^2 - z_pred^2 < 4, where z_placebo = (b_fp - b_noise)/sqrt(se_fp^2 + se_noise^2) and z_pred = (b_fp - log 1.33)/se_fp, the verdict is unsettled unless refute_if is met.",
"support_if": "The primary IRR has a 95% lower bound above 1.00 AND the ratio of the food-poisoning IRR to the Noise - Commercial IRR has a 95% lower bound above 1.00 (independence assumed for the ratio's variance, which is conservative when the two estimates covary positively)."
},
"prior": 0.3,
"registered_at": "2026-11-13T14:12:00-0500",
"registrant": "claude (restaurants agent, 13 November 2026)",
"script_commit": "0c147b3",
"sha256": "6e8e2be9eb4e8187941bcac75955c9add2f3fcc1ff77822bd68ee8cf72610254",
"statement": "Restaurants that score 14 or more points (not A-range) at a routine initial inspection draw more 311 Food Poisoning complaints in the following 365 days than restaurants that score 0-13, after borough, cuisine, chain status and quarter, and the excess is larger than for Noise - Commercial complaints at the same lots."
},
"registration_commit": {
"commit": "a70335a4283e34f0e80c3048f3bcdee40cf7e81e",
"committed_at": "2026-11-13T14:12:00-05:00"
},
"registration_file_sha256": "33eaa939f6d7238359f2b56ff6c2dbc0929d07bf23f201b4dc8befccc0715452",
"registration_path": "claims/registered/restaurants-score-complaints.json",
"registration_sha256": "6e8e2be9eb4e8187941bcac75955c9add2f3fcc1ff77822bd68ee8cf72610254",
"registration_text": "{\n \"id\": \"restaurants-score-complaints\",\n \"kind\": \"confirmation\",\n \"statement\": \"Restaurants that score 14 or more points (not A-range) at a routine initial inspection draw more 311 Food Poisoning complaints in the following 365 days than restaurants that score 0-13, after borough, cuisine, chain status and quarter, and the excess is larger than for Noise - Commercial complaints at the same lots.\",\n \"graph_evidence\": [\n {\n \"association_id\": \"RzNvMLQc9u (variable) protection from contamination violation -> outbreak level\",\n \"pubmed_ids\": [\n \"31928421\"\n ],\n \"direction\": \"+\",\n \"n_findings\": 1,\n \"n_studies\": 1,\n \"summary\": \"Minnesota chain outbreak (Firestone 2020): a protection-from-contamination violation at the last routine inspection, OR 4.92 (1.57-15.39) for outbreak level; 54 restaurants.\"\n },\n {\n \"association_id\": \"FCDuRbXqiy (variable) evidence of vermin -> food-borne outbreaks\",\n \"pubmed_ids\": [\n \"11344897\"\n ],\n \"direction\": \"+\",\n \"n_findings\": 2,\n \"n_studies\": 1,\n \"summary\": \"Miami-Dade 1995 case-control (Cruz 2001): overall inspection outcome and number of critical violations did not differ between outbreak and control restaurants; only vermin, OR 3.3 (1.1-13.1).\"\n },\n {\n \"association_id\": \"z1sSUpT_EZ (variable) complaint rates -- outbreak rates\",\n \"pubmed_ids\": [\n \"21375880\"\n ],\n \"direction\": \"+\",\n \"n_findings\": 1,\n \"n_studies\": 1,\n \"summary\": \"US local health departments (Li 2011): complaint and outbreak rates correlate, r = 0.38; complaints detect a median 69% of outbreaks.\"\n },\n {\n \"association_id\": \"CCatfE2F4k (variable) chain restaurants -- violations per inspection\",\n \"pubmed_ids\": [\n \"28060568\"\n ],\n \"direction\": \"-\",\n \"n_findings\": 1,\n \"n_studies\": 1,\n \"summary\": \"Philadelphia 2013-2014 (Leinwand 2017): chains 6.5 against 9.6 violations per inspection; used here as a covariate.\"\n }\n ],\n \"literature\": [\n {\n \"citation\": \"Irwin K et al. Am J Public Health 1989;79:586-590 (PMID 2705592)\",\n \"doi\": \"10.2105/ajph.79.5.586\",\n \"finding\": \"Seattle-King County, 28 outbreak restaurants and 56 matched controls: poor routine scores 5 times, temperature-control violations 10 times more likely in outbreak restaurants.\"\n },\n {\n \"citation\": \"Jones TF et al. Emerg Infect Dis 2004;10:688-692 (PMID 15200861)\",\n \"doi\": \"10.3201/eid1004.030343\",\n \"finding\": \"Tennessee 1993-2000, 167,574 inspections: mean scores of outbreak restaurants did not differ from others; inspector means ranged 69-92.\"\n },\n {\n \"citation\": \"Petran RL, White BW, Hedberg CW. J Food Prot 2012;75:2016-2027 (PMID 23127711)\",\n \"doi\": \"10.4315/0362-028X.JFP-12-147\",\n \"finding\": \"No overall association between routine inspection scores and outbreak inspection scores at outbreak restaurants; some specific violations more likely.\"\n },\n {\n \"citation\": \"Harrison C et al. MMWR 2014;63:441-445 (PMID 24848215)\",\n \"doi\": null,\n \"finding\": \"NYC Yelp pilot: of 468 illness reviews only 3% had been reported to DOHMH by phone or online, so 311 complaints are a small, selected share of illness.\"\n },\n {\n \"citation\": \"Wong MR et al. Am J Public Health 2015;105:e81-e87 (PMID 25602861)\",\n \"doi\": \"10.2105/AJPH.2014.302404\",\n \"finding\": \"NYC letter grading: probability of an A-range score on unannounced inspection +35% (31-40%) three years after launch.\"\n },\n {\n \"citation\": \"Firestone MJ, Hedberg CW. Emerg Infect Dis 2018;24:2164-2168 (PMID 30457518)\",\n \"doi\": \"10.3201/eid2412.180544\",\n \"finding\": \"Salmonella rates in NYC fell 5.3% a year relative to the rest of New York State after letter grading (2011-2015 against 2006-2010).\"\n },\n {\n \"citation\": \"Jin GZ, Leslie P. Q J Econ 2003;118:409-451\",\n \"doi\": \"10.1162/003355303321675428\",\n \"finding\": \"Los Angeles grade cards raised hygiene scores and were followed by a fall in foodborne-illness hospitalisations (about 20%). Not in PubMed; abstract not read here, cited from the paper's published summary.\"\n }\n ],\n \"prediction\": {\n \"direction\": \"+\",\n \"magnitude\": \"IRR (not-A against A-range) 1.10 to 1.60, midpoint 1.33. The literature is split: Irwin 1989 found poor scores predict outbreaks (about 5 times), Cruz 2001, Jones 2004 and Petran 2012 found no overall association. 311 complaints are a noisy, self-selected signal of illness (Harrison 2014), which dilutes any effect.\",\n \"support_if\": \"The primary IRR has a 95% lower bound above 1.00 AND the ratio of the food-poisoning IRR to the Noise - Commercial IRR has a 95% lower bound above 1.00 (independence assumed for the ratio's variance, which is conservative when the two estimates covary positively).\",\n \"refute_if\": \"The primary IRR has a 95% upper bound below 1.10 (the low end of the band is excluded).\",\n \"rule11\": \"The placebo is the Noise - Commercial model. If its IRR has a 95% lower bound above 1.00 (it meets the first half of support_if), or D_placebo = z_placebo^2 - z_pred^2 < 4, where z_placebo = (b_fp - b_noise)/sqrt(se_fp^2 + se_noise^2) and z_pred = (b_fp - log 1.33)/se_fp, the verdict is unsettled unless refute_if is met.\",\n \"power\": \"Computed before the join from marginal counts (data/derived/restaurants/power_complaints.json): about 417.5 complaints at not-A and 562.6 at A-range restaurants expected; se(log IRR) 0.079 to 0.102 for dispersion 1.5 to 2.5; if the midpoint is true, z = 2.77 to 3.57 and D against no effect = 7.66 to 12.76 (>= 4, so the test discriminates the midpoint from no effect); at IRR 1.10 it does not (z about 1).\"\n },\n \"plan\": \"Script analyses/restaurants/complaints_test.py at commit 0c147b3 (no resampling, no seed). Cohort: first Cycle Inspection / Initial Inspection with a score per CAMIS, 2023-01-01 to 2025-06-30, citywide live 43nn-pn8j, restricted to CAMIS whose 10-digit BBL holds exactly one CAMIS in the live data from 2022 on. Exposure: score >= 14 against 0-13. Outcome: count of erm2-nwe9 complaints, type Food Poisoning, location type Restaurant/Bar/Deli/Bakery, at the BBL, created on days 1-365 after the inspection. Model: Poisson GLM, HC1 robust SE, covariates borough, cuisine group (top 20), chain (dba with >= 10 CAMIS), inspection quarter. Placebo: same model with Noise - Commercial complaints at the BBL. Secondary, deciding nothing: C-range (>= 28) against A; per 10 points; NTA fixed effects; Manhattan only; complaints in the 365 days before the inspection.\",\n \"alternatives\": [\n \"Volume: busy restaurants have more diners, so more complaints, and more chances for violations; the noise placebo and cuisine and chain controls address part of this.\",\n \"Reporting propensity: neighbourhoods and clienteles that complain more to 311 also draw stricter inspections or are in areas inspected differently.\",\n \"Persistent quality: a restaurant's hygiene is stable, so the association reflects the restaurant, not the inspection; the pre-window secondary shows this and does not make the result causal.\",\n \"Survivorship: the live data keep only restaurants open now; restaurants that closed after a bad score and many complaints are missing, biasing the IRR towards 1.\",\n \"Linkage: single-CAMIS BBLs are smaller buildings and differ from food halls and towers.\"\n ],\n \"prior\": 0.3,\n \"registrant\": \"claude (restaurants agent, 13 November 2026)\",\n \"script_commit\": \"0c147b3\",\n \"registered_at\": \"2026-11-13T14:12:00-0500\",\n \"sha256\": \"6e8e2be9eb4e8187941bcac75955c9add2f3fcc1ff77822bd68ee8cf72610254\"\n}",
"statement": "Restaurants that score 14 or more points (not A-range) at a routine initial inspection draw more 311 Food Poisoning complaints in the following 365 days than restaurants that score 0-13, after borough, cuisine, chain status and quarter, and the excess is larger than for Noise - Commercial complaints at the same lots."
}History in this collection
- addedv1.0.0Nov 14, 2026this version