Claim at v1.1.0

restaurants-score-complaints

51a728d526d809076cda3a9a05d22531d5b3a2a46371d1e08b1cf3b8c25eb694

Data

{
  "has_outcome": true,
  "kind": "confirmation",
  "lineage": {
    "audit": "docs/lineage_audit.md; data/derived/lineage_audit.json",
    "class": "a",
    "class_meaning": "exposure or treatment only, not the outcome",
    "classes": [
      "a"
    ],
    "early_sources": [
      {
        "class": "a",
        "dataset": "43nn-pn8j",
        "evidence": "Inspection scores are the exposure. Referee: \"The inspection extract that defines the exposure cohort ... was fetched at 14:08:53 and 14:09:07\"; 311 reads before registration were aggregates for the power check.",
        "fetched": "2026-11-13T14:09:07-05:00",
        "name": "DOHMH New York City Restaurant Inspection Results"
      }
    ]
  },
  "outcome": "restaurants-score-complaints",
  "prereg_branch": {
    "name": "prereg/restaurants-score-complaints",
    "sha": "a70335a4283e34f0e80c3048f3bcdee40cf7e81e"
  },
  "prior": 0.3,
  "registered_at": "2026-11-13T14:12:00-0500",
  "registrant": "claude (restaurants agent, 13 November 2026)",
  "registration": {
    "alternatives": [
      "Volume: busy restaurants have more diners, so more complaints, and more chances for violations; the noise placebo and cuisine and chain controls address part of this.",
      "Reporting propensity: neighbourhoods and clienteles that complain more to 311 also draw stricter inspections or are in areas inspected differently.",
      "Persistent quality: a restaurant's hygiene is stable, so the association reflects the restaurant, not the inspection; the pre-window secondary shows this and does not make the result causal.",
      "Survivorship: the live data keep only restaurants open now; restaurants that closed after a bad score and many complaints are missing, biasing the IRR towards 1.",
      "Linkage: single-CAMIS BBLs are smaller buildings and differ from food halls and towers."
    ],
    "graph_evidence": [
      {
        "association_id": "RzNvMLQc9u (variable) protection from contamination violation -> outbreak level",
        "direction": "+",
        "n_findings": 1,
        "n_studies": 1,
        "pubmed_ids": [
          "31928421"
        ],
        "summary": "Minnesota chain outbreak (Firestone 2020): a protection-from-contamination violation at the last routine inspection, OR 4.92 (1.57-15.39) for outbreak level; 54 restaurants."
      },
      {
        "association_id": "FCDuRbXqiy (variable) evidence of vermin -> food-borne outbreaks",
        "direction": "+",
        "n_findings": 2,
        "n_studies": 1,
        "pubmed_ids": [
          "11344897"
        ],
        "summary": "Miami-Dade 1995 case-control (Cruz 2001): overall inspection outcome and number of critical violations did not differ between outbreak and control restaurants; only vermin, OR 3.3 (1.1-13.1)."
      },
      {
        "association_id": "z1sSUpT_EZ (variable) complaint rates -- outbreak rates",
        "direction": "+",
        "n_findings": 1,
        "n_studies": 1,
        "pubmed_ids": [
          "21375880"
        ],
        "summary": "US local health departments (Li 2011): complaint and outbreak rates correlate, r = 0.38; complaints detect a median 69% of outbreaks."
      },
      {
        "association_id": "CCatfE2F4k (variable) chain restaurants -- violations per inspection",
        "direction": "-",
        "n_findings": 1,
        "n_studies": 1,
        "pubmed_ids": [
          "28060568"
        ],
        "summary": "Philadelphia 2013-2014 (Leinwand 2017): chains 6.5 against 9.6 violations per inspection; used here as a covariate."
      }
    ],
    "id": "restaurants-score-complaints",
    "kind": "confirmation",
    "literature": [
      {
        "citation": "Irwin K et al. Am J Public Health 1989;79:586-590 (PMID 2705592)",
        "doi": "10.2105/ajph.79.5.586",
        "finding": "Seattle-King County, 28 outbreak restaurants and 56 matched controls: poor routine scores 5 times, temperature-control violations 10 times more likely in outbreak restaurants."
      },
      {
        "citation": "Jones TF et al. Emerg Infect Dis 2004;10:688-692 (PMID 15200861)",
        "doi": "10.3201/eid1004.030343",
        "finding": "Tennessee 1993-2000, 167,574 inspections: mean scores of outbreak restaurants did not differ from others; inspector means ranged 69-92."
      },
      {
        "citation": "Petran RL, White BW, Hedberg CW. J Food Prot 2012;75:2016-2027 (PMID 23127711)",
        "doi": "10.4315/0362-028X.JFP-12-147",
        "finding": "No overall association between routine inspection scores and outbreak inspection scores at outbreak restaurants; some specific violations more likely."
      },
      {
        "citation": "Harrison C et al. MMWR 2014;63:441-445 (PMID 24848215)",
        "doi": null,
        "finding": "NYC Yelp pilot: of 468 illness reviews only 3% had been reported to DOHMH by phone or online, so 311 complaints are a small, selected share of illness."
      },
      {
        "citation": "Wong MR et al. Am J Public Health 2015;105:e81-e87 (PMID 25602861)",
        "doi": "10.2105/AJPH.2014.302404",
        "finding": "NYC letter grading: probability of an A-range score on unannounced inspection +35% (31-40%) three years after launch."
      },
      {
        "citation": "Firestone MJ, Hedberg CW. Emerg Infect Dis 2018;24:2164-2168 (PMID 30457518)",
        "doi": "10.3201/eid2412.180544",
        "finding": "Salmonella rates in NYC fell 5.3% a year relative to the rest of New York State after letter grading (2011-2015 against 2006-2010)."
      },
      {
        "citation": "Jin GZ, Leslie P. Q J Econ 2003;118:409-451",
        "doi": "10.1162/003355303321675428",
        "finding": "Los Angeles grade cards raised hygiene scores and were followed by a fall in foodborne-illness hospitalisations (about 20%). Not in PubMed; abstract not read here, cited from the paper's published summary."
      }
    ],
    "plan": "Script analyses/restaurants/complaints_test.py at commit 0c147b3 (no resampling, no seed). Cohort: first Cycle Inspection / Initial Inspection with a score per CAMIS, 2023-01-01 to 2025-06-30, citywide live 43nn-pn8j, restricted to CAMIS whose 10-digit BBL holds exactly one CAMIS in the live data from 2022 on. Exposure: score >= 14 against 0-13. Outcome: count of erm2-nwe9 complaints, type Food Poisoning, location type Restaurant/Bar/Deli/Bakery, at the BBL, created on days 1-365 after the inspection. Model: Poisson GLM, HC1 robust SE, covariates borough, cuisine group (top 20), chain (dba with >= 10 CAMIS), inspection quarter. Placebo: same model with Noise - Commercial complaints at the BBL. Secondary, deciding nothing: C-range (>= 28) against A; per 10 points; NTA fixed effects; Manhattan only; complaints in the 365 days before the inspection.",
    "prediction": {
      "direction": "+",
      "magnitude": "IRR (not-A against A-range) 1.10 to 1.60, midpoint 1.33. The literature is split: Irwin 1989 found poor scores predict outbreaks (about 5 times), Cruz 2001, Jones 2004 and Petran 2012 found no overall association. 311 complaints are a noisy, self-selected signal of illness (Harrison 2014), which dilutes any effect.",
      "power": "Computed before the join from marginal counts (data/derived/restaurants/power_complaints.json): about 417.5 complaints at not-A and 562.6 at A-range restaurants expected; se(log IRR) 0.079 to 0.102 for dispersion 1.5 to 2.5; if the midpoint is true, z = 2.77 to 3.57 and D against no effect = 7.66 to 12.76 (>= 4, so the test discriminates the midpoint from no effect); at IRR 1.10 it does not (z about 1).",
      "refute_if": "The primary IRR has a 95% upper bound below 1.10 (the low end of the band is excluded).",
      "rule11": "The placebo is the Noise - Commercial model. If its IRR has a 95% lower bound above 1.00 (it meets the first half of support_if), or D_placebo = z_placebo^2 - z_pred^2 < 4, where z_placebo = (b_fp - b_noise)/sqrt(se_fp^2 + se_noise^2) and z_pred = (b_fp - log 1.33)/se_fp, the verdict is unsettled unless refute_if is met.",
      "support_if": "The primary IRR has a 95% lower bound above 1.00 AND the ratio of the food-poisoning IRR to the Noise - Commercial IRR has a 95% lower bound above 1.00 (independence assumed for the ratio's variance, which is conservative when the two estimates covary positively)."
    },
    "prior": 0.3,
    "registered_at": "2026-11-13T14:12:00-0500",
    "registrant": "claude (restaurants agent, 13 November 2026)",
    "script_commit": "0c147b3",
    "sha256": "6e8e2be9eb4e8187941bcac75955c9add2f3fcc1ff77822bd68ee8cf72610254",
    "statement": "Restaurants that score 14 or more points (not A-range) at a routine initial inspection draw more 311 Food Poisoning complaints in the following 365 days than restaurants that score 0-13, after borough, cuisine, chain status and quarter, and the excess is larger than for Noise - Commercial complaints at the same lots."
  },
  "registration_commit": {
    "commit": "a70335a4283e34f0e80c3048f3bcdee40cf7e81e",
    "committed_at": "2026-11-13T14:12:00-05:00"
  },
  "registration_file_sha256": "33eaa939f6d7238359f2b56ff6c2dbc0929d07bf23f201b4dc8befccc0715452",
  "registration_path": "claims/registered/restaurants-score-complaints.json",
  "registration_sha256": "6e8e2be9eb4e8187941bcac75955c9add2f3fcc1ff77822bd68ee8cf72610254",
  "registration_text": "{\n \"id\": \"restaurants-score-complaints\",\n \"kind\": \"confirmation\",\n \"statement\": \"Restaurants that score 14 or more points (not A-range) at a routine initial inspection draw more 311 Food Poisoning complaints in the following 365 days than restaurants that score 0-13, after borough, cuisine, chain status and quarter, and the excess is larger than for Noise - Commercial complaints at the same lots.\",\n \"graph_evidence\": [\n  {\n   \"association_id\": \"RzNvMLQc9u (variable) protection from contamination violation -> outbreak level\",\n   \"pubmed_ids\": [\n    \"31928421\"\n   ],\n   \"direction\": \"+\",\n   \"n_findings\": 1,\n   \"n_studies\": 1,\n   \"summary\": \"Minnesota chain outbreak (Firestone 2020): a protection-from-contamination violation at the last routine inspection, OR 4.92 (1.57-15.39) for outbreak level; 54 restaurants.\"\n  },\n  {\n   \"association_id\": \"FCDuRbXqiy (variable) evidence of vermin -> food-borne outbreaks\",\n   \"pubmed_ids\": [\n    \"11344897\"\n   ],\n   \"direction\": \"+\",\n   \"n_findings\": 2,\n   \"n_studies\": 1,\n   \"summary\": \"Miami-Dade 1995 case-control (Cruz 2001): overall inspection outcome and number of critical violations did not differ between outbreak and control restaurants; only vermin, OR 3.3 (1.1-13.1).\"\n  },\n  {\n   \"association_id\": \"z1sSUpT_EZ (variable) complaint rates -- outbreak rates\",\n   \"pubmed_ids\": [\n    \"21375880\"\n   ],\n   \"direction\": \"+\",\n   \"n_findings\": 1,\n   \"n_studies\": 1,\n   \"summary\": \"US local health departments (Li 2011): complaint and outbreak rates correlate, r = 0.38; complaints detect a median 69% of outbreaks.\"\n  },\n  {\n   \"association_id\": \"CCatfE2F4k (variable) chain restaurants -- violations per inspection\",\n   \"pubmed_ids\": [\n    \"28060568\"\n   ],\n   \"direction\": \"-\",\n   \"n_findings\": 1,\n   \"n_studies\": 1,\n   \"summary\": \"Philadelphia 2013-2014 (Leinwand 2017): chains 6.5 against 9.6 violations per inspection; used here as a covariate.\"\n  }\n ],\n \"literature\": [\n  {\n   \"citation\": \"Irwin K et al. Am J Public Health 1989;79:586-590 (PMID 2705592)\",\n   \"doi\": \"10.2105/ajph.79.5.586\",\n   \"finding\": \"Seattle-King County, 28 outbreak restaurants and 56 matched controls: poor routine scores 5 times, temperature-control violations 10 times more likely in outbreak restaurants.\"\n  },\n  {\n   \"citation\": \"Jones TF et al. Emerg Infect Dis 2004;10:688-692 (PMID 15200861)\",\n   \"doi\": \"10.3201/eid1004.030343\",\n   \"finding\": \"Tennessee 1993-2000, 167,574 inspections: mean scores of outbreak restaurants did not differ from others; inspector means ranged 69-92.\"\n  },\n  {\n   \"citation\": \"Petran RL, White BW, Hedberg CW. J Food Prot 2012;75:2016-2027 (PMID 23127711)\",\n   \"doi\": \"10.4315/0362-028X.JFP-12-147\",\n   \"finding\": \"No overall association between routine inspection scores and outbreak inspection scores at outbreak restaurants; some specific violations more likely.\"\n  },\n  {\n   \"citation\": \"Harrison C et al. MMWR 2014;63:441-445 (PMID 24848215)\",\n   \"doi\": null,\n   \"finding\": \"NYC Yelp pilot: of 468 illness reviews only 3% had been reported to DOHMH by phone or online, so 311 complaints are a small, selected share of illness.\"\n  },\n  {\n   \"citation\": \"Wong MR et al. Am J Public Health 2015;105:e81-e87 (PMID 25602861)\",\n   \"doi\": \"10.2105/AJPH.2014.302404\",\n   \"finding\": \"NYC letter grading: probability of an A-range score on unannounced inspection +35% (31-40%) three years after launch.\"\n  },\n  {\n   \"citation\": \"Firestone MJ, Hedberg CW. Emerg Infect Dis 2018;24:2164-2168 (PMID 30457518)\",\n   \"doi\": \"10.3201/eid2412.180544\",\n   \"finding\": \"Salmonella rates in NYC fell 5.3% a year relative to the rest of New York State after letter grading (2011-2015 against 2006-2010).\"\n  },\n  {\n   \"citation\": \"Jin GZ, Leslie P. Q J Econ 2003;118:409-451\",\n   \"doi\": \"10.1162/003355303321675428\",\n   \"finding\": \"Los Angeles grade cards raised hygiene scores and were followed by a fall in foodborne-illness hospitalisations (about 20%). Not in PubMed; abstract not read here, cited from the paper's published summary.\"\n  }\n ],\n \"prediction\": {\n  \"direction\": \"+\",\n  \"magnitude\": \"IRR (not-A against A-range) 1.10 to 1.60, midpoint 1.33. The literature is split: Irwin 1989 found poor scores predict outbreaks (about 5 times), Cruz 2001, Jones 2004 and Petran 2012 found no overall association. 311 complaints are a noisy, self-selected signal of illness (Harrison 2014), which dilutes any effect.\",\n  \"support_if\": \"The primary IRR has a 95% lower bound above 1.00 AND the ratio of the food-poisoning IRR to the Noise - Commercial IRR has a 95% lower bound above 1.00 (independence assumed for the ratio's variance, which is conservative when the two estimates covary positively).\",\n  \"refute_if\": \"The primary IRR has a 95% upper bound below 1.10 (the low end of the band is excluded).\",\n  \"rule11\": \"The placebo is the Noise - Commercial model. If its IRR has a 95% lower bound above 1.00 (it meets the first half of support_if), or D_placebo = z_placebo^2 - z_pred^2 < 4, where z_placebo = (b_fp - b_noise)/sqrt(se_fp^2 + se_noise^2) and z_pred = (b_fp - log 1.33)/se_fp, the verdict is unsettled unless refute_if is met.\",\n  \"power\": \"Computed before the join from marginal counts (data/derived/restaurants/power_complaints.json): about 417.5 complaints at not-A and 562.6 at A-range restaurants expected; se(log IRR) 0.079 to 0.102 for dispersion 1.5 to 2.5; if the midpoint is true, z = 2.77 to 3.57 and D against no effect = 7.66 to 12.76 (>= 4, so the test discriminates the midpoint from no effect); at IRR 1.10 it does not (z about 1).\"\n },\n \"plan\": \"Script analyses/restaurants/complaints_test.py at commit 0c147b3 (no resampling, no seed). Cohort: first Cycle Inspection / Initial Inspection with a score per CAMIS, 2023-01-01 to 2025-06-30, citywide live 43nn-pn8j, restricted to CAMIS whose 10-digit BBL holds exactly one CAMIS in the live data from 2022 on. Exposure: score >= 14 against 0-13. Outcome: count of erm2-nwe9 complaints, type Food Poisoning, location type Restaurant/Bar/Deli/Bakery, at the BBL, created on days 1-365 after the inspection. Model: Poisson GLM, HC1 robust SE, covariates borough, cuisine group (top 20), chain (dba with >= 10 CAMIS), inspection quarter. Placebo: same model with Noise - Commercial complaints at the BBL. Secondary, deciding nothing: C-range (>= 28) against A; per 10 points; NTA fixed effects; Manhattan only; complaints in the 365 days before the inspection.\",\n \"alternatives\": [\n  \"Volume: busy restaurants have more diners, so more complaints, and more chances for violations; the noise placebo and cuisine and chain controls address part of this.\",\n  \"Reporting propensity: neighbourhoods and clienteles that complain more to 311 also draw stricter inspections or are in areas inspected differently.\",\n  \"Persistent quality: a restaurant's hygiene is stable, so the association reflects the restaurant, not the inspection; the pre-window secondary shows this and does not make the result causal.\",\n  \"Survivorship: the live data keep only restaurants open now; restaurants that closed after a bad score and many complaints are missing, biasing the IRR towards 1.\",\n  \"Linkage: single-CAMIS BBLs are smaller buildings and differ from food halls and towers.\"\n ],\n \"prior\": 0.3,\n \"registrant\": \"claude (restaurants agent, 13 November 2026)\",\n \"script_commit\": \"0c147b3\",\n \"registered_at\": \"2026-11-13T14:12:00-0500\",\n \"sha256\": \"6e8e2be9eb4e8187941bcac75955c9add2f3fcc1ff77822bd68ee8cf72610254\"\n}",
  "statement": "Restaurants that score 14 or more points (not A-range) at a routine initial inspection draw more 311 Food Poisoning complaints in the following 365 days than restaurants that score 0-13, after borough, cuisine, chain status and quarter, and the excess is larger than for Noise - Commercial complaints at the same lots."
}

History in this collection

  1. addedv1.0.0Nov 14, 2026this version