# MacroVision: household precision scenarios and selection accounting (v86)

Follow-up: the source reconstruction, matching replicate-weight checks and fixed-spec exploratory association are now documented in [v88](HOUSEHOLD_ASSOCIATION_V88.md). The scenarios below remain the historical count-only planning calculation.

## Decision

Keep effect estimation and model calibration closed. The v85 counts support a reproducible **planning and selection audit**, but do not establish actual survey precision, a liquidity association, or robustness to household changes. Matching microdata and longitudinal replicate weights were not available in this execution environment; they have not been replaced with invented observations or standard errors.

Two findings matter for the next empirical run:

- Of 460 exposed households in the eligible event-year domain, 322 (70.0%) survive the combined outcome-roster, residence and food-quality rules. Of 5,408 controls, 4,205 (77.76%) survive. The difference is **7.76 percentage points lower retention among exposed households**. This is descriptive selection accounting, not a significance test or evidence that selection caused an effect.
- Under an illustrative common food-insecurity prevalence of 10% and baseline/outcome correlation of 0.5, the equal-origin-cohort calculation has a normal planning half-width of **8.59 pp**, above the 5 pp target. A composition-sensitive pooled-cell diagnostic gives 6.82 pp. These are scenarios, **not empirical confidence intervals**; prevalence and correlation are assumptions, not estimates from SIPP.

[Method and code checkpoint](https://github.com/idop1/macrovision/commit/94e933e49993fdf40d8279aea7fe470a86d2cb11) precedes these derived diagnostics. All v85 cell counts were already known when that checkpoint was written; this is not outcome-blind preregistration or independent replication.

## Inputs and estimand distinction

Only `data/processed/sipp_household_food_v85.json` is read. No household identifiers, food values, replicate weights or individual deposits are opened. Its 4,527 retained households form four cells: control/low 2,167; control/high 2,038; exposed/low 174; exposed/high 148. Each origin contributes one window, with no repeated fixed-index key in v85.

Let Y be binary food insecurity and D = Y_outcome - Y_baseline, in [-1,1]. The planning contrast in cohort c is:

T_c = (mean D exposed,low - mean D exposed,high) - (mean D control,low - mean D control,high).

The cohort-standardized planning target averages the five T_c equally. The pooled-cell calculation is a second diagnostic; it mixes cohort composition separately within each cell and is **not the same estimand**. Neither is an adjusted model coefficient or a causal effect. No observed T_c or pooled contrast is calculated here. Equal cohort weights are fixed for this planning exercise; an eventual empirical model and population target still require a separate specification before outcomes are compared.

“Low” remains deposits below one month of baseline income. It is not low net wealth, consumption expenditure, debt service or exposure to automation.

## Precision calculations

For each cohort, H_c = sum over four cells of 1/n_cell. Under independent households and equal within-cell change variance v:

- Equal-cohort variance: v * sum(H_c) / 25.
- Pooled-cell diagnostic variance: v * sum(1/n_pooled_cell).
- Assumed v = 2*p*(1-p)*(1-rho), with equal binary prevalence p at baseline and outcome and pre/post correlation rho.
- Normal planning half-width: z_0.975 * sqrt(variance).
- Approximate 80%-power minimum detectable contrast: (z_0.975 + z_0.8) * sqrt(variance), with fixed variance. This is a normal-approximation design calculation, not measured power.

The binary joint distributions implied by the chosen p and nonnegative rho values are feasible. The common variance, independence and common prevalence assumptions remain unverified. Household survey clustering, unequal weights, cross-cohort dependence and confounding are not represented. A real SIPP confidence interval must account for its sample design; a count-only calculation cannot substitute for matching longitudinal replicate weights. Independence scenarios can either overstate or understate actual precision.

Illustration at **assumed p=10%**:

| Assumed correlation | Equal-cohort half-width (pp) | Pooled-cell half-width (pp) | Equal-cohort approximate 80% MDE (pp) |
|---|---:|---:|---:|
| 0.0 | 12.15 | 9.65 | 17.36 |
| 0.5 | 8.59 | 6.82 | 12.28 |
| 0.8 | 5.43 | 4.31 | 7.76 |

Under the middle equal-cohort scenario, reaching a 5 pp planning half-width would require approximately **2.95 times the sample**, holding cell proportions, prevalence, correlation and all independence assumptions fixed. This is not a recruitment requirement or a forecast of obtainable SIPP cases. The full JSON includes p=5%, 10%, 20%, 50% crossed with rho=0, 0.5, 0.8 and both weightings, without selecting a favorable scenario as the result.

Using only the mathematical variance bound Var(D)<=1 gives independent-count half-width ceilings of 28.63 pp (equal cohort) and 22.74 pp (pooled cells). These are variance ceilings **within that simplified calculation**, not upper bounds on complex-survey uncertainty. A large ceiling does not prove the actual precision target fails. Some favorable scenarios meet the target; others do not. The empirical precision gate is therefore **unresolved**, not passed or definitively failed.

## Selection accounting

The denominator below is the **5,868 households already balanced throughout the event year**, with positive fixed-index longitudinal weights and a known qualifying economic ending or no employer ending. This is itself a selected post-event domain. It excludes earlier sample/weight restrictions, event-year composition changes, unknown histories and non-economic-only endings. It cannot represent all 26,411 baseline eligible households or all displaced households.

| Origin | Eligible exposed | Retained exposed | Retained (%) | Eligible controls | Retained controls | Retained (%) | Smallest retained cell |
|---|---:|---:|---:|---:|---:|---:|---:|
| 2018 | 178 | 131 | 73.60 | 2571 | 1940 | 75.46 | 59 |
| 2020 | 148 | 98 | 66.22 | 765 | 595 | 77.78 | 46 |
| 2021 | 27 | 21 | 77.78 | 566 | 448 | 79.15 | 9 |
| 2022 | 70 | 44 | 62.86 | 965 | 779 | 80.73 | 21 |
| 2023 | 37 | 28 | 75.68 | 541 | 443 | 81.89 | 13 |
| **Total** | **460** | **322** | **70.00** | **5,408** | **4,205** | **77.76** | — |

There are 138 excluded exposed and 1,203 excluded controls. The combined loss includes outcome roster changes, index residence restrictions, unavailable household food links and failures of the strict food-quality path. Aggregates cannot separate these reasons by exposure, identify the role of composition alone, or demonstrate missing-at-random selection. Positive longitudinal-weight eligibility losses are not a pure nonresponse rate. Small cohort cells, including a cell of nine households, also preclude treating a large pooled N as uniformly strong overlap.

### Outcome-support sensitivity without filling in outcomes

For the eligible-domain **exposed-versus-control change contrast**, suppose retained change means were mu_E and mu_C and retention fractions r_E and r_C. If excluded changes can lie anywhere in [-1,1], the eligible contrast lies between:

L = r_E*mu_E - r_C*mu_C - [(1-r_E)+(1-r_C)]

U = r_E*mu_E - r_C*mu_C + [(1-r_E)+(1-r_C)].

The interval's full width is **104.49 pp**; its endpoints cannot be calculated without the retained means. This is an outcome-support ignorance width, not a sampling confidence interval or an observed association. A change contrast can range from -200 to +200 pp, so a width over 100 pp is mathematically possible. For a follow-up binary-level contrast with support [0,1], the analogous width is 52.24 pp. No monotonicity or missing-at-random restriction is assumed.

These bounds concern the exposure contrast in the pooled eligible domain, **not the liquidity interaction, equal-cohort contrast, or a causal effect**. Counts of excluded low/high-deposit households are unavailable, so corresponding resource-specific selection bounds are not identified from this artifact. The calculation shows how weak unrestricted extrapolation can be; it does not assert that excluded households actually have extreme outcomes.

## Concrete next empirical run

1. Reconstruct the frozen v85 membership from official release and weight archives; verify the source hashes recorded in the v85 crosswalk. Keep participant-level extracts outside the public repository. Save a reproducible private cache outside the ephemeral checkout so this reconstruction is not repeatedly lost.
2. Before comparing hardship outcomes, publish a new specification for the target index-person longitudinal domain, cohort aggregation, confounders and a composition sensitivity. The primary strict domain and broader index-tracking domain must be distinguished. Do not silently replace missing indices, recode household splits as unchanged households, or claim the broader analysis is a national household prevalence estimate.
3. Produce a blinded membership/weight audit by exposure, baseline deposit group and cohort, with separate exclusion reasons. A broader design must specify where annual food answers apply when people move and how baseline members' employment histories remain observable after departures. Unknown histories cannot be assigned to controls.
4. Merge the matching fixed-index longitudinal point and replicate weights by source endpoint and index key, check complete replicate coverage and confirm release-specific variance factors. Calculate actual variance for the specified estimator, with justified handling of dependence across cohorts. Refit any fitted adjustment in every replicate. Do not average unrelated replicate numbers across endpoints without establishing compatibility.
5. Then release all specified cohort and pooled/standardized results, including unavailable and reversed estimates, alongside composition/response sensitivities. Keep calibration closed unless precision and robustness justify a defensible empirical parameter. A reported association still does not establish causal protection by liquidity.

Steps 1-4 are pending. No new H1 coefficient, household composition effect, H2 test or model parameter was produced by v86.

## Reproduction and checks

```bash
python scripts/sipp_precision_v86.py --output data/processed/sipp_precision_v86.json
python -m unittest discover -s calibration -p 'test_sipp_precision_v86.py' -v
```

The script uses the Python standard library. The output records SHA256 hashes of the aggregate input, plan and analysis code, with explicit closed replicate/effect/calibration gates. Seven synthetic tests check the four-cell variance identity, empty and duplicate cells, complete/all-missing retention, outcome-support scaling and impossible retention counts. The full 24 scenario rows and cohort accounting are in [the aggregate output](data/processed/sipp_precision_v86.json); the fixed assumptions are in [the plan](data/metadata/sipp_precision_plan_v86.json).

## Sources

- [v85 household design and food-measurement audit](HOUSEHOLD_FOOD_VALIDATION_V85.md): aggregate cell and flow counts; no microdata used here.
- [U.S. Census: SIPP sampling error](https://www.census.gov/programs-surveys/sipp/methodology/sampling-error.html): complex design must be considered; replicate weighting supports variance estimation.
- [U.S. Census: SIPP weighting](https://www.census.gov/programs-surveys/sipp/methodology/weighting.html): weights depend on population and duration of analysis.