# MacroVision: SIPP replication and temporal extension (v83)

## Historical expansion audit (v84)

The [v84 historical sample and precision audit](SIPP_HISTORICAL_PLAN_V84.md) identifies 11,253 candidate people and 468 separations with one window per origin panel. It does not estimate a pooled hardship effect: the working-model precision target is still unmet, and food comparability and household exposure require validation. The estimates below remain the v83 results.

## Conclusion

**The evidence does not provide stable empirical support for H1.** The person-disjoint replication cohort is too small for its prespecified estimation threshold. A separately specified temporal extension can be estimated, but its point estimate changes direction relative to the pilot and remains imprecise. Neither result licenses a causal conclusion or calibration of the liquidity mechanism in the full model.

The v82 pilot's conditional association was **+2.97 percentage points**, 95% interval **-15.92 to +21.87**. The v83 temporal extension gives **-6.06 points**, interval **-23.33 to +11.22**. Positive means that lower baseline deposits are associated with more worsening of hardship following economic separation, relative to the no-separation contrast. Thus the new point estimate is opposite to the predicted direction, but the interval also contains a meaningful positive association. **This is inconclusive, not evidence of zero effect and not a rejection of the entire automation thesis.**

## What was fixed before opening the new outcomes

The replication uses reference years **2022 baseline / 2023 separation / 2024 outcome**, in SIPP releases 2023/2024/2025. Year-end deposits and all covariates precede the separation year. Annual food insecurity is not relabeled monthly consumption. Events, age range, deposit threshold, outcome coding and primary regression follow the v82 design.

The intended person-disjoint replication is restricted to **origin panel 2023**. We checked its person identifiers against every person in the original pilot's baseline release, not merely the old final sample: overlap is zero. This is sample separation, not a new independent macroeconomic episode; the two calendar windows overlap.

The audit extracts **no food-security variables** and reads no outcome-release primary microdata file. It uses only baseline resources, employment events and the separate longitudinal weight files, including 2025 weights. The separate Git checkpoints establish the sequence:

1. [Primary specification fixed before new outcome analysis](https://github.com/idop1/macrovision/commit/779cdd79b483d31bbb587e79e130bb55375ccdcb).
2. The blinded replication audit found **1,123 candidates and only 26 economic separations**. The minimum fit count was already **30**, so the primary replication could not become an effect estimate by relaxing that rule.
3. [A separate extension specified after the blind counts, before new outcome analysis](https://github.com/idop1/macrovision/commit/e0879a0922f067a66ec2c37ee3eca7c9c624815f) adds the other eligible origin panel, 2022. This is explicitly a secondary temporal extension with participant overlap, not a replacement independent replication. It found **2,634 blinded candidates and 70 economic separations**.
4. [Both blinded audits, planning scenarios, crosswalk and analysis code locked](https://github.com/idop1/macrovision/commit/5ef1a32ba1727236d73099483cdff010135866b4) before opening the new food outcomes.

The primary specification was not rewritten. The extension has its own specification file referencing the primary specification hash. At estimation, the tool checks specification, analysis-code, crosswalk and input hashes, as well as the unchanged blind candidate counts. No external study registration is claimed. The new outcome file was downloaded earlier but its household outcome values were not read until after these locks. Baseline/event files had previously been used by v82; this is an outcome holdout, not a claim that all source files were previously unseen.

## Sample and precision planning

The person-disjoint panel starts with 4,290 employed people aged 25-60 in December 2022. Stable baseline residence and no prior economic separation leave 3,757. Complete event-year observations, positive longitudinal weights, the comparison-group rule and valid baseline resources leave 1,123 blinded candidates, including 26 events. The event subgroups contain only **16 below-one-month-deposit candidates and 10 at-or-above-one-month candidates** before outcome exclusions. They are below the public rate-cell minimum of 30.

The extension starts with 8,581 baseline employed people, then 7,474 after baseline residence/prior-event restrictions, then 2,634 blinded candidates. Outcome residence and completeness restrictions leave **2,472 people and 62 economic separations**. The source-level identifier overlap count, 13,422, refers to all ages in the extension's baseline source panel versus all people in the old baseline release. It is not a count of matched final analytic participants. We do not describe the extension as person-disjoint or combine its estimate with v82 as independent evidence.

Before outcomes, eight four-cell planning scenarios varied assumed prevalence (10%/30%), pre-post correlation (0/0.5), and variance multiplier (1/1.5). These are transparent approximations using Kish effective sizes, **not** actual survey-design regression power calculations. They suggested 80%-power minimum detectable effects of **37.35-98.82 points** for the disjoint cohort and **23.66-62.59 points** for the extension. Outcome exclusions can further reduce precision. Both are far from the planning target of a **5-point 95% half-width**. We proceeded once under the fixed plan and report the lack of precision rather than selecting another definition or pooling until a favorable result appears.

## Estimates

All estimates below are conditional associations. The primary outcome is the change in the annual binary food-insecurity indicator (`RFOODS=2 or 3`) between 2022 and 2024. The interaction is economic job separation x deposits below one month of baseline mean income. This is income coverage, not expenditure coverage or total safe liquid wealth.

The core model includes event, lower deposits, interaction, log baseline household income, inverse-hyperbolic-sine baseline household net wealth and debt balance, age, age squared, and baseline hardship. The expanded sensitivity adds baseline education groups, analytic industry groups, employment arrangement, sex, household size and children. Industry and arrangement refer to the lowest-numbered active December job, a stated proxy rather than a verified primary job. All additions were specified before outcomes; missing baseline covariates are handled by complete-case exclusion.

| Analysis | People | Events | Interaction | Design SE | 95% interval |
|---|---:|---:|---:|---:|---:|
| Person-disjoint replication | 1,050 | 22 | **Not estimated under fixed rule** | — | — |
| Temporal extension, core model | 2,472 | 62 | **-6.06 pp** | 8.81 pp | **-23.33 to +11.22 pp** |
| Extension, expanded baseline controls | 2,463 | 62 | -6.19 pp | 8.78 pp | -23.40 to +11.02 pp |
| Extension, reported food components only | 2,291 | 59 | -6.56 pp | 9.04 pp | -24.27 to +11.16 pp |

The fixed 30-event core and 60-event expanded-model thresholds are applied after completeness restrictions. The small disjoint cohort is not assigned a coefficient even if a software package could return one. Small descriptive rates are suppressed. The 5-point half-width target is not met by any of the extension fits.

We use 2025 `FINYR3` (2022-2024) and its **240 longitudinal replicate weights**, verify the replicate reference period and equality of `REPWGT0` with the point weight, then refit each model in every replicate. Fay's factor is 0.5; intervals use the normal 1.96 multiplier. No post-outcome model simplification is allowed if the fixed design or a replicate is rank deficient. Weights account for the documented survey sampling structure; they do not remove confounding or fully capture imputation uncertainty. Panel restrictions do not support a claim of nationally representative household prevalence.

## What the robustness checks establish

Adding education, industry, employment arrangement and household composition barely changes the extension point estimate. Restricting to food components reported rather than imputed also leaves the negative point estimate and wide interval. These checks do not recover the positive pilot direction, and do not make the result causal.

Census documents another change to food-security and adult-well-being allocation income bands in the 2025 release. The reported-food sensitivity therefore requires `AFOOD1..3=1` and conditional items `AFOOD4..6` to be reported or outside their universe (`0 or 1`) in both years. Valid skips are not mistaken for imputed answers. This restriction can itself select a different sample; it is not a complete correction for missing-data bias.

Economic separation is not necessarily unemployment, permanent job displacement or automation exposure. Selection, anticipation, health, local labor-market conditions, changes in household composition and errors in coarse deposit/industry proxies remain. Stable-residence and complete-case restrictions narrow the population. US household hardship does not identify Israeli or global price and fiscal responses.

## Research decision

- **H1:** no precise, stable positive association established. The independent replication is underpowered; the dependent temporal extension changes sign and remains inconclusive.
- **H2:** still untested. A baseline debt balance used as a control is not contractual debt-service burden.
- **H3, automation unemployment, price decline and fiscal feasibility:** not tested by this exercise.
- **Model:** do not insert these coefficients into the full model as validated liquidity protection parameters. Existing simulation mechanisms remain hypotheses and sensitivity assumptions.

The next useful empirical work is a **larger, harmonized set of historical SIPP origin panels with sufficient separation events**, preceded by release/weight compatibility review and a design-based precision calculation. Define person reuse and calendar dependence explicitly, fix the outcome and liquidity measure before further estimation, and verify debt-service availability. A parallel source assessment should seek permitted data with actual consumption expenditure for the broader thesis. Adding another small regression without improving measurement or precision is unlikely to settle H1.

## Files and reproduction

The source archives and selected-column caches stay outside the checkout; no household/person records or identifiers are committed or publicly served. Only aggregate reports, audit counts and variable metadata are published.

- `scripts/sipp_replication_v83.py` (reuses tested helpers in `scripts/sipp_pilot_v82.py`).
- `calibration/test_sipp_replication_v83.py`: blindness, specification lock, December continuation, job selection, power scenarios and fixed-design tests.
- `data/metadata/sipp_replication_spec_v83.json` and `sipp_extension_spec_v83.json`.
- `data/metadata/sipp_blind_audit_v83.json` and `sipp_extension_blind_audit_v83.json`.
- `data/metadata/sipp_crosswalk_v83.json`: selected metadata, source hashes, dictionary presence checks and baseline industry-code inventory.
- `data/processed/sipp_replication_v83.json` and `sipp_extension_v83.json`: full aggregate results, suppressed cells and archive/specification/audit hashes.

Download official public-use `pu2023_csv.zip`, `pu2024_csv.zip`, `pu2025_csv.zip`, `lgtwgt2025yr3_csv.zip` and `lgtrw2025yr3_csv.zip` from each release's Census dataset directory. Also supply `pu2022_csv.zip` for the conservative person-overlap check. Install `requirements-research.txt`.

```bash
python scripts/sipp_replication_v83.py --data-dir /outside/sipp --prior-data-dir /outside/sipp --cohort holdout --stage audit --audit-path /outside/holdout-audit.json --output /outside/holdout-audit.json
python scripts/sipp_replication_v83.py --data-dir /outside/sipp --prior-data-dir /outside/sipp --cohort holdout --stage estimate --audit-path /outside/holdout-audit.json --output /outside/holdout-result.json
python scripts/sipp_replication_v83.py --data-dir /outside/sipp --prior-data-dir /outside/sipp --cohort extension --stage audit --audit-path /outside/extension-audit.json --output /outside/extension-audit.json
python scripts/sipp_replication_v83.py --data-dir /outside/sipp --prior-data-dir /outside/sipp --cohort extension --stage estimate --audit-path /outside/extension-audit.json --output /outside/extension-result.json
python -m unittest discover -s calibration -p 'test_sipp_replication_v83.py' -v
```

The archived audit hashes document the original run; rerunning an audit is reproduction, not a new preregistration. Any changed specification or analysis code must be treated as a new analysis rather than silently relabeled a replication. PSID remains excluded from this workflow.

## Official sources

- [2025 public-use data and longitudinal weights](https://www.census.gov/data/datasets/2025/demo/sipp/2025-data.html)
- [2025 user's guide, especially sections 7.2 and 7.3](https://www2.census.gov/programs-surveys/sipp/tech-documentation/methodology/2025_SIPP_Users_Guide.pdf)
- [2025 longitudinal weight dictionary](https://www2.census.gov/programs-surveys/sipp/tech-documentation/data-dictionaries/2025/lgtwgt2025_dictionary.txt)
- [2025 food-security allocation matrix change](https://www.census.gov/programs-surveys/sipp/tech-documentation/user-notes/2025-usernotes/2025-allocation-matrix-change.html)
- [2025 data dictionary](https://www2.census.gov/programs-surveys/sipp/tech-documentation/data-dictionaries/2025/2025_SIPP_Data_Dictionary.pdf)
- [Previous pilot and its source documentation](EMPIRICAL_PILOT_V82.md)