Data Deposit Preparation

Prepare an AEA Data Editor compliant replication package for the sewage-house-prices project.

Input:

$ARGUMENTS

— output directory (defaults to

Replication/

).

---

Project-Specific Context

Pipeline Structure

The project has a 6-layer data pipeline in

scripts/R/

:

1. 01_data_ingestion/

   — Raw data collection (EDM archives, APIs)

2. 02_data_cleaning/

   — Format standardisation, geocoding, validation

3. 03_data_enrichment/

   — Temporal aggregation, rainfall metrics, dry spill identification

4. 04_feature_engineering/

   — Spatial matching (house/rental ↔ spill sites)

5. 05_data_integration/

   — Merging historical and API EDM data

6. 06_analysis_datasets/

   — Final dataset assembly

Analysis scripts:

scripts/R/09_analysis/

(6 subdirectories by approach) Utilities:

scripts/R/utils/

Python scripts:

scripts/python/

(river network processing) Docker pipelines:

RiverNetworks/

,

upstream_downstream/

Data Layout

data/raw/          — Original immutable data (EDM, Land Registry, Met Office, shapefiles)
data/processed/    — Intermediate pipeline outputs (parquet)
data/final/        — Analysis-ready datasets
data/cache/        — Postcode geocoding cache

Key Dependencies

• R packages managed via

  renv

  (

  renv.lock

  )
• Python environment via

  uv

  in

  scripts/python/

• PostGIS via Docker for river network analysis

---

Workflow

Step 1: Inventory

1. Read all scripts in

   scripts/R/

   and parse data file references
2. Read

   renv.lock

   for package versions
3. Scan

   output/tables/

   and

   output/figures/

   for output files
4. Read the manuscript (

   docs/overleaf/_main.tex

   ) for table/figure references
5. Check

   scripts/python/

   for Python dependencies

Step 2: Analyse Dependencies

1. Parse script dependencies (which scripts create files that others load)
2. Map the execution order (follows the 6-layer pipeline, then analysis scripts)
3. Cross-reference the full execution order documented in

   ReadMe.md

Step 3: Assemble Package

Create in

Replication/

(or specified directory):

1. README.md — AEA format:

   • Data availability statement (which data is public vs restricted)
   • Computational requirements (R version, packages, PostGIS, Python)
   • Program descriptions (what each script does)
   • Replication instructions (step-by-step)
   • Expected runtime

2. master.R — Runs everything in order:

   # Master replication script for "Sewage in Our Waters"
   # Estimated runtime: [X hours]
   
   source(here::here("scripts", "R", "01_data_ingestion", "script.R"))
   # ... through all layers
   source(here::here("scripts", "R", "09_analysis", "subdir", "script.R"))
   

3. install_packages.R — If renv is not used:

   install.packages(c("tidyverse", "fixest", "modelsummary", ...))
   

4. DEPOSIT_CHECKLIST.md — Pre-deposit verification

Step 4: Validate

Run the 10 verification checks (equivalent to

/audit-replication

):

1. Script execution order is correct
2. All data file references resolve
3. All output files are generated
4. Package versions documented
5. No hardcoded absolute paths
6. Data provenance documented
7. README completeness (AEA format)
8. Output cross-reference (every table/figure traced to a script)
9. Restricted data properly flagged
10. Master script runs without modification

Step 5: Present Results

1. Package contents — All files in

   Replication/

2. Script order — Numbered sequence with dependency graph
3. Data availability — Public vs restricted datasets
4. Verification result — X/10 checks passed
5. Deposit steps — openICPSR / Zenodo instructions

---

Principles

• AEA Data Editor standards are the target. README format, versions, data access statements.
• Don't rename scripts without approval. Present ordering first, let the user decide.
• Thorough data provenance. Every dataset documented with source, access date, and restrictions.
• Test before declaring ready. Always validate after assembly.
• Document restricted data clearly. Land Registry and Zoopla data may have access restrictions.