Data Analysis Workflow

Run an end-to-end data analysis in R or Python: load, explore, analyze, and produce publication-ready output.

Input:

$ARGUMENTS

— a dataset path (e.g.,

data/county_panel.csv

) or a description of the analysis goal (e.g., "regress wages on education with state fixed effects using CPS data").

---

Phase 0: Choose Language

Determine language from

$ARGUMENTS

or ask the user:

• User mentions

  tidyverse

  ,

  fixest

  ,

  lm

  ,

  .R

  context → R track
• User mentions

  pandas

  ,

  statsmodels

  ,

  sklearn

  ,

  .py

  or

  .ipynb

  context → Python track
• Dataset is

  .csv

  /

  .parquet

  with no language cue → use AskUserQuestion with a single-select menu:
  • header: "Language"
  • question: "Which language should I use for this analysis?"
  • options:
    • label: "R (Recommended)", description: "tidyverse, fixest, ggplot2 — full plugin support with coding conventions and R reviewer"
    • label: "Python", description: "pandas, statsmodels — supported for analysis scripts and figures"
    • label: "Both", description: "R for figures and tables, Python for data processing"

---

R Track

Constraints

• Follow

  rules/r-code-conventions.md

  for all standards
• Save scripts to

  scripts/R/

  with descriptive names
• Save all outputs (figures, tables, RDS) to

  output/

• Use

  saveRDS()

  for every computed object
• Run

  r-reviewer

  on the generated script before presenting results

Phase 1: Setup and Data Loading

1. Create R script with proper header (title, author, purpose, inputs, outputs)
2. Load required packages at top (

   library()

   , never

   require()

   )
3. Set seed once at top:

   set.seed(42)

4. Create output directories:

   dir.create("output/analysis", recursive = TRUE, showWarnings = FALSE)

5. Load and inspect the dataset

Phase 2: Exploratory Data Analysis

• summary()

  , missingness rates, variable types
• Histograms for key continuous variables
• Scatter plots, correlation matrices
• Panel trends, pre-treatment comparisons if applicable
• Save all diagnostic figures to

  output/diagnostics/

Phase 3: Main Analysis

• Panel data: use

  fixest

  ; cross-section: use

  lm

  /

  glm

• Cluster SEs at the appropriate level (document why)
• Multiple specifications: start simple, progressively add controls
• Report standardized effects alongside raw coefficients

Phase 4: Publication-Ready Output

Tables:

modelsummary

(preferred) or

stargazer

— export

.tex

and

.html

Figures:

ggplot2

with project theme; explicit

ggsave(width = X, height = Y)

; save as

.pdf

and

.png

; add

bg = "transparent"

only if output is for Beamer slides

Phase 5: Save and Review

1. saveRDS()

   for all key objects
2. Run the

   r-reviewer

   agent: "Review the script at scripts/R/[script_name].R"
3. Address Critical and High issues before presenting results

R Script Template

# ============================================================
# [Descriptive Title]
# Author: [from project context]
# Purpose: [What this script does]
# Inputs:  [Data files]
# Outputs: [Figures, tables, RDS files]
# ============================================================

# 0. Setup ----
library(tidyverse)
library(fixest)
library(modelsummary)

set.seed(42)
dir.create("output/analysis", recursive = TRUE, showWarnings = FALSE)

# 1. Data Loading ----
# 2. Exploratory Analysis ----
# 3. Main Analysis ----
# 4. Tables and Figures ----
# 5. Export ----

---

Python Track

Constraints

• Save scripts to

  scripts/python/

  with descriptive names
• Save all outputs (figures, tables, pickles) to

  output/

• Use

  joblib.dump()

  for model objects;

  .to_parquet()

  for DataFrames
• Use

  pathlib.Path

  for all file paths — never hardcode absolute paths
• Set random seeds at the top of the script

Phase 1: Setup and Data Loading

1. Create Python script with header (title, author, purpose, inputs, outputs)
2. Import all packages at the top of the file
3. Set seeds:

   np.random.seed(42)

   and

   random.seed(42)

4. Create output directories:

   Path("output/analysis").mkdir(parents=True, exist_ok=True)

5. Load and inspect the dataset with

   pandas

Phase 2: Exploratory Data Analysis

• df.describe()

  ,

  df.isnull().sum()

  ,

  df.dtypes

• Histograms and distributions with

  matplotlib

  /

  seaborn

• Scatter plots and correlation matrices
• Save diagnostic figures to

  output/diagnostics/

• Save summary stats:

  df.describe().to_csv("output/diagnostics/summary_stats.csv")

Phase 3: Main Analysis

• Cross-section OLS:

  smf.ols("y ~ x", data=df).fit(cov_type="HC3")

• Panel data:

  PanelOLS

  from

  linearmodels

  with cluster-robust SEs
• Multiple specifications: build incrementally
• Document SE choice with a comment

Phase 4: Publication-Ready Output

Tables: Format with

pandas

and export via

.to_latex()

or

stargazer

(Python port) Figures:

matplotlib

/

seaborn

; explicit

fig.savefig(path, dpi=300, bbox_inches="tight")

; save as

.pdf

and

.png

Phase 5: Save and Review

1. joblib.dump(model, "output/model.pkl")

   for fitted models

2. df_results.to_parquet("output/results.parquet")

   for DataFrames
3. Review the script manually against the Python checklist below before presenting

Python Script Template

# ============================================================
# [Descriptive Title]
# Author: [from project context]
# Purpose: [What this script does]
# Inputs:  [Data files]
# Outputs: [Figures, tables, pickle/parquet files]
# ============================================================

import random
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt
import seaborn as sns
import joblib
from pathlib import Path

# Seeds
np.random.seed(42)
random.seed(42)

# Output directories
Path("output/analysis").mkdir(parents=True, exist_ok=True)
Path("output/figures").mkdir(parents=True, exist_ok=True)

# 1. Data Loading
# 2. Exploratory Analysis
# 3. Main Analysis
# 4. Tables and Figures
# 5. Export

Python Quality Checklist

[ ] All imports at top
[ ] Random seeds set (numpy + stdlib)
[ ] All paths use pathlib.Path — no hardcoded strings
[ ] Output directories created with mkdir(exist_ok=True)
[ ] Figures saved with explicit dpi=300, bbox_inches="tight"
[ ] Model objects saved with joblib.dump()
[ ] DataFrames saved as parquet
[ ] Comments explain WHY, not WHAT

---

Shared Principles

• Reproduce, don't guess. If the user specifies a regression, run exactly that.
• Show your work. Compute summary statistics before jumping to regression.
• Check for issues. Look for multicollinearity, outliers, perfect prediction, missing data.
• Use relative paths. All paths relative to repository root.
• No hardcoded values. Use variables for sample restrictions, date ranges, thresholds.