Awesome-Agent-Skills-for-Empirical-Research election-data-source-countypres

install
source · Clone the upstream repo
git clone https://github.com/brycewang-stanford/Awesome-Agent-Skills-for-Empirical-Research
Claude Code · Install into ~/.claude/skills/
T=$(mktemp -d) && git clone --depth=1 https://github.com/brycewang-stanford/Awesome-Agent-Skills-for-Empirical-Research "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/17-DAAF-Contribution-Community-daaf/dot-claude/skills/election-data-source-countypres" ~/.claude/skills/brycewang-stanford-awesome-agent-skills-for-empirical-research-election-data-sou && rm -rf "$T"
manifest: skills/17-DAAF-Contribution-Community-daaf/dot-claude/skills/election-data-source-countypres/SKILL.md
source content

County Presidential Data Source Reference

County Presidential Election Returns 2000-2024 from MIT Election Data and Science Lab (MEDSL). Use when analyzing county-level presidential vote shares, party trends, turnout, or geographic voting patterns. Key join column county_fips enables linking to census, education (CCD/SAIPE), and demographic datasets. Requires Harvard Dataverse API key (HARVARD_DATAVERSE_API_KEY env var). Categorical variables use uppercase strings, not Portal integer codes. Critical caveat: naive mode='TOTAL' filtering silently drops ~1,000 counties in 2020+ data — use 3-pattern reconstruction.

The authoritative source for county-level U.S. presidential election returns spanning 2000-2024. Provides candidate-level vote counts across all 50 states and DC, enabling vote share analysis, partisan trend mapping, and cross-domain geographic research via FIPS code joins.

CRITICAL: Value Encoding

This dataset uses uppercase string codes for categorical variables (party, mode, candidate, state) rather than integer codes. Empty strings (

""
) appear as undocumented values in
party
(501 rows, 2024) and
mode
(2,795 rows, 2024).

Contextpartymodecandidate
Standard values
DEMOCRAT
,
REPUBLICAN
TOTAL
BARACK OBAMA
Aggregate/meta values
OTHER
,
""
""
,
ELECTION DAY
OTHER
,
UNDERVOTES

See

./references/variable-definitions.md
for complete encoding tables.

Prerequisites

API Key Required: This data source requires a Harvard Dataverse API key to fetch data. Unlike education data sources (which use the Urban Institute's free, unauthenticated API), election data is hosted on Harvard Dataverse and requires authentication.

Setup instructions:

  1. Create a free Harvard Dataverse account at https://dataverse.harvard.edu/
  2. Log in, navigate to your account name (top-right) → API Token
  3. Click "Create Token" and copy it
  4. Set the environment variable before launching Claude Code:
    export HARVARD_DATAVERSE_API_KEY="your_token_here"
    
    For Docker users: run this inside the container after
    docker compose exec daaf-docker bash
    but before
    claude
    . To make it persistent across sessions, add it to
    ~/.bashrc
    .

If the key is missing, any fetch script will fail with a

KeyError: 'HARVARD_DATAVERSE_API_KEY'
. The orchestrator should check for this variable's existence before dispatching Stage 5 fetch tasks that use this data source.

What is the MEDSL County Presidential Returns Dataset?

  • Producer: MIT Election Data and Science Lab (MEDSL)
  • Coverage: County-level presidential election returns, 50 states + DC
  • Frequency: Every 4 years (presidential election cycle)
  • Available years: 2000, 2004, 2008, 2012, 2016, 2020, 2024
  • Primary identifier:
    county_fips
    (5-digit FIPS code, stored as integer)
  • Record unit: One row per county-year-candidate-party-mode combination
  • Total records: 94,151 rows x 12 columns (~8.4 MB). Note: rows per year vary dramatically — 2020/2024 have ~2x rows due to mode breakdowns (~22K vs ~9.5K for earlier years)
  • Source: Harvard Dataverse (DOI: 10.7910/DVN/VOQCHQ)

Reference File Structure

FilePurposeWhen to Read
variable-definitions.md
Complete column specs, party/mode/candidate value tablesInterpreting specific columns or coded values
coded-values.md
All categorical value mappings with frequenciesFiltering or recoding party, mode, candidate
columns.md
Detailed per-column profiling (types, nulls, ranges)Understanding column characteristics
quality-notes.md
Known issues, anomalies, duplicates, null patternsAssessing data reliability
mode-reconstruction.md
3-pattern TOTAL mode reconstruction for 2020+ dataCleaning any 2020+ analysis (CRITICAL)
interpretations.md
Preliminary semantic interpretations (flagged for review)Understanding column meanings

Decision Trees

What analysis do I need?

Analyzing presidential election data?
├─ County-level vote shares → Use 3-pattern mode reconstruction (./references/mode-reconstruction.md)
│   └─ Longitudinal (cross-year) → MUST reconstruct TOTAL for 2020+ (naive filter drops ~1,000 counties)
│   └─ Single year (pre-2020) → Safe to filter mode='TOTAL'
│   └─ Single year (2020/2024) → Reconstruct unless analyzing a known TOTAL-only state
├─ Party trends → Group by year + party, use party column (not candidate name)
│   └─ Third parties → See ./references/coded-values.md (GREEN/LIBERTARIAN vary by year)
├─ Turnout analysis → Use totalvotes column (dedup per county-year before summing!)
├─ Joining with other data → Use county_fips as join key (zero-pad to 5 chars first!)
│   └─ Census/ACS data → Join on county_fips (standard 5-digit string)
│   └─ Education data (CCD/SAIPE) → Join on county_fips
│   └─ Null FIPS? → See ./references/quality-notes.md (CT, ME, RI)
└─ Voting method analysis → 2020 and 2024 only, see mode breakdown

Is this a data quality issue?

Unexpected values?
├─ county_fips is null → CT/ME/RI in pre-2020 years (52 rows)
├─ county_fips > 72999 → Kansas City MO (FIPS 2938000, non-standard)
├─ county_fips join failures → Zero-pad to 5 chars! (AR codes = 4 digits as int)
├─ AR FIPS 5135 has two counties → Source data error: St. Francis under Sharp County
│   └─ See ./references/quality-notes.md #arkansas-fips-contamination (BLOCKER)
├─ CT counties missing from shapefile → 2022+ TIGER uses planning regions, not counties
│   └─ See ./references/quality-notes.md #connecticut-fips-geography-mismatch
├─ ~1,000 counties missing after mode filter → Use 3-pattern reconstruction, not naive filter
│   └─ See ./references/mode-reconstruction.md
├─ candidate is not a person → UNDERVOTES/OVERVOTES/SPOILED/TOTAL VOTES CAST
│   └─ Filter these OUT for candidate-level analysis
├─ party is empty string → 501 rows in 2024, undocumented
├─ mode is empty string → 2,795 rows in 2024 (may be totals OR breakdowns per state)
├─ candidatevotes is null → 37 rows (NM 2024, mode breakdown)
├─ sum(candidatevotes) > totalvotes → 49 county-years (minor rounding)
├─ Duplicate rows → 83 exact duplicates exist
└─ Alaska 2004 → District-level data, not county; FIPS = 2001-2099
    └─ See ./references/quality-notes.md #alaska-2004

Quick Reference: Election Variables

Party Values by Year

YearDEMOCRATREPUBLICANLIBERTARIANGREENOTHER
""
2000YY-YY-
2004-2016YY--Y-
2020YYYYY-
2024YYY-YY

Mode Values by Year

YearModes Available
2000-2016
TOTAL
only
2020
TOTAL
+ 15 breakdown modes (11 states)
2024
TOTAL
+ 9 breakdown modes +
""
(varies by state)

For complete mode values see

./references/coded-values.md
.

Key Identifiers

IDFormatLevelExampleNotes
county_fips
Int64 (5-digit)County
6037
(LA County, CA)
52 nulls (CT/ME/RI); join key for census/education data
state_po
String (2-char)State
CA
USPS abbreviation; 1:1 with
state
state
StringState
CALIFORNIA
Full uppercase name

WARNING: FIPS Zero-Padding Required for Joins

county_fips
is stored as Int64. When converting to string for joins with Census, SAIPE, CCD, or other datasets, zero-pad to 5 characters or Arkansas and other small-FIPS states will produce 4-digit codes that fail to match.

df = df.with_columns(pl.col("county_fips").cast(pl.Utf8).str.zfill(5).alias("county_fips_str"))

Missing Data Codes

CodeColumn(s)MeaningFrequency
null
county_fips
FIPS not assigned52 rows (CT, ME, RI pre-2020)
null
candidatevotes
Vote count unavailable37 rows (NM 2024 mode breakdowns)
0
totalvotes
No votes recorded50 rows
0
candidatevotes
Zero votes for candidate3,908 rows (4.15%)
""
party
Party not specified501 rows (2024 only)
""
mode
Mode not specified2,795 rows (2024 only)

Non-Candidate Entries in candidate Column

ValueRowsMeaning
OTHER
27,548Aggregate of minor candidates
TOTAL VOTES CAST
427County total (redundant with totalvotes)
UNDERVOTES
402Ballots with no presidential selection
OVERVOTES
380Ballots with multiple presidential selections
SPOILED
14Invalidated ballots

Filter these out for candidate-level vote share analysis.

Best Practice: Party-Based Identification

Always use

party
column for identifying party affiliation, never
candidate
name. Candidate names are inconsistent across years (e.g., "DONALD TRUMP" in 2016 vs "DONALD J TRUMP" in 2020/2024). The
party
column (
DEMOCRAT
,
REPUBLICAN
) is stable across all years.

# CORRECT — stable across years
dem = df.filter(pl.col("party") == "DEMOCRAT")

# WRONG — misses 2016 or 2020/2024 depending on which name you use
trump = df.filter(pl.col("candidate") == "DONALD TRUMP")

Data Access

Dataset Paths

TopicTypePath
County presidential returnsSingle fileHarvard Dataverse DOI:
10.7910/DVN/VOQCHQ
CodebookSingle fileBundled:
County Presidential Returns 2000-2024.md
Sources per stateSingle fileBundled:
sources-president.tab

Codebooks

DatasetCodebook Path
County presidential returns 2000-2024
County Presidential Returns 2000-2024.md
(bundled in Dataverse)

Codebook is a Markdown file bundled in the Harvard Dataverse deposit. For human reference. The QA methodology paper is at: https://www.nature.com/articles/s41597-022-01745-0

Truth Hierarchy: When interpreting variable values, apply this priority:

  1. Actual data file (what you observe in the TSV) -- this IS the truth
  2. Live codebook (Markdown file in Dataverse) -- authoritative documentation, may lag
  3. This skill documentation -- convenient summary, may drift from codebook

If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

Example Fetch

# Fetch from Harvard Dataverse API
import os, requests, polars as pl, io

api_key = os.environ["HARVARD_DATAVERSE_API_KEY"]
# Get file ID from dataset metadata first, then download
# File: countypres_2000-2024.tab (TSV format)
file_url = "https://dataverse.harvard.edu/api/access/datafile/{file_id}"
r = requests.get(file_url, params={"key": api_key, "format": "original"})
df = pl.read_csv(io.BytesIO(r.content), separator='\t')

# Filter to California, 2020, TOTAL mode only
ca_2020 = df.filter(
    (pl.col("state_po") == "CA") &
    (pl.col("year") == 2020) &
    (pl.col("mode") == "TOTAL")
)

Filtering

# Common filter patterns for county presidential data

# 1. Cross-year analysis: WARNING — naive filter drops ~1,000 counties in 2020+!
#    Use 3-pattern mode reconstruction instead. See ./references/mode-reconstruction.md
#    The single-line filter below is ONLY safe for single-year analysis on a state
#    known to have TOTAL rows (e.g., 2000-2016 data, or a confirmed TOTAL-only state).
longitudinal = df.filter(pl.col("mode") == "TOTAL")  # UNSAFE for 2020+ multi-state!

# 2. Remove non-candidate rows (UNDERVOTES, OVERVOTES, etc.)
candidates_only = df.filter(
    ~pl.col("candidate").is_in(["TOTAL VOTES CAST", "UNDERVOTES", "OVERVOTES", "SPOILED"])
)

# 3. Major party analysis — RECOMMENDED: use party column, not candidate name
#    Candidate names change across years (e.g., "DONALD TRUMP" vs "DONALD J TRUMP")
two_party = df.filter(pl.col("party").is_in(["DEMOCRAT", "REPUBLICAN"]))

# 4. Exclude Alaska 2004 anomaly
clean = df.filter(~((pl.col("state_po") == "AK") & (pl.col("year") == 2004)))

# 5. Exclude rows with null county_fips (for join operations)
joinable = df.filter(pl.col("county_fips").is_not_null())

Common Pitfalls

PitfallIssueSolution
Cross-year mode mismatch2000-2016 has only
TOTAL
; 2020/2024 have breakdowns. Mixing modes inflates counts
Always filter
mode == 'TOTAL'
for longitudinal analysis
Non-candidate rows
UNDERVOTES
,
OVERVOTES
,
SPOILED
,
TOTAL VOTES CAST
appear as "candidates"
Filter out before computing candidate vote shares
Alaska 2004District-level data with non-standard FIPS codes (2001-2099); vote counts overstatedExclude AK 2004 or handle separately; do not join on county_fips
Null FIPS for joinsCT, ME, RI have null
county_fips
in some years; breaks joins
Use
state_po
+
county_name
as fallback join key
Kansas City MO FIPSUses non-standard FIPS 2938000 (not a real county FIPS)Handle as special case in joins; Kansas City is an independent city
Empty string party/mode2024 has
""
in party (501 rows) and mode (2,795 rows)
Treat as missing/undocumented; filter or investigate by state
Vote share > 100%49 county-years where sum(candidatevotes) > totalvotesMinor rounding; use caution with strict validation
Duplicate rows83 exact duplicate rows exist in datasetDeduplicate before analysis
FIPS zero-paddingInt64 → string without padding gives 4-digit codes for AR and other small-FIPS states
pl.col("county_fips").cast(pl.Utf8).str.zfill(5)
before joins
Name-based party IDCandidate names change across years ("DONALD TRUMP" vs "DONALD J TRUMP")Always use
party
column, never
candidate
name
CT FIPS mismatch2022+ Census TIGER uses CT planning regions, not legacy countiesUse 2020-vintage TIGER shapefiles for CT county joins
totalvotes duplication
totalvotes
is repeated per candidate row; summing without dedup inflates by ~3-4x
Deduplicate to one row per (county_fips, year) before aggregating

Critical: Mode Column Behavior Change

WARNING: Naive

mode == "TOTAL"
filtering drops ~1,000 counties in 2020+ data. Multiple states report ONLY mode breakdowns (no TOTAL rows). A simple filter silently removes all their counties. Use 3-pattern mode reconstruction instead. See
./references/mode-reconstruction.md
for the full code pattern and validation.

The

mode
column behavior changed significantly starting in 2020:

  • 2000-2016: All rows have
    mode = 'TOTAL'
    (aggregate county totals only)
  • 2020: 11 states report by voting method alongside TOTAL; 10 states have ONLY breakdowns (AR, AZ, GA, IA, KY, MD, NC, OK, SC, VA)
  • 2024: Additional breakdown states; some have empty string
    mode
    (which may represent totals OR breakdowns depending on the state)

For any cross-year or multi-county analysis, use 3-pattern mode reconstruction:

  1. Pattern 1: TOTAL present → keep TOTAL, drop breakdowns
  2. Pattern 2: Only breakdowns, no TOTAL → sum
    candidatevotes
    across modes
  3. Pattern 3: Empty-string mode = totals → reclassify after per-state verification

Empty-string detection must be per-state — NC 2024 empty-string rows are breakdowns (multiple per county-candidate), while other states' empty-string rows are totals (one per county-candidate). See

./references/mode-reconstruction.md
for detection logic and code.

Related Data Sources

SourceRelationshipWhen to Use
Census/ACSJoin via
county_fips
County demographics, population, income
SAIPE (
education-data-source-saipe
)
Join via
county_fips
County poverty estimates (cross-domain)
CCD (
education-data-source-ccd
)
Join via
county_fips
School district data (cross-domain)
MEDSL Precinct ReturnsFiner geographic resolutionWhen county-level is insufficient
MEDSL Senate/House ReturnsSame producer, different officeWhen analyzing down-ballot races

Note: This is the first election domain dataset in DAAF. Cross-domain joins with education data are possible via

county_fips
. No election-specific explorer or query skills exist yet.

Topic Index

TopicReference File
Column specifications
./references/columns.md
Column types and ranges
./references/columns.md
Party values and year coverage
./references/coded-values.md
Mode values and year behavior
./references/coded-values.md
Candidate name mapping
./references/coded-values.md
Non-candidate entries
./references/coded-values.md
Complete encoding tables
./references/variable-definitions.md
Null county_fips patterns
./references/quality-notes.md
Alaska 2004 anomaly
./references/quality-notes.md
Kansas City MO FIPS
./references/quality-notes.md
Duplicate rows
./references/quality-notes.md
Null candidatevotes
./references/quality-notes.md
Per-state data sources
./references/quality-notes.md
Missing votes flags
./references/quality-notes.md
Mode reconstruction (3-pattern)
./references/mode-reconstruction.md
States without TOTAL rows
./references/mode-reconstruction.md
Empty-string mode detection
./references/mode-reconstruction.md
Row count estimation by year
./references/mode-reconstruction.md
FIPS zero-padding for joins
./references/columns.md
AR FIPS contamination (Sharp/St. Francis)
./references/quality-notes.md
CT geography mismatch (2022+)
./references/quality-notes.md
Party-based identification (best practice)
./references/coded-values.md
totalvotes deduplication
./references/variable-definitions.md
Preliminary interpretations
./references/interpretations.md
Data profiling scripts
./scripts/