Open Science Guide

Implement open science practices including study pre-registration, open data sharing, registered reports, and FAIR data principles to increase research transparency and reproducibility.

Why Open Science?

Open science practices address the replication crisis and increase trust in research findings:

Practice	Problem It Addresses
Pre-registration	Prevents HARKing (hypothesizing after results are known) and p-hacking
Open data	Enables verification, reanalysis, and meta-analyses
Open materials	Allows exact replication of studies
Open access	Removes paywalls that limit access to knowledge
Registered reports	Eliminates publication bias (acceptance before results are known)
Open code	Enables computational reproducibility

Pre-Registration

What to Pre-Register

Pre-registration commits you to your research plan before seeing the data:

Pre-registration template (standard fields):

1. HYPOTHESES
   - H1: [Specific, directional hypothesis]
   - H2: [Another hypothesis]

2. DESIGN
   - Study type: [Experiment / Survey / Observational]
   - Between/within subjects design: [Details]
   - Conditions: [List experimental conditions]

3. SAMPLING PLAN
   - Sample size: [N = X, justified by power analysis]
   - Stopping rule: [When will data collection stop?]
   - Inclusion/exclusion criteria: [List]

4. VARIABLES
   - Independent variables: [List with levels]
   - Dependent variables: [List with measurement details]
   - Covariates: [List any control variables]

5. ANALYSIS PLAN
   - Primary analysis: [Exact statistical test, e.g., "2x3 mixed ANOVA"]
   - Secondary analyses: [Additional planned analyses]
   - Inference criteria: [alpha level, correction for multiple comparisons]
   - Exclusion criteria: [How will outliers or failed attention checks be handled?]
   - Missing data: [How will missing data be handled?]

6. OTHER
   - Exploratory analyses: [Analyses not tied to specific hypotheses]

Where to Pre-Register

Platform	URL	Disciplines	Features
OSF Registries	osf.io/registries	All	Free, flexible templates, versioned
AsPredicted	aspredicted.org	Social sciences, psychology	Simple 9-question form, private until shared
ClinicalTrials.gov	clinicaltrials.gov	Clinical research	Required for clinical trials (FDA)
PROSPERO	crd.york.ac.uk/prospero	Systematic reviews	Health-related reviews only
AEA RCT Registry	socialscienceregistry.org	Economics	RCTs in social sciences

Pre-Registration Workflow

1. Design your study
2. Write the pre-registration document
3. Have a colleague review it
4. Submit to a registration platform
5. Receive a time-stamped registration (URL + DOI)
6. Collect and analyze data following the pre-registered plan
7. Report results transparently:
   - Confirmatory analyses (pre-registered)
   - Exploratory analyses (clearly labeled as exploratory)
8. Link the pre-registration in your manuscript

Registered Reports

Registered Reports are a publication format where peer review occurs before data collection:

Stage 1 (Before Data Collection):
  - Submit introduction, methods, and analysis plan
  - Peer review evaluates the research question and methodology
  - If accepted: "In-Principle Acceptance" (IPA)
  - Paper will be published regardless of results

Stage 2 (After Data Collection):
  - Collect data following the approved protocol
  - Analyze and report results
  - Add discussion section
  - Final peer review checks adherence to protocol
  - Publication

Over 300 journals now accept Registered Reports. Check the registry at cos.io/rr.

Benefits of Registered Reports

• Eliminates publication bias (null results are published)
• Ensures methodological rigor is reviewed before sunk costs
• Prevents post-hoc changes to hypotheses or analyses
• Provides certainty of publication to researchers

FAIR Data Principles

FAIR principles ensure research data is Findable, Accessible, Interoperable, and Reusable:

Findable

- F1: Data are assigned a globally unique, persistent identifier (DOI)
- F2: Data are described with rich metadata
- F3: Metadata include the identifier of the data
- F4: Data are registered or indexed in a searchable resource

Actions:
- Deposit data in a repository that assigns DOIs
- Write a comprehensive README and data dictionary
- Use standard metadata schemas (Dublin Core, DataCite)

Accessible

- A1: Data are retrievable by their identifier using open protocols (HTTP)
- A2: Metadata remain accessible even if data are no longer available

Actions:
- Use established repositories (not personal websites)
- Specify access conditions clearly (open, restricted, embargoed)
- Even if data cannot be shared, publish metadata describing them

Interoperable

- I1: Data use a formal, accessible, shared language (e.g., CSV, JSON, RDF)
- I2: Data use vocabularies that follow FAIR principles
- I3: Data include qualified references to other data

Actions:
- Use standard file formats (CSV, not proprietary Excel)
- Use standard variable names and coding schemes
- Link to related datasets using DOIs

Reusable

- R1: Data are richly described with provenance information
- R2: Data are released with a clear, accessible data usage license
- R3: Data meet domain-relevant community standards

Actions:
- Include a data dictionary with variable descriptions
- Apply a license (CC-BY 4.0 recommended)
- Describe data collection procedures, cleaning steps, and known issues
- Include analysis code alongside data

Data Sharing Platforms

Repository	Disciplines	Max Size	DOI	Cost
Zenodo	All	50 GB	Yes	Free
Dryad	All (focus on sciences)	Unlimited	Yes	Sliding scale
Figshare	All	20 GB (free)	Yes	Free/institutional
OSF	All	5 GB (free)	Yes	Free
Harvard Dataverse	All (focus on social science)	2.5 GB per file	Yes	Free
ICPSR	Social science	Varies	Yes	Free deposit
GenBank	Genomics	N/A	Accession numbers	Free
Protein Data Bank	Structural biology	N/A	PDB IDs	Free

Data Sharing Best Practices

README Template for Data Deposits

# Dataset: [Title]

## Description
Brief description of the dataset and the study it comes from.

## Citation
If you use this data, please cite:
[Full citation of the associated publication]

## File Description
- `data_raw.csv` - Raw data as collected (N = 500, 45 variables)
- `data_processed.csv` - Cleaned data after exclusions (N = 467, 38 variables)
- `codebook.csv` - Variable descriptions, types, and valid ranges
- `analysis_script.R` - Complete analysis code reproducing all results

## Variables (data_processed.csv)
| Variable | Type | Description | Valid Range |
|----------|------|-------------|-------------|
| participant_id | string | Unique participant identifier | P001-P500 |
| age | integer | Age in years | 18-65 |
| condition | categorical | Experimental condition | control, treatment_a, treatment_b |
| score_pre | numeric | Pre-test score | 0-100 |
| score_post | numeric | Post-test score | 0-100 |

## Missing Data
- 33 participants excluded for failing attention checks
- 12 missing values in `score_post` (participants did not complete)
- Missing coded as NA

## License
CC-BY 4.0 International

## Contact
[Name, email, ORCID]

Sensitive Data Considerations

When data cannot be fully shared (e.g., due to participant privacy):

1. Anonymize: Remove direct identifiers (name, email, IP) and indirect identifiers (rare combinations of demographics)
2. Aggregate: Share summary statistics or aggregated data instead of individual-level data
3. Restricted access: Deposit data with access controls (e.g., ICPSR restricted-use data)
4. Synthetic data: Generate synthetic datasets that preserve statistical properties
5. Controlled access: Use data use agreements (DUAs) for sensitive data
6. Code without data: At minimum, share analysis code so methods are transparent

Open Science Badges

Many journals award badges for open science practices:

Badge	Meaning
Open Data	Data publicly available
Open Materials	Research materials publicly available
Preregistered	Study pre-registered before data collection
Preregistered + Analysis Plan	Preregistered with detailed analysis plan

These badges (developed by COS) appear on published articles and signal commitment to transparency.