Methodology Memo

Analytical Question and Framing

Question Type Determination

What kind of question is the agency asking? Is this descriptive (what happened to PM2.5 levels?), inferential (did PM2.5 significantly change after the regulation?), or causal (did the regulation cause the change in PM2.5)?

Alternative Framings Considered

What other framings did you consider? What would each produce? Why were they set aside?

Rationale for Chosen Framing

Why does this framing serve the agency's decision? What are the limits of what this framing can claim?

Data Sources and Quality

Dataset Descriptions

What data are you working with? What does each dataset contain?

Known Data Quality Issues

What quality issues have you discovered? How do they affect the analysis? How are you handling them?

Preparation Decisions

Aggregation Strategy

What time resolution are you using? Why?

Confounder Controls

What confounders are you controlling for? How?

Station Selection

Are you using all stations or a subset? Why?

Methodology

Model Specification

What model are you fitting? What are the key parameters?

Seasonal Controls

How are you accounting for seasonal patterns?

Weather Controls

How are you accounting for weather effects?

Trend Specification

How are you modeling the long-term trend vs the regulation's effect?

Validation Strategy

Assumption Checks

What assumptions does your model require? What tests are you running? What do the results mean?

Sensitivity Analysis

What alternative specifications or parameters are you testing? How robust are the findings?

Cross-Model Review

What did a second AI perspective find when reviewing the methodology?

Limitations

What can this analysis claim and what can't it claim? What would make the evidence stronger?

Findings Summary

What did you find? State the key results with effect sizes and confidence intervals.