What is post-hoc or secondary Analysis?
Post-hoc or secondary analysis examines existing data that was already collected through normal use of an education platform. You're studying patterns that occurred naturally—not running experiments or changing the student experience.
Understanding the Approach
When you conduct post-hoc analysis, you're asking: "What patterns exist in data that already happened?" You're observing and interpreting, not testing interventions.
Example post-hoc questions:
- How does time spent on practice problems relate to test performance?
- What behaviors predict students who will struggle before midterms?
- Do engagement patterns differ between morning and evening learners?
NOT post-hoc (experimental):
- Does giving students personalized hints improve their scores? (requires A/B test)
- Will a new dashboard feature increase engagement? (requires intervention)
Post-hoc vs. Experimental Research
| Dimension | Post-hoc Analysis (SafeInsights v1) | Experimental Research (Future) |
|---|---|---|
| Data source | ✅ Historical platform data | ❌ Data from planned intervention |
| Random assignment | ❌ Cannot assign conditions | ✅ Can randomly assign treatments |
| Causality claims | ❌ Correlation only | ✅ Can establish causation |
| Platform changes | ❌ Cannot modify experience | ✅ Can control student experience |
| Timeline | ✅ Analyze immediately | ❌ Must wait for data collection |
| Scale | ✅ Often very large (thousands-millions) | Varies (may be smaller) |
What This Means for Your Research
You can answer:
- Descriptive questions (what happened?)
- Correlational questions (what goes together?)
- Predictive questions (what forecasts outcomes?)
- Comparative questions (do groups differ?)
You cannot answer:
- Causal questions that require manipulation (does X cause Y?)
- Questions about new features that don't exist yet
- Questions requiring specific experimental conditions
💡 Important: Post-hoc analysis can still yield valuable causal insights using methods like regression discontinuity, instrumental variables, or natural experiments—but the design requires careful thought about confounding variables.
Six Common Post-hoc Research Patterns
Pattern 1: Engagement and Performance Relationships
Research question: How does student engagement with learning resources relate to performance outcomes?
Example: "How does time spent on practice problems correlate with quiz performance?"
Data typically needed:
- Event logs (resource access, page views)
- Assessment scores
- Student identifiers for linking
Common methods: Correlation, multiple regression, structural equation modeling
Key limitation: Students who engage more may differ in motivation—correlation doesn't prove engagement causes better performance.
See full pattern details in research guide →
[Patterns 2-6 would follow same structure...]
Assessing Feasibility for Your Question
Use this checklist to evaluate if post-hoc analysis can answer your question:
✅ Likely feasible if:
- Your question asks about relationships, patterns, or predictions
- The behaviors/outcomes you care about were already captured by a platform
- You're comfortable with correlational rather than causal conclusions
- You need large samples that already exist
⚠️ Questionable feasibility if:
- You need very specific demographic variables that may not be collected
- Your question requires data from multiple platforms (not yet supported)
❌ Not feasible if:
- You need to manipulate or change the student experience
- You want to test something that doesn't exist yet
- You require random assignment to conditions
💡 Still unsure? Browse the Data Catalog to see what data exists, or contact us for a feasibility consultation.
Next Steps
Explore specific research patterns
Post-hoc research guide - Detailed examples with methods and data needs
Check data availability
Data Catalog - See what platforms offer the data you need
Understand the process
Study lifecycle - Learn the steps from idea to results
Get support
Last updated: December 2024