Post-hoc/Secondary Analysis Study Lifecycle
This guide walks through the complete process of conducting a post-hoc or secondary research with SafeInsights.
[Diagram showing 8 steps with estimated timelines]
Step 1: Define Your Research Question (1-2 weeks)
Start with a clear, focused research question that can be answered with observational data.
Good examples:
- "How does frequency of practice predict final exam scores in college algebra?"
- "What patterns of resource use are associated with course completion?"
Not yet supported:
- "Does my new intervention improve learning?" (requires A/B test)
- "How do OpenStax and Khan Academy compare?" (requires fusion)
Resources:
- Post-hoc Research Guide
- Data Catalog to check feasibility
Step 2: Check Data Feasibility (1-2 weeks)
Verify that the data you need exists and is accessible.
Actions:
- Review the Data Catalog for relevant members
- Check what data types are available (events, assessments, demographics)
- Review any limitations or constraints
- Access simulated data to understand data structure
Key Questions:
- Does a member have the variables I need?
- Is the sample size adequate?
- Are there temporal coverage gaps?
- What demographic data is available?
Resources:
- Organization-specific getting started guides in Data Catalog
- Simulated data (links in each org's section)
Step 3: Draft Your Study Concept (1-2 weeks)
Develop a detailed study design.
What to include:
- Research question and hypotheses
- Target population and sample
- Variables and measures
- Analytic approach
- Expected outputs
Resources:
Step 4: Submit Proposal (Day 1)
Submit your proposal through the SafeInsights Management App.
[Link to Management App]
Your proposal will be reviewed by:
- SafeInsights governance (privacy, FEAT compliance)
- Member organization (data access policies, priorities)
Step 5: Proposal Review (2-4 weeks)
Both SafeInsights and the member organization review your proposal.
Review criteria:
- Scientific merit and feasibility
- Privacy protections adequate
- FEAT compliance
- Alignment with member priorities
- Outputs meet privacy thresholds
Possible outcomes:
- ✅ Approved → Proceed to Step 6
- 🔄 Revisions needed → Address feedback and resubmit
- ❌ Declined → Receive explanation and guidance
Step 6: Develop & Test Container (2-6 weeks)
Write your analysis code and package it as a container.
Actions:
- Download simulated data for your target member
- Write analysis code (Python, R, or SQL)
- Test locally with simulated data
- Package as container
- Document your code
- Submit for review
Resources:
- Simulated data in Data Catalog
- OpenStax R Research Container example
- OpenStax Sample Research Container
- SQL Tutorial
Code Review: Member organization reviews your code for:
- Security (no data exfiltration attempts)
- Privacy compliance
- Correct use of data
Step 7: Container Execution (Minutes to hours)
Once approved, your container runs on real data within the secure enclave.
What happens:
- Container spins up in isolated environment
- Reads data (read-only access)
- Executes your analysis
- Generates outputs
- Outputs queued for review
Duration: Depends on data volume and complexity (typically minutes to hours)
Step 8: Output Review & Release (1-2 weeks)
Before you receive results, outputs are reviewed to ensure privacy compliance.
Review process:
- Automated checks for minimum cell sizes
- Human review for any edge cases
- Verification that outputs match approved proposal
You receive:
- Aggregated statistics
- Summary tables
- Statistical model results
- Plots and visualizations
You do NOT receive:
- Individual-level data
- Small cell counts
- Any identifiable information
Resources:
Step 9: Analyze & Publish (Ongoing)
Use your approved outputs for analysis and publication.
Publishing considerations:
- Cite SafeInsights appropriately
- Acknowledge the data organization
- Share code and analysis approach (where possible)
- Consider preregistration (optional)
Typical Timeline
| Phase | Duration |
|---|---|
| Feasibility & Design | 2-4 weeks |
| Proposal Review | 2-4 weeks |
| Container Development | 2-6 weeks |
| Container Execution | Hours to 1 day |
| Output Review | 1-2 weeks |
| Total | ~2-4 months |
Tips for Success
- Start with feasibility: Don't invest time in a full proposal before checking data availability
- Use simulated data early: Develop code locally before submission
- Be specific in proposals: Vague proposals take longer to review
- Plan for iteration: First studies often need revision—that's normal
- Engage early: Use support resources to get feedback