DS Plan (Data Science Planning)

Data Science Planning Phase

ds-plan is the critical second phase of the /ds workflow, sitting between brainstorming and implementation. It focuses on rigorous data profiling and the creation of an executable analysis plan.

Core Methodology:

• The Iron Law: Mandates that a SPEC.md must exist before any planning begins.
• Rigorous Profiling: Moves beyond .head() samples to identify nulls, type drift, and grain problems in the tail and rare groups.
• ETL Strategy: Triggered for large datasets (>1M rows), focusing on filter push-down, parallelism, and intermediate caching.
• External Skill Discovery: Prevents 'greenfield' duplication by forcing a search for existing battle-tested examples in other skills.
• Data Pull Profiling: A mandatory gate for datasets >= 50M rows to decide between raw pulls and server-side aggregation.
• Master Dataset Design: For multi-exhibit projects, it defines minimal canonical datasets to ensure consistency across all tables and figures.
• Parameter Inventory: Centralizes all analysis constants (filters, thresholds) into a single config file to eliminate 'magic numbers'.

Outcome: Produces a machine-executable .planning/PLAN.md with a detailed task breakdown and deterministic verification commands.