UCLA and MIT's PLanet Catches Broken Study Designs Before You Recruit Anyone
PLanet is a domain-specific language that treats experimental designs as matrix constraints, catching hidden assumptions before you run a single trial.
ProTakeaways
- PLanet is a new DSL that treats experimental designs as matrix constraints solved by Z3.
- Static analysis checks which main, interaction, and time-based effects a design can actually test.
- Expressed 14 of 15 sampled CHI and UIST experiments, beating edibble (12/15) and Touchstone 2 (8/15).
- Composable operators include counterbalance, cross, nest, order, limit_plans, and multifact.
- Available via
pip install planet-dsl, with a paper at ACM UIST. - Limits: no power analysis, no adaptive trials, and grammar is tuned to human-subjects HCI studies.
Designing a user study looks straightforward until you realize the Latin square you copied from a previous paper quietly assumed your interface effect does not depend on the task type. Get that wrong and your causal claim falls apart, no matter how clean the p-value looks. A new domain-specific language called PLanet, from researchers at UCLA, MIT, and UMass Amherst, tries to fix this by turning experimental designs into matrix constraints that a solver can reason about, and by statically checking which causal effects your design can actually test.
PLanet implements a grammar of composable operators for constructing experimental assignment procedures grounded in matrix algebra, and compiles programs into constraint satisfaction problems over matrices. The payoff is a static analysis that tells you, before you recruit a single participant, whether the design you wrote down can answer the question you actually care about.
The assumption problem nobody writes down
Every experimental design encodes assumptions about how variables interact and how time affects outcomes. A within-subjects design assumes carry-over effects are manageable. A Latin square assumes the effect of the interface does not depend on the task. Current workflows require the researcher to be aware of a design's implicit assumptions and manually ensure that their domain assumptions and causal queries align. As soon as you add a second or third variable, that manual reasoning becomes intractable, and researchers tend to default to whatever design they saw in a paper last week.
Existing tools do not help much. Some constrain researchers to predefined designs, others offer fine-grained control but require low-level specification, and none support detecting which causal effects the experiment can test for. Tools like PlanAlyzer check validity but target large-scale online field experiments, not the small-N human-subjects studies where within-subjects tricks and counterbalancing dominate.