The call for evidence-based research in education has accelerated in recent years, accompanied by the need to examine the nature of inquiry into student learning. Donovan and Pellegrino (2003) make a strong appeal to improve the quality of classroom research, stating that “education needs high-quality research if the results are to be reliable for the purposes of improving practice”. The challenge for a researcher in the classroom is, how can one generate valid and reliable data to address a hypothesis when the research venue is variable and data are messy? In order to address this question, educators must establish standard methods and experimental designs for research that can be applied in many classroom settings. Our goal in this article is to analyze several different experimental designs and evaluate the validity of each approach in the context of an example research question.