Now, the venerable research institute, SRI International, is taking a crack at this question. In August, it published Phase 1 of an ongoing study to examine "to what extent, why, when, and how [italics in original] online learning models that target Algebra 1 work or do not work for different student populations."
Funded by the Gates Foundation, the report, "Supporting K-12 Students in Online Learning: A Review of Online Algebra I Courses," looks at six popular online providers of Algebra I courses, which present "a mature segment of the online learning market." These six providers offer fully-packaged courses designed as an alternative (and not a supplement) to traditional, in-person classes. The enrollments for these profiled companies range from 18,000+ in Michigan to half a million students around the U.S.
The authors open with an overview of existing pedagogical research on content, instructional approach, implementation and student outcomes, which form the basic framework on which the companies were evaluated on. They also dive into nitty gritty details related to the user experience, such as the number of words in a sample lesson (p. 33) and the number of clicks it takes to get from log-in to lesson (p. 24). (Of the six, "only one remembers where a student left off and takes her directly to the right place.")
Notable areas where the authors found room for improvement include:
The second part of the report profiles each of the six company in deeper detail and offers color-coded grades based on six characteristics (standards alignment, instructional approach, media design, assessment, feedback). Perhaps of most immediate use for teachers are additional info related to basic technology requirements, length of course, and cost.
These profiles were based on interviews with company representatives and SRI's independent review of the courses. Teacher reviews and feedback were not included.
This report does not offer quantifiable measures of efficacy and student success, such as passing rates. Those numbers will have to wait until Phase 2, when SRI says it "will use system use and outcomes data to empirically validate whether or not the features identified in Phase 1 are associated with student success." We'll be keeping a close watch, particularly in light of highly publicized criticisms over one of its profiled vendors, K12.
While the lengthy report demands time to digest, the organized presentation makes it worthwhile not only for teachers and parents, but for researchers and entrepreneurs as well. It offers a useful template on how to provide useful details about product offerings in a transparent manner. A summary of recommendations is available for both developers and adopters of online courses (p. 48).