Postsecondary Learning

Udacity's Lessons Learned

By Betsy Corcoran     Jul 24, 2013

Udacity's Lessons Learned

It doesn't much matter whether you cite Spiderman's Stan Leeor France's Voltaire for that deeply resonate quote: "With great powercomes great responsibility."  Theline was echoing in the heads of high profile Udacity's founders over the pastfew days as news broke of the poor exam results of many students takingUdacity's classes offered in conjunction with San Jose State University.

In a 1:1 interview with EdSurge, Udacity founder and CEOSebastian Thrun said he is drawing many lessons from the experience--startingwith how the classic semester schedule may work against students, the startlinggaps in what students know and the logistical challenges of delivering onlineeducation.

None of those answers soften a harsh truth: many of thestudents who worked at these classes failed. Two SJSU professors describe the dismal passing rates (29%, 44% and 51%) in this piece in the San Jose Mercury News.

That outcome is disappointing, concedes Thrun, who has saidmany times that he hopes that Udacity can genuinely help students learn, regardless oftheir background.  Althoughthe program has "paused" until Udacity and the university can figureout how to best structure it, Thrun maintains that he's determined to bringUdacity back to SJSU.  

So what happened? In January, Thrun and California Gov.Jerry Brown unveileda high profile deal in which Udacity would provide course support for several remedial courses. Udacity would support three classes, created byprofessors at SJSU. Each class would have 100 students, half from SJSU and halffrom nearby community colleges and high schools. The students selected were not"typical" community college students: in many cases, they had alreadyfailed a remedial class or a college entrance exam. SJSU professors woulddesign the course curriculum. Eventually the classes would cost $150 apiece.But at least on this iteration, foundations stepped in to cover that cost forstudents.

The professors creating the curriculum for the programdidn't have much time; they were still writing curriculum when the coursesbegan. "It was really hard on the faculty," Thrun says. "Theamount of work they put in was admirable: they spent about 400 hours creatingthese courses." Whatever gaffs and bugs showed up in the curriculum wasnot a problem--just part of the organic back and forth of courses. "We hada whole bunch of clerical mistakes. In most cases we heard about it, and fixedit on the fly. It happens in the classroom as well," Thrun says. "Mostof those will never show up again."  

Then the unanticipated problems started to crop up. When thecourses started, two of the three classes didn’t give students precise deadlines forassignments. "We communicated our expectations poorly," concedesThrun. "We had two deadline-free courses. Especially in these classes,students fell behind. That was a mistake," Thrun declares.

The Udacity team began setting tougher deadlines andaggressively reaching out to students to help them make up for lost time."We called every single student, sent out text messages and introduceddeadlines," recalls Thrun. The effect was big: students caught up to theirclasses.

That push helped in another way: initially many studentswere unaware of the online tutors (who are real people) who were available online to help, 12 hours a day. But over the weeks, it became clear that thetutoring services were crucial. "The mentoring program was absolutelyessential for every student's outcome," Thrun says.

Those tutors were so important because many students lacked evenelementary-school-level mathematics knowledge, Thrun says. Among the frequentlyheard questions: How to divide twonumbers? How can you subtract a bigger number from a smaller one? Tutorsanswered questions and continued to reach out to encourage students. 

As the class progressed, Udacity also realized that many ofthe students simply couldn't get to a computer regularly enough. For somestudents, says Thrun, "there were none in the home, [and] even in school they couldn't getthe hours needed to make progress. We had to work with the schools directly (toarrange for computer time for Udacity students.) It was actually a bigdeal." A June articlein the Oakland Tribune outlined some of these challenges.

When students did get to the online programs, even navigating the computer systems could be daunting. One of the questions that tutors were frequently asked was how todo exponential notation on a computer. And although the curriculum wasdelivered on Udacity's platform, assessments were delivered via a separatelearning management system used by SJSC. Worse: results on one system tookabout 48 hours to update on the other, says Thrun.

All that said, when students were surveyed, they said thebiggest impediment to succeeding in the class was pacing: they just didn't haveenough time. "Sal Khan has strong datathat says in math in particular, a more flexible pacing is important forsuccess," Thrun says. "He's been preaching go at your own pace andyou can turn a C-level student to an A-level student." Thrun also pointedto Foothill College's"Math My Way" program which has been able to double its student-passrates by giving students more time.

Thrun is proud of the number of students who completed theclass--83% of the students finished, a dramatic increase from the usual,single-digit complete rates that MOOCs have seen. Yet the fact that thestudents continued to try makes the failed final exams even more poignant:"Yes, it's an enormous failure rate," he concedes. "I won'tsleep."

The final analysis of the data from the courses isn't dueback from the independent auditor for another two weeks. Once that data isavailable, Thrun says, he will develop an action plan for carrying on. "Let'simprove infrastructure. And address the timing and pacing. I don't want to lockus into a semester frame if we shouldn't do this." Even so, he adds,there's "no doubt that we will carry on."

Postsecondary Learning

Udacity's Lessons Learned

By Betsy Corcoran     Jul 24, 2013

Udacity's Lessons Learned

It doesn't much matter whether you cite Spiderman's Stan Leeor France's Voltaire for that deeply resonate quote: "With great powercomes great responsibility."  Theline was echoing in the heads of high profile Udacity's founders over the pastfew days as news broke of the poor exam results of many students takingUdacity's classes offered in conjunction with San Jose State University.

In a 1:1 interview with EdSurge, Udacity founder and CEOSebastian Thrun said he is drawing many lessons from the experience--startingwith how the classic semester schedule may work against students, the startlinggaps in what students know and the logistical challenges of delivering onlineeducation.

None of those answers soften a harsh truth: many of thestudents who worked at these classes failed. Two SJSU professors describe the dismal passing rates (29%, 44% and 51%) in this piece in the San Jose Mercury News.

That outcome is disappointing, concedes Thrun, who has saidmany times that he hopes that Udacity can genuinely help students learn, regardless oftheir background.  Althoughthe program has "paused" until Udacity and the university can figureout how to best structure it, Thrun maintains that he's determined to bringUdacity back to SJSU.  

So what happened? In January, Thrun and California Gov.Jerry Brown unveileda high profile deal in which Udacity would provide course support for several remedial courses. Udacity would support three classes, created byprofessors at SJSU. Each class would have 100 students, half from SJSU and halffrom nearby community colleges and high schools. The students selected were not"typical" community college students: in many cases, they had alreadyfailed a remedial class or a college entrance exam. SJSU professors woulddesign the course curriculum. Eventually the classes would cost $150 apiece.But at least on this iteration, foundations stepped in to cover that cost forstudents.

The professors creating the curriculum for the programdidn't have much time; they were still writing curriculum when the coursesbegan. "It was really hard on the faculty," Thrun says. "Theamount of work they put in was admirable: they spent about 400 hours creatingthese courses." Whatever gaffs and bugs showed up in the curriculum wasnot a problem--just part of the organic back and forth of courses. "We hada whole bunch of clerical mistakes. In most cases we heard about it, and fixedit on the fly. It happens in the classroom as well," Thrun says. "Mostof those will never show up again."  

Then the unanticipated problems started to crop up. When thecourses started, two of the three classes didn’t give students precise deadlines forassignments. "We communicated our expectations poorly," concedesThrun. "We had two deadline-free courses. Especially in these classes,students fell behind. That was a mistake," Thrun declares.

The Udacity team began setting tougher deadlines andaggressively reaching out to students to help them make up for lost time."We called every single student, sent out text messages and introduceddeadlines," recalls Thrun. The effect was big: students caught up to theirclasses.

That push helped in another way: initially many studentswere unaware of the online tutors (who are real people) who were available online to help, 12 hours a day. But over the weeks, it became clear that thetutoring services were crucial. "The mentoring program was absolutelyessential for every student's outcome," Thrun says.

Those tutors were so important because many students lacked evenelementary-school-level mathematics knowledge, Thrun says. Among the frequentlyheard questions: How to divide twonumbers? How can you subtract a bigger number from a smaller one? Tutorsanswered questions and continued to reach out to encourage students. 

As the class progressed, Udacity also realized that many ofthe students simply couldn't get to a computer regularly enough. For somestudents, says Thrun, "there were none in the home, [and] even in school they couldn't getthe hours needed to make progress. We had to work with the schools directly (toarrange for computer time for Udacity students.) It was actually a bigdeal." A June articlein the Oakland Tribune outlined some of these challenges.

When students did get to the online programs, even navigating the computer systems could be daunting. One of the questions that tutors were frequently asked was how todo exponential notation on a computer. And although the curriculum wasdelivered on Udacity's platform, assessments were delivered via a separatelearning management system used by SJSC. Worse: results on one system tookabout 48 hours to update on the other, says Thrun.

All that said, when students were surveyed, they said thebiggest impediment to succeeding in the class was pacing: they just didn't haveenough time. "Sal Khan has strong datathat says in math in particular, a more flexible pacing is important forsuccess," Thrun says. "He's been preaching go at your own pace andyou can turn a C-level student to an A-level student." Thrun also pointedto Foothill College's"Math My Way" program which has been able to double its student-passrates by giving students more time.

Thrun is proud of the number of students who completed theclass--83% of the students finished, a dramatic increase from the usual,single-digit complete rates that MOOCs have seen. Yet the fact that thestudents continued to try makes the failed final exams even more poignant:"Yes, it's an enormous failure rate," he concedes. "I won'tsleep."

The final analysis of the data from the courses isn't dueback from the independent auditor for another two weeks. Once that data isavailable, Thrun says, he will develop an action plan for carrying on. "Let'simprove infrastructure. And address the timing and pacing. I don't want to lockus into a semester frame if we shouldn't do this." Even so, he adds,there's "no doubt that we will carry on."

STAY UP TO DATE ON EDTECH
News, research, and opportunities - sent weekly.
STAY UP TO DATE ON EDTECH
News, research, and opportunities - sent weekly.