How Automated Speech Recognition Could Change Studying Forever

Digital Learning

How Automated Speech Recognition Could Change Studying Forever

from Amazon Web Services (AWS)

By John Waters     Jan 7, 2019

How Automated Speech Recognition Could Change Studying Forever

Tech entrepreneur Fred Singer is the CEO and founder of Echo360, a company which he launched after a visit to his son’s classroom. Singer has funded and led a wide range of digital companies over the past twenty years, and the list of business and charitable boards on which he serves is long. But his current venture is something of a personal mission.

“My son has learning issues, and when I walked into his classroom, I began to see it through his eyes,” he says. “The classroom, where the actual learning takes place, was surprisingly void of impactful technologies. As a guy who lives on the cutting edge of technology, I found it very frustrating to see how far behind the school was, not just in terms of their use of one-off tech and old-school tools, like clickers and white boards, but their failure to connect the dots between better outcomes and using things like big data and mobile technologies.”

EdSurge spoke with Singer about inspiration, engagement and how his vision of a classroom video capture and distance learning platform that connects the dots is transforming teaching and learning in more than 500 institutions around the world.

EdSurge: What was it about that visit to your son’s classroom that helped inspire you to found Echo360?

Fred Singer: The question that struck me was, ‘how could it be that kids—born to a generation in which smart phones and laptops are a part of their everyday existence—have to sit in a classroom where someone just talks to them?’ They get bored and they get on their phones and the schools get mad and ban the phones. Most kids today don’t even learn cursive, so note taking itself has already changed.

But the phones aren’t the problem. The real issue is that the teaching and learning process in class is not engaging students via technology that is an integral part of their lives. Why have a laptop policy if there is not software to go with it? And as I began to research this issue, I discovered that most of the technology investment in educational institutions really doesn’t impact learning engagement for learning outcomes.

How does classroom video capture technology address this issue?

This technology captures whatever is going on in the classroom and turns it into easily searchable and referenceable video content that students can access anywhere, anytime, on all their devices—which is one piece of the puzzle. It’s not about the video capture itself, but the ability to use that dynamic technology to capture very complex, live, real-time interactions—moments in the classroom, you could say—and extend them.

How does your platform boost interactivity and student engagement, specifically?

Our system allows you to turn a classroom into a highly interactive experience on more than one level. As I said, we are effectively extending moments in the classroom, which students can review as often as they need to. We’ve found that students spend an average of 18-20 minutes going through the videos. They can scroll to the exact moment they need to review. It’s a time-shifting thing.

But this system also gives teachers the ability to tune in to what’s really going on in their classrooms in real time. If students don’t understand something, instead of having to raise their hands and ask a question, they can click a button that tells the teacher they don’t understand. Instead of hearing only from the same six students, instructors get feedback from many more.

Let’s say you’re teaching about weather and you’re trying to figure out if everyone understands where the eye of the storm is. You’re pointing to the visual, but you might have 40 kids in the class and you really have no idea if they’re following you or not. With our system, you get feedback that tells you, say, only 5 percent of the class gets it, so you immediately shift gears and focus on the underlying learning. When everyone gets it right, you keep moving.

Can you share an example of an educator using this technology?

One example that comes to mind is University of Massachusetts, Boston, economics professor Andrew Perumal, who uses Echo360 to facilitate in-class activities, such as asking students open-ended questions about course material. He effectively provides his students with what he refers to as “low-stakes” testing. This practice helps to keep students engaged, allows them to test their knowledge without impacting their grades, and provides Dr. Perumal with valuable insights into what material students are struggling with.

You said classroom content and activities become searchable through your platform. How does that work?

Echo360 videos have always been searchable if they’re closed captioned, but typically, only a small portion of classroom content is captioned because of the cost. We’re changing that situation with our innovative use of automated speech recognition (ASR) technology.

Our platform is built on Amazon Web Services (AWS), so it made sense for us to use Amazon Transcribe, which is a machine-learning-based ASR service. Amazon Transcribe automatically creates text transcripts from the audio track in the videos and makes them available for students and instructors to view directly from within the classroom player. The main benefit here is obvious; student have access to searchable transcripts, automatically generated, so they get them quickly enough to be useful. And it’s this service that makes it possible for our system to turn a classroom lecture into a study guide.

Of course, this is also essential assistive technology. Amazon Transcribe allows us to provide a cost-effective and scalable approach to closed captioning. Although it’s true that the law requires this kind of assistive technology at some schools, I believe ASR is now more than just assistive technology; it’s on its way to becoming foundational educational technology for all students.


The Echo360 video platform use Amazon Transcribe to provide real-time transcriptions of lectures at the University of Michigan. Source: Echo360.

What do you mean when you say ASR is on its way to becoming “foundational educational technology”?

It’ll become foundational technology because of its ability to link the classroom to other resources and capabilities—search, note taking, study guides, and a lot of big data. Students can attend a lecture, which is analog, and link it to the digital textbooks they’re using in class. And all that detail creates linkages between the learning in the classroom and outside the classroom, on a range of devices.

I predict that this will change the way students take notes and how they interact in class. They won’t just get a recording of the class, but all the keywords they need for sophisticated search—and search, of course, is huge. Our system is also able to turn a classroom lecture into a study guide, automatically.

How are you measuring success?

Today, Echo360 technologies are used by millions of students in approximately 11,000 classrooms. Where we have installed our system—mostly in universities right now, but in a growing number of high schools—we’re getting a 700 percent increase in students interacting with the system. That percentage refers to the increase in student engagement generated when Echo360 is used to transition from a traditional lecture capture system to its current active learning system.

We’re also seeing significant improvements in grades. At Indian River State College, for example, the grades of students who used the system in introductory courses—referred to by the college as Gateway Courses—were 17 percent higher than those who did not use the system. In general, the greater the number of times a student used the system, the higher was his or her course grade.

And after using the system in his political science course, Professor Ivan Ivanov at the University of Cincinnati found that his DFW rates (the percentage of students who received a D or F grade or withdrew from the course) declined from 20 percent down to around 9 percent.


Indian River State College implements Echo360 campus wide to support their students

All that interactivity must generate a ton of data.

It does! Schools have virtually no real-time learning data. There’s a distinction between profile data (who you are) and behavioral data (what you do). Education is awash in profile data, but completely lacking in behavioral data—which is the foundation of predictive analytics in education.

Our system allows you to start turning the classroom into an interactive experience, but it also allows you to gather absolutely invaluable behavioral data. With that data, you can not only predict student success, but also determine why some students succeed and some fail. You can see how they learn: Are they doing their homework? Answering questions in class? Taking notes? Asking for help?

And this data can, in turn, be used to create a feedback loop for teachers and institutions that is also incredibly valuable. You can use it to develop better teaching models that reach all the students.


The Research

The Echo360 platform is based both on Singer’s belief in the potential power of his technology to transform the modern classroom, and on key research around blended and active learning, including:

Tech entrepreneur Fred Singer is the CEO and founder of Echo360, a company which he launched after a visit to his son’s classroom. Singer has funded and led a wide range of digital companies over the past twenty years, and the list of business and charitable boards on which he serves is long. But his current venture is something of a personal mission.

“My son has learning issues, and when I walked into his classroom, I began to see it through his eyes,” he says. “The classroom, where the actual learning takes place, was surprisingly void of impactful technologies. As a guy who lives on the cutting edge of technology, I found it very frustrating to see how far behind the school was, not just in terms of their use of one-off tech and old-school tools, like clickers and white boards, but their failure to connect the dots between better outcomes and using things like big data and mobile technologies.”

EdSurge spoke with Singer about inspiration, engagement and how his vision of a classroom video capture and distance learning platform that connects the dots is transforming teaching and learning in more than 500 institutions around the world.

EdSurge: What was it about that visit to your son’s classroom that helped inspire you to found Echo360?

Fred Singer: The question that struck me was, ‘how could it be that kids—born to a generation in which smart phones and laptops are a part of their everyday existence—have to sit in a classroom where someone just talks to them?’ They get bored and they get on their phones and the schools get mad and ban the phones. Most kids today don’t even learn cursive, so note taking itself has already changed.

But the phones aren’t the problem. The real issue is that the teaching and learning process in class is not engaging students via technology that is an integral part of their lives. Why have a laptop policy if there is not software to go with it? And as I began to research this issue, I discovered that most of the technology investment in educational institutions really doesn’t impact learning engagement for learning outcomes.

How does classroom video capture technology address this issue?

This technology captures whatever is going on in the classroom and turns it into easily searchable and referenceable video content that students can access anywhere, anytime, on all their devices—which is one piece of the puzzle. It’s not about the video capture itself, but the ability to use that dynamic technology to capture very complex, live, real-time interactions—moments in the classroom, you could say—and extend them.

How does your platform boost interactivity and student engagement, specifically?

Our system allows you to turn a classroom into a highly interactive experience on more than one level. As I said, we are effectively extending moments in the classroom, which students can review as often as they need to. We’ve found that students spend an average of 18-20 minutes going through the videos. They can scroll to the exact moment they need to review. It’s a time-shifting thing.

But this system also gives teachers the ability to tune in to what’s really going on in their classrooms in real time. If students don’t understand something, instead of having to raise their hands and ask a question, they can click a button that tells the teacher they don’t understand. Instead of hearing only from the same six students, instructors get feedback from many more.

Let’s say you’re teaching about weather and you’re trying to figure out if everyone understands where the eye of the storm is. You’re pointing to the visual, but you might have 40 kids in the class and you really have no idea if they’re following you or not. With our system, you get feedback that tells you, say, only 5 percent of the class gets it, so you immediately shift gears and focus on the underlying learning. When everyone gets it right, you keep moving.

Can you share an example of an educator using this technology?

One example that comes to mind is University of Massachusetts, Boston, economics professor Andrew Perumal, who uses Echo360 to facilitate in-class activities, such as asking students open-ended questions about course material. He effectively provides his students with what he refers to as “low-stakes” testing. This practice helps to keep students engaged, allows them to test their knowledge without impacting their grades, and provides Dr. Perumal with valuable insights into what material students are struggling with.

You said classroom content and activities become searchable through your platform. How does that work?

Echo360 videos have always been searchable if they’re closed captioned, but typically, only a small portion of classroom content is captioned because of the cost. We’re changing that situation with our innovative use of automated speech recognition (ASR) technology.

Our platform is built on Amazon Web Services (AWS), so it made sense for us to use Amazon Transcribe, which is a machine-learning-based ASR service. Amazon Transcribe automatically creates text transcripts from the audio track in the videos and makes them available for students and instructors to view directly from within the classroom player. The main benefit here is obvious; student have access to searchable transcripts, automatically generated, so they get them quickly enough to be useful. And it’s this service that makes it possible for our system to turn a classroom lecture into a study guide.

Of course, this is also essential assistive technology. Amazon Transcribe allows us to provide a cost-effective and scalable approach to closed captioning. Although it’s true that the law requires this kind of assistive technology at some schools, I believe ASR is now more than just assistive technology; it’s on its way to becoming foundational educational technology for all students.


The Echo360 video platform use Amazon Transcribe to provide real-time transcriptions of lectures at the University of Michigan. Source: Echo360.

What do you mean when you say ASR is on its way to becoming “foundational educational technology”?

It’ll become foundational technology because of its ability to link the classroom to other resources and capabilities—search, note taking, study guides, and a lot of big data. Students can attend a lecture, which is analog, and link it to the digital textbooks they’re using in class. And all that detail creates linkages between the learning in the classroom and outside the classroom, on a range of devices.

I predict that this will change the way students take notes and how they interact in class. They won’t just get a recording of the class, but all the keywords they need for sophisticated search—and search, of course, is huge. Our system is also able to turn a classroom lecture into a study guide, automatically.

How are you measuring success?

Today, Echo360 technologies are used by millions of students in approximately 11,000 classrooms. Where we have installed our system—mostly in universities right now, but in a growing number of high schools—we’re getting a 700 percent increase in students interacting with the system. That percentage refers to the increase in student engagement generated when Echo360 is used to transition from a traditional lecture capture system to its current active learning system.

We’re also seeing significant improvements in grades. At Indian River State College, for example, the grades of students who used the system in introductory courses—referred to by the college as Gateway Courses—were 17 percent higher than those who did not use the system. In general, the greater the number of times a student used the system, the higher was his or her course grade.

And after using the system in his political science course, Professor Ivan Ivanov at the University of Cincinnati found that his DFW rates (the percentage of students who received a D or F grade or withdrew from the course) declined from 20 percent down to around 9 percent.


Indian River State College implements Echo360 campus wide to support their students

All that interactivity must generate a ton of data.

It does! Schools have virtually no real-time learning data. There’s a distinction between profile data (who you are) and behavioral data (what you do). Education is awash in profile data, but completely lacking in behavioral data—which is the foundation of predictive analytics in education.

Our system allows you to start turning the classroom into an interactive experience, but it also allows you to gather absolutely invaluable behavioral data. With that data, you can not only predict student success, but also determine why some students succeed and some fail. You can see how they learn: Are they doing their homework? Answering questions in class? Taking notes? Asking for help?

And this data can, in turn, be used to create a feedback loop for teachers and institutions that is also incredibly valuable. You can use it to develop better teaching models that reach all the students.


The Research

The Echo360 platform is based both on Singer’s belief in the potential power of his technology to transform the modern classroom, and on key research around blended and active learning, including:

 

Trending

Get our email newsletterSign me up
Keep up to date with our email newsletterSign me up