AI for Teachers Designed to Untether and Empower—Never Replace

[MUSIC PLAYING] - Welcome to The Future of Education. And now here's your host, Michael Horn. [MUSIC PLAYING] - Welcome to The Future of Education, where we are dedicated to building a world in which all individuals can build their passions, fulfill their potential, and live a life of purpose. And to help us think through how we get there today, in transforming our K-12 system and supporting educators around the country, around the globe, ultimately, I'm delighted by our guest, Satya Nitta. He is the founder and CEO of Merlyn Mind, which we'll hear more about. Merlyn Mind is one, though, that's been on my radar for quite a number of years for their approach to artificial intelligence, which, as we'll hear, is very distinct from a lot of the hype and conversations around AI at the moment. So first, Satya, thank you so much for being here. It's great to see you. Appreciate you joining us. - Pleasure to be here, Michael. - You bet. So let's dive in. Because you founded Merlyn Mind back in 2018, so well before the current craziness around large language models, ChatGPT, all those things that have just created a lot of hype around AI. But even back then, I think it's fair to say that adaptive learning was a big craze, and people would talk about AI in that context. And when you started Merlyn Mind though, you made an important decision, which was to focus on serving the teacher first. So I'd love to hear about that origin story. Why you made that decision? What was behind the vision when you founded Merlyn Mind? - Sure, yeah. I mean, so before Merlyn Mind, I was at IBM Research. And I was there for 18 years, and the last half of my stint there, so the first half I was advancing Moore's Law, working on chip technologies, and the second half I was working on AI. and I got into AI around the time Watson won Jeopardy!. And I was given the keys to the kingdom around 2012, 2013 when, the Watson Jeopardy! moment was one of those seminal moments in AI, where computer seemingly understood language and all the complex allusions and puns and wordplay and beat the two best players of this very complex quiz game. And that was the next follow up to Deep Blue beating Kasparov. And this also happened at IBM Research, down the hall from where I had an office. And so the second time around, what happened was Watson won Jeopardy!, IBM got approached by all kinds of companies from all over the world-- education companies, healthcare companies, financial services companies, consumer companies. And they said, we need to do something with Watson in each of these industries. And they turned to me and said, do you want to take Watson and do something with it in education? And this was 2013, actually, early 2013. And I thought that was an amazing opportunity. I had previously not had done any work in education at all before that. I was just a deep technologist working on either advancing language modeling and language models have predated large language models, which is the whole ChatGPT revolution. And so I was working on language modeling, and I was working on conversational systems and speech recognition a little bit. And I thought this was a great opportunity to take AI and do something in a particular domain. And behind that was this larger realization that AI only really works, even today, in deeply domain specific ways. It's a general technology, but the last mile is the most important mile. So anyway, when I was given the charter, I spent a lot of time, six months or so, learning the field of education. I wanted to approach the field trying to understand the underlying science. Six months a year, I was reading a whole bunch of papers every single day. I stepped out of what I was doing, and I was reading papers in cognitive psychology, neuroscience, learning science, and I'd already had some exposure to neuroscience then, especially cognitive neuroscience, because of an interest in a branch of computing called neuromorphic computing. So I went back to IBM, and I basically said to them, look, we can do a number of things with AI in education. We can take the Watson system and build question answering applications or chatbots across a number of things. Universities can use it to help students who are onboarding get all kinds of answers to their questions, and so on and so forth. But we're sitting here at IBM Research, one of the places that has really advanced computing, and we need to do something very foundational with AI and education. And so I drew back on a very famous incident that happened in the history of AI. Which was, in 1957, there was a very famous conference called the Dartmouth Conference, where the term artificial intelligence was coined. And so back then, if you were to ask some of the founders of the field, Marvin Minsky, Herb Simon, Allen Newell, some of these luminaries, all of whom went on to win the Turing Award, which is the computer science equivalent of the Nobel Prize, why do you want to build an intelligent machine? They used to throw out a bunch of use cases. We want machines to tell jokes and to think people. But one canonical use case they used to toss out was, we want a machine to teach. OK. And so they were talking about AI tutors. And so that became a grand challenge in AI. It became a grand challenge for a very good reason. And I'll come back to that multiple times during this conversation. OK. And I basically said, look, I'm sitting here at IBM Research, in these hallowed halls where the DRAM was invented, Moore's Law was basically advanced through Dennard scaling, Watson won Jeopardy! Kasparov got beaten by Deep Blue, and much of modern computing has some footprint in this building. And I feel the pressure to do something grand. And basically, we need to go after this grand challenge, build a computer to teach. So build a tutoring system. And I wasn't just making it up. In fact, that mantle of trying to get a computer to teach was picked up by generations of AI researchers. So we were sitting on top of 30 years of work in academia. OK, brilliant scientists like John Anderson, right at Carnegie Mellon, had spent a lot of time thinking very hard about how do you get a computer to teach? What is an AI tutor? And I'm going to throw this sorry, elaborate history, because I want to establish the provenance of ideas, and I want to land it to where we are in this moment in AI and what it's doing, OK? So IBM was thrilled with that vision, and we spent an enormous amount of time, like five years, at its peak we had something like 130 researchers, untold millions of dollars, attempting to build an AI tutor, which we built, OK. Well before the current craze on AI tutoring, we had actually taken all the work in academia and built the first large scale industrial tutor. Carnegie Learning is another company that's advanced. - I was going to say Carnegie Learning, Newton, there had been other attempts at it as well. - That's right. Yeah. So we also looked at Carnegie Learning's work. And we said, OK, what they did was very interesting, and we wanted to build an even broader approach to tutoring well beyond something very hierarchical, like math, and go into lots of topics. And at the heart of it, what we're attempting to do was to get a computer, build a chatbot that a student can chat with in very natural language, and this is well before ChatGPT, but with language models of that time. So the chatbot would ask the student a question. The student would respond in natural language. The chatbot would then analyze the response and tell them what they were missing, and not give them the response, and not give them the answer. And we published all of this work. By the end of 2017, I was leaving IBM. I got recruited to go join Amazon, head of an AI effort there. And then I got this once in a lifetime opportunity to start Merlyn Mind. Some of the major backers in this company reached out and said, we heard you're leaving. What's happening to the team? Would you be interested in starting a company? And I jumped on the whole idea. But I left with a very key realization, which is, we built a tutor. It worked. It did something very complex and profound from a technological perspective, much more than anything today's tutors do. We had elaborate student models, knowledge models. We had an ability to score student responses, and we ran into a fundamental roadblock, which is that students weren't using it, actually. We built this for higher ed, and we couldn't get them to use it despite us, leaning in on topics like multiple representations, which is multiple ways to teach a student a concept and put the student in charge. And we spent a lot of time thinking about the user experience. But we just couldn't solve the last mile, which is the motivation problem. We couldn't get a student motivated enough to use a chatbot. And by the way, that fundamental question remains today. OK, what most people don't ask when you see all these flashy demos of GPT 4 based tutoring, is, are people using it? What's the monthly average use? What's the daily average use? How long are they using it for? Are they sticking with it? I mean, how much have they used it initially and how much do they use over six months time? And nobody asked them very hard questions about, is this thing really solving the problem of teaching these kids something? Are we seeing an improvement in learning outcomes? So all these things became major questions in our head. And we finally concluded that the major problem here is not a technology problem. It was actually something much more profound, which is, students learn from people best. That the teacher becomes the central fundamental role model who's delivering the wisdom and the knowledge and serving as a human example of learning for students. And they're motivating the kids, they're giving them examples. They know the kid. They're situating, learning within their background. And so we learned the hard way what generations of educators had already known. Which is that the teacher is the central and most important figure and factor in improving learning outcomes. So when we started Merlyn Mind, we said, I don't think we really want to do something impactful. It wasn't about doing something flashy, and raising a bunch of money, and being in the news. It was about making a real change, OK. And we said, the best thing we can do is to empower the teacher, OK? Use AI to reduce the friction, give them time back, give them cognitive space back, allow them to be with their students. And we're far more likely to help improve education than by attempting to replace the teacher, OK, which we learned through hard experience is an incredibly complex problem. - So let me pause you there for just a moment, because what you're describing is interesting. And there's been a ton of efficacy studies on Khan Academy, IXL, all these math programs. And they basically show that they work if you use it at the minimum usage, but only 5% of students actually use it, which I think is what you're pointing to, right, in your observation. I hadn't realized that IBM had built a tutor that was actually able to work, but your point was they weren't actually using it. So then you've said, OK, our leverage point is in fact, the teacher. So let's fast forward to today. Tell us, what is Merlyn Mind doing for teachers? What's the role of AI in that? And-- - Sure. - Has it changed at all since the explosion of interest in AI, over the last year and change, because now it's much more comfortable or familiar to people. Teachers are interested in it for sure. I'd love to get a sense of how you're actually helping them. - Yeah, sure. So one of the things that's happened, and this was a problem before COVID, it's been exacerbated by COVID. So if you go and study, the average teacher's day, especially in the classroom, they use something like 30 to 40 different applications on average during the course of a teaching week. OK. All kinds of applications from fun games to attendance applications and everything else in between. And so I made a passing comment about AI being a bespoke last mile business. So we really needed to understand how are teachers actually spending their time? What's actually preventing them? What's their workflow? What's the workflow like now, and what's an ideal workflow? And we realized that one of the things teachers love to do was to walk around the classroom, be with their kids, watch over their shoulders as they're trying to solve a problem, just be untethered and walk around. And because of the fact that technology, ironically, has penetrated the classroom, we now have a teacher has a laptop in the classroom, there's some interactive display in the classroom, 20 student laptops, and the teacher is tethered to their desk, switching between different education applications. And they're not walking around the classroom like they used to anymore. And we said, we can solve the problem. And then they also don't do certain things that they'd love to do. For instance, they're showing something interesting on screen, they'd love to take a snapshot of it and send it to the students, and doing so would interrupt the flow of their day. They have to stop teaching, go back to their desk, copy the link, open a Gmail roster or whatever it is, populate all the student emails, copy the link, and send it. Meantime, the students are sitting around twiddling thumbs. And they've lost the entire flow. So we said a really great thing we can do with AI is allow the teacher to walk around the classroom and allow her to control the computer with voice. She's literally talking to the class in some sense. And so we built an early version of it where you can just wake the device up and talk. And the device is controlling the teacher's laptop, and launching new tabs, playing videos, pausing videos, sharing snapshots, answering questions, all of the above. And then we discovered that what teachers really wanted to do was to just simply hold a small push to talk mic like you have on your TV these days, walk around the classroom, and basically control their computer. So in a sense, it's a workflow tool that's basically meant to liberate the teacher from having to be tethered to the desk and control the computer. We can now have her walk all over the classroom. If there are questions that come within the flow of teaching, you can simply turn to the assistant. It's an assistant that essentially, answers questions, controls your computer, controls your browser, and so on. So that's basically what we're doing in classroom. - Yeah, yeah. So help me understand a couple of things then. It's interesting. It sounds like that in essence, it's less giving the time back and more untethering them from the front of the room, so that they can be mobile. Is the AI agent that you've built able to navigate across different apps, bring up an attendance app, bring up a different lesson plan, bring up a particular video, and things of that nature? Is it able to cut across the different interfaces or apps that one would have on their own computer? - Exactly. That's the whole idea. The whole idea is, so today, the browsers have basically taken over the world. So virtually 80% to 90% of your workflow is on some browser. If you just examine your own computing habits, you're literally, you're doing Gmail on a browser, you're doing spreadsheets on a browser, you're doing everything on a browser. And most of adtech lives on a browser. Not all, but most. And what we're doing is fundamentally controlling the browser. OK. - Got you. - And so we can essentially we are getting to a point where we can operate a browser like a human can operate a browser. Read the screen like a human can read it. Navigate across the various tabs, and across the various applications, and various automations built in. So multiple steps are basically taken care through one voice command. So for instance, an automation that we do today that teachers love is, I'll give you two. One is, they're sharing a link of, let's say they're sharing a YouTube video and they want students to watch it afterwards. The teachers can simply say, share this link with my class. It could be YouTube, could be anything else. The AI takes that link, the URL, opens the teacher's Gmail roster or whatever your mail tool they're using, puts it in there, populates the entire student email list, puts a tagline, and sends it out. All of that happens with one simple voice command. What would take six, seven, eight steps on a computer, we automate it, right? Take a screenshot of what's on the screen and share it. So all these kinds of things we do. There are multiple other automations as we call them, that are being handled by our AI. And the idea here is, the AI takes multiple steps, it enters keywords, it clicks on the hyperlinks, it launches videos, it pauses videos. You can skip to the two minute 30 second mark of a YouTube video. You can pull up your lesson plan that you stored in your Google Drive. You can do all these kinds of things. And yeah, so it is an untethering tool. That's exactly correct. Does it save a lot of time? Yeah, maybe a few minutes. But it saves a lot of cognitive load and that's the more important thing. The teacher-- - Doesn't have to be thinking all the steps I have to take. I just say. - They're not thinking about it. They're simply talking to the technology. So yes, the second part of your question, if you want me to go there. How has the current interest in ChatGPT the last year changed? - Yeah, hold on that for a moment, actually. Let's come back because I want to stay on what you've built. It's interesting in a number of levels. Just help us understand the use case a little bit more. Let's say I teach geometry in high school, say. I have maybe five different classes of students that come through. I'm working in a particular period, and maybe like three students in my class, Michael, Satya, and Kayla, are struggling with a particular concept and I want them to see a video explanation from some Khan Academy or wherever. Can you literally say to it, hey, we're in period three. Send the video to Satya, Michael, Kayla, on such and such concept. It'll help them out, we think. And then boom, it can do that fine tune so that you can actually be personalizing as well with it? - We are getting there. So we actually built POCs that do exactly that. Where we split a class up into groups, and you can send things to certain groups and not everybody else. Right now you can send it to the whole class. But let's say the teacher is teaching five classes. The AI can say which class, if it doesn't know, OK? - OK. - Normally, it should know contextually which class it is the teacher is teaching. But it can also disambiguate. It can say, look, you teach two geometry honors classes, which one of these do you want me to send it to? And you can send it there. But what is coming is exactly that level of granularity. Which is, we break up the class into teams and groups based on the teachers settings. But yeah, we can absolutely. It's something that you can build with our technology, we just haven't built it yet. - Got you. So let's get into the technology and we'll wrap back around to the ChatGPT and general interest question in a moment. But let's talk about the technology. Just take us through how you've built this, how it differs, perhaps from the large language models that are underpinning what OpenAI, what Google with Gemini et cetera, et cetera have been doing, and why you've taken that approach given the K-12 education context. - Yeah, I mean, look, the reality is this has large language models also in it. It's not like it doesn't have large language models. It just has a lot more stuff in it. So this is really an emerging area of AI called AI agents, which you'll hear a lot more about over the next year or two. And it's already starting to hear buzz about AI agents in AI circles very deeply. And so we've been working on this class of what we call agent X systems and a field of AI, a subfield of AI for quite a while. But you can talk to this machine just like you talk to ChatGPT. And we built voice computing and language modeling over the last several years. And we've been actually broadly working in this field for about a decade. So when LLMs came out, we basically said, OK, LLMs are much better way to answer questions, much better way to talk. And so that's all part of the system. It's not divorced from ChatGPT or Gemini or any of these things. But we just have our own class of LLMs for a very good reason, and we can talk about that, which we are making entirely private, safe, and secure for the needs of education. But it also allows us to do other things. So what a journalist LLM like GPT 4 cannot do is, copy a link and share it with somebody, OK. So that's where some of the additional work that we've done, additional lift we've done, which is have agents operate on your browser, control your browser for you. So that technology is basically, it's an area of AI that we used to call it automation it's now being called agents. So it's AI agents meets LLMs. That's basically the technology underpinning behind this. - Got you. Super interesting. So let's fast forward then into how the hype conversation, the adoption perhaps of Merlyn Minds has changed since the emergence of AI in the popular understanding. - Yeah. So before I think, the dawn of ChatGPT has actually been a boon for us because before that, we were going around talking about AI. Mostly we were met by a mix of interest and some bafflement. I mean, there were all these tropes about AI being something that one should be afraid of for various reasons et cetera, et cetera. And so we had the job of both creating a market, and then also educating a market, and then also serving a tool to the market. And so we had two jobs to do. So ChatGPT and the ChatGPT movement did a lot for AI in general, not just for us, for everybody who has been working in AI. There's a whole bunch of companies working in AI, very few real AI companies in education at that time, and even now, by the way. When I say real AI companies, companies that build AI, not build with AI, which is very different. Build with AI is, I'm putting a wrapper on top of ChatGPT like say Khanmigo or various other people, but build proper AI from the foundation, from the ground up, OK. But there were a number of companies, I would say globally, several hundred companies, building proper AI across multiple sectors. And all of us had a very similar problem, which is, in our respective domains we had to go educate the market about what this is, what it isn't, why it's useful, why it's just another tool. And it's just another tool. It's just another automation in the long history of technology. There's nothing very mysterious about AI. If anything, the names are misnomer, but that's a whole other conversation. So it's been very helpful for us. Because now everybody says oh, we understand what AI is, and we'd like to see what you guys are doing and why this makes a difference to us. So that's been very helpful. Yeah. - OK. Last question as we wrap up here, because this is fascinating to get an understanding of the approach, where you're seeing the traction, how it's helping teachers, and so forth. Obviously you're building in the here and now, but where is this going? Right? Like in Merlyn Mind, we come back in two or three years. Just a couple ideas. Where is this going to be? Who is it going to be helping and how? - So we are continuing to flush out the assistant for the teacher to start with. Again, keeping in mind that we started this journey saying, let's focus on the teacher because that's the single biggest lever in improving learning outcomes. So obviously, we are helping the teacher do a whole bunch of out-of-class automations as well. And we're building something for real scale, solving real problems around privacy, safety, security. For instance, we have an entire large language model that we've built, that we've trained, that watches any interaction with the AI between the teacher and the AI, between students and the AI. And it's meant to stay clear of hundreds of sensitive topics around violence, and adult themes, and so on and so forth. Interestingly, as a quick aside, this large language model that we built is currently the best large language model on the planet, better than OpenAI's or Anthropic's or Google's, in terms of having safe civil discourse in education. It's what we call a safety and appropriateness model tuned for the needs of K-12 education. So that's a part of our ensemble system. So before you do anything with Merlyn, you hit that LLM, and then everything else that happens, happens afterwards. Whether the Merlyn assistant answers questions, creates lessons per lesson plans, or sends links to students, it's all first gated by a guardrail by this safety model. OK. So two or three years from now, inevitably we want to take this into students, but we want to take this into students with the full understanding that Merlyn will never be a tutor, it will just be a helpful review tool. For instance, could you, with the teacher's permission and the school's permission, record the lessons, send the teachers links, and package the whole thing for students to review afterwards, and for them to study for their exams from. So that's an idea we could do. So we have the ability to essentially work with the device and with the teacher's content and tools. And we have the brand permission to go in because of a fundamental compact that we have with our users, which is we never monetize the data. We never sell the data. We never use their data to even train our models. We delete all voice data instantly, OK. And we are built from the ground up for privacy, safety, and security. It's a very important value proposition of Merlyn Mind. And so you would never see our data being handled by any third parties. So as a consequence, we can go in, we are COPPA, FERPA, GDPR compliant, and as a consequence, we can go in where people like Alexa and Google Assistant, and so on fear to tread, because we are built with this bedrock of privacy. So what we intend to do is to basically build on this, make the best teacher assistant on the planet possible, and inevitably start helping students with homework help. And we are working with a whole bunch of partners. We already have the Merlyn assistant powering new lines displays. There was a big announcement that came out in March. We are now going to announce a few more major partnerships in the next two or three months to power third parties displays, enable third parties to build student review or student assist tools. I don't want to call them tutors because tutor connotates a whole different level of AI that doesn't exist today. But people who really intend to build student help tools and build them in the right way with human fallback and actual human tutors as a fallback. We intend to work with them and empower them with our AI. And the final bit is, we are deepening our large language model capability. We're already building education's fastest, most bespoke LLMs. We want to expose them to the world and allow other people to build with them. With the idea that you can scale with our models and at scale, these are going to be a lot faster, a lot cheaper and much safer than anything you get from generalist LLMs. So that's basically what the next couple of years are for Merlyn. - Fascinating. Well, we are going to stay tuned, watching eagerly, and hopefully get an update from you in a couple of years to see where things have progressed, what has surprised us, and how you're continuing to support educators across the country. So Satya, thanks so much for joining us. - Oh, pleasure being here, Michael. - You bet. And we'll be back next time, on The Future of Education. [MUSIC PLAYING] PLAYING]