Skip to content
‹ Back to IndustriesEducation Technology

In 21 of 119 Penn State AI tutor chats, reasoning never reached the homework

Penn State's Suman Saha won a two-year, $199,009 NSF grant to test ALAASKA, an AI tutor that gives hints instead of answers. In its first classroom pilot, 21 of 119 chats showed reasoning missing from submitted work. Grading only final answers may miss real thinking.

This story was produced through MarketScale. See how Education Technology teams put it to work with Executive Thought Leadership.

By MarketScale Newsroom · Penn StateAlaaskaNational Science FoundationSuman Saha
Share

Key takeaways

01

In about one of every six tutoring chats, reasoning appeared in the conversation but not in the student’s submitted answer, suggesting final products alone may miss parts of the problem-solving process.

02

Penn State's study compares an AI tutor against equivalent coursework, which isolates the tutor's conversational support from AI use in general.

Free workspace

Turn your Education Technology expertise into content.

Record interviews, organize footage, and write with AI on a free trial of the MarketScale platform for qualifying companies. No demo required, no credit card.

Try it Free

Twenty-one out of 119. That's how many student conversations with an AI tutor at Penn State held evidence of reasoning that was missing from the answer the student turned in. The figure comes from an early analysis of the tool's first classroom run in spring 2026.

The tool is called ALAASKA (Adaptive Learning for All through AI-Powered Student Knowledge Assessment). Its lead developer, Suman Saha, an associate teaching professor of computer science and engineering, now has federal money to test it under controlled conditions. In a November 13 Q&A with Penn State News reporter Sarah Small, he described a two-year, $199,009 U.S. National Science Foundation Research Initiation in Engineering Formation grant. NSF's award record, No. 2605386, lists Stephanie Cutler and Ibukun Osunbunmi of Penn State's Leonhard Center for the Enhancement of Engineering Education as co-principal investigators.

Simple division turns 21 of 119 into a rate of about one in six. In about one of every six tutoring chats, the thinking happened and the submitted answer never showed it. Saha said the finding suggests instructors may learn more by examining the problem-solving process, not only the final product.

A tutor built to hold back the answer

Saha started building ALAASKA in 2025. It grew out of earlier classroom work with short AI-generated quizzes, flashcards, summaries and videos. The platform reads a student's responses as they work, estimates where that student's understanding stands, and serves the next useful thing: a targeted explanation, a question, a hint or a quick knowledge check. It does not solve the problem. It points to the next step and waits.

Saha said students often liked how it broke hard problems into manageable pieces without giving away the answer. His reading of the 21-of-119 result is cautious. It suggests instructors may learn more by looking at the problem-solving process, not only the finished product.

ALAASKA by the numbers

9 homework, 4 projects
Assignments students worked through with ALAASKA in the spring 2026 pilot course
2 sections
Programming Language Concepts sections in the funded study: one with ALAASKA, one with equivalent LMS activities
Sept. 1, 2026
Start of the NSF award period
Aug. 31, 2028
Estimated end of the award, per NSF's record

Penn State News; U.S. National Science Foundation

One design choice matters a lot for anyone thinking about rollout. According to the NSF award abstract, ALAASKA adapts in real time without extensive existing student data or repeated diagnostic testing. Saha made the same point: it doesn't need a history of a student before it can help. For operators, that could make it quicker to put into a new course.

What the two-section test isolates

The grant pays for a comparison inside a large Programming Language Concepts course at Penn State. One section uses ALAASKA. The other completes equivalent activities and reflections through the course learning management system. Both sections share aligned syllabi, assignments, rubrics and grading policies. Saha said the setup tests whether the tutor's conversational support improves learning, as opposed to AI use in general, and it doesn't assume students in the comparison section stay away from outside AI tools.

That last condition is the realistic part. For buyers, a comparison built that way could show what the tutor's conversational support adds, separate from AI use in general.

Conversational microlearning, as Saha describes it, is AI that offers questions, hints and feedback in short exchanges to guide a student's reasoning instead of handing over finished answers. The NSF-funded study asks whether that format builds three habits tied to an engineer's professional formation: value-driven reasoning, persistence and reflection.

The data plan is broad. NSF's abstract lists pre- and post-surveys, course performance measures, assignment and exam grades, platform interaction logs and learning management system traces on the quantitative side. On the qualitative side, the team will code student reflections and selected conversations for how students describe their reasoning, effort, ethical choices and use of feedback. The work draws on Self-Regulated Learning theory and the ABCE AI Literacy Framework, which covers affective, behavioral, cognitive and ethical dimensions.

The early figure has limits. It came from one upper-level course, and it doesn't say whether those 21 students understood the material and just wrote thin answers, or whether the reasoning depended on the tutor's prompting. Results from the funded comparison aren't available yet.

Employers want proof of judgment too

The question ALAASKA is built around, how to see reasoning when AI can produce the output, is the same one engineering employers are asking about new graduates. In an October 29 Semiconductor Engineering report, Liz Allan found universities reworking assessment around oral exams, in-class quizzes, labs and projects. ChipAgents founder and CEO William Wang told the outlet that engineers now carry responsibility for validating AI-generated work and catching incorrect assumptions. Andy Nightingale, vice president of product management and marketing at Arteris, described the workflow he wants graduates to bring: specify the problem, generate alternatives, validate assumptions, test results and document decisions. Axiomise CEO Ashish Darbari pointed to the design review, where an engineer has to explain their reasoning on the spot.

Educators outside the U.S. are making the same argument. Writing for Times Higher Education Campus on November 11, Samer Al Kork of Al Akhawayn University in Morocco argued that when a student can generate code in seconds, the output says little about what they understand. What counts is how they reached a decision and whether they can check and explain it. Saurabh Mehta, chief academic officer at Vidyalankar Institute of Technology in Mumbai, said in an October episode of the institute's podcast that assessment weight should move from the outcome to the process. He said he has tried planting bugs in problems to see whether students can find them, and that VIT has formed a committee to work on a new way of assessment.

Penn State wants to learn whether patterns in student-AI conversations can help instructors recognize when persistence and reflection are emerging and when students may need more support. If they can, that could give instructors in large courses a view of the process without an oral exam for every student. Saha frames large courses as the main use case, since that's where instructors can't give everyone individual support at every moment.

Oral defenses still test something a transcript can't. Al Kork and Darbari both stress explaining a technical choice live, to a client or a review panel. A chat log shows a student reasoning with a patient tutor. It doesn't show whether that student can hold up when a customer pushes back. For programs weighing one method against the other, that suggests the two could measure different things.

Checks for a chat-based tutor pilot

For engineering departments weighing AI tutors in high-enrollment courses, the Penn State design points to three checks. First, check whether a pilot compares the tutor against equivalent non-AI activities with matched rubrics and grading. Without that, a gain could just reflect AI use in general. Second, ask who reads the interaction logs and what rubric defines reasoning. NSF's award abstract lists refined instruments and coding rubrics for studying AI-supported learning among the expected outcomes. Third, find out whether the tool needs historical learner data before it helps, since that decides how fast it can go into a new course.

Ask any AI tutoring vendor how often student chats show reasoning that never reaches the submitted work, and how they measure it. Penn State's 21 of 119, from a single course, is an early reference point for that conversation.

Penn State plans to share design guidelines, assessment instruments and methods for analyzing student-AI conversations through faculty-development work with the Leonhard Center. NSF's abstract adds conference publications and workshops, and Saha hopes to extend the approach to more courses, Penn State campuses and partner institutions. Exam grades are among the data the study will collect. Whether reasoning seen in the chats carries over to exams is the result to watch for.

Featured companies

Your experts belong here

Every story in MarketScale Education Technology starts with a company putting its implementation leads, instructional designers, and district partners on the record. Buyers are already reading this topic. The only question is whose experts they find.

Procurement teams read long before they ever call, and your implementers get to answer their questions first.

Book DemoSee how it works15 minutes, straight to a calendar.

About the author

MarketScale Newsroom
MarketScale NewsroomEditorial Team, MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

B2B Weekly

The week in Education Technology, and sixteen other industries, every Monday.

Ten stories, one-line takes, five minutes. Free.

Education Technology: are you visible to AI?

Before they reach out, Education Technology buyers ask AI engines which vendors to trust. Explore how your experts, customers, and partners can become useful content for buyers and AI search.

Free Trial

You just read one Education Technology expert. Your company is full of them.

This article was produced through MarketScale. The same platform turns your implementation leads, instructional designers, and district partners into the articles, video, and social content Education Technology buyers are searching for. Start a free trial and see it with your own people. For qualifying companies, no credit card, no demo required.

NPS +73 · 1,000+ creators · 38+ countries

What your free trial includes

Hands-on access to the MarketScale platform
Media requests to your crowd, remote recording, AI writing tools
No demo required. No credit card.
For qualifying companies. Company confirmation required.

More Education Technology Insights

Iowa State's $6 million USDA grant covers half of a farm tech training expansion

Iowa State's $6 million USDA grant covers half of a farm tech training expansion

Iowa State University won a $6 million USDA grant toward FIELD Commons, a $12 million, 42,000-square-foot farm tech training addition in Ames. The other $6 million must come as non-federal cash. The plan targets more than 1,000 micro-credentials and 100 employer-connected learning experiences over five years, with hands-on training space as the thing being bought.

  • 01Classroom space is the constraint FIELD Commons is built to remove: Iowa State's farm tech courses, including AI in agriculture, already draw more demand than the lab can hold.
  • 02Only 100 employer-connected learning experiences are planned over five years, about 20 a year if spread evenly, against more than 1,000 micro-credentials, which makes the employer slots the limited resource for ag businesses seeking a technician pipeline.
  • 03Student and educator targets count engagements rather than individual people, so the sharper question for partners is how many credentials get completed and how many lead to jobs.

Nov 20, 2026

Values belong in the core curriculum, says the student leader who resisted Idi Amin

Values belong in the core curriculum, says the student leader who resisted Idi Amin

Olara Otunnu, a former UN ambassador, argues that university education must combine academic rigor with values-based character development and international exposure to produce leaders equipped for uncertain times. His argument draws on his experience leading student resistance to Uganda's Amin regime and his conviction that schools, not just families, bear responsibility for instilling ethics and solidarity.

  • 01International student enrollment creates mutual benefit: foreign students learn democratic values and opportunity in the US, while American students learn from peers across Africa, Europe and Asia, building practical solidarity through direct exchange.
  • 02Top-tier education requires deliberate institutional organizing around two things: students who encounter the world beyond their hometown and teachers who persistently ask ethical questions about right and wrong.

Nov 19, 2026

Domenico Grasso and Scott Shireman on UMCI and public trust

Domenico Grasso and Scott Shireman on UMCI and public trust

On DisruptED, host Ron Stefanski talks with University of Michigan President Domenico Grasso and UMCI Executive Director Scott Shireman. Grasso calls the University of Michigan Center for Innovation in Detroit, set to launch in 2027, a bridge for turning university discoveries into things that help society. Shireman says the center serves the campus, the Detroit community, industry and the university's programming.

  • 01Grasso describes UMCI as a bridge for translating university discoveries into technologies, processes and business models that help society.
  • 02Shireman says about 70 students a year from Detroit schools get into U-M Ann Arbor, out of roughly 4,000 graduates, and UMCI's K-12 work aims to put more on that path.
  • 03Grasso says faculty should act as public intellectuals who explain their work, to help rebuild public trust in higher education.

Nov 18, 2026

Explore More Education Technology Insights

Read more expert perspectives from across Education Technology.

Browse Education Technology Hub

About the Expert

MarketScale Newsroom
MarketScale Newsroom

Editorial Team

MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

For B2B teams

Your experts could be publishing here

Stories like this one run on content MarketScale captures from real practitioners. See how your team's expertise becomes coverage in Education Technology and beyond.

Book a Demo

Or call us. No forms required. We pick up. 214-945-2512