A Map of Generative AI for Education

An update to our map of the current state-of-the-art

Compiled by Laurence Holt and Jacob Klein

For this major update to our map, first published in June 2023, we have added over 90 new logos and 11 new areas. Many previously gray areas (meaning we have yet to see a real-world example) are now yellow (we have).

So much is happening that you could be forgiven for deciding to wait until the dust settles. But that would make you a spectator instead of a participant in the greatest changes in K-12 education in our lifetimes.

Back in 2023, we mused over what that change might ultimately look like:

It’s not easy to predict, but two paths seem possible. The first is what has almost always happened to new technology in the classroom: it rearranges the furniture. Laptops become expensive slide projectors. Personalized instruction winds up meaning worksheets with garish dashboards added. It was recently estimated that the average teacher uses 42 edtech tools regularly.

The second path is that the inefficiency and dullness of the industrial way of schooling begin to disappear. Many of the teaching practices that learning science has shown to be most effective — such as active learning and frequent feedback — and most engaging for students — such as role play and project work — require significant time most teachers just don’t have. Could that change if every teacher had an assistant, a sort of copilot in the work of taking a class of students (with varying backgrounds, levels of engagement, and readiness-to-learn) from wherever they start to highly skilled, competent, and motivated young people?

We will see.

If this map is anything to go by, like a great Robert Altman movie, there are going to be a lot more characters before the story starts to resolve.

(Thanks to everyone who alerted us to what we had missed. Please don’t stop.)

Download PDF here.

Teacher Practice Support

Note: A class of “do everything” tools is emerging, combining several of the features listed here — for instance, MagicSchool.ai and Eduaide.ai—or allow you to build your own — eg, Playlab.ai

Lesson generation

A teacher who wants to incorporate more writing into sixth grade science uses an AI tool to generate a lesson based on an existing OpenSciEd plan but with an embedded writing activity.

  • Studies show that a surprising proportion of teachers do not have a core program but use their own lessons or search TeachersPayTeachers or Pinterest, with results of highly varying quality. For them, AI might represent a sort of TeachersPayTeachers on steroids.

  • Tools such as Nolej, DiffIt, MyLessonPal, Copilot, teachology.ai, and many others can generate lessons on any topic to order. Some will also provide trappings such as quizzes, study guides, or perfectly laid-out slide decks and handouts. Others, such as Curipod, let you deliver the lesson as an interactive presentation.

  • An area of growth may be in generating adaptations of existing lessons rather than wholly new material: “I have to follow the Illustrative Math scope and sequence but can we make this activity a role play?”

  • Tools need to get better at (1) being guided by the teacher — eg, x minutes of group work, y minutes of class discussion, etc—(2) forming a coherent part of the learning experience by, for example, understanding what has gone before and what is coming after, and (3) producing high-quality lessons as measured by Rosenshine or an equivalent yardstick.

  • There is an opportunity for an AI tool that evaluates generated lessons along these dimensions.

Instruction coaching

A teacher records the audio of a lesson and uses a tool to get feedback on skills such as wait time and handling specific student misconceptions.

  • Tools allow a teacher to record a lesson and get automated analysis and feedback. Current tools like TeachFX or Edthena provide analysis after class. Future tools may allow real-time coaching on screen or via an earbud.

  • Tools can help coach teachers on evidence-based frameworks such as Marzano or Danielson, supporting school-wide implementations.

  • Evidence in other sectors (eg, customer service) suggests a significant improvement in performance is possible through AI coaching.

  • Audio quality remains an unsolved problem: the teacher may be audible only part of the time, and students hardly ever. Better audio equipment may be intrusive.

  • Some teachers worry about who will have access to recordings and transcripts.

  • The technique is currently better suited to whole-class pedagogies, whereas effective practice may be more small-group-oriented.

  • Several researchers are working on more sophisticated analysis and feedback, including Dora Demszky at Stanford and, with a focus on mathematics, Abhijit Suresh at the University of Colorado, Boulder.

Teaching advisor

An elementary school teacher finds that a subset of his class did not understand negative numbers and suspects his curriculum’s procedural approach is the problem. He uses AI to evaluate his current lesson plan and students’ responses to in-class checks-for-understanding. The AI advisor connects him with like-minded teachers, recommends articles and research papers, and facilitates a discussion that includes an AI subject-matter expert. He hits on a new, more conceptually grounded approach to negative numbers.

  • Tools such as EduGPT are beginning to emerge to play the role of a teaching advisor, or multiple advisors, for different subjects and topics.

  • If fine-tuned on pedagogy, an AI tool could play the role of coach and advisor to a teacher. For instance, it could give advice on specific ways to teach concepts, suggest alternatives, and diagnose student strengths and misconceptions.

  • Other tools including TeachingLab.ai can ingest an existing lesson and advise the teacher on improvements to it, such as changes of pace, embedded checks for understanding, scaffolds to ensure foundational skills are in place, and connections to concepts students have already learned.

  • AI-powered collaboration tools can inject new life into onnline Professional Learning Communities (PLCs) that otherwise often don’t attract a critical mass of educators. AI can understand each teacher’s focus and challenges, match them with similar educators, and inject relevant research and blog posts into the conversation.

Classroom management simulator (new)

A first-year middle school teacher realizes he needs to hone his classroom management skills. He spends his evenings working through a series of simulated classroom scenarios in which AI role plays students. The AI can recreate situations and student personalities very similar to what he faces during the day. He gets to try out different approaches and gets expert advice he can try out, all in a low-stakes environment.

  • Simulators are increasingly widely used for training pilots, surgeons, and even CEOs; why not teachers? David Weston of the Teacher Development Trust in the UK is working on exactly that.

  • Future versions could employ AI agents to role-play specific student personalities that then interact with each other and with the (human) teacher in real time.

  • They will also incorporate expert advice in various approaches including specific behavior management, mentoring, and motivational techniques.

Competency-based teaching

In a high school English unit, students zoom with refugees in order to write about their stories. The conversations are an opportunity for students to practice competencies including empathy and listening. They are provided with a rubric for the competencies and examples of what proficiency looks like. An AI tool sends relevant snippets to the teacher to support competency-based feedback later.

  • Teachers are unfamiliar with creating lessons that include opportunities for students to learn transferable competencies in context—eg, building empathy in an ELA unit on refugees or improving group work in a math lesson.

  • AI tools could take a lesson or unit outline and suggest which competencies it affords practice on. It could then help teachers embed competency-specific activities in the lesson and create a rubric and example student work as a guide.

Tracking student project work (PBL)

Students work on a project to create financial models for a new business. An AI tool tracks assets that students post to an LMS. The AI supports some students directly and provides daily reports to the teacher on who is making progress, what mathematics is incorporated into each student’s model, and who needs teacher support.

  • Keeping track of the diversity of student work in a project-based learning (PBL) class can be challenging for a single teacher. PBL teachers also have anxiety about which standards are being covered by which students and when. Partly as a result, PBL is still a rare practice. AI supports such as Project Leo could help it become more widespread.

  • If students keep a journal during the project, and/or post assets to a repository such as an LMS, an AI tool can analyze their work and (a) support student direction, (b) provide a synthesis to the teacher, and © alert the teacher when individual students need support.

Analysis of student data

An elementary school teacher uses data from reading running records to analyze growth for students who have been receiving an intervention: Has their growth accelerated? Who is responding to the intervention and who is not? What letter combinations do they most frequently miss as a group? What skills do they no longer need to work on? She uses the data at a meeting with the grade-level team to optimize instruction and to re-assign students to intervention groups that fit them better.

  • Schools collect a lot of data but make use of only a small part of it. AI can ingest student data (eg, in CSV format or via API from an SMS, LMS, or proprietary app) and perform analyses on it, suggesting optimal student grouping, focus areas, and generating other insights.

  • Tools that can perform analysis—including sophisticated statistical methods suggested by the tool itself—and generate data visualizations include OpenAI’s Advanced Data Analysis and Fluent. Tools that understand education data specifically, such as Doowii and Strived.io, are beginning to emerge.

  • Tools allow multiple level of analysis, for example: you could upload NWEA MAP data from several cohorts and ask the tool to identify areas with the strongest and weakest growth; ask higher-level questions such as an analysis of summer loss or whether a tutoring initiative made a difference; or combine data from several sources to build a detailed, cross-subject picture of a class.

Background knowledge refresh

A middle grades science teacher wants to anticipate challenging questions her student might ask while discussing the human circulatory system.

  • Educators may want to refresh or deepen their knowledge of a topic before running an open discussion of it, especially if they didn’t major in the topic.

  • People may be more willing to ask for help from an AI than a peer or their supervisor.

  • Teachers who follow the curriculum closely rather than “teaching the domain” may get less value from a refresh but miss the engagement that can come from pursuing student-led inquiry.

Admin support to free teacher time

An elementary school teacher who used to spend an hour at the end of each week compiling a summary of what students will work on next week for a parent-facing app, now relies on AI. She pastes links to and extracts from reading, math, and other curricula into a chat window and asks for a parent-friendly summary. She reviews the result, adds an anecdotal sentence, and posts within five minutes.

  • A major obstacle to teachers implementing new and better practices is lack of time. Teachers may not have capacity to provide personalized feedback and support, or to plan for new pedagogical methods, even though they would like to.

  • Like a human assistant, AI tools could perform some teacher tasks such as drafting emails and progress reports to colleagues and parents, responding to parent inquiries, inputting grades to the SIS, creating report cards, and tracking homework completion and use of supplemental digital tools.

  • AI can provide a natural language interface to common tasks — eg, “send the class a reminder that their prototype is due this Friday” which generates a post to the LMS.

Incorporating research-based practice

A reading coach finds a journal paper for a method of teaching vocabulary. She uses Elicit to get a summary of the paper and ask questions about the specifics of implementation. Then she generates a protocol customized to the vocabulary words she will be teaching in her next session.

  • Educators are, like doctors, expected to read the latest research on evidence-based practice and incorporate them into their teaching. Very few have the time or skills to do so unaided.

  • AI tools such as Elicit, Humata, Scite, Consensus, and Genei can find relevant papers, summarize them, and answer questions as if they were the paper’s author.

  • They may soon generate sample lesson segments incorporating the practice and customized for the topic a teacher is about to teach. This could potentially help bridge the long-standing research-to-practice gap in K-12 education.

Family connections (new)

A parent, worried about their child’s learning loss during the pandemic, is trying to make sense of school report data. They click on a chatbot icon alongside the report and begin an extended back-and-forth, digging into data from a recent reading assessment, understanding their child’s results, and getting suggested home activities. The chatbot even tells them about an after-school program they qualify for and how to enroll. The entire conversation is in Spanish.

  • Families have a tough time getting the most out of school ecosystems: the data, available resources, options, how to advocate for their children, etc.

  • Existing platforms that help schools communicate with families, such as AllHere, and startups such as Paloma, are providing AI chatbots with access to a knowledge base assembled by the school district together with curated resources (often resources that exist but are underused today).

Team teaching with AI (new)

In a high school biology classroom, three educators — Mr Smith, Ms Johnson, and an advanced AI model — collaborate to deliver a lesson on the human heart’s structure and function. Mr Smith focuses on anatomy, beginning with a 3D model. Ms Johnson, specializing in physiology, takes students in groups through a hands-on activity simulating blood circulation using colored water and a model heart. The AI model’s role is formative assessment, checking for understanding and giving feedback on students’ self-explanations. It suggests real-time adjustments to the other teachers.

  • One way to think about in-class AI, according to Jean-Claude Brizard of Digital Promise, is as “team teaching, with AI on the team.” AI tools can play one of several roles: delivering first instruction, checking for understanding, dealing with misconceptions, scaffolding and supporting students during practice, etc.

  • To be an effective member of the team, and for the enacted lesson to be coherent, the AI tool must have been given sufficient context: the purpose of the lesson, the lesson plan, perhaps data on student background knowledge, precursor skills, or interests.

Para Practitioners (new)

An elementary school paraprofessional, supporting teachers in early reading, is trained to use specially designed AI tools to take on more responsibilities. She interprets early reading data, decides which students should receive additional support, helps plan lessons, and co-teaches the ELA block every morning alongside the classroom teacher.

  • There is now a well-documented effect of AI tools on the performance of workers in programming, customer service, managers’ writing, and even management consulting. The consistent finding is that lower-skilled workers improve most, in some cases matching the performance of experts.

  • Can the same thing happen in education? Could access to AI capabilities allow paraprofessionals to take on some of the responsibilities of full teachers? There is precedent in healthcare where nurses can become nurse practitioners, able to interpret test results, diagnose, and prescribe, under the supervision of a doctor.

Classroom Material

Activity-specific content

A tenth-grade US history teacher wants to find a more engaging method of teaching the Cold War. He uses an AI tool to create a role-play simulation in which students play US and Soviet leaders in a re-enactment of the Cuban Missile Crisis. Before the simulation, students work in groups to research and write briefing documents for the players.

  • Many evidence-based and highly engaging methods of teaching require significant additional teacher effort to prepare. AI tools can dramatically reduce that effort, allowing a much larger proportion of teachers to use them.

  • Apps like Teaching Tools can take a topic or resource and design a jigsaw exercise for student groups with research and discussion prompts.

  • AI can generate role-play materials — eg, you are a 1920s door-to-door vacuum cleaner salesperson (so you’ll have to be able to explain what a vacuum is). AI tools can also play one of the roles. For instance, Mizou and the enormously popular character.ai let you go on a quest with Einstein or interview Napoleon.

  • AI can set up a debate on a topic and help students prepare.

  • AI can support teachers in creating activities based on research-based strategies such as contrasting cases (eg, Dan Schwartz, Stanford), for example contrasting the graphs of equations that differ only in their use of +/- operators.

  • Flipping the classroom, in which students study independently what would previously have been the subject of a teacher lecture and then spend teacher time on applications and problem-solving, can be effective but hard work. Tools such as Mindjoy—which lets you create STEM AI-based tutors—can generate materials to support flipping and work with students while the teacher circulates.

Explanations

A high school biology class includes several students with below grade-level reading comprehension. The teacher decides to augment classroom explanations with those written in considerate text — ie, at the students’ level. She takes existing content and uses AI to rewrite it. She uses questions generated by the AI to get feedback from students on whether they understood the explanation.

  • A basic element of instruction is explanation — of a concept, a big idea, a process, an event, etc. There is evidence that explanations customized to individual learners are more effective.

  • For instance, AI tools such as DiffIt can help a teacher take an existing explanation and rewrite it at any reading level. If the AI has data on the student’s knowledge level (eg, previous assessment data or student work), it can take that into account. Explanations that rely on prior knowledge a student doesn’t have are, of course, not very helpful.

  • The tool can also create explanations that incorporate specific student interests, for instance explaining area and volume in terms of Minecraft.

  • Tools like Revyze and PeerTeach allow students to create explanations for each other and can use AI to ensure the contents are accurate before sharing. Students may find explanations created by peers to be more accessible.

  • To spice things up a little, AI can generate explanations in unusual formats, for instance a ballad, hip-hop song, or story explaining DNA-RNA-protein.

  • A big area of growth in 2024, as AI video-generation becomes real, will be creating animations, YouTube-like videos (Prof Jim), or augmented reality simulations (Ludenso) from explanation text.

Student questions generation

A sixth grade student practicing unit rate questions asks her AI for questions based on Pixar movies. The AI says to complete her assignment she can answer either six moderately difficult questions, four challenging questions, or two formidable questions. She takes a deep breath and plunges in.

  • US classrooms use millions of questions and prompts for practice and formative assessment every day. In both cases, variety is good. But it is time-consuming to generate the perfect question set. AI tools such as PrepAI, to teach_, Conker, Formative, QuestionWell, Mindgrasp, Quiz Makito, WorksheetsAI and many others can take a content area and generate questions together with rubrics and model answers.

  • Popular classroom response tools such as Kahoot and Quizizz have added the ability to generate questions with AI, though they are still at the stage where you should check the accuracy of questions before using them.

  • Tools can generate a variety of questions: multiple choice, short answer, essay prompts, exit tickets, etc.

  • EdPuzzle can generate questions for a video. Students encounter the questions as they watch.

  • Some are able to generate questions customized to student interests and—soon—different levels of challenge, different levels of Bloom’s taxonomy, open-ended Fermi problems, and mini-projects.

  • (Importantly, they can also automatically grade these types of problems and give students detailed feedback. See Feedback on student work, below.)

ESL student content

An elementary school teacher uses an AI tool daily to create content specifically to augment her existing lessons for the ESL students in her class. The tool knows each student’s home language and traditions so it can ingest any lesson and build supports such as sentence stems, translations of vocabulary words, explanations of background knowledge, etc.

  • Some core curricula include specific supports for students learning English as a Second Language, but they are often uninspiring or missing altogether. An AI tool such as Twee or Speakable can fill the gap.

  • AI language models can “speak” dozens of languages but, unlike existing translators, they can combine that ability with context such as the lesson being taught and, perhaps, knowledge of how best to support ESL students at different levels.

  • Providing tools to support different student populations—a requirement often enshrined in state frameworks but difficult to enact in class—is a use-case that seems likely to grow.

Active learning embeds

A middle school science teacher uses AI to take an existing lesson on plate tectonics and generate several challenging questions. The teacher selects one about the implications of tectonics. Students turn and talk then record their answers with feedback from AI. The teacher gets an evaluation of which students have understood the lesson so far.

  • Active learning is a much more effective method than traditional classroom teaching, but it has proven difficult to train teachers to convert their lessons to active ones. AI tools could take an existing lesson and suggest active adaptations.

  • For instance, an AI tool can take the text of a traditional lesson and suggest active learning embeds such as having students break into groups to research a topic or work on problems. Students can receive feedback in real-time and AI could alert the teacher to groups who need further challenge or support.

  • Alternatively, the AI tool could convert an existing lesson into engaging media such as video with embedded questions, or a student interview with a character from science or history.

Focus on big ideas

An elementary school teacher, worried that her current unit on fraction focuses too much on algorithms and manipulations rather than the big idea uses AI to generate an alternative sequence. The AI recommends a number line game from the research literature designed to emphasize that a fraction is a number.

  • Off-the-shelf curricula sometimes attempt to cover so much ground that the big ideas get lost. For example, it is common to find entire units on fractions that fail to drive home the point that a fraction is a number.

  • AI can identify big ideas — in existing content, from standards, or by topic — and generate lesson material such as video, animation, or checks-for-understanding in varying contexts to ensure students have a solid grasp on the idea before going on to apply it.

Focus on transfer

An elementary school teacher notices that his students are adept at solving fractions problems but not at using them in real-world situations. He uses AI to generate fractions problems at fourth grade level across a wide range of contexts and has groups of students select three different contexts to work on.

  • Transfer is the ultimate goal of learning — enabling the learner to apply skills in new situations. Research shows that transfer is enhanced by practicing skills in varied contexts, for instance solving equations in abstract, word-problem, and authentic real-world situations.

  • AI tools can generate examples — including questions with and without solutions — across varying contexts, including the real world.

  • Tools can identify connections with similar concepts in other subjects, aiding transfer.

  • And tools can interleave examples of two or three different skills, so that students don’t always know what skill to expect.

Worked examples

A fifth grade math teacher wants to provide extra support to three of her students. She uses AI to generate worked examples interleaved with practice problems and compile them into a booklet she sends home with the students.

  • Worked examples — step-by-step demonstrations of how experts solve problems — improve students’ ability to solve similar problems. Without examples, students sometimes reinforce flawed strategies.

  • AI tools like Sizzle can generate worked examples (both correct and incorrect — students identify the misstep) based on a problem you take a photo of. Examples can be interleaved with practice problems, similar to those included in professional programs such as Algebra By Example.

Flashcard generation

A global history teacher takes a video on the industrial revolution and uses AI to generate flashcards based on the video transcript. She includes the flashcards in a digital study guide she posts to her class via the LMS.

  • AI can take text or video-transcript content and generate flashcards from it. For some types of material — eg, vocabulary — flashcards can be a helpful way for students to learn. Many flashcard apps provide practice based on spaced repetition which aids retention. Podsie, for instance, applies the method to classroom content, often learned and forgotten quickly. Kinnu does it for curated topics.

  • Flashcard apps like Quizlet and Anki, new AI-first flashcard apps such as Gizmo and Wisdolia, and classroom engagement apps such as Kahoot are incorporating AI to generate flashcards and other formats such as quizzes and games.

  • A student can highlight any term, from any class, that they are unsure of, to add to their personal spaced-repetition flashcard bank.

Culturally responsive content (new)

In a high school math class, students engage with statistics using a lesson adapted by AI to examine real-world datasets on racial profiling in neighborhoods, including their own, alongside other community datasets that spotlight issues of justice and injustice. For instance, students analyze traffic stop data, comparing the frequency of stops by race and gender and the outcomes of these encounters.

  • There is evidence that adapting lessons to incorporate culturally responsive content—both window and mirror—by centering students’ customs, experience, and perspectives, can improve engagement and learning.

  • Emerging tools from Reconstruction Onyx and Planning Period use AI to offer teachers help in revising lessons and activities to be based on frames suggested by the teacher—or by students (or student interests gleaned by AI from their work). Tools could also suggest culturally responsive ways to approach a topic such as statistics or genetics.

Vocabulary / glossary

An elementary school teacher beginning a unit on the weather uses AI to create a glossary of terms with definitions, examples, and etymologies at a fourth-grade reading level and with translations to Spanish. She asks the AI to create claymation images for each of the examples which she includes in the glossary.

  • AI can take text or video-transcript content and generate a vocabulary list or glossary for it including definitions, usage examples, and etymology.

  • The glossary can include definitions written at a specific reading level and/or translated into a student’s home language.

Quiz questions

An elementary school teacher wants to check that students have read and understood their homework reading: a short book on Paul Revere’s ride. The teacher uses AI to generate four questions per chapter, one at each Depth of Knowledge (DOK) level 1 through 4. He includes the quiz questions in a take-home pack for students.

  • AI can generate quiz questions based on a text or video transcript. Questions could be multiple choice, short answer, etc, at a specified Depth of Knowledge level or Bloom’s Taxonomy level. They can include model answers for the teacher.

Graphic organizers

A middle school science teacher is teaching a unit on ecosystems. She uses AI to generate a graphic organizer for a food web from producer to decomposer. She includes multiple blank versions of the organizer in a handout for students together with one model food chain, completed by the AI a pond ecosystem.

  • With help from LLM plug-ins like Show Me and apps like Algor and Heuristica, AI tools are already capable of rendering diagrams, such as concept maps or graphic organizers for a topic. They can also create a partially complete version of the graphic for students to fill in.

Just-in-time skill builder

A student working on a project to build and tune a wind instrument realizes she can’t succeed through trial and error tuning. At the prompting of her teacher, she collects data on the frequencies produced by different lengths of tube. But she is stuck in figuring out how to plot the data and fit a curve to it in a spreadsheet. She turns to an AI tool that walks her through the process and explains the underlying math in a way that gets her back to the project quickly.

  • Highly engaging learning experiences — projects, role plays, simulations, etc — often deliver students to a moment where they are motivated to upgrade their skills. Ideally, a teacher is right there but that can be difficult to orchestrate, especially across a whole class.

  • AI tools could step in providing just-in-time skill-specific instruction. That could be content that is part of a curriculum, provided by the teacher, generated by AI, or curated by AI from high-quality open content.

  • Just-in-time content is likely to be more effective if it refers to the specific context the student is in. For instance, if a student wants to fit a curve to air pollution data, the AI could incorporate that context into the instruction.

Extended learning

A middle school student who appears already to have a good grasp of natural selection is given a choice of extension questions to research. She is concerned about the environment and so chooses to find and report on an example of human activity influencing species via natural selection. She creates a video describing pesticide resistance in insects. The AI asks for more detail about the long-term consequences and strategies to mitigate them which the student enthusiastically provides in a follow-up video.

  • AI tools can provide extended learning, enrichment, and new challenges to students who are ready to go further. The AI can offer a set of directions for a student to pursue, enhancing engagement. Rather than just previewing the next unit, extensions can go deeper into the existing topic.

  • Extensions can build autonomy, for instance by generating a big question for the student to research. The student can present ideas to be evaluated by the AI which also reports progress to their teacher.

Connecting new content to old

A high school history teacher wants to make a strong connection from the ideas in US founding documents to the Enlightenment precursors. An AI tool suggests that students read excerpts from John Locke that have been curated to highlight the relevant ideas and create a graphical representation showing the connections. The AI pinpoints the excerpts, generates a rubric, provides a model answer for the teacher, and gives feedback on student responses.

  • The press to get through content in subjects such as history can leave students with a feeling of disconnected silos. To offset that, teachers can make deliberate connections across material.

  • AI can help identify connections based on, for example, the full course syllabus. It can also generate content and activities to deepen the connection such as a graphic organizer mapping the ideas driving the American Revolution and founding documents to the Enlightenment precursors that inspired them.

  • This approach can also be used to “spiral” — ie, revisit prior material but with increased richness and complexity.

Less-cheatable questions​​

A high school English teacher, worried that students may be using a chatbot to write essays, employs AI to interview students individually on their essay: what research they did, how they decided to structure the essay, what they left out, etc.

  • Students are already using ChatGPT to write essays and answer worksheet questions. GPTZero and others offer AI detection.

  • AI tools can, though, be used to make cheating difficult. For instance, an AI tool can question a student about their essay, what research they performed, decisions they made, their writing process, etc.

  • Teachers can also give alternate format questions: instead of having students summarize an article — something an AI does easily -ask them to record a presentation with audio or video, using AI to automatically generate a transcript and act as evaluator of the result.

  • If the purpose of teaching writing is, in some large part, to teach analytical thinking, there may be other ways to do the same. For example, some teachers embrace AI-as-essay-writer and ask students to analyze, fact-check, and improve on the generated essays.

Evaluation + Feedback

Holistic assessment (based on longitudinal student work)

A state agency proposes releasing multiple hours of formal assessment time to be used for instruction. Science faculty get together to develop a series of authentic performance tasks such as designing, building, and launching a rocket. Students use AI to curate a portfolio of work on the tasks, including blog posts, video transcripts, and spreadsheets. The AI produces data for each student that mirrors and exceeds the traditional assessment data. After two years, the state drops the formal assessment requirement.

  • The dream of educators is that assessment as a separate, invasive moment could disappear and instead be fully embedded in instruction. (Formative assessment, embedded in instruction, is an important part of learning and should not disappear, of course.) AI tools may bring that dream closer to reality.

  • An AI tool could have access to the complete corpus of a student’s work across multiple years of development. The tool could track a student’s growth with respect to state standards (and other competency-based dimensions such as creativity), providing both the student and their teachers with a much richer view of what they know and can do.

  • Initially, formal assessment will continue in order to provide ‘ground truth’ to calibrate the AI. Over time, the AI’s insights will become more valuable than those of a single, two-hour snapshot which often will not accurately represent what the student is capable of.

  • This ‘holistic’ approach also allows more authentic assessment — eg, performance tasks and real-world projects rather than multiple choice questions and essays.

  • Note that the approach will only work if the student has been assigned grade-level, rigorous work to evaluate.

Feedback on student work

Elementary school students in a class studying the run-up to the Civil War write two pages summarizing their understanding of events and causes. They get feedback from an AI tool that helps them improve their essay across several dimensions: their argument (eg, do they cite evidence), the clarity of reasoning, their understanding of specific events, and the completeness of their work. Their teacher “tunes” the feedback to match his own style, for instance, saying “provide evidence” rather than “citations”.

  • Learners advance by means of feedback on their work that is (a) immediate, or close to it and (b) includes an opportunity for them to try again. Since this requires a great deal of teacher effort, students typically don’t receive the optimal amount of feedback. Even when they do, they may check their grade and ignore the feedback. This has led to a proliferation of low-rigor exercises that can be automatically graded.

  • AI tools can generate feedback instantly and repeatedly including for high-rigor prompts such as making persuasive arguments and solving multi-part problems.

  • AI is especially good at language, so feedback on writing (Grammarly, Ethiqly, Pressto, Writable, Class Companion, Vexis, CoGrader) is already strong.

  • Some tools (Brisk) that can grade across subjects, drop feedback directly into a Google doc essay. Others (AutoMark) let you upload a specific rubric for the AI to use. Still others (EnlightenAI) learn to mimic the teacher’s feedback style. Tools vary on whether teachers must first review feedback before students see it.

  • Automated feedback allows students to iterate: not just to answer and find out if they were correct but to revise and extend (Quill) until they have a high-quality response.

  • Feedback on short-answer questions across subjects is also already very good (sAInaptic), though hallucinations sometimes occur. Recent research has shown AI to be as good as humans at grading student short answers to reading comprehension questions.

  • Feedback on high-rigor, open-ended math problems is less advanced (Mathnet) since student work often takes the form of sketches and handwritten computations.

  • Tools can also evaluate student self-explanations (Snorkl), a powerful research-based learning technique. For instance, after working in a simulation to produce proteins in a cell, students could talk aloud about what they just did, get an automatic transcription from Whisper, and instant feedback from an AI including terminology such as transcription versus translation, and a mnemonic (‘c’ comes before ‘l’).

  • More frequent feedback may allow teachers to separate it from grading. This can be beneficial because (a) students tend to focus on the grade and ignore the feedback and (b) teachers may provide feedback more as a justification for the grade than as a vehicle for improvement.

  • A future goal for feedback tools is to shift student- (and teacher-) thinking even further: from what-you-need-to-improve toward a process in which students seek and use feedback as a habit and that incorporates peer feedback (giving and receiving) and self-reflection (see, for example, Floop).

Identification of student thinking

A middle school class working on unit rate answers an exit ticket on paper, drawing diagrams, tables, number lines, solving long division problems, scrawling arrows connecting parts, crossing out and starting over. They take a photo of their work and an AI tool takes a few seconds to identify thinking, whatever solution path they take, and separates conceptual understanding from computational error in an instant report to the teacher.

  • The last 20 years of proliferation of machine-scored assessment have had the perhaps unintended consequence that students seldom encounter deeper, open-ended problems, especially in STEM subjects. This, in turn, puts the emphasis back onto procedural thinking, often just tricks students have memorized (flip-and-multiply) and away from conceptual understanding.

  • Initiatives like Mathnet are developing AI tools to do what teachers can do: look at a student’s written approach to an open-ended problem and identify (a) evidence of conceptual understanding, (b) gaps in understanding, (c) computational errors. In this analysis, judging the solution correct or requiring of student students a single, ‘official’ solution path is not as important as uncovering the student’s mathematical thinking.

  • We have yet to see tools that evaluate student thinking based on having them draw, for instance an animal cell. There are pedagogical benefits of drawing-to-learn.

  • Sorcerer is a tool in beta that engages students in a dialogue on a topic and gradually pushes them towards deeper conceptual understanding. It, or similar tools, could reveal greater insight into student thinking in a way that can inform both subsequent lessons and subsequent teaching of the same lesson.

Competency-based feedback (eg, collaboration, critical thinking)

A middle school teacher wants to improve student critical thinking. He uses an AI tool to identify that a segment on video game links to aggression in an upcoming lesson would be a good target. He has students analyze statements for and against the proposition with the help of an AI tool that reframes their causal explanations as questions — eg, “If one person played video games and was aggressive does it follow that everyone who plays violent video games will be aggressive?” Students reported that the AI guide improved their reasoning.

  • Tools such as NXTLVL can help students build transferable competencies such as critical thinking, problem solving, generating creative solutions, understanding other perspectives, etc. Typical school learning experiences, focused on academic standards, may not offer students opportunities to practice and get feedback on competencies.

  • For instance, having an AI tool reframe feedback on causal explanations as questions has been shown to help improve critical thinking.

  • AI tools could analyze student written work or presentations to generate feedback on specific competencies.

Tracking student progress

A sixth-grade mathematics teacher gets a detailed report for a new class based on longitudinal data from elementary school. The report identifies critical precursor work, leveraging data on previous sixth-grade cohorts in the sixth grade curriculum. It takes into account predictions of summer loss based on prior data.

  • For any given learning experience, some students master it and others need more time. Teachers sometimes have red-yellow-green dashboards reflecting the fragmentation of a class day by day. But few teachers have time to pore over dashboards and even fewer have time to solve the knotty problem that is captured there.

  • Like an expert assistant, AI can synthesize data across diverse tools and assessments into the most critical, specific recommendations for a classroom. It can take into account which gaps must be addressed before moving on, and which can safely wait til later, when the curriculum spirals back or, if the choice has to be made due to lack of time, let go.

  • Given access to longitudinal data for a student, AI could detect patterns that are not visible in single assessments such as a student whose conceptual understanding is masked by persistent computational errors.

  • AI can explain areas that require targeted practice in terms the student themselves or a family member can understand and act on, expanding the amount of learning time beyond class.

Rubric generation with model answers

A high school history teacher has developed a performance task in which students curate a museum display for the Great Depression. In previous years, some students did not understand what they were asked to produce, even though the teacher thought they were capable of doing so. This year, the teachers uses AI to ingest the task description together with previous student work and suggest rubrics for the coherence of the exhibit and how well it reflects the key ideas of the Great Depression unit.

  • For complex, multi-faceted skills that do not lend themselves to a correct/incorrect judgment, students may struggle simply because they are not clear on the expected performance. Providing a rubric and model answers at different levels of performance is time consuming for a teacher, but easy for an AI tool.

  • This is especially useful for competency-based skills such as creative thinking, critical thinking, and communication.

Social Tools

Small group facilitation

An elementary school teacher notices that some students do not make much progress on workbook problems. He uses AI tools to run small group instruction on comparing fractions (a topic he introduced today) for those students and finds that they are much more engaged discussing problems with each other than working in a book.

  • Small group instruction is very widely used in early reading and somewhat less frequently in math. A common problem is that only one group can work with a teacher at once and other groups may not be academically engaged.

  • AI tools such as Oko can manage a small group of students by monitoring video and recognizing speech so that students are engaged in a task chosen by the teacher—for instance, practicing skills introduced in a whole class lesson.

  • In the near future, tools will be able to engage in discourse directly with students, for instance directing a discussion on a topic while ensuring everyone contributes.

Discourse support tool for student groups

A group of students are working together to solve a puzzle in a simulated physics world. An AI tool follows their conversation. When one student asks it for help, instead of giving physics support it suggests how to improve their discourse and collaboration. It notes that they have a habit of pursuing non-productive suggestions by group members. It offers to alert them when they next do that. They return to the task in a more focused way.

  • An AI tool could follow the conversation of a student group and give on-demand advice on group collaboration. For instance, it could point out which group members’ ideas are not being tapped, or highlight that the group doesn’t follow through on directions they identify, or don’t seem clear on the problem they are solving.

  • Sidney D’Mello at the University of Colorado Boulder leads a team working on this use case.

  • The same approach could give domain-specific support to a group, for instance clarifying terminology or offering a starting point or an alternative point of view.

Facilitated student discussion board

Students in a unit on Newton’s Laws read a paper and post their questions and confusions to an AI moderated discussion board. The AI facilitates a discussion in which new understandings surface.

  • An AI-facilitated discussion board, such as StudyHall, could help students discuss questions, wonderings, confusing points, challenging problems, project ideas, connections with other topics, etc.

  • Students who do not always contribute in class may be very active in a discussion board and the asynchronous modality can encourage more thoughtful responses.

Facilitating whole class discussion

A middle school math teacher is using Illustrative Math. Students begin by working on a difficult problem that often surfaces misconceptions. An AI tool monitors student work on the problem and automatically creates a slide show of student examples together with the key points the teacher should highlight during a whole class discussion.”

  • An effective teaching strategy — known as productive struggle — is to have students work on a problem individually or in small groups and then facilitate a whole class discussion on what they found, guiding them towards an accurate and formalized (or perhaps more than one) solution. Many teachers find it challenging to orchestrate such discussions in real time and so may not capitalize on the value of this pedagogy.

  • An AI tool with access to each student’s work can quickly recognize common misconceptions, strengths, and computational slips and produce a step-by-step discussion guide that the teacher can follow right away. The guide is similar to a skilled teaching assistant who was able to follow every student’s thinking simultaneously.

  • The guide can suggest which student to call on, in what order, and could project student solutions from the teacher’s laptop.

Interest-based networks (new)

A high school student has begun to build a following for her video channel on pre-Columbian cultures. But she realizes she has a lot to learn on topics as diverse as video editing, rights management, writing powerful headlines, and social media. Her school doesn’t offer any of those but she joins an online community designed to help her build her network of like-minded entrepreneurs and experts.

  • A growing number of young people develop a passion for learning, but not for traditional school topics. They want to build a following online (and perhaps earn money from it), a video game, a music album, a new way to learn a language, interactive stories, or an animal sanctuary.

  • Communities like buildspace are developing AI tools to support these learners, for instance matching them with like-minded network members and bringing aggregating demand for outside experts.

Student Support

24/7 Tutor

An ELL middle schooler trailing in math knowledge is assigned Adam, an AI tutor that speaks his home language, Spanish. The student meets with the tutor three times per week for 45 minutes. The tutor has access to the main class curriculum and tailors topics to support grade-level work. The AI tutor is also available 24/7 on the student’s phone to help with independent work in class or at home.

  • One-to-one human tutoring is perhaps the most effective educational approach we have. But it is expensive. AI holds the promise of being the tutor in your pocket that isn’t just another drill-and-practice app. It feels like interacting with a real human tutor.

  • Today, AI tools are strongest at language. AI tutors for writing (Quill, StoryBird.ai, Caktus, StorySeed) are already here. Foreign language tutors are also available today (Duolingo, LangoTalk, Iago, Supernova).

  • AI models are also strong at coding. Tools to support students learning to code (CodeSignal Learn, Replit) interact more as a copilot than a tutor.

  • Math is harder. AI is, strangely, better at conceptual math than procedures, and conceptual understanding is more important for learning, but tools have yet to take advantage of that. Chatbot science tutors have not yet arrived.

  • There is still much to solve: costs for tools like Khanmigo are prohibitively high, though certain to come down—CK-12’s Flexi is less powerful but free. And the user experience for so-called tutors often feels more like a treadmill than the trusting relationships that can develop with human tutors. For instance, they use a text chat interface, in part because text-to-speech is still too slow to feel natural.

  • Existing tools in the “homework help” category—such as Brainly, CourseHero, Project Chiron, Studdy, CheggMate, Symbolab, and many more—practice apps like edia and test prep, such as r.test and Archer, offer step-by-step solutions (and sometimes access to a live human) but are not yet close to an authentic tutoring experience since they deny the learner a chance to find their own pathway.

  • No tools are yet designed for the intensive, three-times-per-week scheduled sessions that are most effective in human tutoring. That will be solved in time and AI tutors may go further: providing the kind of immersive worlds (eg, through VR) and narrative-based learning scenarios (eg, EngageAI Institute) that are highly engaging and better reflect the real world.

  • Avatar-based tools are beginning to emerge, such as Kyron which lets teachers create a tutor from video of themselves, so your students need never be without your dulcet tones.

  • Chatbots can’t draw or see, so they lack the ability to respond to a student sketch, which makes many topics challenging. This is likely to change in 2024 since multi-modality is a central focus of innovation in AI models.

Early reading coach (new)

An elementary school teacher transforms daily reading time by giving students a tool to create their own on-level books aligned with a unit on Greek myths they are studying. The AI tool generates a mini-book for each student based on their favorite mythological character and creates illustrations to match. The books include comprehension checks embedded in the text. The teacher can either print the books or have children read them aloud with instant feedback from the tool.

  • Early readers need lots of reading material for practice. But curating a book set that combines narrative and non-fiction text matching both the student’s reading level and interests is challenging.

  • AI tools such as LitLab, Project Read, and Storywizard.ai can identify suitable texts in the classroom library or generate new texts that fit. They can target phonics skills and embed comprehension checks. They can ensure vocabulary words are reinforced across texts rather than appearing only once, which makes learning more difficult.

  • As speech-to-text capabilities have dramatically improved, a gaggle of tools such as Microsoft Reading Coach, Ello, Edsoma, and Amira have incorporated the ability to listen to a student read, give real-time feedback, and build a learning path based on the science of reading.

  • Children can customize characters, put themselves in the story (NeonWild), choose how the plot unfolds (learning about story structure), and even change illustration styles.

Curiosity coach (new)

A student has just finished a classroom lesson on radio transmission. She is curious why antennas broadcast signals but wires in DC circuits do not. She launches her knowledge explorer app and asks. That leads to questions about frequency: radio-frequency waves travel long distances, but others do not. Why? As she continues, the app creates an illustrated concept map of where she has been and adds suggestions for other directions. She winds up, by way of gamma rays, at black holes, linking up with an exploration from two weeks ago that ended in the same place. She was going to have to look more into black holes.

  • There is little time in the life of a K-12 student to be curious, to explore on your own. The world of knowledge has been carefully curated for you. Occasionally, an adventurous teacher will notice an interest and suggest a resource; something that isn’t covered in the state standards. AI tools will excel at doing the same: they already have memories that far exceed any human.

  • HelloWonder is aimed at the curiosity of little kids, though its current form is a safe browser with a chatbot. Curio and Moxie are similar but in the form of a voice-enabled robot. Portola, also for little kids, focuses more on creativity.

  • Some early attempts ath a true curiosity coach, such as SocratiQ, a sort of Miro + Wikipedia, show promise. They feel rather like wandering the halls of a large library, or perhaps the Young Lady’s Illustrated Primer in Stephenson’s The Diamond Age.

  • They could also become social tools—connecting you with kids your age interested in the same topics (see Interest-based networks, above)—spark motivation for extended research or a new project that could become a life-long passion, feed summaries to parents, teachers and experts so they can extend your curiosity, and connect you with certifications and experiences that build toward a career or induct you into the “World of X”, whatever your X is.

Teachable agents

A high school student is assigned Martha, a teachable agent for a physics course. Martha asks a lot of questions, especially about everyday things such as objects “bending” as they are immersed in water. (Her interests are coordinated with the physics course syllabus.) It’s the student’s job to teach Martha and address her misconceptions. Martha gets confused when she finds inconsistencies but evolves and grows as she gains deeper understanding.

  • An AI tool could play the role of a learner that the student has to teach a given topic. Teaching something is one of the most effective ways of learning it. (See “teachable agents”, Dan Schwartz, Stanford.)

  • A variant on this is to have the AI tool can act as a Socratic questioner to deepen a student’s understanding.

  • A further variant is to teach non-playing characters (NPCs) in educational games to fulfill relevant quests, acting as teachable agents.

Support for students with special needs

A high school student diagnosed with ADHD had found it challenging to stay organized and focus on tasks at hand. Now he uses an AI assistant that can log in to and understand the school’s LMS. It helps him break down assignments into manageable tasks, plan towards deadlines and get reminders. The tool tracks his behaviors, offers suggestions for optimal work periods, and coordinates times when teachers are available to provide extra support. It incorporates game-like features that reward focus and task completion.

  • AI tools could be particularly effective at providing additional support to students with special needs throughout their education journey.

  • Tools could provide assistive writing support such as starter prompts, alternative input methods such as voice recognition, executive function support such as planning tools, visual support, and real-time classroom support such as text-to-speech and speech-to-text.

  • This is as well, of course, as personalized learning tools that adjust the pace of instruction, offer alternative explanations, and scaffold specific skills. Goblin.tools, for example, uses AI to help neurodivergent people with tasks such as writing a book report, which it can break down into a sequence of simpler steps.

Mental health support

An adolescent student feeling social and academic pressures accesses an AI tool provided by her school to deal with anxiety and stress issues. The tool is an AI chatbot counselor, ready to listen 24/7, and programmed with cognitive-behavioral therapy techniques, providing immediate coping strategies and relaxation exercises. Chatting with the AI allows her to open up about her feelings, a significant first step in acknowledging and addressing her struggles. It monitors patterns indicative of heightened stress and provides her with early interventions, personalized resources including videos, and connections to local mental health professionals.

  • AI tools such as Woebot Health and Koko can provide always-available, anonymous mental health support to students who may be hesitant to reach out to a human counselor.

  • AI-powered chatbot tools such as Edsights and Mainstay can offer advice and support techniques. They can provide a non-judgmental space for students to discuss their feelings and emotions, practice social skills, and receive encouragement and motivation. They can facilitate connections between students who are going through similar experiences.

  • Tools can monitor patterns that might indicate declining mental health and recommend professional help or alert a designated support person.

College / career advisor

A high school senior aspiring to become a tattoo artist has always thought of it as art. But she encounters an AI tool one day that helps her see what specific credentials she will need including, for instance, an understanding of infection. She suddenly develops an interest in biology that shocks—and delights—her teacher. The tool is able to recommend extracurricular activities and suggests a local internship to help bolster her credentials and introduce her to like-minded people.

  • In many schools, students have only infrequent access to a counselor. AI could in some cases provide an alternative.

  • AI tools can provide students with tailored college and career advice, help identifying academic pathways, evaluate career options (eg, CareerDekho, Unschooler, Coach), provide job market insights and future skills demand that may be geography-specific, and recommend networking events and internships.

  • Tools can also help ensure that, for a desired college or career path, a student is taking the courses necessary to succeed.

  • And tools can support students in navigating the college application process, help with essay writing (ESAI), identifying scholarships and financial aid opportunities, and guidance on essays.

Smart student portfolio (new)

A high school student installs a new app that allows her to send copies of her work in every course to a digital portfolio she controls. Using the app in an earth science class on natural hazards, she can also capture notes, add highlights, and build her own collection of hazards to use as a jumping-off point for a project on researching hazard prediction technologies. The app employs AI to help her track progress toward her college application, identify gaps and help her fill them, celebrate her progress over time, and curate her best work to share as she chooses.

  • An increasing amount of student work, especially in middle and high school, is digital. But it is often stored in siloed systems and quickly forgotten. An AI tool could capture it all and begin to build a portfolio, not just to show off the best work, but to document the student’s journey through knowledge structures.

  • Students could see their journey, and their growth, more coherently. The tool might help them uncover recurring patterns and trajectories on which to build.

  • Students could add their own notes and sources—as tools such as Google’s NotebookLM are beginning to do—ask questions, discover related ideas and resources, and connect ideas from different areas in order to deepen understanding.

  • Though recorded lectures are not common in high school, as they are in college, especially since mobile phones are ever less welcome, tools like Jamworks may begin to help students mark, learn, and inwardly digest each lesson.

  • The portfolio could also become a source of holistic formative assessment (see Holistic assessment, above), generating an evaluation of their work without the need for a set-piece test.

Meta-cognitive support (new)

A high schooler realizes, to get where he aims to get, he needs to improve his self management. Lacking someone in his life to mentor him, he turns to an AI tool to help him assess his current situation, decide what’s important, and learn skills to make a plan and monitor his way toward it. The tool assesses his study habits, for instance, and sets weekly goals that start easy to build his confidence. He learns how to manage his assignment load and how better to review and retain new knowledge.

  • There are a set of meta-cognitive skills that are essential to academic success and yet are seldom addressed directly in school. They include self-awareness, growth mindset, seeking and absorbing feedback, identifying goals and making plans to achieve them, monitoring performance toward them, persisting in their pursuit, and more. (See, for example, XQ Institute’s Competency framework.)

  • AI tools are emerging to support students by understanding their specific context and making gradual, actionable recommendations, like an empathetic mentor. An early example is SchoolAI’s “Explore Spaces” that incorporate study plans and exam-taking strategies. More such supports will grow out of to-do-list managers and study tools.

Eyes Wide Open

We should be aware of the risks to students and educators as we explore the many positive possibilities of AI in K-12:

  • The Null Hypothesis. Most promising edtech interventions do not scale. In this past year, generative AI’s acceleration has seemed almost magical, but so did television, computers, the internet, and mobile — previous foundational technologies that became part of K-12 education but didn’t necessarily improve it.

  • Hallucinations. Preventing AI from making up facts and sources may prove to be difficult. Teachers are now assigning students to critically investigate AI output, but some may lack the media skills or background knowledge to do so successfully.

  • Atrophy of Critical Thinking. Even if AI resources become extremely accurate, using them mindlessly will bypass productive struggle and negate writing’s potential as a tool for deep thinking and self-expression. Calculators enabled students to avoid computation; will AI do the same for thinking? As science fiction writer Ted Chiang wrote about AI more generally, “the desire to get something without effort is the real problem.”

  • The Deluge. As the cost of creating content and digital tools approaches zero, the web is already becoming more flooded with books, lesson plans, flashcards, study guides, and videos of varying quality, making it more time-consuming for educators to select and align to research-backed pedagogy. Eventually, trusted curation, integration frameworks, and rollups into unified curricula may bring quality and coherence, but until then students and teachers may suffer from more cognitive load deciding across resources and tools.

  • Distraction. “Attention Is All You Lack.” AI has already been sneaking into most K-12 classrooms for the past few years, powering addictive, distracting TikTok and YouTube feeds. As entertainment engagement algorithms improve, the battle for attention will become more difficult and more critical.

  • Bias. A corollary of the loss of critical thinking skills. Students may be more susceptible to the biased output of current AI models.

  • Information Bubbles. Hyper-personalization could lead to students learning from a narrow set of sources that never force them to grapple with divergent values and experiences.

  • Dehumanization. (See Mitch Resnick and Jennifer Carolan’s warnings.) Schools that employ AI tutors may overlook the ways that teachers care for students, motivate them, and model what it is to be a healthy adult. Ideally AI will free up more educator time for human connection; to enhance the community at the heart of school, we’ll need a lodestar of learning that’s about more than skills and information transfer.

Compiled by Laurence Holt and Jacob Klein. Laurence has spent the last two decades leading innovation teams in for-profit and nonprofit K-12 organizations. Jacob is currently Head of Product at TeachFX.

Previous
Previous

The 5 Percent Problem: Online mathematics programs may benefit most the kids who need it least