A Tale of Two AI Mice

Background

With the advent of strong LLMs like ChatGPT, Age of Learning launched two different conversational child-facing chatbot initiatives to further the success of ABCmouse.

The goal of these initiatives was to make an educational AI companion that would bring in more revenue through clever use of emerging technology combined with our classic flagship character.

‍

In-App Ask ABC

The goal of this initiative was to create an AI-powered chatbot companion inside the ABCmouse app that children can have conversations with. This feature was intended to increase ABCmouse app engagement KPIs, keeping existing subscribers playing longer through the addition of this new, interactive content.

‍

External App Ask ABC

The goal of this initiative was to create a standalone AI chatbot app for children featuring ABCmouse that children can have conversations with. This feature was intended to increase the number of ABCmouse subscribers through character advertising. In theory, parents would download this free app for the novelty, have their child chat with ABC, then gain better brand awareness which would lead to more subscriptions.

‍

Generative Research

Our department knew early in development that some generative research would be required to guide both development teams to better understand how children and their families would interact with any sort of AI-powered chatbot.

‍

We triangulated several using several different methods to target key research questions.

‍

What type of conversational agents are children already exposed to?

To address this question, our team conducted a literature review and shared it out to the product teams. This research showcased that children are already familiar with voice-based conversational agents, such as Alexa and Siri, and are generally successful at communicating with them. Our research identified areas where existing conversational agents were weak. Specifically, existing conversational agents sometimes struggled to understand child speech patterns and did not carry on with the type of conversations children wanted to have. Additionally, some research showed that it is possible for childrens to grow emotionally attached to conversational agents and have fun interacting with them. 

‍

What characters do parents want their children to talk with?

We conducted a survey that identified parents judge character preference based on visual design quality, knowledge of their child's preferences, perceived character values, and character realism. This research helped identify how certain segments of parents felt about certain characters, which then was used to shortlist which characters in the ABCmouse brand family would be featured in upcoming AI prototypes. 

We discovered that parent preferences for characters varied depending on their child's age, leading the product teams to choose "cuter" cartoon characters like Gooloo Gooloo and ABCmouse for a product aimed towards younger children. If our product teams were to age the AI product up, they would have chosen a more human "educator" coded character instead of a "cute friend for your child" character.

‍

How do children communicate with a voice chat conversational companion, and what do they talk about?

Our department ran 20 semi-structured interviews with families of children ages 4 - 8, some of which have used the ABCmouse app while others have never used the ABCmouse app. These interviews contained a task where the moderator pretended to have ABC the character on the phone and, using a custom ABCMouse Google Voice Agent, had the child talk with ABC.

Before taking part in this Wizard of Oz-style activity, most children demonstrated that they did not have a strong attachment to the ABCmouse flagship character, ABC. They knew him as "the learning mouse" or just "the learning app": they thought he was a "nice" mouse that "likes learning" but he would not be their first choice of character to invite to their birthday party. This supports other research by our team showing that most children who use ABCmouse do not have a strong attachment to the character, identifying an opportunity in the ABCmouse product for more character-child interaction. 

When children had the opportunity to have conversations with ABC through the custom Google Voice Agent, they asked questions primarily about himself, his family, his friends, and appearance. This indicated to our stakeholder teams that while their AI ABCmouse prototypes must be able to talk about themselves and have a good understanding of the ABCmouse character family background.

‍

Evaluative Research

Once development was on the way, our task was to work with the different product teams to ensure they could iterate on the design and technical specifications.

During this phase we primarily ran semistructured remote moderated playtesting with children ages 4 - 7. We would run a playtest every two weeks, syncing with each product update.

‍

Throughout this we discovered:

• LLMs struggle with child speech patterns: Large language models, the backbone of any AI-p, are trained off of adult communication patterns. Children communicate fundamentally differently than adults do: They talk in sentence fragments, talk in a non-linear fashion, don't enunciate words, make up their own words, and ask questions by using statements. This caused both Ask ABC prototypes to struggle holding basic conversations. In the end, Ask ABC struggled holding meaningful conversations with children in our target age range. 
• Trust & Safety: We had several instances of AI hallucinating and adult product testers bypassing guardrails. Additionally some content, while not exactly harmful or offensive, is simply not the type of content you want to hear coming out of ABCmouse's mouth. 
• No Clear Job to Be Done: The development teams hoped that children would seek out the ABCmouse conversational agent and hold conversations about educational topics. However, this theory did not come true: most children struggled holding open-ended, unprompted conversations with ABCmouse. If they want to know something specific, they are able to ask a family member or use an existing conversational agent, like Alexa or even ChatGPT. 
• No Parasocial Relationship: For this product to work, our organization expected that children would develop a parasocial relationship (one-way affection) towards Ask ABC, even if they were aware it's not a "real person". However, user testing showed that was not the case. ABCmouse was a fact-finding tool for older children and too frustrating to interact with for younger children. 
• Monetization: Watching their children talk with a kid-friendly AI mouse was a novelty that parents would simply not be willing to pay for. Some parents were even concerned about the AI aspect: they would not want their child talking to an AI mouse, even for free, and indicated that they would cancel their ABCmouse subscriptions if this product was implemented.

The Pivot: Ask ABC as a product search guide.

In the end, both the in-app and external app Ask ABC teams had to drastically pivot. The evaluative research made it abundantly clear that the developed solution would not meet the goals of this project, which was furthered by the beta app gaining little traction. The Ask ABC external app was pulled from the app store and is no longer functioning.

One thing we observed repeatedly in user testing was children saying, "I want to play a game." Children and their parents consistently tried to use Ask ABC as a method to access activities even if that feature was not available. This presented an opportunity for a safe, creative use of the AI model already built.

Now, Ask ABC has been redeveloped into an ABCmouse search feature, solving the issue of a child-friendly voice search. ABCmouse children and parents are now able to access Ask ABC and have ABCmouse search through over 10,000 activities to find something that meets their search terms. This new iteration is a far cry from the intended disruptive "Your Child's First AI" idea. However, all the development labor put towards making it was well put to use in a major quality of life update. Even fervent anti-AI parents don't mind using AI for speech recognition and content search purposes. Furthermore, just because Ask ABC the AI Chatbot wasn't a good fit for children in the ABCmouse age range doesn't mean that it's not a good fit for all children. Learnings from this research & development cycle were put to use in Age of Learning's line of School Solutions targeted towards an older age range.

This feature is live in both ABCmouse and ABCmouse 2.0.

‍