AI art in teaching emotional intelligence to kids with ASD. Interactive prototypes for a smartwatch and a responsive website. Created with Figma.
UI & UX Design | Figma | Smartwatch | Website
Get a Quick Overview of the Prototype & Concept Idea from the Concept Video on Youtube
The main idea is to
- help especially kids with Autism Spectrum Disorder (especially Asperger) to understand and express their emotions better with the help of AI generated art
- enable children to process their emotions with their guardians through those images, practice emotional intelligence and at the same time create art with it.
Example interaction:
- Child interacts with their smartwatch --> smartwatch saves data as text --> app uses that data to AI generate an image via API
- --> saves image to user profile --> from the user profile the image can be released to public webpage, which is an artistic wall of images
- Data which is being sent to the API can also include such data as heart rate info and other sensor info that you might get from wearables
- Target group: Kids with Autism Spectrum Disorder (especially Asperger)
- Team: 2 members (me & Mika Pelvola)
- Course project: “HTI.800 Human-Technology Interaction Research and Innovation Project”, Tampere University
- Theme: "Generative AI & Art"
- As parents, we wanted to create something useful for kids/families
- We have ASD kids within our close circle
- Generative AI and emotional intelligence are hot topics, and emotional intelligence is also taught at schools in Finland
- Kids with ASD often have problems with recognizing, naming, expressing, and handling emotions, and visual help is useful for them
- Responsible for all the design related things and literature reviews
- Did basically everything else expect for programming and the concept videos
- Ideation and presentations were done together
To help kids with ASD
- to understand and express their emotions with the help of AI generated art through their smartwatch, which doesn’t get lost easily
- to manage their emotions with their guardiands, and practive emotional intelligence
- to create art that they can also share for the public, if wanted
- to maintain a digital emotional diary created based on their emotions, heart rate, and other available data with the help of generative AI
- Interactive prototypes of the smartwatch and website (mobile + main views on desktop) created with Figma
- Technical demo
- Concept video + technical demo video
- User personas and usage scenarios
- Literature reviews & empathizing
- Reading relevant litterature and empathizing the target group
- Defining
- Defining project scope and creating user personas and use scenarios
- Ideation
- Further ideation of the concept
- Prototyping & demo
- Creating interactive prototypes with Figma and a technical demo with the Bangle.js 2 smartwatch
- User testing / prototype evaluation
- Testing the prototypes and concept idea with 2 kids and 1 parent
- Iteration & Concept Videos
- Small iterations to the prototypes, and then creation of the concept videos
- Every kid with ASD is different, so different kinds of interaction techniques are needed to practice emotional intelligence in addition to parents’ help
- Images, graphic signs, objects, signs, simplified language, technical device, computer programs, mobile apps, etc. are helpful
- IDEA: A smartwatch + exploring emotion art generated by AI on mobile devices and computer
- Images are super important, because they contretize and clarify emotions, but they should be used together with text
- IDEA: Using emojis (with text) for expressing the emotion
- Having an emotional diary can be useful
- IDEA: A digital emotional diary and artwall that are created by the AI generated art based on the kid’s emotions and other available data, like heart rate
- Help for discussion with the child is, for example, Tony Attwood’s “CAT kit - Cognitive Affective Training”
- IDEA: Tips for discussion about emotions to parents
- During the meltdown breathing, walking away from the situation, and, for example, counting to 10 can help
- IDEA: The smartwatch can help the child in calming down by counting together to 10
- Praises are important
- IDEA: The smartwatch could give instant positive feedback
- Using the application is not tied to any certain physical location (can be e.g. school or home, inside or outside)
- Sunlight may affect the usage
- Child can use the device/app on their own or with their guardians
- Devices with a browser (main focus is on a smart watch)
- Sam goes to school wearing a smartwatch, which conveniently stays with him at all times. He has a PE class where they play soccer.
- Sam's team is losing 2-4, and when Sam unexpectedly gets the chance to kick the ball into an almost empty goal, he misses. This triggers an emotional breakdown: he starts crying, raging, and banging his head against a nearby light pole.
- The watch on his wrist vibrates and notifies him that his heart rate is elevated. It also offers him the option to select an emoji, a color, and an animal that best represent his current emotion.
- Sam notices this and selects an angry emoji, the color red, and a lion. The watch collects this data, along with other available information like his heart rate, and sends it forward to generate AI-created art.
- An AI-generated image, reflecting Sam's emotions at that moment, appears on the watch but also gets uploaded to a website where previous emotional artworks are stored. It first lands on a private wall, from which it can be shared to a communal, public art wall if desired.
- Sam’s mother, Martha, has been at work all day and is finally home.
- She tries to ask Sam about his day but doesn’t get much of a response.
- She decides to check how Sam has expressed his emotions through AI-generated art and opens the EmotionAI website on her phone.
- She notices that Sam experienced negative emotions in the afternoon and shows him the artwork he created using his smartwatch to express his feelings.
- Sam becomes interested in the result and explains what happened during his PE class. They then discuss the emotions Sam felt at that moment, what triggered them, and what he could do to calm himself down if something similar happens again.
- Sam feels proud that he was able to create something so visually impressive by expressing his emotions, and Martha is happy that she was able to get her son to open up and talk about his feelings.
See the smwartwatch proto created with Figma
See the mobile proto created with Figma
Only the main views (home page and art wall) of the prototype were created due to the scope of the course.
See the desktop proto cretead with Figma
I created the prototypes and my partner the video.
See the concept video on YouTube
My partner did both the technical demo and the video.
See the technical demo on YouTube
User testing wasn’t mandatory in this course, but we wanted to gather at least some feedback on our concept idea and prototypes. As a result, two children from the target group and one parent tested them. The children tested only the smartwatch prototype, while the parent tested all three prototypes. The prototypes were slightly iterated based on the feedback, but some were left for the future due to the scope of the course.
- One of the children really liked the idea, the usability, and the process of creating art, while the other wasn’t as enthusiastic (note: that child was already in a bit of a bad mood).
- Counting to ten was perceived as too fast by both children, although they agreed that counting could likely help them calm down.
- Distinguishing between different font family options was difficult for both.
- Using possible voice commands was seen as potentially challenging.
- One child wished for the option to select specific animals, such as a penguin, a horse, and a pig.
- The other mentioned that it’s good the watch (the clock face) can be digital, as it’s easier to understand.
- It was appreciated that it’s a smartwatch rather than a mobile app since, on a phone, it’s easy to get distracted and start doing other things, which isn’t possible with a watch.
- Clearer positive feedback, ("You successfully calmed yourself down").
- Counting was too fast.
- More control for the user, such as adjusting font size, deleting data. Also more information about data usage.
- The art wall was highly praised, especially in the desktop view, where more content was visible at once.
- A more prominent link to the art wall.
- Additional filters, such as emotions.
- On the private wall, the ability to review artwork from a specific week and mark images as favorites.
- "Design first, but with tech along all the time" approach was a great way to keep the focus on important things with realistic perspective
- It's not that simple to design for a watch – especially for kids
- The small screen size limits the amount of information that can be included and you need to think about how the user would interact with the watch
- This target group, ASD kids, was new to us, and now we can also in the future design better for that target group and kids in general
- More about prototyping with Figma in general
- Will be useful also in future works
- Simply a lot of new observations related to APIs, Javascript, data transfer etc.
- The project was ambitious and the topic was important to us, and we succeeded quite well!
- The group work went really smoothly
- Would be nice to take this even further, and in that case, for example
- more evaluation with representatives of the target groups (ASD kids and their guardians)
- prototype iteration based on the results from the next evaluation sessions
- development