Digital approaches for user testing with girls
5 tips from Oky India on remote girls' consultations and user testing during the time of COVID
COVID-19 has impacted girls in unprecedented ways. The pandemic may result in more than 11 million girls dropping out, or lacking access to school. Reaching girls has never been more important, and digital platforms play a huge role in providing support and information to girls, through the devices they use.
Oky, the period tracker app for girls, has been co-created with adolescent girls based on user-centered design principles. Initially launched in Indonesia and Mongolia, Oky is now also live in India and Mexico, and will be released in more countries in 2022 and 2023.
Girls’ ideas and preferences are central to Oky’s design in every market. Girls shape features and technical specifications, and the look and feel of localized app versions. Much of girls’ feedback is received through user testing of prototypes or mock up designs to get insights what works and what does not work for girls in a specific context.
User testing is usually done in person. Oky prototypes were user tested with girls across Indonesia and Mongolia in 2019, with the user testing teams traveling across the two countries to reach and meet girls.
But COVID pandemic measures have posed new challenges: how can out-of-school girls learn about important menstrual and reproductive health topics? How do we engage girls and test digital prototypes with them, to ensure a digital tool, like Oky, meets their needs and wants, when in-person interaction is not possible?
Splash, an Oky partner in India, faced these challenges when user testing Oky with girls in West Bengal, during the height of a COVID surge in 2021. Splash developed a solution by leveraging digital approaches to engage with girls on Oky remotely.
Splash International is a non-profit organization with the goal to change the lives and health of children living in dense urban areas. Splash does this by focusing on WASH and menstrual health and hygiene interventions, in public schools in urban areas, to impact the highest concentration of children.
Splash has trialed and tested digital approaches for girls’ engagement and user testing, and identified 5 key tips to engage girls remotely:
1: Create diverse user groups on familiar platforms
Ensure that remote engagement and user testing includes diversity in participants and local languages and is conducted on a digital platform that girls already use. For example, in West Bengal, 117 adolescent girls from 11 different schools in Kolkata joined the user groups. The selected schools covered a range of relevant demographics including the most common languages, Bengali, Hindi and Urdu, as well as other groups. Splash created six WhatsApp groups, and girls were assigned to the groups based on the language they spoke.
“In 2020, we conducted a digital landscape survey with Quilt.AI which provided evidence that in Kolkata girls are highly engaged on digital platforms. During our interactions with schools, we found that WhatsApp was used widely by schools to maintain communication with parents and students. Teachers also shared lessons and conducted ‘tests’ using WhatsApp amid school closures. Given its prevalence, we chose WhatsApp as our primary platform to engage with girls remotely and for Oky user testing.”
Selecting a platform that girls already understand and use reduces access or digital literacy issues for remote engagement. However, the use of digital platforms requires safeguarding considerations. Plan International has included a useful risk assessment of different platforms in their Safeguarding Children and Young People Policy Guidance (p.9). Anyone engaging in remote consultations and user testing with girls needs to perform a risk assessment and to put safeguarding measures in place.
2: Engaging adults as moderators and supporters
Adult moderators are important to support young users and engage with parents. It is recommended to engage moderators who are already known to the girls participating in the remote user testing exercise. For safeguarding purposes, two moderators should be assigned per group.
Splash worked with focal teachers from the participating schools to act as moderators, to support girls, and ensure parents were comfortable with the process. A two-day workshop was conducted on Zoom to build teacher capacity on menstrual health and hygiene, how to discuss sensitive topics with adolescent girls, and they received an introduction to the global Oky app.
The second adult moderator for each group was a team member from Splash, who facilitated the remote engagement and activities.
3: Spend time breaking the ice
To get the most out of remote testing groups it is important to build trust and rapport among users. Splash set up an initial introductory group Zoom call, followed by 5 days of interactions on WhatsApp, before the formal user testing process began.
Informed consent was explained using Zoom calls and voice notes before consent was solicited.
“We hosted on-boarding and pre-testing sessions from September 15, 2021 - October 5, 2021. These on-boarding sessions were an opportunity to introduce ourselves so that the girls knew the ‘faces’ behind the WhatsApp messages and instructions sent by us, this enabled us to build rapport and trust.”
This pre-test period was also used to ensure girls were comfortable and engaged. It allowed them to ask questions about WhatsApp features and the user testing process. Facilitators used an encouraging tone and positive reinforcement through both voice and text messages to encourage girls to speak up and share.
4: Assign specific tasks and give time for responses
It is important to allow for sufficient time for the user testing process and to give girls ample opportunity to explore the app and provide feedback. Splash engaged girls to test Oky India over the course of a month, starting with a detailed introductions of the Oky app, followed by specific tasks for users to complete. Periodic touchpoints enabled Splash to keep participants engaged. Tasks started simply, with voting on avatar graphics and moved on to examine specific features, from creating an account, to using the period cycle tracker, to the ‘Help’ function and the ‘Encyclopedia’.
Girls were encouraged to explore the app, test features, and provide feedback on the ease of navigation and use. This enabled Splash to better understand the girls’ interpretations of instructions, definitions, and their general perceptions of Oky. If users did not understand how to complete a task, moderators shared voice notes, screen shots, and video tutorials to address users’ questions. Splash moderators logged insights, suggestions, pain points, and crash reports.
5: Create space for verbal feedback
To conclude the user testing, Splash organized group wrap-up Zoom calls, in which users shared verbal feedback about their experience as testers. This was an important step, as insights not yet captured via WhatsApp chats or voice notes could be documented during these final calls. It was also an opportunity to express appreciation and thanks to the testers for their valuable inputs.
“Girls engaged in the remote user testing shared rich insights that ensured their perceptions and experiences were represented in the final design of the Oky India app. Girls suggested adding more quizzes, notifications, and add tutorial links in the sign-up page of the app. Girls were excited to participate in the user engagement and testing process and appreciated that their views would shape the app for girls across India. Some girls suggested adding more languages, so girls in other Indian states (especially the states in Southern India and North-East) could also benefit from the Oky app."
The future of remote engagement and user testing
Virtual settings for engagement and user testing have limitations, for example, in reaching girls who don’t have smartphones, or who only have access to shared phones. However, when in-person consultations are not possible, remote user testing can contribute invaluable insights and findings. Remote user testing approaches will therefore become an important part of the Oky toolbox and can be used as a supplementary channel to in-person engagement.
“In many ways remote testing provided insights that in-person testing would have missed, as girls engaged with Oky India over a longer period and in a more natural setting than the usual ‘lab’ environment of user testing.”
