Young and Resilient Research Center Indicator Framework

Case Studies

We need less on our feeds

Implementation Examples

Example 1: Including a quick data collection poll during Swipe Safe training

Steps 1, 2, & 3: review and choose relevant indicators and factors

Seven indicators and associated plain language questions are selected from the overall list. Six of the indicators align with Swipe Safe concepts and the seventh (CI-OS-PS.01.sb) addresses potential gaps in training.

Step 4 & 5: pick methodology & devise questions

Adapting plain language question options to scale-type designs create questions that can be answered quickly and easily for better application during Swipe Safe training. In this example, all young Swipe Safe trainees will be invited to individually complete a paper-based quick poll with the following seven questions, answered on 1-100 point ‘temperature gauge’ response scales (e.g., 1 = 0% worried, 100 = 100% worried).

Original | Revised How worried are children about online bullies/hackers/stalkers/predators? | How worried are you about cyberbullying or online predators? When they are online, how careful are children about making sure the things they do won't harm them/other people? | How much do you think about how things you say or do online affect other people? When they are online, how careful are children about making sure information/people are trustworthy? | How careful are you about making sure you can trust the things you see or hear online? How careful are children about who they share information with online? | How careful are you about sharing things about yourself online, like your name, photos, the town where you live? How are children dealing with risky/harmful/inappropriate people online? | How confident are you that you know what to do if someone tries to hurt you online? How are children dealing with risky/harmful/inappropriate content online? | How confident are you that you know what to do if you see something that is inappropriate or harmful online? What do children say they need to know to help them stay safe online? | What are the three most important things you would like to know to help you stay safe online?

Steps 6: prepare materials

Individual worksheets will be prepared for each Swipe Safe trainee presenting the poll questions and response scales, brief instructions, and space for participants’ anonymous identifiers. Swipe Safe facilitators will distribute worksheets, introduce the activity, and allow participants 5 minutes to complete.

Steps 7 & 8: collect and analyse data

Swipe Safe facilitators will collect participants’ completed poll worksheets. Worksheet data will be cleaned and entered into the analysis platform (e.g., Microsoft Excel) and poll answer values computed. Analysis results will be interpreted and presented as relevant and appropriate based on participation (e.g., males vs females, age ranges, etc.).

Step 9: repeat data collection

The same quick poll will be repeated across all workshops and phases of Swipe Safe training with young people (with a separate, tailored poll administered with adults, if applicable). Comparing repeat poll data (e.g., training at Phase 1 vs training at Phase 2, Swipe Safe trainees vs non-trainees) will help assess any changes in indicators.

Example 2: Running a data collection workshop in addition to Swipe Safe training

Steps 1, 2, & 3: review and choose relevant indicators and factors

As per Example 1, above.

Step 4 & 5: pick methodology & devise questions

In this example, a separate assessment workshop will be organized and run in addition to Swipe Safe training workshops. A workshop type methodology provides opportunities and raises challenges. Workshops allow collection of in-depth data in greater quantities, including using mixed-methods activities to generate both quantitative and qualitative, and individual and small-group data. Data about the same indicators generated in different ways can help triangulate analysis. On the other hand, workshops require more design, planning, and organisation, and necessitate further recruitment of participants for an activity outside regular Swipe Safe training. Designing and planning a workshop includes: 1) deciding on a duration and schedule that will be accessible and engaging for participants (2 hours is a suitable good minimum to work towards); 2) designing quantitative and qualitative activities that address indicators and adapting questions accordingly; 3) scheduling to allow for introductions, wrap-up, and energiser/short breaks. A workshop schedule for the selected indicators could include:

Introduction & Icebreaker (20 mins)
Qualitative activity (35 mins): Before & Now – individually, participants write brief statements explaining their thoughts and actions about trusting things online and sharing things online, before Swipe Safe training and after Swipe Safe training, then quick break
Qualitative activity (45 mins): Scenarios – in small groups, participants respond to 3 brief scenarios describing a character’s online experience reflecting online risks/harms associated with indicators (e.g., online bullying, misinformation, contacts from strangers), then quick break
Qualitative activity (25 mins): Bricks & Ladders – in small groups, participants draw a wall of bricks representing things that they and other young people can do online that could harm others, then draw ladders representing how those harms might be avoided or addressed
Quantitative activity (10 mins) – Quick Poll – individually, participants complete the quick poll outlined in Scenario 1 above, then quick break
Qualitative activity (25 mins): Worries – individually, anonymously participants write down on slips of paper their worries about online hackers, scams, bullies, strangers, or harmful/inappropriate content. Worries are placed in a box and 3-4 randomly drawn out and discussed
Qualitative activity (15 mins): Best & Worst: individually, participants write down the three things they enjoyed most about Swipe Safe training, the three worst things about training, and the three most important changes to make training better
Wrap-up (5 mins)

Steps 6: prepare materials

Materials to support data collection workshops include: information, consent, and registration forms; detailed workshop schedule and facilitator guidance; worksheets for all activities; as well as ancillary items like pens and paper, and other logistical assets (venues, etc.). Facilitators play a central role in ensuring workshop success so clear guidance and understanding about workshop purpose and goals is essential.

Steps 7 & 8: collect and analyse data

Workshop facilitators will collect and/or record (e.g., during group discussions) participants’ completed worksheets. Worksheet data will be cleaned and entered into the analysis platform (e.g., Microsoft Excel) and analysed as relevant. For qualitative data in particular, an anticipated analysis framework should be clearly understood well before data analysis commences.

Step 9: repeat data collection

Ideally, as in the quick poll example, indicator data collection workshops will be repeated with participants from all Swipe Safe training workshops and phases. Comparing repeated workshop data will help assess changes in indicators.

Example 3: Distributing a take-home data collection activity pack

Steps 1, 2, & 3: review and choose relevant indicators and factors

As per Scenario 1, above.

Step 4 & 5: pick methodology & devise questions

Take home activity packs are paper or online materials that Swipe Safe training participants can take home and complete unsupervised. Like workshops, take home packs can include both quantitative and qualitative activities and so help triangulate analysis. However, because participants complete activities in their own time, take home activities should be simple to understand, easy to complete, and involve relatively few different activities (e.g., two to four simple things). In this example, participants who have enrolled in the Swipe Safe program, but not yet undertaken training, are given take home packs before training starts (e.g., 2 weeks before). Over a 2-month period, they are asked to complete activities at home according to the supplied schedule. A take home activity pack for the selected indicators could include:

A diary-type booklet that participants are asked to complete once a fortnight over a 2-month period. The diary could include the quick poll from Example 1 and a prompt asking participants to record any significant experiences over the past week related to online risk/harms
Brief worksheets participants complete once a week where they describe any worries they have about online risks or harms
Worksheets that participants complete at the start, mid-way, and end of the 2-month period where they describe their thoughts and actions about trusting things online and sharing things online
Worksheets that participants complete at the start and end of the 2-month period where they describe the three most important things they would like to know about how to keep safe online

Steps 6: prepare materials

Materials to support take home activity packs include: information, consent, and registration forms; activity FAQs; worksheets for all activities; ancillary items like pens and paper; and an implementation schedule recording participants’ anticipated start and end dates, and schedules for periodic prompts/reminders and support checks to facilitate participants’ ongoing engagement.

Steps 7 & 8: collect and analyse data

Unlike in workshops, completed data remains with participants until the end of the entire nominated period. Completed materials can be collected in person or returned by mail, however, data will likely be received over an extended period rather than all at once. Data will be cleaned and entered into the analysis platform (e.g., Microsoft Excel) as it is received. Preliminary analysis can begin while data is still being returned, but complete analysis will not be possible until all data has been received.

Step 9: repeat data collection

Requirements to repeat data collection will depend on the initial implementation plan but may not be necessary if the original home activity period encompasses all phases of Swipe Safe training. Return of completed take home packs will allow for comparisons between participants (and potentially, Swipe Safe phases) to help assess changes in indicators.