Page 1 of 1 Embargo Copy - Not for Distribution Apps As Learning Tools: A Systematic Review Shayl F. Griffith, PhD, Mary B. Hagan, BA, Perrine Heymann, BPhil, Brynna H. Heflin, BA, Daniel M. Bagner, PhD (doi: 10.1542/peds.2019-1579) Embargo Release Date: Monday, December 23, 2019 - 12:01 am (ET) Embargo Policy: Information in this article is embargoed for release until the date indicated above. Interviews may be conducted prior to the embargo release date, but nothing may be aired or published. If you are a media representative and have questions about the embargo, upcoming press events, or other matters, please contact AAP Communications staff at 630-626-6177, or via e-mail at commun@aap.org The American Academy of Pediatrics, 141 Northwest Point Blvd., Elk Grove Village, IL 60007 Apps As Learning Tools: A Systematic Review Shayl F. Griffith, PhD, Mary B. Hagan, BA, Perrine Heymann, BPhil, Brynna H. Heflin, BA, Daniel M. Bagner, PhD CONTEXT: Young children have increasing access to interactive applications (apps) at home and at school. Existing research is clear on the potential dangers of overuse of screens, but there is less clarity around the extent to which interactive apps may be helpful in supporting early learning. abstract In this systematic review, we present a narrative synthesis of studies examining whether children ,6 years can learn from interactive apps. OBJECTIVE: DATA SOURCES: The PsycInfo, PubMed, ACM Digital Library, and ERIC databases were searched. STUDY SELECTION: Studies were included if the study design was randomized or nonrandomized controlled (quasi-experimental), the sample mean age was ,6 years, the intervention involved children playing with an interactive app, and academic, cognitive, or social-emotional skill outcomes were measured. DATA EXTRACTION: Of 1447 studies, 35 were included. RESULTS: Evidence of a learning benefit of interactive app use for early academic skills was found across multiple studies, particularly for early mathematics learning in typically developing children. Researchers did not find evidence of an intervention effect for apps aiming to improve social communication skills in children with autism spectrum disorder. LIMITATIONS: Risk of bias was unclear for many studies because of inadequate reporting. Studies were highly heterogenous in interventions, outcomes, and study design, making comparisons of results across studies difficult. There is emerging evidence to suggest that interactive apps may be useful and accessible tools for supporting early academic development. More research is needed to evaluate both the potential of educational apps to support early learning, and their limitations. CONCLUSIONS: Department of Psychology and Center for Children and Families, Florida International University, Miami, Florida Dr Griffith conceptualized and designed the study, developed the data extraction instrument, adapted the risk of bias instrument, collected and coded the data, and drafted the initial manuscript; Ms Hagan participated in developing the risk of bias instrument, collected and coded the data, drafted portions of the initial manuscript, and reviewed and revised the manuscript; Ms Heflin and Ms Heymann participated in developing the risk of bias instrument, collected and coded data, and reviewed and revised the manuscript; Dr Bagner conceptualized and designed the study, participated in the development of the data extraction instrument, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. DOI: https://doi.org/10.1542/peds.2019-1579 Accepted for publication Aug 30, 2019 Address correspondence to Daniel M. Bagner, PhD, Department of Psychology and Center for Children and Families, Florida International University, 11200 SW 8th St, Miami, FL 33139. E-mail: dbagner@fiu.edu PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275). Copyright © 2020 by the American Academy of Pediatrics To cite: Griffith SF, Hagan MB, Heymann P, et al. Apps As Learning Tools: A Systematic Review. Pediatrics. 2020;145(1):e20191579 PEDIATRICS Volume 145, number 1, January 2020:e20191579 REVIEW ARTICLE Touchscreen mobile devices (eg, smartphones, tablets) have become ubiquitous for young children.1 Interactive applications or “apps” considered “educational” for young children have similarly gained in popularity2 and are increasingly being integrated into early childhood classrooms as learning tools because of perceived advantages for child engagement and active learning.3 The integration of interactive app technology into children’s lives at home and school has outpaced research needed to inform comprehensive recommendations for its use. Recommendations have thus far focused on preventing overuse of screens4 rather than opportunities for maximizing learning. Research on whether young children can learn from interactive apps; the academic, cognitive, or social-emotional skill domains that may be best supported by interactive apps; and the conditions under which this learning may be maximized; is still emerging. engaging.2 Additionally, research shows that young children benefit from repeated and varied practice of foundational skills.12,13 Interactive apps have potential to support active engagement with learning material through embedding learning concepts into gamelike activities, scaffolding children’s learning through adaptive learning technology, providing feedback and rewards through gameplay, and promoting repeated practice of important foundational skills.2,14 Additionally, the ubiquity of touchscreen devices could facilitate more equitable and convenient access to educational materials in the home, in contrast to other types of technology (eg, computers), which still show marked income gaps in access.1 Research on whether interactive educational touchscreen media can support early learning is therefore essential for updating our understanding of “high-quality educational programming” for the new digital age. (I2 .90%), limiting interpretability of a meta-analytic effect size. Instead, a narrative synthesis of results is the recommended approach when studies are highly variable in interventions, outcomes, study design, and risk of bias.18,19 In 2016, the American Academy of Pediatrics (AAP) recommended screen time (including touchscreens) for 2- to 5-year-olds be limited to ,1 hour per day of “high-quality educational programming” and that parents coview screens with children.5 Given the scarcity of studies on interactive screen media, the AAP statement largely referenced studies from the more-extensive literature on television, in which researchers documented the detrimental effects of overconsumption of television on sleep,6 obesity,7 and cognitive and language development8 but also showed that high-quality and developmentally appropriate programming (eg, Sesame Street)9 can support vocabulary, math, and social-emotional skills in children .30 months.10,11 Studies in diverse fields, including psychology, education, and computer science, have begun to investigate children’s learning from apps, with most studies published in the past 3 years. The findings of these studies have not yet been well synthesized. In existing review articles, authors have provided commentaries on children’s use of mobile technology, rather than systematic reviews of the literature.15,16 In one recent metaanalysis, authors reported a significant positive effect across studies of young children’s learning from touchscreen devices.17 However, there was significant clinical and methodological heterogeneity in the included studies, which encompassed varied types of interventions (eg, electronic books, interactive game apps), outcomes targeted, comparison groups, and study designs (randomized and nonrandomized, controlled and noncontrolled). Statistical heterogeneity in this study was high 2. What content subjects or skills are particularly well suited to touchscreen app learning? Theories on learning suggest children learn best when activities promote active “minds-on” learning, are appropriately scaffolded, and are 2 Thus, in this study, we aimed to systematically review the rapidly expanding but heterogeneous body of evidence for young children’s learning from touchscreen interactive apps. Given the popularity of interactive educational apps for young children; the importance of early learning and school readiness for later academic, behavioral, and emotional functioning20,21; and the existing questions about whether young children can apply information learned on a screen to real life15; we focused on children younger than 6 years in this review. The specific research questions were as follows: 1. Is there evidence that children ,6 years can learn from using interactive touchscreen apps? 3. How does learning from touchscreen interactive app games compare with children’s learning in other instructional contexts, including in-person instruction or educational video? METHODS Data Sources An electronic database search was conducted in 4 databases popular to the fields of psychology, pediatrics, education, and computer science (PsycINFO, PubMed, ERIC, and ACM Digital Library) by using the following Boolean search phrase: (tablets OR iPads OR touchscreen OR apps OR interactive media) AND (learning OR literacy OR mathematics OR skills OR development OR functioning OR outcomes OR achievement) AND (toddlers OR young children OR GRIFFITH et al preschoolers). Articles published before January 1, 2008 (the year Apple and Android app stores opened), and written in languages other than English were excluded. The database searches were conducted on January 22, 2019. Additionally, the reference sections of included articles and relevant review articles were examined to identify any other studies meeting inclusion criteria (n = 2). Study Selection Study inclusion and exclusion criteria are presented in Table 1. The search initially identified 1447 unique records. After initial screening, 115 records were subject to full text review by 2 independent coders from a group of 4 (authors S.F.G., B.H.H., M.B.H., P.H.). Disagreements were discussed, and consensus was reached. Randolph’s k22,23 was 0.84, indicating excellent interrater reliability. Thirty-four articles, describing 35 studies, were included (see Fig 1 for Preferred Reporting Items for Systematic Reviews and Meta-analyses diagram). Data Extraction and Synthesis The following information was extracted from each study: design, sample size, location of study, language of participants, length of intervention, type of control group, name of app(s), study context, targeted subject(s) of intervention, outcome measures, and results. Given FIGURE 1 Preferred Reporting Items for Systematic Reviews and Meta-analyses flowchart of the study selection. the number of articles (n = 12) for which there was not sufficient information to calculate an effect size and the clinical, methodological, and statistical heterogeneity of the included studies in terms of design, outcomes measured, and comparison group used, formal meta-analytic strategies were not used to summarize overall effects.18,19 Instead, we presented a narrative synthesis of the included studies according to the targeted subject of the intervention.19 Additionally, to provide context as to clinical significance of study findings, where possible, we presented ranges of effect sizes (using Cohen’s d) TABLE 1 Population, Intervention, Comparison, Outcome, and Study Design Chart of Inclusion and Exclusion Criteria Category Inclusion Criteria Children 5 y and under (mean age of sample ,72 mo or identifiable subgroup with mean ,72 mo). Studies conducted in any country, with samples of any language background, were included, as long as the article was published in English. Clinical and nonclinical samples were included. Intervention Intervention involves use of an interactive game app on a touchscreen. “Interactive” suggests that children’s interaction with the screen has an effect on the material presented. Studies were excluded if they involved only interventions with electronic books, computers, noninteractive video, or multimedia or transmedia interventions. Studies in which a touchscreen device or app was used only as an assistive technology (eg, iPad version of a Picture Exchange Communication System) were excluded. Comparison Any comparison group type (eg, treatment or instruction as usual, no treatment, attention control, waitlist control, noninteractive video control, alternative app condition control, etc). Outcome Academic achievement, executive functioning skills, socio-emotional skills, and task-specific learning (eg, learning nonsense names, learning to find an object, learning facts). Studies were excluded if only process outcomes (eg, engagement) were measured. Study design Randomized controlled designs (including laboratory experiments and field trials) and quasi-experimental designs. Case studies, reviews, multiple baseline or multiple probe studies, correlational studies, and descriptive studies were excluded. Population PEDIATRICS Volume 145, number 1, January 2020 3 representing post-test differences between the app intervention and control groups for the randomized studies. Assessment of Risk of Bias Because most included studies had a randomized design, risk of bias was assessed with the Cochrane’s risk of bias tool for randomized trials24 that was adapted to include only categories relevant to psychological and educational research. Studies were rated in the following categories: randomization (selection bias), blinding of outcome assessment (detection bias), missing data (attrition bias), and selective reporting (reporting bias). Quasiexperimental studies, for which assignment to group was not random (eg, administrative convenience; n = 5), were also evaluated by using this tool to capture risk of bias due to nonrandomized assignment in the randomization category. Ratings of “high,” “low,” or “unclear” risk were assigned for each domain. Each study was coded for risk of bias independently by 2 coders from a group of 4 (coders stated above). Kappa values were 0.91, 0.61, 0.51, and 0.96 for randomization, missing data, outcome assessment, and selective reporting, respectively, indicating moderate to excellent agreement.25 All disagreements were discussed, and a consensus was reached between coders. RESULTS Description of Studies The total number of participants across studies was 4639. Eighteen studies were conducted in the United States, 5 in Australia, 4 in the United Kingdom, 2 in Canada, and 1 in each of the United Arab Emirates, Italy, Greece, Croatia, the Netherlands, and Germany. The primary language of most of the samples (n = 30) was English (for studies conducted in the United States, Australia, the United 4 Kingdom, Canada, and the United Arab Emirates). The 5 remaining studies included samples in which the primary language spoken was not English (Italian, Greek, Croatian, Dutch, and German, respectively); in each case, the interventions and assessments were delivered in the primary language of the sample. Samples included typically developing children, except for the studies in which researchers examined social communication (n = 3), which all consisted of children diagnosed with autism spectrum disorder (ASD). In 12 studies, researchers used apps developed by the study authors (noted in Table 2); in 4 of these 12 studies, the app was commercially available, suggesting potential financial profit for the author.26–29 In the remaining studies, researchers used existing apps developed by nonrelated companies. outcome; or groups receiving no treatment. Five studies included interventions delivered at home, 18 included interventions delivered in the school, and 12 were delivered in a laboratory. The studies had a variety of outcomes in targeted educational domains, including mathematics (n = 15), language arts (LA) (n = 11), executive functioning (n = 5), health and science knowledge (n = 3), and social communication (n = 3). One study included targeted outcomes in both mathematics and science,62 and another included math and LA outcomes.60 Each of the studies examined potential learning or skill benefits of interactive apps. Although not specifically excluded from the search strategy, no identified study was designed to examine harmful effects of interactive app use on functioning. See Table 2 for a summary of studies. Twenty-nine studies had randomized designs, including 18 randomized controlled trials (RCTs) with app interventions delivered over an extended period (ranging from 5 days to 1 academic year) and 11 laboratory-based experimental studies, which included interventions (ie, learning phases) delivered over a short time span (ranging from 5 to 20 minutes). For simplicity, studies with randomized designs and interventions delivered over an extended time are referred to as RCTs. Laboratory-based experiments (all of which also have randomized designs but a short-term intervention phase) are referred to as experimental studies. The remaining 6 studies included in this review were quasi-experimental (ie, nonrandomized controlled) trials. Comparison groups varied across studies, including in-person instruction (eg, classroom-based, inperson, using manipulatives), video instruction, or other non–app-based instruction (eg, pen and paper practice); attention controls receiving apps not targeting the identified Risk of Bias Within Studies Overall, 19 of the 35 studies (54%) were at high risk of bias in at least 1 domain. High risk of bias ratings were given for nonrandom assignment to groups, substantial missing data and data not missing at random (eg, imbalances in missing data between groups), and lack of blinding of outcome assessors. Additionally, risk of bias was judged to be unclear due to inadequate reporting of methods in at least 2 domains for 19 of the 35 studies. Only 3 studies had low risk of bias across the randomization, missing data, and outcome assessment domains, and no studies were given low risk ratings across all 4 domains. The results of the studies should be interpreted with caution given the literature lacks rigorously designed RCTs. Narrative Synthesis of Studies by Learning Outcomes Early Math Learning In 12 randomized studies (9 RCTs) and 3 quasi-experimental studies, researchers examined children’s math GRIFFITH et al PEDIATRICS Volume 145, number 1, January 2020 5 Brown and Harmon,44 2013a Spencer,41 2013 Learning domain: English and LA Blackwell,42 2015 Schacter and Jo,27 2017a Schacter et al,35 2016a Schacter and Jo,26 2016a Park et al,40 2016a Outhwaite et al,38 2017a Papadakis et al,39 2018a Outhwaite et al,36 2019a Miller,35 2018a Kosko and Ferdig,32 2016a Mattoon et al,33 2015a Han,31 2018 Learning domain: early math learning Aladé et al,30 2016a Author, Year 1y 1 wk Quasi-experimentale RCTe N = 352, mean age = 5.56 y, 51.1% male N = 20, range = 48–59 mo 6 wk RCTe 5d 22 wk 10 sessions over 2–3 wk 15 wk 14 wk 16 wk 12 wk 10 d 6 wk 3 wk 12 training trials 3 trials Length of Intervention RCTe Quasi- experimentale RCTe RCT e Quasi- experimentale RCTe RCTe RCT e RCTd Experimentalb Experimentalb Study Design RCTe N = 13, range = 4–5 y, 53.8% male N = 389, mean age = 60.64 mo, 49.6% male Range = 50–61 mo, 48.1% male N = 365, mean age = 62.0 mo, 48.5% male N = 103, mean age = 4.87 y, 51.5% male N = 227, mean age = 56.00 mo, 46.3% male N = 433, mean age = 4.55 y, 50.6% male N = 100, mean age = 55.85 mo, 48% male Range = 4–5 y N = 40, mean age = 42.08, 50% male N = 73, mean age = 51.34 mo N = 24, range = 4–5 y, 62.5% male N = 60, mean age = 58.06 mo, 42% male Study Population TABLE 2 Studies Included in the Systematic Review Shared iPad in classroom with set of 70 apps 3 educational iPad apps targeting upper- and lowercase alphabet knowledge, matching, and No. concepts Know Number Free Math Shelfc Math Shelfc 16 math apps designed for the studyc Math app designed for studyc Math Shelfc Maths 3–5, Maths 4–6 Math app developed for studyc Zorbit’s Math Adventure DinoKids – Math Lite, Hungry Fish, Motion Math, Math Express, Maths Age 3–5, Park Math Package of 15 math apps Maths 3–5, Maths 4–6 Measure That Animal App Intervention Condition and Name of App Educational apps unrelated to target academic content Classroom instruction Classroom instruction Top 5 math apps in the app store Classroom instruction Classroom instruction Memory game app Classroom instruction Classroom instruction Classroom instruction Classroom instruction Instruction with traditional math manipulatives No treatment Noninteractive video of gameplay in Measure that Animal No treatment Control Condition Alphabet knowledge (PALS Pre-K45 Upper and Lower Case Alphabet Knowledge subtests and measures created for study) Early literary skills (Star Early Literacy Assessment43) Numeracy (correctly written No. 1–10) Broad math skills (iPad assessment created for the study) No. sense abilities (iPad assessment created for the study) Broad math skills (iPad assessment created for the study) Broad math skills (TEMA-3) Broad math skills (48-item and 50-item measures created for study) Broad math skills (TEMA-3) Broad math skills (22-item measure created for study) Broad math skills (Progress Test in Math Level 537) Quantity knowledge (quantity recognition and Give-N tasks) Broad math skills (19-item measure created for study) Broad math skills (TEMA-334) Measurement (measuring tasks similar and less similar to app gameplay) Outcome (Measure) No Yes Yes Yes Yes Yes Yes Yes Yes Yes No No No No Yes (for measuring task similar to app) Intervention Condition Outperformed Control Condition? 6 GRIFFITH et al N = 48, mean age = 45.19 mo, 52.1% male N = 170, mean age = 51.9 mo, 43% male Neumann,49 2018a Learning domain: science and health Kwok et al,58 2016a Walter-Laager et al,56 2017a Vatalaro et al,53 2018a Teepe et al,52 2017a Russo-Johnson et al,51 2017a N = 98, mean age = 27.3 mo, 53.1% male N = 71, mean age = 41.05 mo, 48.2% male N = 71, mean age = 40.05 mo, 40.0% male N = 63, range = 3–5 y, 52.5% male N = 30, mean age = 56.5 mo, 50% Male Gilliver et al,48 2016a Patchan and Puranik,50 2016 Quasi- experimentale N = 281, kindergarten, “relative balance of boys and girls” Cubelic and Larwin,46 2014a 6 1-min trials Experimentalb 8 training trials 20 min Experimentalb Experimentalb 8 wk Quasi- experimentale 2 10-min intervention sessions over 2 wk 8 wk RCTe RCTb 9 wk 6 wk One academic year Length of Intervention RCTe RCTe Study Design Study Population Author, Year TABLE 2 Continued Endless Alphabet, Noodle Words, Goodnight ABC, ABC GO, Beck and Bo, Draw and Tell, Don’t let the Pigeon Run this App, Alien Assignment Lingua Kidz used with an adult Classroom instruction Apps targeting first sound fluency (ABC Phonics, Build a Word, Magic Penny, Starfall), phoneme segmentation fluency (First Word Sampler, Word Wall HD, Pocket Phonics, Skills Builder Spelling, Phonic Monster 1), and nonsense word fluency (ABC Touch and Learn, Bee Sees, Kindergarten Lite, Starfall, Super Why) Phonological awareness app created for the studyc Endless Alphabet, Letter School, the Draw App Writing practice with iPad app (finger drawing) Word learning app designed for the studyc Jeffy’s Journey Apps chosen and used by the study preschool (Letter School, Gazzili Science, Yumilo Rainbow Power, Faces I Make, Counting Bear) Vocabulary picture cards, used with an adult Noninteractive video of gameplay in word learning app No treatment Writing practice with pen and paper Classroom instruction Vocabulary app created for the study Control Condition App Intervention Condition and Name of App No. correct answers on 4 animal facts Language development (abridged version of the standardized “Language and Development Test for 2-Year-Olds”57) Receptive vocabulary skills (PPVT54) and expressive vocabulary skills (EVT55) Expressive and receptive vocabulary (measure created for study) Word learning (No. correct responses in object naming) Alphabet knowledge (naming 26 letters and writing 26 uppercase letters) Letter name and sound knowledge, name writing (measure created for study) Phonological awareness and vocabulary (measure created for study) Phonological awareness (DIBELS47 Next Assessment Phoneme Segmentation Fluency and Nonsense Fluency subtests) Outcome (Measure) No No No No No Yes Yes Yes Yes (for Phoneme Segmentation Fluency and Nonsense Fluency) Intervention Condition Outperformed Control Condition? PEDIATRICS Volume 145, number 1, January 2020 7 N = 75, range = 23.5–25.6 mo, 44% male Learning domain: executive functioning Choi and Kirkorian,63 2016a Tarasuik et al,66 2017a Huber et al,65 2018a Study 1: N = 18, mean age = 5.4 y, 50% male; study 2: N = 50, mean age = 5.1 y, 46% male N = 96, mean age = 36.6 mo, 56.5% male N = 49, mean age = 4.8 y, 45% male N = 100, mean age = 55.85 mo, 48% male Learning domain: math and science Schroeder and Kirkorian,62 2016a Huber et al,64 2016a N = ∼750, kindergarten N = 114, mean age = 4.99 y, 50% male N = 86, mean age = 66.93 mo, 62.8% Study Population Learning domain: math and ELA Bebell and Pedulla,60 2015a Putnam et al,59 2018a Author, Year TABLE 2 Continued Experimentalb Experimentalb Experimentalb Experimentalb Experimentalb Quasi-experimentale RCTd Study Design 4 trials 9 min of screen time 4 5-min trials 4 trials Don’s Collections: 15 questions total; Life Cycles: 5 trials 1y 5d Length of Intervention Practice with the app Extra Tower of Hanoi Early math app (Shiny Party) App that required children to tap on specific location to reveal the hiding place of a characterc Practice with the app Extra Tower of Hanoi Two mini games in the app Mesozoic Math Adventures (Don’s Collections [Math] and Life Cycles [Science]) Set of commercially available apps targeting ELA and math used in the classroom (during school-wide iPad implementation year) App developed for study to teach animal factsc App teaching about healthy and unhealthy foods (D.W.’s Unicorn Adventure) App Intervention Condition and Name of App Practice with 3D Tower of Hanoi Model Cartoon (Penguins of Madagascar) Practice with 3D Tower of Hanoi Model App that showed a noncontingent video of the hiding place of a character Noninteractive video of gameplay in word learning app Classroom instruction as usual (in the year before the iPad implementation) Face-to-face instruction on animal facts No treatment Control Condition No. extra moves on 3D Tower of Hanoi task Delayed gratification (gift-delay task) and working memory (working memory task) No. extra moves on 3D Tower of Hanoi task Responses on object retrieval trials Quantity; knowledge of growth cycle (measures created with items similar to app gameplay) Early literary and broad math skills (CPAA61 ELA and Math Subtests) Recall (No. times correctly names healthy and unhealthy foods) Outcome (Measure) No Yes No Yes (for younger toddlers) Yes (for knowledge of the growth cycle for older preschoolers); no (for quantity task for younger preschoolers) Yes Yes Intervention Condition Outperformed Control Condition? 8 GRIFFITH et al N = 80, mean age = 34.83 mo, 79.6% male Whitehouse et al,72 2018a 6 mo 2 mo RCTd RCTd 4 wk Length of Intervention RCTd Study Design Standard ABA 1 3 apps created for the study to promote attention, imitation, and vocabularyc App targeting social communication skills, attending to people and following social cues (FindMe)c Community therapy 1 App designed to enhance ABA therapy social communication outcomes (Therapy Outcomes by You) App Intervention Condition and Name of App Community therapy Treatment as usual Standard ABA Control Condition Social communication skills (MCDI, CSBS-DP), restricted and repetitive behaviors (RBSR73), behavioral flexibility (BFRS74), developmental skills (ATEC75), and adaptive skills (VABS-276) Social communication skills (MCDI68; CSBSDP69; BOSCC70), (), and presence of Autism (ADOS-271) Attention, vocabulary, and imitation (mastered targets in standard ABA) Outcome (Measure) No No No Intervention Condition Outperformed Control Condition? ABA, Applied Behavior Analysis; ADOS-2, Autism Diagnostic Observation Schedule; ATEC, Autism Treatment Evaluation Checklist; BFRS, Behavior Flexibility Rating Scale; BOSCC, Brief Observation of Social Communication Change; CPAA, Children’s Progress Academic Assessment; CSBS-DP, Communication and Symbolic Behavior Scale, Developmental Profile; DIBELS, Dynamic Indicators of Basic Early Literacy Skills; ELA, English Language Arts; EVT, Expressive Vocabulary Test; MCDI, MacArthur Communicative Development Inventory; PALS Pre-K, Phonological Awareness Literacy Screener, Preschool; PPVT, Peabody Picture Vocabulary Test; RBS-R, Repetitive Behavior Scale, Revised; TEMA-3, Test of Early Mathematics Ability, Third Edition; VABS-2, Vineland Adaptive Behavior Scales, Second Edition. a Peer-reviewed. b Study context is laboratory. c App was developed by study authors; italic script indicates the app is commercially available. d Study context is home. e Study context is school. N = 54, mean age = 49.63 mo, 79.6% male N = 30, mean age = 47 mo, 90% male Study Population Fletcher-Watson et al,29 2016a Learning domain: social communication Esposito et al,67 2017a Author, Year TABLE 2 Continued outcomes after using math interactive apps. In 11 of the 15 studies, researchers examined school-based interventions; a home-based intervention was examined in 1; and 3 were laboratory-based experiments. Outcomes included measures of early math skills (including number recognition, number naming, and basic addition and subtraction). School and home interventions ranged from 5 days to 1 academic year. Overall, 7 of the 12 randomized studies (including 1 of the 3 laboratory-based experiments) and all 3 quasi-experimental studies showed better outcomes in the math app intervention group compared with the control group used. Specifically, of the 9 studies (6 RCTs) that compared a math app intervention to in-person classroom instruction, 7 (4 RCTs) reported significantly greater math learning gains on early math measures (n = 8) and a number-writing task (n = 1) for groups using interactive math apps,26,27,36,38,39,41,60 whereas the remaining 2 RCTs reported statistically equivalent gains between the groups.33,35 Of the 4 studies (3 RCTs) in which researchers compared a math app intervention to control groups using apps not targeting math (ie, attention controls) or not receiving treatment, 3 reported better outcomes on early math measures for app intervention groups compared with controls,28,31,40 and 1 reported no significant differences between groups.32 Effect sizes, when reported (n = 9, from 9 RCTs with intervention periods between 5 days and 22 weeks), ranged from 0.04 to 0.94. In 2 experimental studies, researchers examined math learning from interactive apps compared with noninteractive video, with mixed results. Younger preschoolers’ (3–4.5 years) performance on a quantity math task improved after watching an instructional video, but not after playing a math app, whereas older preschoolers (4.5–5.5 years) PEDIATRICS Volume 145, number 1, January 2020 improved in neither condition.62 In the other study, preschoolers’ (3.5–5.5 years) performance on a measuring task improved in both interactive app and noninteractive video conditions.30 LA Learning In 7 randomized studies (5 RCTs) and 4 quasi-experimental studies, researchers examined children’s LA outcomes (letter knowledge, phonological awareness, letter writing, or vocabulary). Eight of the 11 studies were based in schools, whereas 3 were laboratory-based experiments. Interventions ranged from 1 week to 1 academic year, and most used commercially available apps. Overall, 3 of the 7 randomized studies (including 0 of the 3 experimental studies) and 3 of the 4 quasi-experimental studies showed better outcomes in the app intervention group compared with the control group, whereas the remainder showed statistically equivalent outcomes between groups. Specifically, in 3 studies (2 RCTs), researchers examined alphabet knowledge and phonological awareness after children used apps targeting these skills, compared with apps not targeting these skills. No significant differences in learning were found between groups for 2 studies,44,53 whereas the other found children’s phonological awareness and vocabulary skills improved more after using a phonological awareness app compared with a vocabulary app.48 In 5 studies (2 RCTs), researchers found that children’s letter naming and writing skills50,60,77,78 and phonological awareness46,60,78 improved after using a set of LA apps compared with a classroom or in-person instruction control group. In 2 experimental studies, researchers examined children’s vocabulary learning using an app compared with nonapp instruction (noninteractive video and picture card instruction) and found no differences in learning.51,56 In 1 final study, researchers found that children’s receptive and expressive vocabulary did not improve after 2 10-minute sessions using a storytelling app with their parents compared with no treatment.52 Effect sizes for the RCTs, when reported (n = 12, from 3 school-based interventions over 1–9 weeks), ranged from 0.07 to 0.94. Effect sizes for the experimental studies, when reported (n = 3, from 2 studies), ranged from 0.41 to 0.60. Health and Science Facts In 3 randomized studies (1 homebased RCT and 2 experimental studies), researchers examined children’s learning of health or science concepts. In the studies, researchers examined children’s learning of healthy versus unhealthy foods, the growth cycle, and facts about animals. Overall, the 3 studies indicated children learned from the apps, with some exceptions. In 1 study, researchers found no difference in learning facts about animals between an interactive app group and a group receiving inperson instruction,58 whereas researchers in another found the app group learned more about healthy foods compared with a no-treatment control.59 In a third study, researchers found that older preschoolers (4.5–5.5 years) learned about the growth cycle better from interactive app play compared with noninteractive video instruction, but younger preschoolers (3–4.5 years) were not able to learn in either condition.62 Executive Functioning In 4 experimental studies, researchers examined children’s performance on executive functioning tasks. In 2 studies, researchers found that children improved their performance on the traditional threedimensional (3D) version of the 9 Tower of Hanoi task after practice trials with a touchscreen app version of the task, and this improvement was not significantly different from children who both practiced and were tested with the 3D version.64,66 In the third study, researchers found that young toddlers (mean age = 25 months) were better able to learn how to retrieve objects in a real-life situation when they received training with an interactive app compared with a noninteractive video; however, this effect was reversed for older toddlers (mean age = 33 months).63 In the fourth study, researchers found that children were more likely to pass a gift-delay task and performed better on a working memory task after playing an educational app game compared with watching noneducational television.65 communication outcomes ranged from 0 to 0.40. DISCUSSION In this article, we systematically reviewed 35 studies on young children’s learning from interactive educational apps. The studies were heterogeneous in targeted outcomes, length of intervention, context of intervention, and experimental design. Evidence for learning from touchscreen interactive app use was strongest for math-related app interventions and weakest for interventions targeting social communication skills. Study quality was mixed overall, with the risk of bias for many studies unclear in more than 1 domain, often from deficits in methods reporting. Social Communication In 3 RCTs, researchers examined whether social communication skills improved after using apps designed to improve social skills, such as imitation, attention to social stimuli, and following social cues. Each study included a clinical sample of children with ASD; no studies with social development outcomes in typically developing children met inclusion criteria for this review. In these 3 studies, researchers compared outcomes in a treatment group receiving the app at home in addition to standard applied behavioral analysis or other community therapies to a control group receiving only treatment as usual. For 2 of the 3 studies, children and parents used the app together,67,72 whereas in the third study, children used the app alone.29 None of the 3 studies reported significant improvement in the primary social communication skills outcome measures for the app treatment group compared with the control group. Effect sizes for gains in the app groups on social 10 Can Young Children Learn From Interactive Touchscreen Apps? A majority of studies comparing math or LA (letter naming and writing, phonological awareness) outcomes in children using interactive apps to controls receiving equivalent regular classroom or small-group instruction found more favorable outcomes for the interactive app condition. In addition, although study effect sizes varied widely, large effects were reported for school-based studies of math and LA learning. For example, the largest effect sizes in math were from a school-based 22-week intervention using the app Math Shelf,27 whereas the largest effect sizes in LA were from a school-based 6-week intervention using a phonological awareness app.48 These findings are notable because they suggest the addition of interactive touchscreen technology may afford a learning advantage for early academic learning exceeding traditional modes of instruction and are consistent with the theory that interactive apps are well suited to support active, repeated, and varied practice of skills.2,16 Results of other studies reviewed suggest touchscreen app games have the potential to support learning in other skill areas, including science knowledge and executive functioning. However, there were a small number of studies targeting these skill areas, and further research is needed to establish whether children’s understanding of science concepts could be promoted using interactive apps, and similarly, whether improvement in executive functioning task performance is task specific or would generalize to other measures of executive functioning. Results from the 3 RCTs in which researchers examined app interventions targeting social communication skills for children with ASD suggest these skills may not be best supported by touchscreen apps. This finding is particularly important to note because touchscreen technology is popular in ASD treatment, including electronic augmentative exchange communication systems.79 Social communication skills may be less easily generalized to the real world, compared with foundational early math and preliteracy skills, which is consistent with research showing improvements in app gameplay but not real-world social communication skills.29 However, given the small number of studies examining this outcome, further research in this area is needed. Finally, findings from laboratory experimental studies on children’s learning from interactive apps compared with noninteractive video were mixed. In general, researchers in these studies examined learning in younger children who have difficulty transferring learning from media.80 Although in some studies researchers found that task performance improved with an interactive learning GRIFFITH et al phase,63 others found the interactive component may have interfered with learning because of overtaxed attentional resources.62 Although other studies show older children may benefit from learning through interactive app play, further research is needed to establish the lower age limit of this potential benefit. Future Research and Clinical Implications Important gaps were identified and should be addressed in future research. First, more large-scale randomized trials of apps are needed, with focus on clarifying what app features and content may best support learning. Thousands of apps are added to app stores every month,81 so large-scale trials of each new educational app may not be feasible. Understanding features of interactive apps supporting learning in different content areas will allow clearer standards regarding apps to be deemed “educational” by app makers. Second, most studies reviewed were conducted in schools and examined math and LA outcomes. Given touchscreen devices are widely present in homes and the AAP’s recommendation that screen time at home be educational, it will be important to examine educational benefits of touchscreen media use at home. More studies examining other learning content areas are also needed. Third, almost all app interventions reviewed involved child solo use of apps. Children learn best when coviewing television with parents,82 and parent behaviors while co-using apps have been linked to child positive engagement and affect.83 Thus, examining children’s learning from app games with parental co-use is an important area for future research. Fourth, all but 5 of the reviewed studies included app interventions PEDIATRICS Volume 145, number 1, January 2020 delivered in English to Englishspeaking populations, and most included studies based in the United States, making it difficult to evaluate the impact of country or language on learning from apps. There was no apparent pattern to the results of studies according to country or language; for example, of the 4 studies in which researchers examined children’s math outcomes conducted outside of the United States, 3 showed better performance in the app condition, including the 1 study conducted in Greek. Further research is needed on the extent to which educational apps can support learning in a child’s nonpreferred language as well as on the generalizability to other languages of the findings presented on LA learning from apps. Fifth, further research is needed to evaluate important moderators of treatment effects. There is emerging evidence from the studies reviewed that age may be an important moderating factor, and children below preschool age may have more difficulty learning from screen media. Future researchers should more closely examine moderating factors that may impact learning from touchscreen apps, including children’s pretest ability and demographic characteristics, such as sex, socioeconomic status, and previous experience and familiarity with technology. Finally, in a separate body of research on television, researchers have identified negative effects of excessive screen use on children’s development.6–8 Although the authors of the articles in this review did not explicitly examine negative effects of use or overuse, as the popularity and diversity of digital media for children increases, it will be necessary for researchers and clinicians to balance the potential positives of interactive screen media for learning with the imperative to prevent overuse. Further research examining the relative advantages and disadvantages of various types of screen media (eg, television versus interactive touchscreen) will be important to update recommendations to better reflect the new digital context. Limitations There were some limitations to this systematic review. First, risk of bias was unclear for many studies, and many studies had incomplete reporting of study methods. Second, although the study included unpublished research, it is possible additional unpublished research was not identified. Unpublished studies may have fewer positive outcomes as a result of publication bias.84 Third, because of the nascent state of the literature, the summarized studies were highly heterogeneous. Thus, it was not possible to compare results across studies or investigate moderators of these effects. CONCLUSIONS In this systematic review, we examined the literature on educational app learning in young children. Evidence of learning benefits from educational touchscreen apps was found across multiple studies, particularly for early mathematics. Conversely, multiple studies did not find evidence of an intervention effect for apps aiming to improve social communication skills in children with ASD, providing some preliminary evidence that concrete academic skills may be better supported by interactive apps than more-complex social-emotional skills. Findings were mixed in studies in which researchers examined children’s short-term learning from interactive touchscreen media, particularly for toddlers. With the results of this review, we highlight both the potential for interactive apps 11 to play a useful role in promoting some types of early skills as well as the substantial gaps in knowledge that still exist regarding their use. Continued research in this area will be critical to inform the debate around young children’s screen time, as clinicians and researchers try to strike a balance between taking advantage of the potential benefits of new technology, while encouraging limits in screen time. ACKNOWLEDGMENTS We thank David H. Arnold, PhD, for his helpful feedback on a draft of this article. ABBREVIATIONS AAP: American Academy of Pediatrics App: application ASD: autism spectrum disorder LA: language arts RCT: randomized controlled trial 3D: three-dimensional FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose. FUNDING: No external funding. POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose. COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2019-3503. REFERENCES 1. Rideout V. The Common Sense Census: Media Use by Kids Age Zero to Eight. San Francisco, CA: Common Sense Media; 2017. Available at: https://www. commonsensemedia.org/sites/default/ files/uploads/research/csm_zerotoei ght_fullreport_release_2.pdf. Accessed March 13, 2019 2. Hirsh-Pasek K, Zosh JM, Golinkoff RM, Gray JH, Robb MB, Kaufman J. Putting education in “educational” apps: lessons from the science of learning. Psychol Sci Public Interest. 2015;16(1): 3–34 3. Haßler B, Major L, Hennessy S. Tablet use in schools: a critical review of the evidence for learning outcomes. J Comput Assist Learn. 2016;32(2): 139–156 4. McCarthy C. Pediatricians and television: it’s time to rethink our messaging and our efforts. Pediatrics. 2013;131(3):589–590 5. Council on Communications and Media. Media and young minds. Pediatrics. 2016;138(5):e20162591 6. McDonald L, Wardle J, Llewellyn CH, van Jaarsveld CHM, Fisher A. Predictors of shorter sleep in early childhood. Sleep Med. 2014;15(5):536–540 7. Cox R, Skouteris H, Rutherford L, FullerTyszkiewicz M, Dell’ Aquila D, Hardy LL. Television viewing, television content, food intake, physical activity and body mass index: a cross-sectional study of 12 preschool children aged 2-6 years. Health Promot J Austr. 2012;23(1):58–62 the unknown. Pediatrics. 2015;135(1): 1–3 8. Zimmerman FJ, Christakis DA, Meltzoff AN. Associations between media viewing and language development in children under age 2 years. J Pediatr. 2007;151(4):364–368 16. Radesky JS, Christakis DA. Increased screen time: implications for early childhood development and behavior. Pediatr Clin North Am. 2016;63(5): 827–839 9. Anderson DR, Bryant J, Wilder A, Santomero A, Williams M, Crawley AM. Researching Blue’s Clues: viewing behavior and impact. Media Psychol. 2000;2(2):179–194 17. Xie H, Peng J, Qin M, Huang X, Tian F, Zhou Z. Can touchscreen devices be used to facilitate young children’s learning? A meta-analysis of touchscreen learning effect. Front Psychol. 2018;9:2580 10. Anderson DR, Pempek TA. Television and very young children. Am Behav Sci. 2005;48(5):505–522 11. Anderson DR, Hanson KG. What researchers have learned about toddlers and television. Zero Three. 2013;33(4):4–10 12. Whitehurst GJ, Lonigan CJ. Child development and emergent literacy. Child Dev. 1998;69(3):848–872 13. Purpura DJ, Baroody AJ, Lonigan CJ. The transition from informal to formal mathematical knowledge: mediation by numeral knowledge. J Educ Psychol. 2013;105(2):453–464 14. Neumann MM, Finger G, Neumann DL. A conceptual framework for emergent digital literacy. Early Childhood Education Journal. 2017;45(4):471–479 15. Radesky JS, Schumacher J, Zuckerman B. Mobile and interactive media use by young children: the good, the bad, and 18. Fletcher J. What is heterogeneity and is it important? BMJ. 2007;334(7584): 94–96 19. Ryan R, Cochrane Consumers and Communication Review Group. Cochrane consumers and communication review group: data synthesis and analysis. Available at: https://cccrg.cochrane.org/sites/cccrg. cochrane.org/files/public/uploads/ Analysis.pdf. Accessed April 13, 2019 20. Duncan GJ, Dowsett CJ, Claessens A, et al. School readiness and later achievement. Dev Psychol. 2007;43(6): 1428–1446 21. High PC, Klass P; Council on Early Childhood. Literacy promotion: an essential component of primary care pediatric practice. Pediatrics. 2014; 134(2):404–409 22. Randolph J. Free-marginal multirater kappa: an alternative to Fleiss’ fixed- GRIFFITH et al marginal multirater kappa. In: Proceedings from the Joensuu University Learning and Instruction Symposium 2005; October 14–15, 2005; Joensuu, Finland. 23. Warrens MJ. Inequalities between multi-rater kappas. Adv Data Anal Classif. 2010;4(4):271–286 24. Higgins JPT, Altman DG, Gøtzsche PC, et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928 25. Fleiss JL. Statistical Methods for Rates and Proportions. Hoboken, JN: Wiley; 1981 26. Schacter J, Jo B. Improving low-income preschoolers mathematics achievement with Math Shelf, a preschool tablet computer curriculum. Comput Human Behav. 2016;55:223–229 27. Schacter J, Jo B. Improving preschoolers’ mathematics achievement with tablets: a randomized controlled trial. Math Educ Res J. 2017; 29(3):313–327 28. Schacter J, Shih J, Allen CM, et al. Math Shelf: a randomized trial of a prekindergarten tablet number sense curriculum. Early Educ Dev. 2016;27(1): 74–88 29. Fletcher-Watson S, Petrou A, ScottBarrett J, et al. A trial of an iPadTM intervention targeting social communication skills in children with autism. Autism. 2016;20(7):771–782 30. Aladé F, Lauricella AR, Beaudoin-Ryan L, Wartella E. Measuring with Murray: touchscreen technology and preschoolers’ STEM learning. Comput Human Behav. 2016;62:433–441 31. Han A. How Training Set and Prior Knowledge Affect Preschoolers’ Perception of Quantity and Early Number Learning [dissertation]. West Lafayette, IN: Purdue University; 2018 32. Kosko K, Ferdig R. Effects of a tabletbased mathematics application for preschool children. J Comput Math Sci Teach. 2016;35(1):61–79 33. Mattoon C, Bates A, Shifflet R, Latham N, Ennis S. Examining computational skills in prekindergarteners: the effects of PEDIATRICS Volume 145, number 1, January 2020 traditional and digital manipulatives in a prekindergarten classroom. Early Child Res Pract. 2015;17(1) 34. Ginsburg H, Baroody A. Test of Early Mathematics Ability. Austin, TX: PRO-ED; 2003 35. Miller T. Developing numeracy skills using interactive technology in a playbased learning environment. Int J STEM Educ. 2018;5(1):39 36. Outhwaite LA, Faulder M, Gulliford A, Pitchford NJ. Raising early achievement in math with interactive apps: a randomized control trial. J Educ Psychol. 2019;111(2):284–298 37. Math Assessment Resource Service. Progress Test in Maths Level 5. London, UK: GL Assessment; 2015 38. Outhwaite LA, Gulliford A, Pitchford NJ. Closing the gap: efficacy of a tablet intervention to support the development of early mathematical skills in UK primary school children. Comput Educ. 2017;108:43–58 39. Papadakis S, Kalogiannakis M, Zaranis N. The effectiveness of computer and tablet assisted intervention in early childhood students’ understanding of numbers. An empirical study conducted in Greece. Educ Inf Technol. 2018;23(5): 1849–1871 44. Brown M, Harmon MT. iPad intervention with at-risk preschoolers: mobile technology in the classroom. J Lit Technol. 2013;14(2):56–78 45. Invernezzi M, Sullivan A, Meier J. Phonological Awareness Literacy Screening - Prekindergarten. Charlottesville, VA: University Press; 2001 46. Cubelic CC, Larwin KH. The use of iPad technology in the kindergarten classroom: a quasi-experimental investigation of the impact on early literacy skills. Compr J Educ Res. 2014; 2(4):47–59 47. Good RH, Kaminski RA. Dynamic Indicators of Basic Early Literacy Skills Administration and Scoring Guide, 6th ed. Eugene, OR: Institute for the Development of Educational Achievement; 2003 48. Gilliver M, Cupples L, Ching TYC, Leigh G, Gunnourie M. Developing sound skills for reading: teaching phonological awareness to preschoolers with hearing loss. J Deaf Stud Deaf Educ. 2016;21(3):268–279 49. Neumann MM. Using tablets and apps to enhance emergent literacy skills in young children. Early Child Res Q. 2018; 42:239–246 40. Park J, Bermudez V, Roberts RC, Brannon EM. Non-symbolic approximate arithmetic training improves math performance in preschoolers. J Exp Child Psychol. 2016;152:278–293 50. Patchan MM, Puranik CS. Using tablet computers to teach preschool children to write letters: exploring the impact of extrinsic and intrinsic feedback. Comput Educ. 2016;102:128–137 41. Spencer P. iPads: improving numeracy learning in the early years. In: Steinle V, Ball L, Bardini C, eds. Proceedings from the 36th Annual Conference of the Mathematics Education Research Group of Australasia; July 7–11, 2013; Melbourne, Australia 51. Russo-Johnson C, Troseth G, Duncan C, Mesghina A. All tapped out: touchscreen interactivity and young children’s word learning. Front Psychol. 2017;8:578 42. Blackwell CK. Technology use in early childhood education: investigating teacher access and attitudes toward technology and the effect of iPads on student achievement [dissertation]. Evanston, IL: Northwestern University; 2016 43. Renaissance Learning. STAR Early Literacy Accessment. Available at: https://www.renaissance.com/ products/assessment/star-360/starearly-literacy-skills/. Accessed November 1, 2019 52. Teepe RC, Molenaar I, Verhoeven L. Technology-enhanced storytelling stimulating parent–child interaction and preschool children’s vocabulary knowledge. J Comput Assist Learn. 2017;33(2):123–136 53. Vatalaro A, Mcdonald A, Debbie C, Vaughn LH, Barnes AC. A quasiexperiment examining expressive and receptive vocabulary knowledge of preschool head start children using mobile media apps. Early Child Educ J. 2018;46(4):451–466 54. Dunn L, Dunn D. Peabody Picture Vocabulary Test. 4th Ed. Minneapolis, MN: NCS Pearson; 2007 13 55. Williams K. EVT-2. Minneapolis, MN: Pearson Assessments; 2007 56. Walter‐Laager C, Brandenberg K, Tinguely L, Schwarz J, Pfiffner MR, Moschner B. Media‐assisted language learning for young children: effects of a word‐learning app on the vocabulary acquisition of two‐year‐olds. Br J Educ Technol. 2017;48(4):1062–1072 57. Grimm H, Aktaş M, Frevert S. SETK-2. Göttingen, Germany: Hogrefe; 2000 58. Kwok K, Ghrear S, Li V, Haddock T, Coleman P, Birch SAJ. Children can learn new facts equally well from interactive media versus face to face instruction. Front Psychol. 2016;7:1603 59. Putnam MM, Richmond EM, Brunick KL, Wright CA, Calvert SL. Influence of a character-based app on children’s learning of nutritional information: should apps be served with a side of media characters? Games Health J. 2018;7(2):121–126 children’s executive functioning. J Exp Child Psychol. 2018;170:72–85 66. Tarasuik J, Demaria A, Kaufman J. Transfer of problem solving skills from touchscreen to 3D model by 3- to 6year-olds. Front Psychol. 2017;8:1586 75. Rimland B, Edelson S. Autism Treatment Evaluation Checklist: Statistical Analyses. San Diego, CA: Autism Research Institute; 2000 67. Esposito M, Sloan J, Tancredi A, et al. Using tablet applications for children with autism to increase their cognitive and social skills. J Spec Educ Technol. 2017;32(4):199–209 76. Sparrow S, Cicchetti D, Balla D. Vineland Adaptive Behavior Scales, 2nd Edition Manual. Minneapolis, MN: NCS Pearson; 2005 68. Fenson L, Dale P, Reznick J, et al. Macarthur Communicative Development Inventory: User’s Guide and Technical Manual. San Diego, CA: Singular Publishing Company; 1993 69. Wetherby A, Prizant B. Communication and Symbolic Behavior Scales: Developmental Profile. Baltimore, MD: Paul H. Brooks Pub.; 2002 60. Bebell D, Pedulla J. A quantitative investigation into the impacts of 1:1 iPads on early learners’ ELA and math achievement. J Inf Technol Educ Innov Pract. 2015;14:191–215 70. Grzadzinski R, Carr T, Colombi C, et al. Measuring changes in social communication behaviors: preliminary development of the Brief Observation of Social Communication Change (BOSCC). J Autism Dev Disord. 2016;46(7): 2464–2479 61. NWEA. Children’s Progress Academic Assessment. 2009. Available at: https:// www.nwea.org/cpaa/. Accessed November 12, 2019 71. Lord C, Rutter M, DiLavore P, Risi S. Autism Diagnostic Observation Schedule, Second Edition (ADOS-2). Los Angeles, CA: WPS; 2012 62. Schroeder EL, Kirkorian HL. When seeing is better than doing: preschoolers’ transfer of STEM skills using touchscreen games. Front Psychol. 2016;7:1377 72. Whitehouse AJO, Granich J, Alvares G, et al. A randomised controlled trial of an iPad-based application to complement early behavioural intervention in Autism Spectrum Disorder. J Child Psychol Psychiatry. 2017;58(9):1042–1052 63. Choi K, Kirkorian HL. Touch or watch to learn? Toddlers’ object retrieval using contingent and noncontingent video. Psychol Sci. 2016;27(5):726–736 64. Huber B, Tarasuik J, Antoniou MN, Garrett C, Bowe SJ, Kaufman J. Young children’s transfer of learning from a touchscreen device. Comput Human Behav. 2016;56:56–64 65. Huber B, Yeates M, Meyer D, Fleckhammer L, Kaufman J. The effects of screen media content on young 14 validity. Res Dev Disabil. 2008;29(5): 398–407 73. Bodfish J, Symons F, Lewis M. The Repetitive Behaviour Scale: Test Manual. Morganton, NC: Western California Center Research Reports; 1998 74. Peters-Scheffer N, Didden R, Green V, et al. The behavior flexibility rating scale-revised (BFRS-R): Factor analysis, internal consistency, inter-rater and intra-rater reliability, and convergent 77. Neumann MM. Using tablets and apps to enhance emergent literacy skills in young children. Early Child Res Q. 2018; 42:239–246 78. Blackwell CK. Technology Use in Early Childhood Education: Investigating Teacher Access and Attitudes Toward Technology and the Effect of IPads on Student Achievement [dissertation]. Vol 76. Evanston, IL: Northwestern University; 2016 79. Alzrayer N, Banda DR, Koul RK. Use of iPad/iPods with individuals with autism and other developmental disabilities: a meta-analysis of communication interventions. Rev J Autism Dev Disord. 2014;1(3):179–191 80. Zack E, Gerhardstein P, Meltzoff AN, Barr R. 15-month-olds’ transfer of learning between touch screen and real-world displays: language cues and cognitive loads. Scand J Psychol. 2013;54(1): 20–25 81. 42matters. Google Play Store stats. Available at: https://42matters.com/ stats. Accessed July 12, 2019 82. Strouse GA, O’Doherty K, Troseth GL. Effective coviewing: preschoolers’ learning from video after a dialogic questioning intervention. Dev Psychol. 2013;49(12):2368–2382 83. Griffith SF, Arnold DH. Home learning in the new mobile age: parent–child interactions during joint play with educational apps in the US. J Child Media. 2019;13(1):1–19 84. Rosenthal R. The “file drawer problem” and tolerance for null results. Psychol Bull. 1979;86(3):638–641 GRIFFITH et al