Efficiency, Effectiveness and Fairness Narratives of Education Technology: A Synthesis of Claims and Evidence from the Asia-Pacific Region

Abstract

For decades, the field of educational technology (ed-tech) has been characterised by over-optimism and lofty promises that often fail to materialise.  This research investigates the narrative claims or hype of ed-tech across diverse education contexts in seven Asian-Pacific countries. It uses a qualitative meta-analysis of research reports developed by Master of Education Students at Monash University in 2023. These reports examined narratives and evidence of particular ed-techs in their contexts. We found five common narrative claims made for new educational technologies (Effectiveness, Efficiency, Fairness, Transformation, and Limitations) and ten narrative sub-themes that were quite similar across different nations, contexts and technologies. However, there were differences in the specific expressions of the narratives, especially within the efficiency narratives, due to different implementations, strategic approaches, and cultural contexts. Overall, the evidence fell a long way short of the narrative claims made for the various technologies. While some narratives produced a reasonable number of studies supporting them in limited contexts, many narrative sub-themes were unmatched by evidence. To support more informed dialogues, we offer a Critical Thinking framework of Education Technology, which summarises the most common hype narratives and evidence gaps from our study. We hope this might support more informed planning and investment conversations, enhance educators' tools to teach critical appraisal of education technology in university programs, and inspire further research into areas where evidence for improvements is limited.