AI Literacy Level of Students: A Scale Development Study

Abstract

This study aimed to develop a valid and reliable scale to measure artificial intelligence literacy among high school students in Türkiye. By establishing a multidimensional measurement framework and providing robust psychometric evidence, the study sought to contribute an assessment tool capable of supporting research, educational practice, and evidence-based policy development in AI literacy. Based on an extensive literature review, an initial pool of 21 items was generated and evaluated by experts to ensure content validity. Following a pilot study with 211 students, the refined instrument was administered to 645 high school students. Exploratory factor analysis revealed a five-factor structure comprising 20 items, accounting for 63.3% of the total variance. The identified factors were Basic Knowledge and Awareness, Ethical and Safe Use, Application Skills, Critical Perspective, and Evaluation. Confirmatory factor analysis indicated good model fit (CFI = 0.94; RMSEA = 0.05; GFI = 0.93; RMR = 0.06; χ²/df = 2.729). Reliability analyses demonstrated high internal consistency across the overall scale and its subscales. Significant differences in AI literacy scores were observed across gender, grade level, and patterns of AI use. Despite being limited to secondary school students in Türkiye, the scale provides a robust measurement tool for assessing AI literacy and offers empirical evidence to inform curriculum design, teacher education, and national digital literacy initiatives.