Predicting AI-Assisted Student Academic Improvement Using K-Nearest Neighbors

Abstrak

The rapid integration of Artificial Intelligence (AI) assistants in higher education necessitates empirical methods to evaluate their actual impact on student academic performance. A significant challenge in Educational Data Mining (EDM) is interpreting these impacts accurately, especially given the inherently imbalanced nature of educational datasets. This study proposes a predictive framework utilizing the K-Nearest Neighbors (KNN) algorithm to classify student academic improvement based on AI usage patterns and Learning Management System (LMS) engagement. To ensure model robustness, the Boruta algorithm was applied for feature selection, alongside the SMOTE-Tomek technique to address class imbalance. The optimized KNN model achieved a high predictive performance with an F1-Score of 86.8% and a Balanced Accuracy of 86.1%. Furthermore, the integration of Explainable AI (XAI) via SHapley Additive exPlanations (SHAP) revealed that using AI for conceptual clarification, rather than direct task completion, is the primary driver of academic success. The findings demonstrate that this explainable KNN framework effectively identifies at-risk students, providing educators with transparent, actionable insights to deliver personalized interventions and foster responsible AI usage.