Design and Evaluation of an AI-Based Conversational Agent for Adaptive Feedback in Educational Robotics

Artificial intelligence (AI) is increasingly adopted in education to deliver structured, real-time feedback, especially in complex, collaborative learning scenarios. Large Language Models (LLMs), in particular, have enabled the integration of chatbots into a wide range of educational settings. This study presents the design and implementation of an AI-empowered chatbot, based on the Anthropic Claude 3.5 Sonnet, within educational robotics activities, aiming to explore its potential as a valid support system for real-time, contextualised feedback delivery. The system was deployed in a field study involving 80 students (grades 5–7), organised into 26 collaborative groups. Three hypotheses were evaluated: whether chatbot interaction (i) correlates with students' perceived usefulness (PU), (ii) supports more efficient programming behaviour, and (iii) is related to task success. While the number of attempts did not significantly decrease, chatbot interaction showed mixed associations with success, highlighting that usage frequency alone does not predict effectiveness. Students also consistently rated the system as highly useful (PU=4.04 \pm 0.46). Additionally, the chatbot infrastructure enables continuous tracking of student activity, opening the way for advanced student modelling and the development of digital twins within educational systems. This work contributes to the field of intelligent tutoring by demonstrating that LLM-based agents, when pedagogically aligned, can improve both perceived feedback quality and learner engagement in collaborative, task-based learning environments. Future research should refine feedback strategies and evaluate system performance across more homogeneous learner groups to improve modelling accuracy and scalability.