How students solve problems during Educational Robotics activities: identification and real-time measurement of problem-solving patterns

This dissertation aims to provide the results through the utilisation of data mining and machine learning techniques for the assessment with Educational Robotics (ER). This research work has three main objectives: identify different patterns in the students’ problem-solving trajectories; predict the students’ team final performance, with a particular focus on the identification of learners with difficulties in the resolution of the ER challenges; analyse the correlation of the discovered patterns of students’ problem-solving with the evaluation given by the educators. We analysed the literature on Educational Robotics’ traditional evaluation and Educational Data Mining for assessment in constructionist environments. An experimentation with 455 students in 16 primary and secondary schools from Italy was conducted, through updating Lego Mindstorms EV3 programming blocks in order to record log files containing the coding sequences designed by the students (within team work), during the resolution of two preliminary Robotics’ exercises (Exercise A and B). The collected data were analysed based on data mining methodology. We utilised five machine learning techniques (logistic regression, support vector machine, K-nearest neighbors, random forests and Multilayer perceptron neural network) to predict the students’ performance, comparing two approaches: - a supervised approach, calculating a feature matrix as input for the algorithms characterised by two parts: the team’s past problem-solving activity (thirteen parameters extracted from the log files) and the learners’ current activity (three indicators for Exercise A and four indicators for Exercise B); and - a mixed approach, applying an unsupervised technique (the k-means algorithm) to calculate the team’s past problem-solving activity, and considering the same indicators of the supervised approach representing the students’ current activity. Firstly, we wanted to verify if similar findings emerged comparing younger students and older students, so we divided the entire dataset in two subsets (students younger than 12 years old and students older than 12 years old) and applied the supervised and mixed approach in these two subgroups for the first exercise, and a clustering analysis for the second exercise. This process demonstrated that similar problem-solving strategies were applied by both younger and older students, so we aggregated the dataset and performed the supervised and the mixed approach comparing the performances of these two techniques considering the entire dataset. The results have highlighted that MLP neural network with the mixed approach outperformed the other techniques, and that three learning styles were predominantly emerged from the data mining. Furthermore, we deeply analysed the pedagogical meaning of these three different approaches and the correlation of the discovered patterns with the performance obtained by learners. We denote the added value of data mining and machine learning applied to Educational Robotics research and highlight the significance of further implications. Finally, we discuss the future further development of this work from educational and technical view.

Questa tesi presenta l’utilizzo di tecniche di data mining e machine learning per la valutazione di attività di Robotica Educativa. Gli obiettivi di questo lavoro di ricerca sono tre: identificare differenti pattern durante le attività di problem-solving degli studenti; predire il risultato finale ottenuto nella risoluzione delle sfide di programmazione (e annotato dagli educatori) utilizzando tecniche machine learning; analizzare le correlazioni tra i pattern ottenuti e la valutazione assegnata dagli educatori. Per raggiungere questi obiettivi è stata svolta una sperimentazione con 455 studenti di 16 scuole primarie e secondarie italiane: è stato aggiornato il software del kit Lego Mindstorms EV3 così da registrare le sequenze di programmazione create dagli studenti in una scheda SD all’interno del robot, durante la risoluzione di due esercizi introduttivi di Robotica. I dati raccolti sono stati analizzati con una metodologia di data mining. Sono state utilizzate cinque tecniche di machine learning (regressione logistica, support vector machines, K-nearest neighbors, classificatore random forests e rete neurale Multilayer perceptron) così da predire la performance ottenuta dagli studenti. I risultati ottenuti hanno mostrato che la rete neurale MLP ha superato le altre tecniche in termini di predizione e che 3 stili di problem-solving sono emersi all’interno del dataset considerato; questi 3 stili sono stati analizzati in dettaglio sia da un punto di vista educativo che in relazione ai risultati ottenuti dagli studenti nella risoluzione degli esercizi.