1 Introduction

Academic dropout is a major challenge in current educational institutions [1]. College dropout statistics remain a concern in the United States, and many other countries with quality resources devoted to education [2]. Recent data show that up to 40% of undergraduate students leave universities and colleges every year [3]. While high school dropout rates are decreasing, for instance in the United States, the phenomenon is becoming more prominent in higher education [4, 5]. Therefore, predicting and preventing academic dropout has become one of the main concerns in educational institutions [6], and naturally has caught the attention of many researchers. One of the main aspects in predicting students’ dropout is the need to identify those at risk of dropping out as early as possible [7, 8].

Academic dropout is an extremely complex, multidimensional phenomenon that can be related to personal characteristics (abilities, competences and motivations), family characteristics (first-generation students, sociocultural and socioeconomic situation), academic background (achievement and vocational choices), and contextual characteristics (curriculum organization, size and quality of institutions) [9,10,11]. Addressing this problem requires high-quality data composed of different aspects that influence dropout as much as possible.

The use of Artificial Intelligence (AI) approaches is a promising line for analysing that data [12], producing models that can predict which students are more likely to drop out, and identifying effective interventions for those students. However, many teachers have barriers in applying those AI techniques in education. In this context, regarding AI literacy for teachers, UNESCO highlights the need of a re-examination of teachers’ roles and the competencies they need in the AI era [13]. They state that these competencies should be categorized into three progression levels of a reference competency framework (AI CFT): Acquire (rudimentary exploration of AI tools), Deepen (confident use of AI tools), and Create (customization and modification of AI tools). In the third level we find teachers who are experts in AI applied to education and they obviously can easily apply AI techniques for educational problems. However, non-expert teachers belonging to the first two levels find it challenging to access AI, because they do not know how to apply the algorithms or how to interpret the results, particularly counterfactuals - hypothetical scenarios that simulate alternative actions or events; in our case, attributes that can be changed in order to reverse the drop-out risk and put students back on track for academic success. Making this process accessible for users who are not experts in AI, which this paper deals with, is currently a challenging, open research problem [14], which aims for users with little or no knowledge about AI to be able to apply related methods - in this particular case, machine learning algorithms. This would allow effective, user-friendly access to AI for anyone, not only AI experts [15] (note that the term “accessible” generally refers to the possibility that everyone can use a certain resource regardless of their technical, cognitive or physical condition; in the context of this manuscript, we are only referring to and addressing the first of those dimensions -technical accessibility-, while cognitive and physical dimensions, both of major importance, are out of the scope of this paper because they require a particular further analysis).

To address this gap, we have identified interesting lines that can be followed. First, Auto Machine Learning (AutoML) has allowed automation of the application of machine learning techniques in different domains [16], easing users’ work in difficult IA tasks such as algorithm selection or tuning, improving on expert human performance in certain machine learning tasks often in a shorter amount of time [17], as well as understanding and interpreting results [18]. It is based on the idea that the user only has to provide the system with input data and then the results will be displayed back to them. Moreover, in domains like healthcare it has been shown that that AutoML techniques can match or improve upon expert human performance in certain machine learning tasks, often in a shorter amount of time [17], an essential variable to take into account in the academic context and practice.

Second, there is a growing interest in models’ explainability. ML models can be classified as white-box models or black-box models [19]. White-box models, such as decision trees, provide details on how the prediction is performed and can be understood by humans. However, black-box models, such as deep learning models, are hard or impossible for humans to understand [20]. Although black-box models are usually more accurate, many educators do not trust them because they do not understand how the model arrives at its prediction [21]. In this sense, the proposed approach aims to incorporate Explainable Artificial Intelligence (XAI) to help to take advantage of those models. XAI techniques seek to allow users to understand the decisions made by machine learning models. This helps users feel more confident about the model’s recommendations in decision-making processes [22]. XAI has caught the attention of the scientific community and a lot of research in this area is currently underway, particularly in the Education domain [23].

XAI in education shares much of the same impact as broader uses of IA, however, the educational field in general, and the phenomenon of dropping out in particular, has distinctive needs for XAI [24]. Making ML techniques accessible to teachers with no previous experience in DM is one of them - and the challenge of this paper. Nevertheless, our aim is that the role of XAI in this proposal is seen not just as a product, but as a process that requires social interactions and a knowledge transfer process from an explainer to an educational agent, as seen in Miller’s study [25]. In that respect, by including XAI, we aim to have an impact on educational stakeholders not only in terms of dropout prevention but also intervention. Firstly, by making the results of the ML prediction accessible to teachers, and secondly, by encouraging teachers to feel more able to trust models, particularly black-box models. XAI techniques can be used to interpret the predicting models and to help make predictions about certain individuals - such as those who are likely to drop out of a course. Thirdly, explaining the predictions and interpreting the results also help make the model more transparent and trustworthy to decision-makers, an essential issue in deploying any intervention in real education environments. However, there is one more open front in this work to effectively intervene with those individuals, and so the fourth impact is opening up significant opportunities for personalized intervention through the understandable individual predictions.

Third, apart from basic XAI approaches, more powerful XAI techniques have arisen that can provide personalised recommendations to guide interventions for at-risk students. One promising line in this regard is using counterfactual explanations. The idea behind counterfactuals is that by slightly changing the input feature values for an individual, the system’s prediction for those individuals changes too [26]. For instance, if we identify a student who is at risk of dropping out, we can determine which of their input values we could alter slightly in order to be able to change the predicted class value (e.g. drop-out or non-drop-out). This action provides teachers with indications about the action needed to successfully intervene in that case.

Based on that background and given the research gap identified previously, the main contribution of the current work is the proposal of a web-based visual interactive tool for eXplainable Automated Prediction of Dropout. It allows non-expert users to easily apply the entire data mining process in a single step, producing and displaying the prediction model and all the explanations. It also includes interactive, tailored dashboards that can be used easily by users who are not data mining experts, such as teachers or other stakeholders. Our system provides two different interfaces aligned with the first two levels (“Acquire” and “Deepen”) of UNESCO AI CFT framework, depending on the user: (a) users with no or low knowledge of data mining, and (b) users with certain knowledge, but not experts. In this regard, the interface is intended to be able to adapt to the type of user and therefore be more useful and easier-to-use. The dashboard can discover the best predictive model for the dataset provided and predict each students’ dropout likelihood, as well as make inferences about the changes in some attributes that would need to be implemented for those students in the form of interventions to reverse the predicted dropout condition. Our approach does not attempt to analyse the role of those attributes but rather strives to bring a powerful methodology closer to educational actors to promote early prediction and initial intervention. To the best of our knowledge, at the time of writing this paper, our approach represents the first dashboard (intended for non-expert users) for predicting students’ academic dropout that incorporates AutoML and advanced XAI techniques.

Note that our approach is not conceived as a tool to reduce teachers’ involvement, but to provide them with knowledge that can be useful for early intervention. Our tool does not decide or take measures by itself, instead it automates the knowledge discovery process, and it is the teacher who finally makes a decision and intervenes with each particular student, given the dropout prediction and the suggested changes in attribute values, maybe in combination with other sources of knowledge and the teachers’ own criteria.

In this scenario, the final purpose of this research is to demonstrate that the proposed dashboard can be effectively utilized by users who are not experts in AI to identify and intervene on students at risk of dropping out [27,28,29,30,31]. To make it feasible, we formulated three Research Questions (RQ) that tried to shed some light onto the previously identified research gap:

  • RQ1. Can teachers without expertise in AI accurately identify students at risk of dropping out with the proposed dashboard, regardless of their knowledge of ML?

  • RQ2. Can those teachers use the dashboard to find ways to recover those at-risk students in a timely manner?

  • RQ3. Is the proposed dashboard usable according to the users’ experience?

Note that effectiveness (RQ1 and RQ2) and usability (RQ3) are considered in the literature as challenges and aspects that need to be assessed when it comes to the proposal of systems including AutoML techniques [32], which justifies the inclusion of those RQs to demonstrate the validity of our dashboard. Also note that those RQ are related with the identified gap, which is about the scarcity of usable tools (RQ3) that can be used by non-expert users to manage to identify (RQ1) and intervene (RQ2) on at-risk students. Finally, consider that our work presents a usable tailored dashboard (RQ3) adapted for non-expert users including, as described later, elements based on XAI to help those users intervene (RQ2) on students identified as at-risk of dropping out (RQ1) by means of AutoML techniques. This way, the research gap is perfectly aligned with the proposed dashboard and the results of our experiments, obtained by answering the three RQ of our study.

The paper is organized as follows: In Sect. 2, we present the background, describing the main approaches related to our research; Sect. 3 describes the generic approach proposed in this paper; Sect. 4 provides a practical example of using the approach in a real scenario; Sect. 5 contains an explanation of the materials and methods used; Sect. 6 describes the results obtained after conducting the validation of our approach; finally, in Sect. 7 we present the conclusions of our research and future lines.

2 Related work

As noted, educators have to face various pedagogical issues every day, and this includes the current high rates of academic dropout in higher education. There are several computer-based approaches that teachers can use to mitigate the negative effects of this problem. In particular, EDM (Educational Data Mining) and LA (Learning Analytics) techniques are promising solutions, since they can predict students at risk of dropout [33]. However, many teachers are not familiar with the concepts of EDM/LA, such as algorithms, parametrization, or evaluation metrics. Using dashboards that make those details transparent for non-expert users is a key factor for making IA more accessible in this field. These dashboards should be able to automate the process of applying algorithms and provide users with information so that they can understand the models produced and the prediction for each individual.

This section includes an analysis of the literature that is relevant to the aspects above in the Education domain. In particular, it covers the approaches that help teachers automate the application of machine learning algorithms, known as Auto Machine Learning - AutoML; the understanding of machine learning algorithms’ predictions, which is called Explainable Artificial Intelligence - XAI; and lastly, the use of LA dashboards to address that issue.

2.1 AutoML in education

Most of the literature applying AutoML in Education, which we will be addressing in this section, note how demanding and difficult the task of selecting and setting a proper model for each dataset is. AutoML is intended to address precisely that challenge.

For instance, the work presented by Tsiakmaki et al. [34] proposes a fuzzy-based active learning method for the prediction of academic performance that uses AutoML to automate the choice of ML methods and their hyperparameters. According to the authors, the use of AutoML methods opens up new horizons for extracting knowledge in educational environments. The same authors had previously presented a work where they used AutoML for the task of predicting students’ learning outcomes based on their participation in online courses [18]. In that work, AutoML was used to select algorithms (limited to tree-based and rule-based models), tune hyperparameters, select the features, and preprocess data by using the Auto-Weka tool.

Zeineddine et al. [35] applied AutoML for predicting academic performance based on data available in advance to the start of the academic program. In particular, they used AutoML to determine the best selection model and its hyperparameters, relying on a tool called Auto-Weka that was run with the hyperparameter optimization option. Their results showed that ensemble models produced the best prediction accuracy. This study does not report any dashboard that can be effectively used by teachers.

In the study conducted by Novillo Rangone et al. [36], the authors highlighted the need to have data scientists who take part in the stages of a conventional EDM project that applies machine learning. To tackle this issue, they proposed AutoML to automate the stages of data pre-processing, choice of machine learning models, and models’ hyperparameters setting. To do so, they relied on Auto Sklearn, an open source, Python-developed library with great capabilities for Bayesian optimization, meta-learning and the construction of sets, as well as good performance indicators. In this work, the authors also highlighted the importance of interpreting the results obtained by the models, so it combined AutoML and XAI. However, this was a conceptual work and these ideas were only presented but not developed as a real tool or framework. The same authors, in [37] present a new data mining educational model with a minimum presence of data scientists. The model used AutoML to obtain an optimal algorithm for classification or regression tasks, showing that gradient boosting was the most accurate approach in their experiments. A user-friendly prototype was developed for non-expert users but very few details were presented on how AutoML ideas were applied in the data modeling stage, including choice of algorithm and hyperparameter optimization substages, among others.

The work by Mnyawami et al. [38] presented a comparative study of an AutoML approach against a conventional ensemble learning method for the case study of predicting student dropout in Sub-Saharan African countries. In this paper, AutoML was used for model selection and hyperparameter optimization. The results show that AutoML approach lightly outperformed the traditional ensemble method when applied to their particular dataset.

Apart from model selection and hyperparameters setting, other different aspects have also been automated by using AutoML. For instance, in the study conducted by Bosch [39] used AutoML for feature-engineering purposes, which is a time-consuming and difficult task. Two AutoML feature engineering approaches were used, called Featuretools and TSFRESH (Time Series FeatuRe Extraction on basis of Scalable Hypothesis). Featuretools extracts features from tabular data, mainly aggregate features such as means, modes, and others. TSFRESH is a method for extracting features from numerical sequence data (time series), including features such as mean, standard deviation, coefficient of polynomial functions, slopes, and frequencies (Fourier transform), among others. The results showed that TSFRESH features were more effective than those generated by Featuretools. However, given the heterogeneity of the educational data used and the reduced number of student samples, the authors also developed a novel feature extracting that outperformed the two mentioned existing approaches.

Finally, in the study conducted by Cañete-Sifuentes et al. [40] compared AutoML against an off-the-shelf pattern-based classifier in a class imbalance problem such as predicting university dropout. In particular, they showed that the off-the-shelf method PBC4cip was as good as Microsoft Azure AutoML when dealing with class imbalance problems, with the advantage that off-the-shelf methods are less time-consuming and do not have the potential environment-ethical issues of AutoML approaches.

Most of the above-mentioned papers highlight the power of AutoML but also present interpretability as a major challenge that needs to be faced in systems implementing AutoML, which is precisely one of the main contributions of our work.

2.2 XAI in education

In educational environments, XAI is used for various purposes, such as being accountable to students, parents, and authorities, providing personalized feedback, identifying classroom needs, and collaborating with parents to improve the learning process [23]. Feedback, both at the individual and institutional levels, can contribute to educational success [41]. Additionally, clear explanations of learning activities significantly contribute to student satisfaction and academic outcomes, while XAI systems assist teachers and administrators in designing effective educational courses and content.

Khosravi et al. [23] argued that although XAI in education shares similarities with its application in other fields, it also has specific needs. In their research, they analyzed key aspects that must be considered when implementing XAI in education, such as stakeholders, benefits, explanation presentation methods, types of AI models used, and user-centered interface design. This underscores the importance of understanding the specific educational context to be analyzed, including the educational level and teaching modality. It is essential to consider these aspects to ensure effective application of XAI solutions, adequately tailoring them to the needs and characteristics of the corresponding educational environment.

XAI is particularly important when dealing with academic dropout, where teachers need to understand why certain students are identified as at-risk and what factors contribute to that determination [42]. However, it is equally important to know what a student who has been identified as at-risk can do to avoid dropping out [43]. In this regard, the use of counterfactuals seems interesting. Khosravi et al. [23] stated that artificial intelligence algorithms with a counterfactual approach in education are used to improve decision-making and the design of personalized educational interventions. These algorithms allow educators to simulate hypothetical scenarios and assess the impact of different pedagogical strategies on students’ academic performance. As a result, these models can help identify the most effective interventions for addressing students’ individual needs and improving educational outcomes. Similarly, the study by Forney and Mueller [44] highlighted that counterfactual expressions are valuable for reasoning agents as they allow them to go beyond observed data, establish precedents of necessary and sufficient causes, and quantify the agent’s regret. This is useful for understanding how actions affect the environment and decision-making in complex situations regarding educational issues.

In the context of academic performance, counterfactuals can be used to determine what actions could be taken to improve a student’s performance [45]. A counterfactual explanation presents a hypothetical scenario in the form of “what-if,” which depicts a cause-and-effect relationship. Counterfactuals enable us to take steps towards achieving a specific outcome by modifying the features of the model to produce the desired response. Counterfactual explanations are widely sought-after because they are easy to understand and user-friendly [26]. Afrin et al. [46], proposed using a counterfactual explanation approach that contributes to analyzing model outcomes in relation to students’ academic performance. Their findings suggest that certain adjustments in the use of social networks could improve student performance. Additionally, they conducted a study among educators to gather their opinions on using counterfactual explanations for student success predictions through artificial intelligence. Participating educators considered this analysis a powerful tool to improve student success prediction and guide future educational actions and support.

Focusing on academic dropout, XAI approaches, such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations), are having a notable impact in this area [47]. For instance, SHAP has recently been used for feature importance analysis in dropout prediction in programming courses [48]. In the study by Hur et al. [49], explanations produced by SHAP were applied in interventions for students at risk of dropping out based on analysis of the features with greater negative impact on student performance. In other work, Alwarthan et al. [12] used methods such as LIME, SHAP, and global surrogate models to clarify proposed prediction models, which were originally considered black box models. These methods were used to provide a comprehensive explanation of model outputs. Additionally, Melo et al. [22] discussed several XAI methods such as LIME, SHAP, and global surrogate models that can be applied to develop interpretable models for predicting school dropout. All of these techniques enable educators and policymakers to understand and interpret the factors that contribute to student dropout and effective intervention to support at-risk students. However, Swamy et al. [42] showed that for XAI models to be effective, teachers need to trust the models’ predictions and recommendations, and this is the only way to ensure their effectiveness and adoption in education.

In addition to LIME and SHAP, which are two of the most popular XAI methods, counterfactual explanations are very useful for producing interventions and recommendations as well as predicting dropout [50]. Tsiakmaki and Ragos assessed the effectiveness of explainable machine learning methods, using counterfactual explanations as an example. The results highlighted the usefulness of these methods in identifying key factors influencing student performance, such as participation in the course forum and access to specific resources. They concluded that counterfactuals provide a clear understanding of cause and effect, allowing concrete actions to improve learning outcomes. The efficacy of local model-agnostic LIME and SHAP methods for producing counterfactual explanations was assessed. The methods were applied to two randomly-chosen at-risk students, and SHAP was found to be more stable than LIME. Nevertheless, to be effective, the counterfactual explanations need to be based on actionable and causal features [51]. Garcia-Zanabria et al. [26], proposed a system called SDA-Vis, which comprises a set of linked views, allows users to identify variable alteration (counterfactuals) to change predefined students’ situations of dropout. This involves perturbing the variables of a dropout student to achieve synthetic non-dropout students. They are concerned about diversity (width of the range of the generated counterfactuals) and closeness (similarity between the counterfactuals and the original instance), and therefore rely on the DiCE (Diverse Counterfactual Explanation) approach to build counterfactuals. DiCE balances the objective function incorporating terms that define Proximity and Diversity via Determinants Point Processes (DPP-Diversity).

The above works highlight the relevance and utility of XAI and counterfactuals in the field of Education. However, most of them are based on visual explanations that do not provide the tutor with real tools for intervention. There are hardly works like the one presented in our paper, where it is possible to take advantage of counterfactual scenarios to potentially reverse the condition of at-risk students.

2.3 Learning analytics dashboards

In the educational sphere, vast amounts of information are generated, facilitating trend analysis and identification for both teachers and students [52]. In recent years, dashboards used to analyse that information have become increasingly common in the education sector [53], used by school inspectors, teachers, school leaders, and other educational stakeholders, including students and parents. These dashboards track various data points such as student-produced artifacts, social interactions, resource usage, time spent, and test and self-assessment results. In other words, they provide educational stakeholders (students, teachers, institutions) with information that can be easily interpreted for decision-making [54]. The primary purpose of these dashboards is to assist teachers in detecting and understanding behavioral patterns in their classes, enabling them to tailor learning tasks based on student preferences [55]. They are a particularly powerful tool for stakeholders who are not familiar with statistics or data analysis techniques because they present information in an intuitive and concise manner.

Learning Analytics Dashboards (LADs) are technological tools designed to support the educational process by integrating four essential components: measurement, data collection, analysis, and reporting related to students and their educational environments [56]. These dashboards play a crucial role in monitoring various outcomes such as academic performance, student progress, learning difficulties, early identification of at-risk students, identification of areas for improvement in learning materials [57], motivation, and other emotional and behavioral aspects, enabling students and educators to gain a deeper understanding of learning processes and predict significant outcomes [58].

There are many approaches to designing LADs in education, depending on the type of educational analysis that needs to be conducted [59]. One main approach uses different visualisation techniques that contribute to the main purpose of the dashboard. For instance, different colour codes may be used to identify students depending on the severity of their risk at a certain time, and other representations such as bar plots, time series, and trend plots have been used.

One of the most important applications of LADs is the identification of situations that affect students’ performance. The literature has a number of dashboards used in education for dealing with the problem of identifying students who are at risk [60]. Chounta et al. [61] used traffic light-inspired representations for identifying each student’s situation (green, low risk; orange, medium risk; and red, high risk). Bañeres et al. [62] reported a dashboard providing two views, a simple one for students and another more complete one for teachers. The dashboard in Chen et al. [63], called DropoutSeer, can be used to identify the most important features indicating at-risk students in MOOCs (Massive Online Open Courses). Ferreira et al. [64] proposed a dashboard, FICAvis, showing indicators with a notable influence on dropout, and monthly tracking of students.

An important dashboard to highlight, due to its relationship with the one presented in our research, is the one presented by Garcia-Zanabria et al. [26], called SDA-Vis, which included the use of counterfactuals for early intervention with at-risk students, based on the generation of synthetic non-dropout options. In this section, we will only describe the visual functionalities of the dashboard described in [26], since the counterfactual part of that work has already been explained in Sect. 2.2. SDA-Vis is composed of a series of main visual resources including: histograms to represent the distribution of students’ characteristics; a projection to represent the probabilities for students based on the model threshold; a projection to represent the counterfactuals and their probabilities; counterfactual exploration to represent the synthetic values; a table showing all real dropout students’ values; and finally, a visualization of the impact of some counterfactuals on a certain group of students.

Recently, trends in learning analytics dashboard research have been observed, including the increasing use of additional student data under multimodal learning analytics. Examples like EMODA illustrate how students’ emotions are monitored during online learning sessions using various types of data such as audio, video, self-reports, and interaction traces [65]. Within this framework, Franzoni et al. [66] stated that information presentation on dashboards includes pie charts, bar charts, histograms, timelines, and radar diagrams, aiming to display key performance indicators to the instructor. The inclusion of gamification elements, such as leaderboards and badges, has also been suggested as a way to enhance digital learning environments [67].

After analyzing all the background literature related to our research, we can conclude that we found no educational dashboards that incorporate AutoML with XAI and Counterfactuals, which constitutes the core of our proposal.

3 Proposed approach

This section describes the generic approach for making AI more accessible for the prediction of students’ academic dropout, using a web-based interactive dashboard with AutoML and XAI functionality, as shown in Fig. 1. The main goal of the dashboard is that teachers who are not experts in programming, Data Mining and/or Machine Learning, can create their own dropping-out prediction model. Furthermore, using XAI, users will be also able to easily understand the results, interpret them and tackle the students with high probability of dropout.

Thanks to the approach shown in Fig. 1, teachers would only need to interact with the system to upload the data because once uploaded, the model is automatically created and trained, and they would be able to manage the results. The main functionality that the user will find in the tool is as follows:

  • Prediction of drop-out as a group: the tool clearly shows the teacher a percentage of students at risk of drop-out;

  • Visualize the importance of students´ group features for dropout prediction: Teachers will be able to see a bar chart showing how each of the features affects the prediction model. This chart will allow users to determine the main causes of dropout among students as a group;

  • Visualize individual student predictions: teachers can select a specific student and check individual dropout predictions;

  • Manage student features to check how they affect the prediction: teachers will be able to check the contribution of each particular feature to the prediction for the selected student and modify features to ascertain how they impact dropping-out percentages and model predictions. This opens up the possibility of contact and/or deciding to intervene individually with students at risk of drop-out;

  • Generate counterfactual scenarios: Automatically managing the student’s features included in the model, the users will be able to generate between 1 and 10 counterfactual scenarios for avoiding drop-out from the students ID.

Fig. 1
Fig. 1
Full size image

Proposed approach for our Interactive Dashboard

Figure 1 shows the main elements involved in our dashboard that can be exploited by the users by following the steps below:

  1. 1.

    In order to generate the Interactive dashboard, a user or teacher only has to select the dataset of interest (Fig. 1a). The only restriction is that the dataset should be in a CSV (Comma Separated Values) format.

  2. 2.

    After dashboard generation, AutoML (Fig. 1b) is executed automatically to select the best prediction model (Fig 1c). We use mljar-supervised [68], which is an automated machine learning Python package designed to work with tabular data. It saves data scientists time by abstracting the common ways to preprocess data, construct machine learning models, and perform hyper-parameter tuning to find the best model. This package selects the best attrib-utes from the dataset so the user does not have to worry about this task, and also automates the process of best model selection and parameter tuning. In this regard, it implements several algorithms such as Baseline, Linear, Ran-dom Forest, Extra Trees, LightGBM (Light Gradient-Boosting Machine), Xgboost, CatBoost, Neural Networks, Nearest Neighbours and a weighted en-semble from all previous algorithms. By default, the most common split ratio is used, 80:20—80% of the dataset goes into the training set and 20% of the dataset goes into the testing set. The package also generates markdown re-ports from AutoML training prediction models with details and charts about the Machine Learning process. Depending on the dataset size, this step may take a long time to complete.

  3. 3.

    The best prediction model that is automatically selected is used to apply XAI methods (Fig 1.d). We used Explainerdashboard [69], which is a library for quickly building interactive dashboards to analyse and explain the predic-tions and workings of (scikit-learn compatible) machine learning models. It provides various graphs, interactive information panels, and other visual tools to visualise explanations of machine learning results. The library's func-tionality includes interpretive tools such as determining the importance of features, explaining a local forecast, What-If analysis, dependence of fea-tures, and others. It allows us to investigate SHAP values, permutation impor-tances, interaction effects, partial dependence plots, all kinds of performance plots, etc. Note that this stage can also take a long time since all graphs need to be generated.

  4. 4.

    Then, we also produce counterfactual explanations for each student as part of the XAI procedure (Fig 1.d). This is done using DiCE -Diverse Counterfac-tual Explanations- [70] to implement counterfactual explanations, which provides information by showing feature-perturbed versions of the same case. In other words, it discovers automatic "what-if" explanations for model output that can be a useful complement to other explanation methods.

  5. 5.

    Finally, we generate two different Dashboards, a basic interface and an ad-vanced interface. The users can select the one they want to run (Fig 1.e). The basic Interface (Fig 1.f) provides explanations and 2 different tabs: “Predic-tions” and “What If…”. It is aimed at beginner users or users with no DM knowledge at all. The advanced interface (Fig 1.g) is designed for teachers with some intermediate DM knowledge, does not have explanations and has 5 different tabs: “AutoML Report”, “Feature Impacts”, “Classification Stats”, “What If…” and “Counterfactual Scenarios”.

Regarding step 5, it is important to justify the need for two different views and the different elements included in each of them. As we introduced at the beginning of the paper, our proposal is aligned with UNESCO AI CFT framework [13]. Expert users belonging to “Create” level do not need a tool like the one proposed in this paper for obvious reasons. However, the first two levels’ users, “Acquire” and “Deepen” can undoubtedly take advantage of it. Regarding beginner users of the basic interface (aligned with “Acquire” level), UNESCO talks about exploratory use of AI tools. Based on that idea and the expertise of our teamwork (composed of teachers in all the three AI CFT levels), we decided to include in the basic interface functionalities for exploring the predictions made and basic what-if scenarios. On the other hand, for “Deepen” level users, UNESCO presents a more confident use of AI tools and that is why we decided to provide the rest of functionalities for this group of users in the more advanced interface of our dashboard. As we will see later, these decisions seem appropriate in the light of the positive results obtained when testing our dashboard with real users (teachers).

4 Practical example

In this section we describe the dataset used for the experiment and the implementation of our generic approach to that dataset, showing a practical example of the dashboard for predicting student dropout and intervention.

4.1 Data

We used a freely available public dataset about student dropout and academic success [71]. The dataset contains information about 4424 undergraduate students at a higher education European institution, including students enrolled in 17 programs in 8 different knowledge areas: agronomy, design, education, nursing, journalism, management, social service, and technology. Data were collected over a period of 10 years, from academic year 2008/9 (after the application of the Bologna Process to higher education in Europe) to academic year 2018/19. The dataset is provided in CSV (Comma Separated Values) format and coded in UTF8 (8-bit Unicode Transformation Format). The dataset contains non-academic and academic student data, with a total of 34 features. The 17 non-academic data features are divided into demographic, socioeconomic, and macroeconomic data. The 17 academic data features can be grouped into three categories: at enrolment, at the end of the first semester, and at the end of the third semester. There is also information on three levels of achievement (dropout, enrolled, graduated) at the end of the standard course duration, to be used for forecasting students’ dropout. For the present study, we merged the levels of achievement into two categories (dropout and no-dropout) by merging enrolled and graduate as no-dropout students. As a result, the number of students in the dropout class was 1421 (32.2%), while the number of students labelled as no-dropout was 3003 (67.8%). The attributes are shown in Table 1 and we refer the reader to the original source for a comprehensive explanation of each one [71].

Table 1 Information of all subjects
Full size table

We implemented the theoretical approach proposed in this paper through an interactive web-based dashboard with two interfaces for basic and intermediate users. The code is publicly available at https://github.com/0Kan0/Democratizing-the-prediction-of-student-s-dropout-The-Interactive-Visual-DashBoard-tool. The dashboard was generated starting from the best prediction model obtained by mljar-supervised [68] when applied to the student dropout dataset described in the previous Sect [71]. As we mentioned before, prior to the generation of the dashboard, the user must load a dataset, as depicted in Fig. 2. The user does not have to specify the range of attributes to be used as input values or the class attribute. By default, the first attributes are expected to be the input attributes, and the last attribute should be the class attribute to be predicted (in our case “Target” in Fig. 2).

Fig. 2
Fig. 2
Full size image

Main Web page for generating the Interactive Dashboard

This way, with no need to generate the dataset itself, teachers can use the dashboard directly with a specific existing dataset. Then, they select a basic or advanced setup (see Fig. 3) according to their prior knowledge of DM. Two study cases, for basic and advanced interfaces, with practical examples of the use of them are described in the following two subsections.

Fig. 3
Fig. 3
Full size image

Main page for accessing the two views of our Interactive Visual Dashboard

4.2 Case 1: basic interface

As Fig. 4 shows, the basic interface provides a dashboard with 2 tabs (“Predictions” and “What If”) displaying basic graphs and descriptions for each one. It is designed for teachers with basic or no prior knowledge of DM, contributing to the goal of making the prediction of student dropout an accessible task for those users.

Fig. 4
Fig. 4
Full size image

Basic Interface tabs

  • Predictions

In the “Predictions” tab, teachers can see two graphs about prediction measurements: “Confusion Matrix” and “Feature Impact”.

The “Confusion matrix” (See Fig. 5) shows the percentage (and absolute number, in brackets) of True Positives (%TP or dropout students correctly predicted as dropout), True Negatives (%TN or no-dropout students correctly predicted as no-dropout), False Positives (%FP or no-dropout students incorrectly classified as dropout) and False Negatives (%FN or dropout students incorrectly classified as no-dropout). There is explanatory text for this graph in the basic interface (Fig. 5).

Fig. 5
Fig. 5
Full size image

Confusion Matrix from the Predictions tab

”Feature Impact” is an XAI method that shows the importance of each attribute by indicating how much each feature or attribute affects the prediction (see Fig. 6). To obtain the importance or impact of each feature, the mean absolute SHAP value is also calculated.

Fig. 6
Fig. 6
Full size image

Feature Impact from Predictions tab

  • What-If

In the “What If…” tab, teachers can select a specific student and see their dropout prediction and how some changes in attribute values affect the model’s prediction. First, we select the student we want to predict via a dropdown box showing all students (we can also randomly select one student). For example, in Fig. 7 we selected the student with ID 6,725,799 A (see Fig. 7). Then, the prediction section shows the pie chart with the percentage or probability of that student dropping out or not (in blue and orange respectively). Figure 8 shows that the selected student has a 60.9% probability of dropping out and a 39.1% probability of not dropping out.

Fig. 7
Fig. 7
Full size image

Select Student from What If tab

Fig. 8
Fig. 8
Full size image

Prediction from What If tab

Then, the “Contributions Plot” (Fig. 9) shows the contribution of each particular feature on the prediction for the selected student. The contributions (starting from the “population average” on the left side of the graph) add up to the final prediction in different degrees. This allows the user to explore how each specific prediction has been built up from the different ingredients in the model.

Fig. 9
Fig. 9
Full size image

Contributions Plot from What If tab

The first yellow bar in Fig. 9 shows the average percentage of all students predicted. In this case, it indicates that the percentage is around 65% for no dropout. The next bars will indicate whether the percentage increases or decreases. The green bars mean increasing the percentage of no dropout, while red bars decrease it and the final, blue bar shows the final percentage for the student. The larger the bar, the stronger its impact on the final prediction. Figure 9 shows that “Tuition fees up to date” is the largest red bar, which intuitively means that if we change its value, that will dramatically change the student’s percentage and greatly increase the chances of them being labelled no dropout.

Finally, by using the “Feature Input” functionality (see Fig. 10), the user can directly change attribute values and see how they impact the prediction and change the prediction graph in real time.

Fig. 10
Fig. 10
Full size image

Feature Input from What If tab

For example, for student 6,725,799 A, if the user changes “Tuition fees up to date” from 0 to 1, the prediction changes and the percentage probability of dropout drops from 60.9% to 17.3%, whereas the percentage of no dropout rises from 39.1% to 82.7% (see Fig. 11).

Fig. 11
Fig. 11
Full size image

Prediction Graph after changes were made

4.3 Case 2: Full/Advanced interface

The full interface provides a dashboard with 5 different tabs (Fig. 12): “AutoML Report”, “Feature Impact”, “Classification Stats”, “What If…” and “Counterfactual Scenarios” without comments. These tabs are described below, with the exception of “Feature Impact” and “What if” tabs, which have the same functionality as described for the basic interface. The advanced interface provides additional tabs. They are suitable for teachers with some previous intermediate knowledge of DM, again contributing to a more accessible IA for the prediction of student dropout for those users.

Fig. 12
Fig. 12
Full size image

Advanced Interface tabs

  • AutoML

In the “AutoML Report” tab, the user can see the report produced by the MLJAR-Supervised AutoML library. This report contains the leaderboard (see Fig. 13) showing the names of all models used, training time, evaluation metric type, metric value and which model is best. MLJar uses 10 models and an ensemble of all the models. The ensemble uses a greedy approach to create a weighted ensemble starting from all previously trained models. The performance of all generated models is also presented as scatter and box plots.

Fig. 13
Fig. 13
Full size image

AutoML Report tab

  • Classification Stats

The “Classification Stats” tab provides additional information about common evaluation metrics and plots related to the classification. For example, “Model performance metrics” show the score of the prediction model evaluation metrics such as accuracy, precision, recall, f1-score, roc-auc-score (Area Under the ROC -Receiver Operating Characteristic- Curve), pr-auc-score and log loss.

In addition to the “Confusion matrix” (also included in the “Basic Interface”), this tab also shows the “ROC AUC Plot” (Fig. 14). The “ROC AUC curve” is a performance measurement for classification problems at various threshold settings. ROC is a probability curve and AUC represents the degree or measure of separability. It indicates how far the model is capable of distinguishing between classes. The TPR (True Positive Rate) and FPR (False Positive Rate) values indicate the rate of correct and incorrect classification of the model for each decision threshold.

Fig. 14
Fig. 14
Full size image

ROC AUC plot from the Classification Stats Tab

  • Counterfactual scenarios

Finally, the “Counterfactual Scenarios” tab lets teachers select a student and automatically generate between 1 and 10 counterfactual scenarios (Fig. 15). These scenarios will automatically indicate which are the variables to change so that the student’s situation will change (from the originally predicted class value to the opposite class value). That means that if the student has a high chance of dropout (say, over 50%), it will recommend the most suitable variables to vary so the percentage of dropout falls and the student prediction can change to the no-dropout class value.

Fig. 15
Fig. 15
Full size image

Counterfactual Scenarios tab with inputs

This way, after selecting a student and the number of scenarios we want, the dashboard displays the variables from the original scenario as well as the number of counterfactual scenarios we selected. Comparing the variables of the original with the counterfactual scenario, we could achieve that “Tuition fees up to date” changes from 0 to 1 (note that the counterfactual scenarios can change any number of variables). So, if we go back to the “What If…” tab and change the variable in “Feature Input”, we can see the percentages according to the change.

5 Materials and methods

In this section we will describe the materials and methods that we used to validate our dashboard. In particular, we present the sample, the instruments and the procedure we followed.

5.1 Sample

For our study we used a population of 49 teachers (14 women, 35 men) with different knowledge backgrounds: Computer science, Psychology, Speech Therapy and Education. They can be split into two groups based on self-reported prior knowledge of DM. There were 24 with basic or no knowledge of DM (6 Psychology, 15 Computer Science and 3 Education) and 25 with a medium or high level (2 Psychology, 19 Computer Science and 4 Education). The teachers ranged between 30 and 55 years old with an average age of 43.

5.2 Instruments

In order to answer our three research questions, we used four traditional quality metrics: success rate (whether teachers can perform the task in terms of drop-out detection or not) for the 1st RQ and performance time and again success rate (whether teachers can perform the task in terms of finding ways to recover students from drop-out or not) for the 2nd RQ. In fact, to be able to successfully complete the entire task proposed involves being able to take advantage of all the resources available at the dashboard and therefore fulfil the 1st and 2nd RQs. For the 3rd RQ, the dashboard’s usability was measured using the well-known System Usability Scale - SUS [72, 73]. When selecting a scale for measuring usability, many approaches were considered [74], such as QUIS (Questionnaire for User Interface Satisfaction) or CSUQ (Computer System Usability Questionnaire), some of them specific for web-based tools. However, we finally opted to use SUS since, despite it being a general-purpose approach, it is a traditional, popular, simple, accurate and valid approach that has been already successfully employed for assessing usability, particularly in the web [75].

We asked the participants to complete the questionnaire immediately after the task. It has ten questions designed to be answered after the user’s interaction with the system, each using a 5-point Likert scale (from strongly disagree to strongly agree). Additionally, we collected information about users’ self-reported experience of use. The users were asked two open questions to get their feedback about the interface.

5.3 Procedure

We emailed instructions about the case study task. We gave the users a general explanation of the interactive, web-based dashboard application, and the following instructions:

  1. 1.

    Please click on the following URL, where you can find the main page of the Interactive Visual Dashboard tool (Fig. 3).

  2. 2.

    Select either the basic or advanced AutoML DashBoard Dropout Explainer interface (see Fig. 3), depending on your level of knowledge about data mining (low or medium/high). Browse through all the interface tabs and look at all the charts.

  3. 3.

    Then try to do the following task and monitor the time you need to complete/do it successfully: “Look for a student predicted as dropout and modify the values of the student attributes in order to avoid dropout, i.e., changing the student’s class value from dropout to no dropout”. Before starting the task press the Start button in the “Start timer” panel included in the “What if” tab (see Fig. 16) and press Stop when you finish the task in order to measure the total time used.

  4. 4.

    Finally, fill in a questionnaire with your personal data (age, sex, academic degree, and level of knowledge in data mining), 10 questions about the usability of the tool, and two open questions about your point of view about the dashboard.

Fig. 16
Fig. 16
Full size image

Start and Stop buttons for start timer in What-if tab

6 Results of assessing dashboard use

This section shows the results of the case study structured by the three research questions. Table 2 shows the general results means and standard deviations (SD) from the two groups (basic and advanced) in the different metrics, but the different RQs individually below. This table will be referred to as we develop each of the results questions in the subsequent subsections of the manuscript.

Table 2 Descriptive statistics for the two groups of users
Full size table
Table 3 Classification evaluation metrics of all the algorithms with the dataset
Full size table

6.1 RQ1. Can teachers without expertise in AI accurately identify students at risk of dropping out with the proposed dashboard, regardless of their knowledge of ML?

To answer the first RQ we consider the task completion/success rate in terms of risk of dropping-out detection. A simple binary indicator (completed or not completed) indicates if the user is able to make two steps: (1) select the dataset, and (2) ask the dashboard to automatically execute and select the best algorithms. The general completion rate was 93.9%: particularly, 100% for the advanced interface’s users and 87.5% for the basic knowledge group; three of the 49 users (6.1%) were not able to complete the task, all from the basic knowledge group.

Also note that the task completed by the users is based on the results provided by the AutoML module used in this research. Therefore, to complete the analysis and answer this RQ, it is important to highlight that the predictions made for the AutoML module of our dashboard were accurate. However, as the performance indicators are not strictly part of teachers’ responsibility on this task (which is what this RQ is assessing) but something automatically provided by our tool, the performance analysis has been included as an appendix out of the main body of this manuscript.

6.2 RQ2. Can those teachers use the dashboard to recover at-risk students from drop out in a timely manner?

To answer the 2nd RQ we again turned to success rate and to time on task: whether teachers can perform the task in terms of finding ways of recovering students from drop-out or not, and how long they spend on that.

The task instructions asked the teachers to “…modify the values of the student attributes in order to avoid dropout, i.e., changing the student’s class value from dropout to no dropout”. As noted in the previous RQ, the overall success rate was 93.9%, 100% for the advanced interface’s users and 87.5% for the basic knowledge group where three of the 49 users from the basic knowledge group (6.1%) were not able to complete the task.

We selected an example to illustrate how to theoretically “rescue” a student from drop-out, guiding teachers’ decision-making for intervention. Given that academic dropout is an extremely complex, multidimensional phenomenon, this is a crucial tool for drop-out prevention. After selecting a student and the number of scenarios we want, the dashboard displays the variables from the original scenario as well as the number of counterfactual scenarios we selected (for example, student 52112655D (see Fig. 15)). Comparing the original variables with the counterfactual scenario, we see that “Tuition fees up to date” has changed from 0 to 1 so going back to the “What If…” tab and changing this variable in “Feature Input”, we can see how the 60.9% dropout probability drops to a mere 17.3%, while the no-dropout likelihood rises from 39.1 to 82.7% (see Figs. 8 and 11, before and after the change). This may mean that if the student had paid their tuition fees, they would not have been as likely to drop out, which provides teachers with some clues for actions that may help interventions succeed.

Given that time is precious in educational practice, it is important to determine whether teachers can complete the task in a timely manner. This was done by measuring time on task - the time the users needed to complete the task. Time is measured in seconds by a web-based timer. The mean time taken by the basic knowledge group to complete the task was M = 402.25 s (about 6 and a half minutes), SD = 334.87. The mean time in the advanced interface knowledge level group was M = 329.57 (around 5 and half minutes), SD = 192.34. Additionally, and most importantly, there were no statistically significant differences between the two groups in terms of time taken to complete the task (F(1,47) = 3.368; p = .063). In addition, the qualitative data showed that all of the users indicated that finding a dropout student was time-consuming, as most of the students were in the no dropout class.

6.3 RQ3. Is the proposed dashboard usable according to the users’ experience?

Usability was assessed via the Spanish adaptation of the SUS, a 10-question scale with Cronbach α = 0.812 [72, 73, 76, 77]. The overall score in the questionnaire can range from 0 to 100, a score of 85 indicates excellent usability, 68–84 indicates good usability, scores between 51 and 68 suggest improvements are needed, and scores below 51 indicate usability issues.

The mean score for the basic interface in the SUS questionnaire was M = 74.6, SD = 10.15, and for the advanced interface it was M = 78.7, SD = 12.10. Overall, both scores indicate good usability. Both groups seemed to be comfortable with the interface performance. We also tested whether there were statistically significant differences between the two interfaces in terms of usability, finding that there were none (F(1,47) = 1.657; p = 0.204).

Finally, the two compulsory open questions we included gave users the opportunity to provide open feedback about the interface and the freedom and space to answer in as much detail as they liked. The questions were asked immediately after using the tool and were: “(1) Please discuss any positive aspects of the interface; (2) Please discuss any improvable aspects of the interface.” This allowed users to give their personal positive and improvable comments about the dashboard and their experience using it. Some of the most indicative comments about the about both the basic and advanced interfaces are shown below, due to the richness of the information collected.

For the basic interface, the users said that the dashboard can be understood by teachers with no knowledge of Data Mining. The charts and descriptions seem to have been suitable for giving information to non-expert users. For instance: “It is easy to navigate through the tabs and graphs, easy to understand in terms of functionality, and it looks quite good.”; “With a quick glance and a couple of minutes, it is enough to be able to use the interface without any problems”; “In general, the interface can be used in an appropriate manner, within a short period of time, because the graphics are very visual”; “Each section/graph has an accurate description, so it is clear what each chart represents in every section”. However, there were also some improvement comments about the basic interface, such as: “In the case of the basic interface, the descriptions take up half of the screen, making the text small, and the size of the graphics was not properly optimised, and they are not displayed at full width”. There were frequent comments along these lines, so it seems that there is scope for action in terms of web design and usability. In this regard, users considered other possibilities such as showing information as popup text.

Users of the advanced interface also appreciated graphs and descriptions: “The system provides a lot of graphical information that is very useful for advanced interface’s users. Additionally, it provides a lot of information, for example, in the first tab of the AutoML, you can view all the charts for each prediction model”; “The What-if and Counterfactuals tabs are very useful and should be integrated together so that the counterfactual recommendation can be directly applied”; “The ‘counterfactual scenarios’ tab greatly facilitates the process of finding which changes in the student’s characteristics would make their potential dropout situation change.” Bearing this in mind, the What-if and Counterfactuals could be integrated in the near future for the basic version as well. Users also said that ROC curves were very useful: “The elements in the classification-stats tab, such as ROC, are very informative and allow a quick assessment of the performance of the model used”. They also suggested improvements, but once again, more in terms of the dashboard design than the methodology of the tool: “It would be good to highlight the values that change in each counterfactual. I believe that when generating ‘Counterfactual Scenarios’, it would be better to either highlight the changed parameters or only display those, instead of having to search for them visually.” We are currently working on these aspects of the dashboard design; on the one hand by showing only the values that change and not all values in the counterfactual, and on the other, by allowing values to be directly applied with a button that changes them automatically in What-If without having to do it manually. However, there are also some improvements suggested for the advanced tool that go beyond dashboard design: “The autoML report and classification stats tabs have some charts that are not entirely clear. It would be helpful to provide a more detailed explanation of the meaning of the charts for better understanding.”. In the future, it would be an improvement to add comments in all the charts in the advanced interface as in the basic interface.

In both the basic and advanced interfaces, users said that, once again, the graph in the “what if” tab was a strong point: “the graph is very simple yet highly illustrative regarding the modified values in feature inputs; “the differentiation between ‘dropout and no dropout’ through graph colors is highly visual” and the “Feature-Importance and Contributions-Plot greatly help in understanding which factors carry more weight for prediction of drop out”.

However, users of both interfaces said that there was room for improvement in terms of the variables used. For example, users expressed their interest in the meaning of the variables: “In the What-if tab, when changing variables, some of them may be unknown or their value meanings are unclear.”

An interesting improvement expressed by users of both interfaces was related to screening dropouts: “manually searching through all students to identify dropouts becomes cumbersome if you want to investigate everyone. It would be beneficial to have a filter that only displays those with a higher probability of dropout.”, this is an extremely easy issue to show in terms of the dashboard design. Another improvement in visualisation basics, e.g. “Perhaps if the prediction graph was at the same height on the screen as the Feature-Importance, it would be possible to immediately and visually assess the effect of the change on the graph”.

7 Discussion and conclusions

In this paper, we presented an interactive visual dashboard that allows non-expert users to apply user-friendly data mining techniques to big data sources. We designed two interfaces (basic and advanced) aligned with UNESCO AI CFT framework [13], intended for two different levels of expertise in Data Mining (low or no knowledge, and intermediate). Our approach aims to make it possible to automate the whole data mining process for predicting, explaining, and intervening in drop out but also make the data mining process accessible to any educator, with or without data mining expertise.

This means that our system is not only capable of being used for prediction, but it can also be used to accurately identify each student at risk of dropping out, and recognize the causes behind it, simulate an intervention aimed at those variables, and find recommendations/actions that may be considered to put that student back on track for success.

Putting the results in practice and positively answering the 1st RQ, users without previous knowledge in DM were able to apply ML techniques thanks to the AutoML approach, a methodology which would otherwise be completely out of the reach of non-expert users. In line with previous results from the educational field [17], we found that users with basic or no knowledge on ML were able to complete the first part of the task then accurately identify students at risk of dropping out with the proposed dashboard. So, it is good news that this methodology contributes to preventing the phenomena of academic drop-out. Beyond that, exploring the 2nd and 3rd RQ, we made available to any educational agent (non-DM expert) a tool for finding ways to recover students from dropping-out and do so in a reasonable time and user-friendly way. This means providing educational agents with very valuable information to prevent and intervene in academic drop out and apparently doing so in a way that is trustworthy and positive. As Waring et al. [17] noted, often a lack of transparency in small decisions made by the AutoML methodology, such as what model configurations have been searched, leads machine learning experts and novices alike to question the automatic results. If users do not trust the AutoML system, they will hesitate to apply the results of the dashboard in critical applications [78], especially in educational settings where interpretability and transparency of algorithms are crucial for a practice to be adopted into real contexts.

The promising results suggest that the proposed dashboard fulfils its mission and contributes to making IA more accessible for any non-expert user, teacher or stakeholder, regardless of their previous IA background. In particular, users with low or no previous knowledge of IA have been shown to be able to predict student dropout with our tool and explore what-if rules with no differences in time or usability compared to their fellow intermediate users. If this were not the case, in the words of Chirkova and Yang [79], we would be contributing to a “big data divide”, in which only few experts are able to take advantage of data mining, while regular education actors miss out on this opportunity.

The intermediate users also had a positive experience by being able to take advantage of more advanced tools such as counterfactual explanations. In this regard, one of the main goals of our approach, making IA more accessible for predicting academic dropout, could be considered as achieved.

Finally, with regard to the future, we are currently working on:

  • Performing further experiments to complete the evaluation of our dashboard. First, note that we used a public dataset about traditional face-to-face classroom conducted only in Europe and this might affect the model’s prediction because the algorithms consider the weighting of the importance of each variable. Note that this may vary depending on the scenario and, therefore, other datasets and variables representing other social and geographical realities could be analyzed. Second, we would like to evaluate our dashboard with more teachers from other knowledge areas, in particular, other domains that are less related to EDM/LA, such as medicine, biology, physics, economics, and architecture. Third, it would be interesting to benchmark our dashboard’s performance, usability, accessibility and interpretability against similar existing dropout predictions systems, like for instance the commercial ones developed by MoodleFootnote 1 or IntelliboardFootnote 2;

  • As part of our dashboard, we have used the MLJAR-Supervised AutoML library [68] to automatically select the best attributes, the best model and hypermeter tuning. However, it is a free tool that, although powerful, has certain limitations. There are other similar tools with more advanced techniques for attribute selection and parameter tuning, and a more complete set of available ML models. Those could be explored to check if it is feasible to include them in our dashboard and obtain potentially better predictive performance results than the ones presented here;

  • Adapting the dashboard methodology for prediction, understanding, and taking action to other phenomena that could benefit from this, such as learning disabilities or school failure. It would be very useful for non-expert users such as teachers and therapists, etc., to be able to exploit the vast amount of available data from different sources in order to extract information and make well-informed decisions in their professional life. In line with Espinosa et al. [80], the value of previously discovered knowledge could also be very valuable if it is available for subsequent understanding or applications;

  • Improving counterfactuals. Counterfactuals are undoubtedly powerful, useful elements in the decision-making process. However, there are several challenges faced by the systems that provide counterfactuals to output. One is the fact that they provide a list of values for all attributes for each counterfactual but most of them are the same values as the original. One solution may be indicating only the features that are being changed with a different color or displaying them alone. Another problem is that users can select a number of counterfactuals to generate and then it can be perceived as a complex set of elements without selection, order or priority. This can overwhelm a user with the amount of information at that step of the process and prevent them from making useful decisions. Therefore, in line with You et al. [81], one important future line of research is not to ask the user about a number of counterfactuals to discover but automatically show the most useful set of counterfactuals that meet the needs of the user at that point;

  • Based on the counterfactuals obtained, it would be interesting to design potential interventions based on them. The use of LLMs (Large Language Models) can be an interesting approach to provide the instructor with recommendations on which interventions should be made for each student. In this regard, some works start to appear in the literature [82] and it is a line worth exploring.