Latest / 1 More Paper / Towards Designing AI-Enabled Adaptive Learning Systems — Kabudi (2023)

Transcript

Towards Designing AI-Enabled Adaptive Learning Systems — Kabudi (2023) · 1 More Paper

• You’re listening to “Towards Designing AI-Enabled Adaptive Learning Systems,” by Tumaini Kabudi, from University of Agder. Published in 2023. Towards Designing AI-Enabled Adaptive Learning Systems Tumaini Kabudi (Half title page) Towards Designing AI-Enabled Adaptive Learning Systems Tumaini Kabudi Towards Designing AI-Enabled Adaptive Learning Systems Dissertation for the degree philosophiae doctor University of Agder Faculty of Social Sciences 2023 Doctoral dissertations at the University of Agder 407 ISSN: 1504-9272 © Tumaini Kabudi, 2023 Print: Make!Graphics Kristiansand This thesis is dedicated to. my beloved Papa, Palamagamba John Kabudi my beloved Mama, Amina Matthew Kabudi and my handsome brother, Aidan Imani Kabudi. Utukufu kwa Mungu Juu. Acknowledgements. The long and winding road towards a PhD has been a remarkable experience. It was a journey of unceasing yet interesting challenges. For this, I would like to give primary thanks and praise to God the Almighty (Elohim), My Jehovah-Jireh. He has given me strength and encouragement throughout all the challenging moments of completing this dissertation. I am truly grateful for His unconditional and endless love, mercy, and grace. To God be the Glory! Apart from my efforts, the success of this PhD depends largely on the encouragement and guidance of many other people. I take this opportunity to express my gratitude to those who have been instrumental in the successful completion of this thesis. I hardly know where to start expressing the gratitude, but I am certainly grateful to all who have assisted me in the process of completing this PhD. It would be impossible to list everyone by name, but several people deserve sincere and special thanks. First, I am extremely grateful to my supervisors, Prof. Ilias Pappas and Prof. Dag Håkon Olsen for their invaluable advice, constant support, and patience during my PhD study. Their immense knowledge and plentiful experience encouraged me throughout my academic research and daily life. I could not imagine having better advisors and mentors for my PhD study. Besides my supervisors, I would like to thank Prof. Sandeep Purao, Dr Leona Chandra Kruse, and Dr Zacharoula Papamitsiou not only for their insightful comments and encouragement but also for the hard question that inspired me to widen my research from a variety of perspectives. My gratitude extends to the Faculty of Social Sciences at the University of Agder for the funding opportunity to undertake my studies at the Department of Information Systems. The wonderful people in that department, led by Prof. Carl Erik Moe, deserve all the best. Their hospitality, commitment, and support have made me feel at home. My fellow PhD students have always been a welcoming and convivial group with which to socialize and have professional discussions. I thank my fellow PhD students in the department and faculty for the stimulating discussions and plain old fun we have had in the last three years. Enormous appreciation is extended to St. Ansgar Catholic Church in Kristiansand, especially to Father Oliver Obinna Izuogu and the entire English community in our church. I would also like to acknowledge the enormous contribution of the missionaries at Misjonshuset Kristiansand, especially the International Christian Fellowship community. Thanks to all of you for opening your doors to me and welcoming me into your communities. I appreciate the prayers, fellowship, and words of encouragement. On that note, I do not want to forget my parish church back in Tanzania and my friends and colleagues in that country who constantly prayed for me during my PhD. A special thanks to my Tanzanian community in Kristiansand. Mungu awabariki sana kwa upendo wenu kwangu. Mungu adumishe undugu wetu. I also offer special thanks to my “African Union” community in Kristiansand for the laughter, our African food, and the dancing to our African tunes. We may have been far from home, but the “Africa” in us never left. I will always cherish the special moments we shared. Mungu ibariki Tanzania; Mungu ibariki Afrika. Above all, this thesis is dedicated to my beloved family. Meine liebe parents, Prof. Palamagamba John Kabudi and Dr Amina Matthew Kabudi, have been extraordinarily supportive and made countless sacrifices throughout my life. My one and only brother, Aidan Imani Gabriel Kabudi, has also motivated me to accomplish this PhD in a timely manner. As a family, you have been there for me: during challenging times you have been a shoulder to lean on. You constantly checked in on and prayed for me. WhatsApp can testify to the hours of calls we would take to talk and laugh but mostly to let the PhD candidate in the family to express her frustrations, insecurities, and concerns. Together, each in your own unique ways, you all encouraged me, sang songs to me, and cheered me on. I could not have asked for better parents or a better brother. Without such a supportive team behind me, I know I could never be where I am today. I hope I have made you proud. It would be difficult to find adequate words to convey how much I appreciate you all. Lots of love and thanks to all of you. Abstract. Among the many innovations driven by artificial intelligence (AI) are more advanced learning systems known as AI-enabled adaptive learning systems (AI-ALS). AI-ALS are platforms that adapt to the learning strategies of students by modifying the order and difficulty level of learning tasks based on the abilities of students. These systems support adaptive learning, which is the personalization of learning for students in a learning system, such that the system can deal with individual differences in aptitude. AI-ALS are gaining traction due to their ability to deliver learning content and adapt to individual student needs. While the potential and importance of such systems have been well documented, the actual implementation of AI-ALS and other AI-based learning systems in real-world teaching and learning settings has not reached the effectiveness envisaged on the level of theory. Moreover, AI-ALS lack transferable insights and codification of knowledge on their design and development. The reason for this is that many previous studies were experimental. Thus, this dissertation aims to narrow the gap between experimental research and field practice by providing practical design statements that can be implemented in effective AI-ALSs. The research is guided by the main research question: How should AI Adaptive Learning Systems be designed and developed? The main research goal will be investigated by examining five sub-questions: i. What are the core research problems and educational practice concerns in AI-ALS and the interventions and solutions proposed to address them? ii. What are the underlying principles for designing AI-ALS in the academic literature? iii. How can researchers, educational practitioners, designers, and developers successfully integrate and implement AI technologies to promote quality education? iv. What aspects should one follow consider when designing AI-ALS? v. Which of these are considered important for the design and development of AI-ALS? To address the research question and its objectives, I examined the above questions by eliciting knowledge from both literature and practice. An interpretive research (exploratory) study using expert interviews was conducted in conjunction with a ranking-type analysis of the formulated propositions I also used Gupta and Bostrom's (2009) theoretical model for technology-mediated learning, which is based on adaptive structuration theory (AST), to understand AI-ALS as a mediating tool for adaptive learning. There is a need to obtain a comprehensive understanding of AI-ALS in terms of enhancing the whole teaching–learning process in a real educational setting. The theoretical model was chosen because it is suitable for investigating all the elements of a socio-technical system – technology and learning techniques, process, actors, actions, and outcomes – that are crucial elements in AI-ALS. This thesis’s findings are presented in five published academic articles. The first paper is a systematic review of the literature on contemporary learning environments. The paper explains the concerns and issues in 21st-century learning environments that need to be addressed. The second paper is a systematic mapping of the literature on AI-ALS, which provides a deeper understanding of AI-ALS by highlighting recent research, the research gaps that still exist, and future directions in the field. The third paper is an empirical study, focusing on areas in the field of artificial intelligence in education (AIEd) that need to be addressed to improve its implementation. The paper’s findings identify the practical benefits of adopting and implementing AIEd contexts. The fourth paper is also empirical and derives design requirements for AI-ALS from the literature and thus generate preliminary proposition (in form of preliminary design principles) for designing such systems. The fifth paper is another empirical study that derives design requirements from expert interviews and then formulates preliminary design principles that address these requirements. The findings from the interviews revealed new design requirements that had not emerged in the earlier literature review. As the body of preliminary propositions were composed of multiple sources, the list of propositions needed to be synthesized and refined to produce a coherent list. Through this iterative process and the evaluation of the preliminary propositions, nine propositions for designing AI-ALS are identified as important to the successful design and development of AI-ALS by promoting its adoption and implementation in educational settings. The theoretical contributions of this thesis advance the field by conducting a systematic mapping of AI-ALS to understand and synthesize extant research contributions on this topic. It also offers an overview of AI-ALS design and implementation, which will be of value for the information systems (IS) research community. Moreover, it provides insights by drawing on Gupta and Bostrom’s (2009) theoretical model to understand the nature of AI-ALS as a mediating tool for improved learning. The empirical part of this thesis provides practical insights for practitioners, systems developers and educators in education settings who are interested in AI-ALS. This thesis’s practical insights are suggestions for better AI-ALS implementation in educational settings. The following practical recommendations are presented: 1) teachers are crucial in the design and the implementation of AI-ALS; 2) the need for interdisciplinary and transdisciplinary collaborations and research; 3) cultural design issues in the AI-ALS field; 4) algorithm bias in the system should be resolved; 5) the ethics and privacy issues of AI in education (AIEd) must be addressed; 6) understanding education as a complex ecosystem; 7)training of teachers in using AI-ALS; 8) buy-in from governmental institutions and educational institutions management is needed; and 9) there needs to be a thoughtful, creative and incremental approach to deploy AI-ALS. System developers, educators, and practitioners can all benefit from these practical implications. In addition, the combination of two different research methods – expert interviews and a ranking-type evaluation questionnaire – provided a richer understanding of the investigated topic by identifying core requirements and propositions towards improved design, development, and implementation of AI-ALS in educational settings. Thus, this research addresses a gap in transferring insights and codification of knowledge on AI-ALS design and development by comprehensively explaining AI-ALS scenarios and deriving general, transferable propositions for designing AI-ALS scenarios. This knowledge is particularly valuable for developers, designers, educators, and researchers planning to adopt and implement AI-ALS in classrooms. Norwegian Abstrakt Blant de mange innovasjonene drevet av kunstig intelligens (AI) er mer avanserte læringssystemer kjent som AI-aktiverte adaptive læringssystemer (AI-ALS). AI-ALS er plattformer som tilpasser seg læringsstrategiene til elevene ved å endre rekkefølgen og vanskelighetsgraden på læringsoppgaver basert på elevenes evner. Disse systemene støtter adaptiv læring, som er personalisering av læring for studenter i et læringssystem, slik at systemet kan håndtere individuelle forskjeller i evner. AI-ALS får økt betydning på grunn av evnen til å levere læringsinnhold og tilpasse seg individuelle studentbehov. Mens potensialet og betydningen av slike systemer er godt dokumentert, har den faktiske implementeringen av AI-ALS og andre AI-baserte læringssystemer i virkelige undervisnings- og læringssettinger ikke nådd den effektiviteten som man har sett for seg i teorien. Videre mangler AI-ALS overførbar innsikt og kodifisering av kunnskap om design og utvikling. Årsaken til dette er at mange tidligere studier var eksperimentelle. Derfor har denne avhandlingen som mål å redusere gapet mellom eksperimentell forskning og praksis ved å gi praktiske designbeskrivelser som kan implementeres i effektive AI-ALSer. Forskningen styres av hovedforskningsspørsmålet: Hvordan bør AI adaptive læringssystemer designes og utvikles? Hovedforskningsspørsmålet vil bli undersøkt ved å undersøke fem delspørsmål: i. Hva er de sentrale forskningsproblemene og utfordringer knyttet til utdanningspraksis i AI-ALS og intervensjonene og løsningene som foreslås for å løse dem? ii. Hva er de underliggende prinsippene for design av AI-ALS i den akademiske litteraturen? iii. Hvordan kan forskere, pedager, designere og utviklere suksessfullt integrere og implementere AI-teknologier for å fremme kvalitetsutdanning? iv. Hvilke aspekter bør man følge for å designe AI-ALS? v. Hvilke av disse anses som viktige for design og utvikling av AI-ALS? For å adressere forskningsspørsmålet og dets mål, undersøkte jeg de ovennevnte spørsmålene ved å undersøke kunnskap fra både litteratur og praksis. En fortolkende forskningsstudie (eksplorativ) ved hjelp av ekspertintervjuer ble gjennomført i kombinasjon med en rangeringstype analyse av de formulerte designanbefalingene. Jeg brukte også Gupta og Bostroms (2009) teoretiske modell for teknologimediert læring, som er basert på adaptiv struktureringsteori (AST), for å forstå AI-ALS som et medieringsverktøy for adaptiv læring. Det er behov for å få en omfattende forståelse av AI-ALS når det gjelder å styrke hele læringsprosessen i en reell pedagogisk setting. Den teoretiske modellen ble valgt fordi den er egnet for å undersøke alle elementene i et sosio-teknisk system -teknologi og læringsteknikker, prosess, aktører, handlinger og utfall - som er avgjørende elementer i AI-ALS. Funnene i denne avhandlingen presenteres i fem publiserte vitenskapelige artikler. Den første artikkelen er en systematisk gjennomgang av litteraturen om moderne læringsmiljøer. Artikkelen forklarer bekymringene og problemene i læringsmiljøer fra det 21. århundre som må tas opp. Den andre artikkelen er en systematisk kartlegging av litteraturen om AI-ALS, som gir en dypere forståelse av AI-ALS ved å fremheve nyere forskning, forskningsgapene som fortsatt eksisterer, og fremtidige retninger på feltet. Den tredje artikkelen er en empirisk studie, med fokus på områder innen kunstig intelligens i utdanning (AIEd) som må undersøkes for å forbedre implementeringen. Artikkelens identifiserer de praktiske fordelene ved å adoptere og implementere AIEd. Den fjerde artikkelen er også empirisk og utleder designkrav for AI-ALS fra litteraturen og genererer dermed forslag til design (i form av foreløpige designprinsipper) for slike systemer. Den femte artikkelen er enda en empirisk studie som utleder designkrav fra ekspertintervjuer og deretter formulerer foreløpige designprinsipper som adresserer disse kravene. Funnene fra intervjuene avdekket nye designkrav som ikke fremkom i den tidligere litteraturgjennomgangen. Ettersom listen av designforslag var sammensatt av flere kilder, måtte listen over foreløpige forslag syntetiseres og bearbeides for å produsere en ensartet liste. Gjennom denne iterative prosessen og evalueringen av de foreløpige designforslagene, ble ni forslag identifisert som viktige for vellykket design og utvikling av AI-ALS for å fremme adopsjon og implementering i utdanningsinstitusjoner. De teoretiske bidragene fra denne avhandlingen fremmer forskningsfeltet ved å gjennomføre en systematisk kartlegging av AI-ALS for å forstå og syntetisere eksisterende forskningsbidrag om dette emnet. Det gir også en oversikt over AI-ALS design og implementering, som vil være av verdi for forskningsmiljøet innen informasjonssystemer (IS). Videre gir det innsikt ved å bruke Gupta og Bostroms (2009) teoretiske modell for å forstå hvordan AI-ALS virker som et formidlingsverktøy for forbedret læring. Den empiriske delen av denne avhandlingen gir praktisk innsikt for praktikere, systemutviklere og lærere i utdanningsinstitusjoner som er interessert i AI-ALS. Denne avhandlingens praktiske innsikt er forslag til bedre AI-ALS-implementering i utdanningkontekster. Følgende praktiske anbefalinger presenteres: 1) lærere er avgjørende i utformingen og implementeringen av AI-ALS; 2) behovet for tverrfaglig og transdisiplinært samarbeid og forskning; 3) kulturelle designtemaer i AI-ALS-feltet; 4) algoritmeskjevheter i systemet bør løses; 5) etikk og personvernsspørsmål knyttet til AI i utdanning (AIEd) må tas opp; 6) forstå utdanning som et komplekst økosystem; 7) opplæring av lærere i bruk av AI-ALS; 8) aksept fra statlige institusjoner og ledelsen av utdanningsinstitusjoner er nødvendig; og 9) det må være en gjennomtenkt, kreativ og inkrementell tilnærming for å implementere AI-ALS. Systemutviklere, lærere og praktikere kan alle dra nytte av disse praktiske implikasjonene. I tillegg ga kombinasjonen to forskjellige forskningsmetoder - ekspertintervjuer og et evalueringsspørreskjema av rangeringstypen - en rikere forståelse av det undersøkte emnet ved å identifisere grunnleggende krav og preskriptive retningslinjer for vellykket utvikling og implementering av AI-ALS innen utdanningsinstitusjoner. Derfor adresserer denne forskningen et gap ved å overføre innsikt og kodifisere kunnskap om AI-ALS-design og utvikling, ved å forklare AI-ALS-scenarier grundig og utlede generelle, overførbare anbefalinger for utforming av AI-ALS-scenarier. Denne kunnskapen er spesielt verdifull for utviklere, designere, lærere og forskere som planlegger å ta i anskaffe og implementere AI-ALS i klasserom Contents. List of Tables 1 Introduction Even before the dawn of the 21st century, as information technology (IT) advances, higher education institutions (HEIs) have seen significant changes in their traditional role of knowledge dissemination. Combining IT resources with instructional methods creates increasingly innovative services that have significant impacts on teaching and learning. Among these impacts are increased motivation, improved achievement, and the opportunity to develop communication skills (R.Raja and P. C. Nagasubramani, 2018; Ghory and Ghafory, 2021). As a result, technological innovation is increasingly becoming a hallmark of academic research and teaching in HEIs. In fact, the rapid emergence of learning analytics, virtual reality, and technology-mediated learning (TML) systems have been noted in the past few years in HEIs. TML systems are technology-based environments that integrate teaching and learning methods to promote the development of students’ knowledge and skills via technological implementations of tutors and educational resources. A number of learning systems are already widely used in TML, including Blackboard, Moodle, WebCT, and Canvas. These systems offer constant availability and access to course materials, savings in cost and time, better engagement and retention delivery, and other benefits and advantages. Students and academics have, however, often taken a negative attitude towards these learning systems. A majority of those systems focus on achieving technical goals (e.g., content delivery) and ignore pedagogical issues (e.g., the granularity of the learning objects in the system) related to the entire learning–teaching process. Due to the dominance of the technical aspects of these learning platforms, students and lecturers perceive them as not adaptive; that is, they are not able to adjust the design of the interface and content to each user’s needs and preferences. Therefore, advanced learning systems other than learning management systems (LMSs) have been developed in recent years. Technologies such as mobile internet, cloud computing, big data, and artificial intelligence (AI) are incorporated into the research and development of TML environments. These environments are characterized by students thriving in digital environments, where current technologies shape their expectations of and capacity for information access, acquisition, manipulation, construction, creation, and communication. Contemporary learning environments make use of physical and virtual resources to facilitate student learning. Such advanced learning systems include advanced adaptive learning systems, which are systems that strive to incorporate analysis of historical data about the previous users of the system by modelling learning process from the learners’ viewpoint, and, thus, be able to adapt to a rapidly changing environment by providing learners not only accurate and high-quality learning material, but also taking into account the individual learner’s needs”. As students interact with the content, these systems dynamically adapt and modify that content, which allows students to study at their own pace and to receive immediate feedback on how they are doing. Adaptive learning systems were originally designed to teach new concepts rather than simply reinforce memorization. However, these systems have evolved, resulting in the creation of new adaptive learning systems that are based on recent advancements in learning science, data science, and AI algorithms. Adaptive learning technologies encompassing elements of AI to allow the system and users to interact with one another, track students’ progress over time, allow stakeholders (teachers and administration) to make well-informed, data-driven decisions, and help students develop skills in a particular learning area. The capabilities and benefits mentioned above highlight the potential of the adaptive learning system enabled by artificial intelligence (AI-ALS). Previous research has described the impact of AI-adaptive learning technology in many learning scenarios and identified it as an important educational trend. Many companies have committed to this area, with Guan et al. (2020) reporting that a total of US$1.047 trillion has been invested in AI-based education from 2008 to 2017. Moreover, the literature shows that the AI technologies in these learning systems can be used to ensure equitable and inclusive access to quality education. By utilizing emphatic systems, educational robots, assistive technology, teacher modelling, and multimodal interaction, AI can help meet Sustainable Development Goal 4 (SDG4) for quality education by addressing issues such as diversity, inclusive education, equitable quality education, and ethical concerns. The use of AI in education also contributes to long-term educational goals, such as developing 21st century skills, providing universal access to global classrooms, supporting lifelong and life-wide learning, generating interaction data for learning, and providing mentorship to all students (all part of SDG4). Other authors contend that the future of higher education is inextricably linked to the development and growth of intelligent machines capable of managing large amounts of information, self-learning, and improvement. Thus, AI has become a focus of global competition of countries in the educational market. As a result, we are seeing a steady increase in the use of AIEd, whether it be through robots, algorithms, or new generations of adaptive learning systems. Although AI in adaptive learning systems has many advantages, such as providing tailored content to each learner in the shortest possible time, there are challenges that hinder their successful implementation. First, many educators, administrators, institutional leaders, and other HEI stakeholders still find it difficult to understand the relevance and impact of AI-ALS in real educational settings. While the potential and importance of such systems are well documented, the actual implementation status of AI-ALS and other AI-based learning systems in real-life teaching and learning settings is low and not fully understood. It has also been difficult to see where we stand in terms of practice and where we want to advance, as discussions and debates over AI-ALS emerged somewhat unexpectedly – before many were conscious of the relevant developments – and involved high-level, abstract discussions. Despite the efforts invested, there are still few working systems or real-world applications. In this sense, AI-ALS are in need of empirical research to be adopted more widely and used more intensively in education. A retrospective look reveals that IS research focuses largely on other types of TML systems such as LMS and less on adaptive learning systems. Figure 1 shows research published by the Association of Information Systems (AIS) on adaptive learning, which has received little attention in the past. This graph shows the search results in the AIS eLibrary database for “learning management” and “adaptive learning.”1 Furthermore, many research studies on AI-ALS are experimental. Experimental research has been used heavily across the field; it involves researchers grouping individuals into an experimental scenario, determine their responses to the conditions that the researchers have set, and using averaging and statistical techniques to extrapolate a general conclusion. Because of this methodological focus, the design of AI-ALS failed to consider basic educational problems such as the theory–practice transformation. Zawacki-Richter et al. (2019) suggest that the majority of AI-based learning systems (including AI-ALS) have been created by computer scientists, at least in research papers. Unsurprisingly, they use models that are based on the way computers and computer networks operate (since AI algorithms must, by definition, be operated by computers). These AI applications adopt a highly behaviourist learning model: present–test–feedback. Developers assumed that learning was a formally described and controllable process. The critical aspects of the entire teaching–learning process can be overlooked by experimental research when that kind of perspective is adopted. Therefore, researchers and practitioners have been unable to develop AI-ALS in practice. It is thus essential to obtain a comprehensive understanding of AI-ALS in terms of enhancing the entire teaching–learning process in an actual educational setting. Overall, I consider the lack of research on AI-ALS in IS field to be a significant shortcoming. It is necessary to address the research gap by comprehensively explaining AI-ALS scenarios and deriving general, transferable propositions for designing AI-ALS. Several stakeholders in HEIs need to engage in more inclusive conversations to identify specific and minimally viable requirements for AI-ALS applications. Thus, as part of my contribution to IS research, I investigate and establish a set of propositions for design and development of AI-ALS, based on both qualitative and quantitative methods that support the interpretive nature of the research. As stated previously, AI-ALS have been designed and developed using experimental research, that has overlooked critical aspects of the teaching-learning process. Educators and practitioners have not gained much knowledge about AI-ALS based on what has worked and what has failed during the development and use of the technology. Hence, research aimed at gathering knowledge from existing literature and practice (such as evaluating existing AI-ALS) is essential. From such a study, the knowledge gained can be codified into propositions and recommendations on developing, designing, and implementing AI-ALS, based on its existing benefits and challenges. Thus, this dissertation addresses this research gap by exploring AI-ALS with the goal of providing propositions for designing AI-ALS so that those with the necessary expertise can develop such systems. Moreover, the goal is to collect and analyse propositions from a socio-technical standpoint by considering the technical aspects of the system and the social aspects of students, teachers, and the educational sector. This research is situated at the intersection of adaptive learning technologies, AI, interaction design, and IS, all of which have relevance to the design, development, and implementation of AI-ALS. The study contributes with propositions to support the design and development of AI-ALS by collecting and synthesizing data from both literature and practice. 1.1 Research Questions Although there are many examples of adaptive learning systems that have been modelled by AI in the literature, the guiding design propositions, guidelines, and principles for the development of AI-ALS are not widely understood or thoroughly researched. The majority of AI-ALS are “limited to academic projects and a few commercial applications” despite their proven potential, and design difficulties with these systems continue to be highlighted in the literature. Thus, the objective of this dissertation is to provide an understanding of the importance and relevance of AI-ALS for students, teachers, educational practitioners, and researchers. With AI evolving rapidly in the education field, issues such as the integration of AI-ALS in real-world education contexts need to be addressed. One approach is to narrow the gap between experimental research and practice by codifying knowledge obtained so far in the form of propositions. In the field of IS, there are various ways of codifying knowledge, based on which a designer or developer progress from a list of requirements to a model that depicts an IS artifact. Common approaches to codify knowledge into recommendations, propositions, or design principles in IS field include design science, action research, case studies and qualitative studies. For the work at hand, (i.e., codify knowledge and to make it available as propositions for designing AI-ALS), I take an interpretive approach. Specifically, I analyzed projects that deal with the design and development of AI-ALS and accumulated design requirements from their specific implementations. In this way, it is intended to unveil implicit knowledge, in the form of propositions, concerning the socio-technical mechanisms with respect to AI-ALS, which so far have been little addressed in the literature. Therefore, this dissertation thus aims to address the following research question: How should AI Adaptive Learning Systems be designed and developed? Coming up with appropriate propositions for designing AI-ALS will help demonstrate the relevance of AI-ALS in mediating adaptive learning in classrooms while helping to resolve the issues in the education system that currently affect its quality. To answer the main RQ, I began by reviewing recent literature on adaptive learning systems. I needed to understand the problems and concerns these contemporary learning systems attempted to address. Additionally, I had to determine the kinds of AI-enabled interventions that have been developed and used and why those approaches were. Thus, the first sub-question (SQ1) was crafted to facilitate a systematic mapping of the literature on AI-ALS: SQ1: What are the core research problems and educational practice concerns in AI-ALS and the interventions and solutions proposed to address them? From the literature review, it was understood that few studies in the field have addressed design issues in these contemporary learning systems. Due to the necessity to simultaneously optimize multiple objectives such as delivering complex knowledge to students as quickly as possible, designing AI-ALS has become more challenging (van der Vorst and Jelicic, 2019). It can be difficult for designers and developers of AI-ALS to achieve a balance between creating advanced AI-ALS and ensuring that students receive a quality education that truly enables them to develop their learning skills. Additionally, given how quickly AI technology is advancing, there is still a need for research in AIEd to develop principles and support for AI-ALS that are founded on evidence. In light of this, I carried out a literature review to establish the current state of the principles and guidelines for AI-ALS. Therefore, SQ2 was defined as follows: SQ2: What are the underlying principles for designing AI-ALS in the academic literature? Furthermore, research gaps on the current state of AI-ALS implementation are regularly reported in the literature. There have been initiatives to acknowledge AI as an emerging technology that is pervasive and can assure inclusive and fair quality education. However, AI systems have yet to be implemented in deprived neighbourhoods to offer inclusive and equitable quality education. UNESCO (2021) reports have linked AI to the fourth Sustainable Development Goal (“Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”), but there has been little research on the evidence-based implications of AI in schools and universities. The growing need for AI-ALS to be adopted in education necessitates further empirical study of the implementation and evaluation of these systems. I therefore looked into the practical benefits of current AI-ALS efforts and the issues that need to be resolved to use AI to advance inclusive and quality education. SQ3 was thus framed as follows: SQ3: How can researchers, educational practitioners, designers, and developers successfully integrate and implement AI technologies to promote quality education? In addition to investigating existing practical benefits and challenges, I also examined the practical and evidence-based propositions in existing projects, which led to SQ4: SQ4: What aspects should one follow consider when designing AI-ALS? I address the above questions by eliciting knowledge from both the literature and experts in AI-ALS and TML. An exploratory study using qualitative analysis of interviews with experts was conducted, and 13 propositions, in form of preliminary propositions for designing AI-ALS were formulated. In conjunction with the expert interviews, I also investigated whether these propositions are relevant and important to the design and development of AI-ALS. Thus, a ranking-type analysis of the formulated propositions was conducted, and the last sub-question (SQ5) was developed to help guide that analysis: SQ5: Which of these are considered important for the design and development of AI-ALS? An exploratory study using qualitative analysis of the expert interviews, in conjunction with ranking-type analysis of the formulated propositions, provides the empirical basis for the thesis. The results of this study are presented and discussed in five research publications (see Appendix B). A coherent presentation of the research findings is provided in the dissertation summary that integrates the five publications. It should be noted that the findings on SQ5 are presented in Chapter 4 as additional findings; they are being prepared for journal submission. 1.2 Thesis Structure There are two main parts to this dissertation. The first part provides a summary of the research conducted to answer the research questions outlined earlier. It is comprised of six chapters clustered into three development layers: research foundation, empirical analysis, and implications. Chapter 2 positions this study within IS research by reviewing the relevant literature to identify knowledge gaps Chapter 2. Chapter 3 presents the research approaches used. Following this, the findings of the five papers are summarized in Chapter 4, as are the results of the ranking-type analysis of the formulated propositions for designing AI-ALS. A discussion of the findings according to the research questions appears in Chapter 5, along with the implications for theory and practice. The final chapter of the thesis summarizes its conclusions, contributions, and limitations. Figure 3 presents the thesis’s layered structure. The five original research papers serve as the second part of the dissertation and provide full accounts of the research conducted for this project. 2 As a result of literature review and semi-structured interviews with experts, I developed design statements for AI-ALS that were presented as AI-ALS design principles published in papers 4 and 5. During my work, I realized that the design and development of an IT artifact is required in order to formulate design principles. This was not the case with my study since I conducted a more traditional interpretive study. Thus, to avoid misunderstanding, this thesis uses the term "propositions" for conceptual clarity. 2 Positioning the Study This chapter positions this dissertation in relation to the existing literature to illustrate how it can advance existing research, especially in the IS field, to realize the significance and relevance of using AI-ALS in educational environments. Before understanding, designing, and implementing AI-ALS for teaching and learning, a solid knowledge of the history of these systems is essential. Thus, we need to define and elaborate on the primary IT artefacts discussed: AI and adaptive learning systems. I then briefly review the existing literature on the benefits, challenges, and concerns associated with AI-ALS. The last part of this chapter presents a model used to understand an adaptive learning environment mediated by AI. 2.1 Background With the application of IT in education, teaching–learning methods have undergone sweeping changes. Thus, to understand how LMS are embedded in the learning process and relate to learning outcomes, we first take a step back and understand TML, which refers to “an environment in which the learner’s interactions with learning materials (readings, assignments, exercises, etc.), peers, and/or instructors are mediated through advanced information technologies” (Alavi & Leidner, 2001 p. 2). TML can take many forms and may combine different learning strategies and methods in practice, including web- or computer-based learning, asynchronous or synchronous learning, instructor-led or self-paced learning, and individual- or team-based (collaborative) learning. TML is an umbrella term that incorporates different approaches to using computers in learning and teaching: computer-aided/assisted learning, computer-mediated communication, and computer-supported research tools. Examples of learning environments under the umbrella term TML include web-based learning environments, e-learning environments, virtual learning environments, mobile learning environments, and massive open online courses. TML systems can support learning-related behaviours such as active participation in a learning environment through technology. The active participation and interactions between students can occur through interactive whiteboards, emails, video games, discussion forums on LMS, social networks like Facebook and Slack on personal and mobile computing devices, exhibits or installations that feature digital media, wearable technology, and video chat applications like Zoom, Microsoft Teams, and Skype (Ross, 2019; Lal, Dwivedi and Haag, 2021). The most common TML system is the LMS, which is a software package that provides an integrated system for the administration, planning, and delivery of online courses. LMSs allow educators to create more robust educational materials that can be shared with students at any time. Figure 5 shows many types and examples of TML applications used at various stages of education, including LMS types such as BlackBoard, Moodle, and Canvas. TML systems are also used in organizational settings to oversee training and e-learning throughout the operation, train new employees, organize the organization’s learning content in one place, track individual progress, and present data. TML systems are successfully embedded in businesses of all sizes, government agencies, electronic commerce sites, and educational institutions. Although IS research on TML has mainly analysed web-based learning, e-learning, LMSs, and virtual learning environments, the technologies used in this mediation are very diverse. In the TML context, the term “adaptive” refers to a wide variety of system capabilities, making it important to specify the feature(s) to which one refers when discussing adaptive learning and TML; a learning environment is adaptive if it is capable of monitoring the activities of its users; interpreting these on the basis of domain-specific models; inferring user requirements and preferences out of the interpreted activities, appropriately representing these in associated models; and, finally, acting upon the available knowledge on its users and the subject matter at hand, to dynamically facilitate the learning process. Thus, personalized learning support services are made possible by adaptive learning and the dynamic creation of learner models, learning path recommendations, and data evaluation, which should all improve performance. There is no unified view on the concept of adaptive learning in the research literature. It is generally accepted that adaptive learning aims to deliver content that specifically caters to a given student’s needs, differences, circumstances, preferences, and competencies. The understanding that different individuals learn in different ways provides the foundation for adaptive learning, which is a promising domain for academics and practitioners because it can provide meaningful experiences to learners. Section 2.2 describes the history and evolution of adaptive learning systems. 2.2 History and Evolution of Adaptive Learning Systems Adaptive learning first emerged in the educational context in the 1950s and was developed by the noted behaviourist B.F. Skinner, who created a teaching machine that focused on efficiently teaching new concepts rather than reinforcing memorization. The machine functioned by giving the student the opportunity to practice new concepts by answering questions. If a question was successfully answered, feedback and positive reinforcement were provided. Rather than merely having the student try again if an answer was incorrect, smaller steps were taken towards the correct answer through a succession of clues. If Skinner’s programmed instruction from the 1950s can be considered the genesis of the adaptive learning system, computer-aided instruction (CAI), which emerged in the 1970s, is its prototype. Working in the United Kingdom, Pask developed an adaptive teaching machine using computers, which is regarded as a primitive ancestor of CAI. Intelligent tutoring systems (ITS) emerged in the 1980s and are often cited as the earliest true adaptive learning system; their basic framework was proposed by Hartley and Sleeman (1973). In the 1990s, virtual reality (VR) and agent technology were applied to ITS. Intelligent agent teaching systems, also known as intelligent student self-study software systems, emerged from ITS. As the 20th century became the 21st, the combination of AI and hypermedia technology produced a new learning system, the adaptive hypermedia system. In 1996, an adaptive educational hypermedia system was called the first real adaptive learning system. Table 1 depicts the early trajectory of adaptive learning systems through the stages of program teaching machines, computer-aided teaching, ITS, intelligent agent teaching systems, intelligent hypermedia teaching systems and adaptive intelligent learning systems. Recent breakthroughs in big data, learning analytics, and scalable architectures have created opportunities for adaptive learning systems to be radically redesigned. The opportunity to design the next generation of adaptive learning systems is supported by developments in learning science, data science, and AI algorithms, these new forms of learning are what I refer to as AI-ALS. AI is defined as a set of computer programs and technologies that mimic the human brain’s function and intelligence. The most appealing characteristic of AI is its ability to rationalize and take actions that have the best chance of achieving a specific goal. Machines or systems that are mechanically intelligent, performing repetitive tasks efficiently, and/or which self-learn from a vast sources of data and adapt their performance accordingly are all AI systems. Systems can become intelligent by learning from a variety of data types, including text, audio, and video. This repository of big data enables AI systems to learn through computational methods like machine learning (ML) and deep learning. ML, which is a subset of AI, is the study of algorithms that enable computer programs to improve automatically through experience. ML uses computers to simulate human learning by identifying and acquiring knowledge from the real world and improving performance based on this new knowledge. Within the AI domain, ML is the most widely used technique among researchers who develop algorithms for applications such as recommendation systems, autonomous vehicle control, image recognition, computer vision, and natural language processing. This is because researchers realized that it is much easier to train a system with data consisting of desired inputs and outputs than to manually program a system by predicting the desired outputs in the context of all possible inputs. AI-ALS are the most advanced generation of adaptive learning systems that use more sophisticated data analytics (learning analytics) and AI (ML algorithms) to provide just-in-time insights and embed ongoing data collection to improve the quality of adaptive learning. AI-ALS are designed to provide “just-in-time feedback in the learning moment to learners and instructors”. Researchers such as Essa (2016), Li et al. (2021), and K. Zhang and Aslan (2021) describe these next-generation adaptive learning systems as having the ability to provide scaffolding for improving learning strategies and optimizing knowledge acquisition and recall. Adaptive learning can be achieved by providing targeted services, such as learning content and path recommendation and intelligent tutoring, to provide personalized learning support for learners. The learning content and path recommendations are made through by AI in the form of ML algorithms. Table 1 presents the evolutionary trajectory of adaptive learning systems. In the early days of adaptive learning, much of what was considered “adaptive” comes from the perspective of traditional pedagogy without mentioning the influence of technology like AI. The notion of “AI and adaptive learning” has become a new proposition in global research as AI technology continues to be integrated. Adaptive learning has also taken on new meanings as a result of this technological integration. Although elements of AI-ALS applications can be traced to the 1950s, as is detailed above, they began gaining popularity in this century. More recent adaptive learning systems enabled by AI have emerged, such as AutoTutor, Knewton and QuizBot in the United States, Knowre in South Korea, Smart Sparrow in Australia, the online teacher training platforms Declara and Cogbooks in the United Kingdom, and Yixue Squirrel AI, Classba Education and Homework Help in China. In Paper 2, I refer to examples and case studies in which AI-ALS were developed for the scenarios being studied. For example, investigated providing immediate adaptive feedback with IntelliMetric, an AI system that grades writing at or above human reliability rates. Another example is AL, an adaptive IT tool that provides students with feedback on the argumentative structure of a given text by leveraging recent developments in natural language processing and ML. Thus, adaptive learning systems are already capable of capturing a wide range of user cases and steering users in the desired direction, thanks to their AI elements. As new technologies continue to emerge, such as human-computer interaction, sentiment analysis, big data processing in online education, and AI is even more deeply integrated into educational science and psychology, adaptive learning research has kept growing. 2.3 Functions and Components of AI-ALS As shown in section 2.2 (and Papers 1 and 2), there are many forms and examples of AI-ALS. Each adaptive learning system described above is distinctive at the level of detail and generally at the design level. However, most of these systems tend to be constructed from these key elements: i. The capability to automate processes related to assessment, evaluation, remediation, and competency attainment (i.e., automation;; ii. The ability to structure a finite learning path in a limited or unlimited amount of time based on a sequenced progression of skills and competencies (i.e., sequencing; Hou & Fidopiastis, 2017;; iii. Using benchmarks, diagnostics, and formative assessments on a continuous and immediate basis (i.e., assessment;; iv. Collecting, calculating, and evaluating data in real time or nearly real time from a wide range of sources, with a given assumed inference method (i.e., real-time data collection; Holstein et al., 2019; Hou & Fidopiastis, 2017; Maravanyika et al., 2017;; v. Self-organization of information and data occurs as a result of inferences that lead to continuous and persistent feedback in the classroom (i.e., self-organizing; Hou & Fidopiastis, 2017;. These five functionalities are integrated into most AI-ALS to enable them to address and achieve their aims. Moreover, AI-ALS have traditionally been divided into separate components and models that should be accounted for when designing and developing AI-ALS. While different components can be involved, most systems include some or all of the following components, occasionally with different names: environmental model; student model (also known as user or learner model); domain model (also known as expert or content model); and adaptation model (also known as adaptive, instructional, pedagogical, or tutoring model;. The domain model refers to the “content domain to be taught by the system, containing the set of topics and the corresponding learning objectives or outcomes, skills, knowledge, and strategies needed to learn them”. Using this model, students learn the pedagogical material’s data, game model, teaching method, content, and hierarchical structure of the topic and how to provide personalized feedback. The student model refers to learner characteristics; that is, what the student knows and does. In this model, students’ data, such as name, contact information, personal characteristics, skills, activity logs, profiles, self-estimation of topics to be learnt or prerequisite skills (novice, advanced, expert), motivation to learn, and even handicaps, may be used to adapt the presentation of texts, illustrations, and animations, to select exercises, and to compute an appropriate learning path. The pedagogical model refers to “the algorithm that assists in adapting the instruction based on the content and learner mode”. This model provides the actual adaptation through its characterization of the adaptive mechanism and adaptation engine, which is attained by combining the information processed from the learner model with information from the domain model. These components of AI-ALS are depicted in Figure 6. The human machine interface in the diagram brings together the student, domain, and adaptive models in a single user interface with which a student can interact. 2.4 Theoretical Lens to Understand AI-ALS as a Tool that Mediates Adaptive Learning In the above sections, I have defined and discussed the meanings of adaptive learning and AI-ALS, along with the functionalities and components of AI-ALS, to provide a broad understanding of what AI-ALS is intended to achieve. However, despite multiple references in the literature indicating the potential of these next-generation adaptive learning systems, most AI-ALS are still “restricted to research projects and a few commercial applications”. The first chapter of this thesis and Papers 1 and 2 show that there are few studies of AI-ALS implemented in classrooms practice. Moreover, most research studies on AI-ALS have designed them in experimental contexts. As highlighted in the introduction, this has some disadvantages, as developers assume that learning is a formally described and controllable process. Critical aspects of the entire teaching–learning process (i.e., users, technology, and learning technique structures) can be overlooked by experimental research. There is thus a need to provide a comprehensive view of AI-ALS in terms of enhancing the entire teaching–learning process in a real educational setting. In this section, I draw on the TML model of Gupta and Bostrom (2009) to understand AI-ALS as mediating tools for adaptive learning. According to the leading constructivist learning theories, students require individual tutoring to learn well. Teachers’ meaningful feedback not only helps students achieve specific learning outcomes but also helps minimize undesired attrition. As a result, numerous researchers have advocated for a process perspective of learning artefacts to better understand how students learn and to discover potential learning support interventions. Learning processes centred on TML are a significant research stream in IS research. TML was defined by Alavi and Leidner (2001) in their call for TML research 20 years ago as “an environment in which the learner’s interactions with learning materials, peers, and/or instructors are mediated through advanced information technology” (p. 2). TML includes – by definition – “all the elements of a social-technical system: technology and learning techniques, process, actors, actions, and outcomes”. TML systems aim to improve learning outcomes by creating settings in which students can engage with teachers, peers, and educational materials through the use of technology. By identifying students’ learning paths, these systems enable teachers to improve their learning experiences. The variations in the learning process result from the fact that learning requires the construction of new ideas or concepts by the individual based on his or her knowledge, skills, talents, and/or experience. AI-ALS are aware of these variations and adjust to the different learning strategies, techniques, and aptitudes of the students. The foundation for adaptive learning is the understanding that different people learn in different ways. Adaptive learning may be accomplished by developing learning objects, which are tiny units of customizable digital resources that contain pertinent information for a particular student. Content is personalized using ML algorithms that increase the model's prediction abilities with each interaction. Through these means, the appropriation of technology supports students’ learning processes and are thus especially important for TML, which mediates personalized and adaptive learning. The preceding views are founded on two premises. The first is concerned with the impact of structures integrated into a particular context and is characterized as the rules, resources, and capabilities in that environment. In the context of TML, we examine the learning methods and structures that are reflected, for example, by the deployment of IT like AI-ALS. The second premise focuses on the user of an IT artefact, such as how a student interacts with the structures offered. Participants in this interaction learn and adapt the learning methods and structures. This appropriation process is a complicated event in itself, involving cognitive processes and interactions with previously introduced learning methods, support in the learning process, and other learning scenario factors that influence outcomes. The latter refers to “the goal assessment or measures for determining the accomplishment of learning goals” and is the primary outcome measure used in every kind of TML (Janson and de Gafenco, 2015), including AI-ALS. Gupta and Bostrom (2009) also note scaffolding as a well-known technique for guiding and facilitating learning processes. Wood et al. (1976) define scaffolding as temporary instructional assistance for learners to overcome barriers within their zone of proximal development, which involves altering the learners’ individual learning processes and experiences. Process scaffolds are methods that provide and offer initial assistance to support learning processes. Scaffolding, which has its roots in social constructivist theory, holds that intersubjectivity between instructional designer and individual learner and between learners is essential for learning. Typically, scaffolds are designed by an instructor. According to the IS and education literature, there are three types of process scaffolds: meta-cognitive scaffolding that supports individual reflection on learning; procedural scaffolding that help students make navigation decisions such as how to use available resources and tools; and strategic scaffolding, which supports students by anticipating their interactions, such as analysing, planning, and making tactical decisions. Thus, process scaffolding can help ensure a successful appropriation of learning methods. Scaffolds support learning processes by offering guidance on how to use applicable methods and structures, organize tasks, and self-monitor learning processes by providing cues or hints for accomplishing a task. Ifenthaler and Gibson (2019) advocate for the implementation and further investigation of personal and adaptive learning environments based on concepts such as scaffolding to improve individualized learning and personalized feedback whenever necessary. The learning process is also influenced by each student’s specific aptitudes, which are the initial states and abilities of people that influence their behaviour. Aptitudes can be broadly distinguished into two categories: motivation and cognitive abilities. Motivation is defined as the direction, intensity, and persistence of learning-directed behaviour in a learning context. Cognitive abilities include a student’s capacity to perceive, think, and process. Cognitive abilities focused on in the literature include learning strategies, self-efficacy, and learning orientation. Gupta and Bostrom considered learning outcomes in their theoretical model. Learning outcomes are learning goal assessments or measures for determining the success of a learning program, while learning goals are defined as the knowledge, skills, or competencies expected to be attained as a result of the learning process. Learning goals not only influence the complexity of learning techniques but also drive the entire learning design. Complexity deals with “the level of critical thinking and number of decision factors that participants need to go through as they perform a learning technique”. Gupta and Bostrom (2009), they categorized learning goals into skill, cognitive, affective, and meta-cognitive goals. Thus, the complexity of the learning technique should be driven by the learning goals. For instance, a simple task such as multiple-choice questions or reciprocal questioning is likely to be more effective for a novice; for an expert, meanwhile, more advanced tasks such as real-life projects are likely to be more effective. This process view considers learners’ interactions with the structures of TML described above, such as by means of a learner’s adaption to the applied learning methods and materials provided by AI-ALS. Figure 7 illustrates the AST theoretical model of TML described by Gupta and Bostrom (2009). Gupta and Bostrom’s model has been identified as an appropriate tool to engage with to understand the findings of the study around issues like the design of AI-ALS in terms of the incorporated components, functionalities, and features. Moreover, the model will assist in understanding issues around users (both students and teachers) interacting with and using AI-ALS and issues around AI-ALS being adopted, implemented, and used in the entire learning process. Other IS theories like activity theory, the DeLone and McLean IS success model, the technology acceptance model, the expectations confirmation model, and IS design theory are widely used to investigate user satisfaction, design, and the use of online learning environments. However, there has been limited empirical data to provide a coherent, comprehensive picture of TML phenomena. Thus, Gupta and Bostrom articulated a theoretical model, for TML that explicitly configures users, technology, and learning technique structures in learning In their work, Gupta and Bostrom (2009) emphasize the main components of learning and take a variance-based perspective as they conceptualize “learning techniques as structures and operationalizing them as ordinal variables with levels of complexity ”. In the same line, extant literature has referred to this model and examined it typically through positivistic studies. With regard to this thesis, the objective is to explain how adaptive learning happens, that is, a method for personalizing the learning experiences of students; how AI-ALS is employed as a mediating tool to enhance adaptive learning; and how teachers and students interact with such technology (a relatively neglected area of study). As such, the study is less concerned with exploring theoretical relationships among the elements (propositions) in the TML model. Rather, the aim is to improve how AI-ALS are designed and developed. There have been studies using the TML model that did not use a positive perspective. One such study, Choudrie, Zamani and Obuekwe (2022), explored the digital divide in older adults when accepting and using smart devices within an organization. They examined how three learning models (behavioural modelling, constructivist learning, and collaborative learning) could be applied within the context of TML and the digital divide using a TML-based approach. Thus, with an interpretive perspective, I use TML model as an approach to explain and understand how to design better AI-ALS as mediating tools for adaptive learning. 2.5 The Role of AI-ALS in Fostering Adaptive Learning: Benefits and Challenges Before I conclude this chapter, it is important for readers to understand the benefits that AI-ALS offer. AI-ALS are designed to dynamically adjust to the level or type of course content based on an individual student’s abilities and skill attainment in ways that improve a learner’s mastery with both automated and instructor interventions. These systems achieve this by helping stakeholders (students, teachers, administrators, etc.) to address learning challenges such as autonomous learning goals, stress, lack of motivation, diverse backgrounds in terms of demographics and level of knowledge, and resource limitations. This makes AI-ALS unique regarding the advantages they can offer. For example, AI-ALS enable more engaging and effective ways to teach factual knowledge, as they deliver factual content that is tailored to the learner and the learner can identify themselves in it. This also affects their level of motivation and engagement. Moreover, in contexts where content lacks relevance for the learner (e.g., it is uninteresting) or if the content is very far from the learner’s existing competence level (e.g., too difficult or too easy), AI-ALS are built to identify and address the specific needs of each learner. This has implications for the mastery of skills. In addition to the advantages noted in Papers 1 and 2, adaptive learning systems can decrease teachers’ workloads, and teachers receive data with insights into individual students’ needs. Moreover, AI-ALS have the potential to solve the primary and perennial problem in public education: the overwhelming challenge of teachers or faculty being responsible for accomplishing learning mastery among a demographically diverse set of students. Other advantages of AI-ALS and AI-based learning systems more generally include offering rapid feedback and dynamic assessments, which facilitate meaningful group collaborations and engagements in learning settings. In addition, AI-ALS provide human–computer interactions, support engagement and motivation, and help students establish goals All these advantages make AI-ALS valuable for learning and underline their growing importance in the learning environment. Compared to traditional technology-enhanced learning systems, students become increasingly engaged and motivated when using AI-ALS. In addition, designers and developers of AI-ALS acknowledge that differences in the learning process might arise because learning entails the construction of new ideas or concepts by an individual based on his or her specific knowledge, skills, abilities, and/or experience; thus, AI-ALS adapt to students’ differential learning strategies. The ability of AI-ALS to enable adaptive learning in students can spark their interest in the field of education. However, the application of AIEd, specifically in the form of AI-ALS, comes with challenges and disadvantages. First, there no comprehensive and robust structuring of the design elements of AI-ALS. Research is scattered across a variety of technical and socio-technical perspectives, resulting in an urgent lack of an integrative perspective. A good example is the algorithms developed by Brusilovsky to develop systems used to teach Java, an important programming language. The algorithms allow for setting test question parameters. Based on test results, links for students are adapted by showing colourful targets. While this concept is a good idea for an introduction to a programming language, it is hardly possible to calculate variations of test questions that can be analysed by algorithms in other fields such as education. In addition, different teaching methods are not considered at all, so dynamic learning pathways cannot be created. The system offers all information for free navigation and considers the freedom of the learner in that way. Further, as illustrated throughout the previous chapter, AI-ALS have usually been designed in experimental settings. Therefore, previous findings are not easily transferable to AI-ALS designed for use and adoption in a complex educational setting. The examples in Hsiao et al. (2010) illustrate the problem with the lack of an integrative perspective. This is a challenge that conversational AI and natural language processing approaches could help address. Studies have shown that algorithms in AI-ALS are limited to isolated cases and small content areas like mathematics and programming. Nevertheless, most contemporary AI-based applications for teaching and learning focus heavily on content presentation and testing for understanding and comprehension. Bates et al. (2020) state that “comprehension and understanding are indeed important foundational skills, but AI so far is not helping with the development of higher order skills in learners of critical thinking, problem-solving, creativity and knowledge-management” (p. 5). Certainly, most educators believe that to develop the high-level intellectual skills of critical thinking, creativity, and problem-solving, and emotional skills such as empathy – skills that are crucial in the digital age – a more learner-centred, constructivist approach to education is required. AI techniques have been employed to make systems adapt to students’ learning strategies, preferences, and so on, but have not done much to accommodate learners’ competencies and skills. In their systematic literature review, Mousavinasab et al. (2018) noted how few AI techniques had been used in contemporary learning systems to accommodate learner competencies and skills. It can be difficult to attain the skills and competencies taught in a course. Xie et al. (2019) also noted that designers of adaptive learning systems have paid little attention to courses that have practical or technical skills as a prerequisite. Moreover, Afanasyev et al. (2016) found that adaptive learning systems had problems integrating theory taught in class and continuous practical training assessments of the students’ competency levels. Thus, algorithms and cognitive models should be broad enough to teach the sciences, language arts, humanities, and language acquisition. It is important to understand that narrow content areas like mathematics and programming are well structured in terms of knowledge components; in those areas, factual knowledge is easier to model. There have been efforts towards open learner models that attempt to target other disciplines. Moreover, though most AI applications for teaching and learning have focused more on “basic” levels of learning such as memorization and testing comprehension, other technologies like simulations, game-based learning, and virtual reality have had more success in teaching skills such as problem solving, critical thinking, and creativity Other concerns in implementing AI-ALS include ethics and transparency in the collection and use of learners’ data. AI has raised significant ethical concerns, such as replacement of teachers, job loss, and algorithmic bias. Students might not feel comfortable with all their personal data being collected by institutions in the name of adaptive learning. Trust and biases are another issue, along with the overall explainability of the learning adaptation so that learners can build trust in computerized systems and awareness of their own learning needs. Issues such as the effectiveness of AI-ALS interventions, the choice of pedagogies used in these interventions, gender equity, and data ethics have all yet to be addressed to sufficient degree. One reason that AI has had so little influence on teaching and learning in higher education is that education often lags when it comes to new technology. The unwillingness to take chances or embrace new innovations, combined with a lack of funds for anything other than traditional teaching techniques, can work against the adoption of new technology in all areas of education, learning, and personal development. Many educators must be persuaded that a new concept can enhance or expand learning outcomes and experiences, so the education sector is quite cautious when it comes to new technology. Furthermore, while many successful systems have been built (with a peak in the 1990s) and are now used, many other systems have only been evaluated and employed in laboratory settings. Table 3 depicts the most important challenges in this regard. 2.6 Summary and Research Gaps in the Literature AI-ALS are becoming increasingly important in education and could become an integral part of education in the future. This chapter has provided an overview of the state of the literature on AI-ALS and the potential of AI-ALS for students and teachers. Moreover, it describes several studies highlighting the benefits of using AI-ALS in educational. In addition, it lists the functional goals and components of AI-ALS that should be considered in the AI-ALS design. I have also highlighted the challenges of and concerns with implementing and using AI-ALS in classrooms. These challenges and concerns with design constitute research gaps that need to be addressed. Thus, as I stated in the Introduction, it is necessary to address these gaps by comprehensively explaining AI-ALS scenarios and deriving general, transferable propositions for designing AI-ALS. As part of my contribution to IS research, I intend to investigate and establish a set of propositions for the design and development of AI-ALS AI-ALS can be intuitively classified as IS, but they represent a contemporary and advanced form such systems, characterized by a high degree of interaction and intelligence from a socio-technical perspective. As highlighted throughout the chapter, the critical aspects of the entire teaching– learning process have been overlooked by experimental research on AI-ALS. It is important to have a comprehensive understanding of AI-ALS in terms of enhancing the entire teaching–learning process in a real-world educational setting. Therefore, I draw on the TML model by Gupta and Bostrom (2009) to understand AI-ALS as a mediating tool for adaptive learning. By using the model to understand the findings obtained, I follow an interactive learning perspective based on the socio-technical system view, as it allows me to classify a given IS into relevant elements: people, task, structure, and technology. 3 Research Approach and Methodology This study seeks to establish a set of propositions for designing AI-ALS. The previous two chapters provide a foundation by looking at the research regarding AI-ALS in education. The purpose of this chapter is to detail the research design and approach used to construct propositions. How study participants were chosen, methods used to collect data, and how the data were analysed are all explained. 3.1 Research Design A research design can be defined as an action plan that the researcher uses as a guide to get from “here” to “there,” where “here” means as the initial set of questions to be answered and “there” is some set of conclusions or answers. The plan or method of inquiry aids in the transition from a philosophical perspective to the gathering and interpretation of data. It is possible to think of a research design as a blueprint for addressing four issues: what to examine, what data to collect, what data to acquire, and how to analyse the data (Philliber;, Schwab; and Sloss, 1980). The research reported in this dissertation is about crafting propositions that should be followed in designing and developing AI-ALS. It also focuses on understanding the perceptions of lecturers, designers, developers, and experts in AIEd regarding the implementation status of AI-ALS. To establish propositions for designing AI-ALS that are useful and make them available as codification of knowledge statements that support AI-ALS as both process and product, I chose to follow interpretivism as a research framework. In contrast to positivism, interpretivism contends that reality is subjective, socially formed, and made up of various points of view. The researcher inherently shapes research through his or her own lens by bringing subjective views of observable events based on personal experience to the research process The aim of interpretive research is to understand phenomena by accessing the meaning that participants assign to them. Interpretivism is concerned with sense making and understanding. The knowledge of design, development, and implementation status of AI-ALS has been socially constructed by people and organizations. Thus, AI-ALS propositions are related to people and specific settings (here, education). Thus, I decided to acquire knowledge or AI-ALS based on the interpretations of existing knowledge and experts involved in my area of study. The chosen research design enables me to provide a detailed elaboration of the research phenomena and provides the opportunity to document the perceptions of the chosen sample population. Moreover, understanding the perceptions of both researcher and researched is another reason to use this kind of research design. I used both qualitative and quantitative methods that support the interpretive nature of the research. A qualitative design (interviews) and quantitative design (questionnaire) were used to answer the second, fourth, and fifth sub-questions. The first and third sub-questions were answered by a systematic mapping of the literature. This research design allowed me to produce a more comprehensive picture by amalgamating information obtained from a range of data sources. Using multiple approaches with different data sources are also helpful in terms of gaining a richer understanding of the phenomena studied (Munkvold, B.E. and Bygstad, B., 2016). The research design and methods chosen to offer detailed information that enable the research sub-questions to be answered. The structure of the research approach process consists of three complementary pillars: a systematic mapping of literature identifies challenges and gaps in the field researched; a literature review and expert interviews to create preliminary propositions; and a ranking-type evaluation of the formulated propositions. I approached the study by first conducting a systematic literature mapping to collect, analyse, and synthesize all the relevant research. This mapping resulted in Papers 1 and 2. The review contributed to refining the problem definition in this thesis. As Figure 8 shows, an extended literature review was conducted to specifically extract underlying principles mentioned in the existing pool of knowledge. The review resulted in Paper 4 and constituted the identified knowledge base for propositions formulation. Second, qualitative interviews using the expert interview technique in Bogner et al. (2009) resulted in Papers 3 and 5, which are exploratory studies. That method enabled this thesis to be grounded in current practice and provides rich and in-depth information regarding the domain of interest. The outcomes of the exploratory studies addressed sub-questions two and four. Lastly, a ranking-type evaluation of the preliminary propositions was conducted to prioritize the final propositions and addressed the last sub-question. The outcomes of each pillar and process led to findings that are discussed in the chapters below. Figure 8 highlights the activities or steps conducted as the chosen research design for this study. As noted in the previous section, I also use a well-established theoretical model to support and understand the observed phenomena and inform the findings of the study. 3.2 The Study Area and Participants As noted above, the first empirical study in this thesis consists of qualitative expert interviews followed by a ranking-type evaluation. Conducting any empirical study involves recruiting informants, who are study subjects providing critical information or interpretations about the domain of interest and suggesting other sources of evidence. From a qualitative perspective, researchers naturally select informants who can discuss the study area of interest. The application of AIEd through adaptive learning systems is the focal area of this study, and that includes HEIs. The target population includes designers and developers of AI-ALS, lecturers who have used these systems to teach, and researchers in the field of AI-ALS and AIEd. It is important to choose the right primary sources of data to obtain correct, meaningful answers to research questions. The study participants were thus drawn from a list of authors who wrote the coded papers involved in the systematic mapping of the literature (Paper 2), based on their expertise and background. Designer, developer, lecturer, and researcher backgrounds can all bring meaningful and proportionate differentiation to the study. The information provided in publications in the systematic mapping of the literature, Google Scholar profiles, and present positions, the expertise of the informants could be discerned. Thus, an invitation letter to the study were sent to 143 experts through their work e-mails. Some declined the invitation out of scheduling concerns, while others did not respond at all. A total of 38 experts were recruited (Table 3); they were between 28 and 80 years of age and had either designed, developed, used, or researched AI-ALS and/or TML systems. Detailed information about the experts is presented in Table 3. Informants were also needed for the ranking-type evaluation process. This involved a survey in which the experts rated the preliminary propositions and choose the most important ones to consider during the design and development of AI-ALS. To recruit participants for this phase, I circulated the survey to the 38 experts previously selected and other experts in the domain of AI-ALS to ensure the relevance of the knowledge obtained about the field. Communication with experts was done mainly through email for convenience. By using the information provided on their publications in the literature review, Google Scholar Profiles, and present job positions, I was able to gauge the expertise of the informants. A total of 23 informants took part, among whom 13 were from the previous round of interviews. Table 4 provides more information about the informants. 3.3 Data Collection Methods The methods employed to collect primary data from participants were expert interviews and a survey questionnaire, both of which are convenient for collecting data on demographic information, experiences, perceptions, and desirable propositions. Both data collection processes were applied in close cooperation with my supervisors. 3.3.1 Expert Interviews Expert interviews are the primary data source in this thesis. Generally, interviews are used to understand the perspectives of the chosen population. The expert interview technique in Bogner et al. (2009) was used to conduct semi-structured interviews with AI-ALS experts from educational institutions around the world. This approach is used in qualitative empirical research to explore expert knowledge; it was chosen because it enables this thesis to be grounded in current practice and provides rich and in-depth information regarding the domain of interest. The semi-structured interview method was selected to combine the merits of using a list of predetermined themes in a structured interview while ensuring adequate flexibility to enable interviewees to talk freely about any topic. A total of 38 interviews were conducted. The data collection took place from May 2021 through March 2022. Participants specified interview times that were convenient for them. Due to COVID-19, all interviews were conducted using a video conferencing tool; they lasted from about 30 to 60 minutes. In exploratory research, personal interviews are recommended because they allow for comprehensive discussions. All interviews were conducted in English. The questions were mainly open-ended, giving informants the ability to explore their experiences and views. Prior to each interview, information regarding its purpose and participant rights and responsibilities was provided, including permission to audio-record the interview, to which all 38 informants consented. The experts were then briefed about the interview flow and set of questions. The interview guide contained three sets of questions. The goal of the first set was to collect background and expertise information, and the goal of the second set was to identify the participant’s current stance regarding their experiences with AI-ALS. I identified the various problem areas with which the participant was associated based on their experience using current AI-ALS tools. The third set of questions was used to understand the implementation status of AI-ALS at universities and the reasons why AI-ALS would (or would not) be implemented at full scale. Thus, the interview guide focused explicitly on the design, development, and implementation of AI-ALS in HEIs (Appendix X). Depending on the information obtained during the interview, additional questions were asked for clarification. The focus of the interviews varied depending on each interviewee’s expertise. Each participant was free to ask questions and offer insights to help improve the propositions. Immediately after the interviews, notes were reviewed to identify important points. 3.3.2 Questionnaire An online questionnaire was also used in this research effort. An online questionnaire was implemented due to its efficiency in collecting data during the COVID-19 pandemic. Also, it enabled the researcher to reach the target sample population easily. The major benefit of using an online survey is efficiency and opportunity for participants to respond at their own pace but within a compressed amount of time. The online questionnaire was deployed via SurveyMonkey and took approximately 10–20 minutes to complete. The questions in the survey were articulated based on the findings from the expert interviews. The survey consisted of close-ended questions designed to gather demographic information such as gender, age, current occupation, expertise, and type of institution with which a respondent was affiliated. The survey also contained a list of preliminary propositions, and participants chose those they felt were of the highest priority. The propositions were derived from the literature and interviews. The participants then selected their most important propositions; thus, the responses reflect their perceptions. Statistical significance is not examined in the rating and ranking done by experts. The purpose of the rating and ranking is to allow a better understanding of their opinions on the formulated propositions. The questionnaire started with non-complex questions, then progressed to more complex ones to make it easier to complete. The survey ended with an expression of thanks, and the researcher provided her contact information in case there were any questions. All surveys were sent separately through email to ensure the experts did know who else was completing it. The experts were given at least two weeks to answer the survey, and nearly ten responded within a week. Several reminders were sent to other experts. Some declined to participate due to workload issues, while others did not respond. Two weeks were given to experts to complete this task. In the end, 23 completed the survey. The main challenge of using a survey is to ask the right questions in the appropriate way to obtain the desired information. Therefore, I ran a pilot of the survey to check its content validity; that is, whether the measures used in the survey covers all of the content in the underlying construct. Thus, based on the pilot and feedback received, I restructured the questions to improve the chances of obtaining the desired data. A well-designed survey must meet the following requirements: use of accurate and correct terminologies when wording questions; stating the questions in a simple manner; avoidance of making unwarranted assumptions about the sample population; provision of background information about the study; avoiding double-barrelled questions; choice of suitable answer format(s); and a survey pre-test. Both the expert interviews and online questionnaire were carried out in close cooperation with my supervisors, who were actively involved through the design, data collection, data analysis of the interviews results, and the online survey. 3.4 Data Analysis Methods As described above, the data collected were transcribed interviews and open-ended question responses and statistical data from the surveys. Hence, user insights from both the interviews and additional comments from the surveys were analysed through thematic content analysis. For the survey, rating results were extracted and analysed using statistical analysis techniques like descriptive statistics. Thematic content analysis was used to analyse the data transcribed from interviews and open-ended questions in the questionnaire. A large amount of data was expected to be collected. Thematic content analysis works with the data by organizing it, breaking it into manageable units, coding it, and searching for patterns. The main aim of this type of data analysis is to determine, discern, and verify the themes, patterns, meanings, and concepts present in the data. Thus, user quotes are extracted verbatim and grouped by similarity until a general problem statement can be formed. The expert interviews were transcribed and analysed using NVivo12 and Microsoft Excel. The initial stage of thematic analysis was familiarization, in which data were valued as data per se and related in various ways. During this phase, opportunities and linkages between individuals, data, and existing literature were discovered. The data were reviewed and reread several times, with notes taken on individual items and the entire dataset. In the next phase, all relevant information generated by the experts was logged into a spreadsheet, discussed, and coded. Coding data is a technique that categorizes data and thus identifies codes, which Miles and Huberman (1994) describe as Inferential or descriptive tags or labels for organizing the information collected. Codes were generated on the basis on semantic similarity, and interesting data features were coded systematically and collated with the associated data, as seen in Figure 9. To continue the previous phase’s active process, theme construction was performed. In this phase, similar codes were collated, together with relevant information from the literature review on AI-ALS propositions. Thus, themes were built, moulded, and tested in relation to the five sub-questions. The final phase in the content analysis was revising and defining themes. This phase helped clarify the essence and scope of each theme. All coded data were compiled for each candidate theme and reviewed to ensure that each theme and theme name clearly and comprehensively captured the meaning of the data within it and how those data related to the sub-questions. The final phase produced a set of preliminary propositions and five important themes to consider to increase the implementation of AIEd. These findings are presented in Papers 3 and 5. For the questionnaire, the rating results were extracted and analysed using statistical analysis, such as descriptive statistics. 3.5 Validation of the Research Approach All the data collection consequences were validated earlier in this chapter: study design (interview conduction, survey formulation), and selection of experts. Therefore, I discuss validity issues associated with the research approach in the rest of this chapter. To validate the methodological approach in IS research, it is necessary to address plausibility, criticality, credibility, authenticity, and quality of the dataset. These aims were met by publishing the results and findings of this research in a peer-reviewed journal and presenting them at a conference. According to Benbasat et al. (1987), research data can achieve richness and better accuracy if it is evaluated by multiple researchers. Their data interpretations and evaluations are more critical for supporting or contradicting the findings, thus allowing readers to benefit from the latest innovations or advancements and fulfilling the criticality requirement. I also discussed the findings with my supervisors, interviewees, and colleagues during a research seminar. To ensure the accuracy and consistency of the research, my supervisors guided the development of the interview guide and online survey. These academics have long experience in constructing survey questionnaires in IS fields. I consulted them about how detailed or condensed to make the interview guide and survey, how easy-going the survey presentation should be, and how extensive the research explanations should be. After revising the initial version several times, the survey was finalized for distribution. My role as a researcher was to conduct the study rather than to participate in it. I sent the questionnaires to all the experts, handled enquiries, offered clarification when asked, and sent deadline reminders. As a researcher, I was cautious about the subjective interpretations of the consolidated list of propositions. Instead of simply naming them in the questionnaire, I concisely described the carefully listed propositions to avoid differences in understanding. To ensure the credibility of the research, the propositions, which were largely suggested by interviewees, were evaluated by the survey experts to ensure the credibility of an entire study in terms of validity, objectivity, and reliability. By using multiple participants, data sources, and methods, the data sets substantiate one another. It is important to choose the right research techniques to ensure the quality of any research. The combination and use of the various research techniques detailed in this chapter ensure the triangulation of data. The experts were free to criticize the listed propositions in addition to rating them; they were also able to propose missing information that they deemed essential. Thus, expert feedback on the consolidated list of propositions was received, acknowledged, and reflected on properly, especially when it was descriptive and meaningful feedback. This effort enhanced the credibility of the research findings. Although the sample size of the experts was not large, I found it reasonable to proceed further since the panellists’ expertise in and perspectives on AI-ALS were both demonstrated by their scholarship and employment and crucial to the goals of the study. In both qualitative interviews and the ranking-type evaluation survey, size did not depend on statistical power but rather on group dynamics for reaching saturation and expert consensus. There is no clear definition of an ideal sample size of interviews in the literature, but most scholars suggest between 15 and 50 interviewees. Experts were anonymous to each other but never to the researcher. This offered more opportunities to clarify the qualitative data. Moreover, all experts interviewed were comfortable enough to express themselves in English. Thus, language barriers were not an issue in the qualitative interview. Moreover, the study encompassed participants from around the world involved in research, design, development, and teaching in TML. Therefore, instead of simply obtaining the understanding of a certain group and a specific contextual background, the propositions for designing AI-ALS were evaluated from different occupational kinds of expertise (e.g., developer, researcher, or lecturer) and global viewpoints. The experts recommended and evaluated the propositions based on their views about AI-ALS and their participation in research, teaching, and designing. Thus, a shared or even universal understanding of the evaluated propositions could be used to shape the core principles for AI-ALS design and even its implementation in the learning process. As the number of experts from some countries was small, cultural comparisons were not undertaken. However, the societal and cultural contexts of communities play a vital role in the implementation of AI-ALS. Conducting surveys on a larger scale may generate better results in this regard. Table 5 illustrates how Klein and Myers’s (1999) guidelines for conducting and assessing interpretive IS research were used to evaluate this study. In addition to guiding new researchers, these principles can also be used for evaluating research methods after they have completed research. Each of these principles will be discussed in light of this study, in order to determine how applicable, they are. 3.6 Conclusion This chapter has articulated the research design for this study in terms of approach and strategies of enquiry and research methods. The research approach chosen to develop the propositions for designing AI-ALS is interpretive and uses qualitative expert interviews and survey as its research methods. Qualitative interviews with experts and surveys are established methods of generating data in the IS field. The knowledge base from which the data were drawn are interviews with experts from the field and issues identified in the current literature. This chapter shows that AI-ALS and IS researchers in general should adopt a pragmatic approach for finding workable solutions to the domain’s practical problems. Instead of employing quantitative (e.g., surveys, literature review) and qualitative (e.g., interviews, field trips, observations) strategies individually, researchers should blend them to obtain better results in data collection and analysis. By validating the study’s research methods, this chapter has demonstrated the suitability of the chosen research design in conducting IS research to craft propositions for designing AI-ALS. 4 Presentation of Findings This chapter presents results based on the five papers that comprise the core of this dissertation by summarizing the findings and contributions of each. The papers address specific RQs in order to achieve their aims and report related findings. This section briefly describes these findings. As a conclusion to this chapter, the connection between the papers and their contributions to the overall project is explained. An overview of the five papers is provided in Table 6, and the full texts are provided in the Appendix. The papers are arranged according to their relevance to the dissertation, not by publication year. 4.1 Paper 1 – Identifying Research Problems and Concerns in Contemporary Learning Environments Kabudi, T., Pappas, I., & Olsen, D. H. (2020). “Systematic Literature Mapping on AI-Enabled Contemporary Learning Systems”. AMCIS 2020 Proceedings. Article 4. the linked source Summary: The first paper maps recent literature to understand contemporary learning environments, which are digital contexts in which students learn and current technologies shape students’ expectations and their capabilities in acquiring, manipulating, creating, constructing, and communicating information. Good examples of recent contemporary learning environments are adaptive learning systems, intelligent tutoring systems, and recommender systems. However, few examples of these learning interventions have been identified as having been implemented in real-world contexts. Moreover, updated research on the newest generation of AI-enabled adaptive learning systems is lacking. The purpose of the paper thus was to map recent literature and present the summarized findings of research-related problems and concerns in contemporary learning environments. A systematic mapping of the literature on AI-enabled adaptive learning systems was performed, and 122 studies published between 2014 and 2019 were analysed. Findings: This paper explains the concerns and issues that exist in the 21st-century learning environment in and thus need to be addressed. These challenges include learning isolation, difficulty in attaining learners’ skills, backgrounds, profile of student issues, inappropriate information load, design issues, and personalization of information. Some challenges such as student disengagement and poor student motivation were addressed by AI-enabled learning interventions such as adaptive learning systems. However, there is still a small number of implemented interventions designed to address major concerns such as difficulty in attaining learners’ skills, backgrounds, and profiles of student issues. Hence, there is a need to address this research gap. A summary of interventions for AI-enabled contemporary learning systems is provided. 4.2 Paper 2 - Recent Research, Research Gaps, and Future Directions in the Field of AI Adaptive Learning Systems Kabudi, T., Pappas, I., & Olsen, D. H. (2021) AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Education: Artificial Intelligence, 2, Article 100017. Summary: The second paper aimed to provide a better comprehension of AI-ALS by highlighting recent research, the research gaps that remain, and future directions in the field. Previous literature reviews regarding AI-ALS concentrate on the existence of these systems (du Boulay, 2019; Moreno-Guerrero et al., 2020), technological trends in and approaches to adaptive learning, targeted outcomes like student performance and identification of personal traits, and how AI and ML techniques are integrated into learning systems. However, they do not examine the implementation status of AI-enabled learning systems and whether they are fully deployed to address the challenges that students face. Thus, to better understand the status quo of AI-enabled learning systems, the paper maps the recent literature and presents the findings related to the use of such systems. A total of 147 studies published between 2014 and 2020 (an extension of the previous dataset) were analysed. Findings: The paper reports recent research by presenting types of AI-enabled learning interventions, examples of these systems, the aims of AI-enabled learning systems, and the AI and data analytics techniques employed. It identifies research gaps and provides insights in three main areas. The first is a visualization of the co-occurrences of authors associated with major research themes highlighted in AI-ALS. There is a discrepancy between what an AI-enabled learning intervention can do and how it is used in practice. Arguably, users do not understand how to use such systems to their full extent, or such systems do not actually overcome complex challenges in practice, as the literature claims. Therefore, this is a research gap that needs to be addressed. The topic analysis based on themes is useful for identifying what areas of concentration related to AI-enabled learning systems have and have not yet been addressed to a sufficient extent. The second area is a matrix of the types of AI-enabled learning interventions, which shows that problems still exist; for example, there is difficulty in attaining learners’ skills and issues related to students’ backgrounds and profiles. The third area involves analytical methods, their accompanying techniques, and how they are used in AI-enabled learning systems. The topic analysis based on AI-enabled learning interventions is useful because it helps identify the problems to which AI-enabled learning interventions have been applied and those that have yet to be addressed. These areas are discussed in detail in Section 4 of the paper. 4.3 Paper 3 – Identification of Practical Benefits and Challenges of AI Implementation in Education Kabudi, T. (2022). Artificial Intelligence for Quality Education: Successes and Challenges for AI in Meeting SDG4. In: Zheng, Y., Abbott, P., Robles-Flores, J.A. (eds) Freedom and Social Inclusion in a Connected World. ICT4D 2022. IFIP Advances in Information and Communication Technology, vol 657. Springer, Cham. the linked source. Summary: The third paper is an empirical study focusing on areas in AIEd that need to be addressed to improve its implementation. The recent wave of technological innovations in education is based on the application of AI. The United Nations has acknowledged the role of AI in pursuing its SDGs, specifically SDG4, which deals with education: “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”. AI is already changing the education sector. Students now have the ability to find information at their fingertips through educational software and reactive products such as Leapfrog, Amazon’s Siri, and Google’s Alexa. AI offers unprecedented opportunities to humanity and thus helps solve some educational challenges, such as expanding the availability of education, making learning more interactive, and personalizing learning. Despite its potential to provide quality education, AI applications in education raise significant concerns about equity and inclusion, ethics, job loss, and algorithmic bias. Moreover, research reports such as UNESCO (2021) have connected AI and SDG4, but little research has been done on the evidence-based implications of AI in schools and universities. With AI in the education field evolving rapidly, issues such as the integration of AI-ALS systems in real educational contexts need to be addressed. Hence, more research is needed to understand how to increase the adoption of AIEd. The goal of this paper was to understand how to increase AI implementation in education by identifying the practical benefits and challenges that must be retained and addressed, respectively, if AI is to be harnessed to provide quality education. Twenty-two interviews were conducted with AIEd technological experts who are knowledgeable about the design and development of AI-ALS. Findings: The findings of this paper identify the practical benefits of implementing AIEd. It also stated the challenges that must be addressed if AI is to be successfully harnessed in education. Five major themes emerged from the interviews: 1) the role of teachers in AIEd, 2) the inclusion of students with intellectual disabilities, 3) racial and data bias in AIEd, 4) design issues of AI-enabled learning systems, 5) and commercialization of AI-enabled learning systems. The paper discusses these themes in detail, supported by quotes from the experts interviewed for the paper (Section 4). Based on these themes, the study provides important insights and recommendations for future research for practitioners and educators who are interested in using AI for education. 4.4 Paper 4 – Deriving propositions for designing AI-ALS from the Literature. Kabudi, T. (2021). Identifying Design Principles for an AI-enabled Adaptive Learning System. In PACIS 2021 Proceedings, Article 26. the linked source Summary: The fourth paper establishes the empirical foundation of the research by elucidating design requirements and generating propositions for designing such systems. According to the literature, AI-ALS designs are inspired and facilitated by advancements in cognitive theories, big data analytics, learning analytics, AI, and educational data mining techniques. They are also guided by learning theories, motivation theories, metacognition theories, pedagogies, measurement theories, social theories, and most recently by design theories. The fundamental design characteristics of these systems include a customized user interface that handles the interaction between students and the learning system, tracking students’ goals and progress, monitoring and inferring students’ internal state (cognitive, emotional, physical, behavioural, etc.), observing and deducing the external state of the learning environment, monitoring feedback, and facilitating adaptation. However, while the literature shows that numerous adaptive learning systems have been modelled by AI, the underlying propositions that guide the design, development, and implementation of AI-ALS are not clearly known and have not been sufficiently investigated. Thus, the purpose of the study is to establish a set of propositions that would guide the design, development, and implementation of AI-ALS to serve certain purposes and contexts in a university. A systematic literature review of principles and guidelines for AI-ALS was carried out. From a set of 224 retrieved articles, 16 papers published in the past five years were analysed in depth. Findings: The paper presents five design clusters that comprise a total of 24 propositions. They were clustered to better understand the relevance of the different kinds of underlying principles found in the results. The design requirements and principles were based on the design and presentation of the system, the learning content, learning assessment, data processing in the system, and the personalization of learning. Another interesting insight from the paper is the AI-ALS design concerns and requirements that scholars believed were necessary to address. In the first papers, I highlighted concerns and issues in the learning environment that should be addressed. In an attempt to address such concerns, several propositions were presented to address those overlooked challenges and concerns. These include the difficulty of students attaining necessary learning skills, outdated and complex models in systems, background and learner profiles issues, and engagement issues. Selected challenges and propositions that address them are presented in paper 4. Paper 5 - Deriving propositions for designing AI-ALS from Expert Interviews Kabudi, T., Pappas, I., & Olsen, D.H. (2022). Deriving Design Principles for AI-Adaptive Learning Systems: Findings from Interviews with Experts. In The Role of Digital Technologies in Shaping the Post-Pandemic World (pp. 82–94). Springer. the linked source Summary: The fifth paper reports on another empirical study by deriving design requirements from expert interviews and then formulating propositions that meet these requirements. As noted in regard to the other papers, the potential and importance of such systems is well established; however, AI-enabled learning interventions and applications, especially AI-ALS, remain largely at an experimental stage. In addition, there is still a gap in the AIEd research as far as providing evidence-based propositions and support for AI-ALS, even with the rapid advancements in AI technology. Thus, the lack of evidence-based propositions for designing AI-ALS applications affects its large-scale implementation. With AI evolving rapidly in the education field, issues such as the integration of AI-ALS systems within real education contexts need to be addressed. Thus, to further advance AI-ALS in education, the article narrows the gap between experimental research and practice by establishing a set of propositions for design and development of AI-ALS. Twenty-two interviews were conducted with experts knowledgeable about the design and development of AI-ALS Findings: In the fifth paper, 13 features and functionalities that worked well (F), five features and functionalities that had issues (C), and 11 purposes of building AI-ALS (P) were presented. It should be noted that there were 23 Fs that worked well, 14 Cs, and 21 Ps identified as design requirements after several rounds of coding and deep analysis. The functional requirements include the 26 Fs and 14 Cs emphasize the features and functions AI-ALS should have and perform. The 21 Ps comprises non-functional requirements and emphasize performance characteristics of AI-ALS; that is, what the system is intended to do, and how it is supposed to help. The findings from the interviews revealed new design requirements that had not emerged in the literature review process. Combining the requirements derived from experts and those from the literature led to 30 propositions, in the form of preliminary propositions. The term "preliminary design principles" in the paper refers to propositions based on the requirements gathered, with each preliminary design principle addressing at least one requirement. The term does not refer to the design principles term as it is used in design science research, but simply intends to assist future readers within our community to better understand the statements provided as propositions that may be further evaluated and developed as supportive design principles for AI-ALS using a design science approach. Therefore, supportive design principles should address at least one or more design requirements in the paper. As the body of propositions was constructed from multiple sources, the list of propositions needed to be synthesized and cleaned up, with duplicated removed, to produce a coherent list. Thus, the propositions were evaluated to focus on specific recommendations that should guide the design and development of AI-ALS. The evaluation of propositions is an essential step in developing such constructed artefacts in IS research. Various non-structured filters such as comparing names, descriptions and semantic meanings were deployed. This was done to mitigate misunderstanding and eliminate redundancy. The iterative process of reviewing the propositions was done by the candidate and her supervisors, who are subject matter experts in the area. During the review and elimination process, each proposition was examined as to whether it aligned with the descriptions and definitions of AI-ALS. If a proposition was considered to not relate to the definition of AI-ALS, it was removed from the list, and if a proposition did not contribute to the description of AI-ALS, it was not considered. This process was applied to all 30 propositions, to standardize naming, content, and meaning. Despite the diligence with which the most appropriate descriptions were selected, no extensive analysis of the coherence of the collected definitions was conducted at the time. The synthesis was conducted based on the assumption of scientific accuracy from my supervisors. Through this iterative process, the list was reduced to 13 AI-ALS propositions. It should be noted that another round of interviews was conducted after the first 22 experts. This was done not only to gain perspectives from lecturers with experience in TML environments but also to further shape the design knowledge that informed the propositions. Thus, 13 preliminary propositions for designing an AI-ALS based on these results were formulated. A table of those 13 preliminary propositions and their requirements is presented in the paper. 4.5 Additional Findings 4.5.1 Expert Rating and Ranking of the 13 Preliminary Propositions. To further investigate the importance of each proposition for designing AI-ALS, a survey was conducted. The survey was circulated to the 38 experts selected during the expert interview phase and to other experts in the AI-ALS domain to ensure the propositions’ relevance to the field. The other experts were identified using the same process that identified the first group; the information provided in their publications from the literature review, their Google Scholar Profiles, and their employment positions. In total, 23 experts accepted and completed the survey, 13 of whom were from the initial pool of 38 experts. Thirteen of the 23 were male, and 10 were female; they ranged in age from 26 to 80 years. Their profiles are presented in table 5 in the Methodology section. Experts were asked to pick the five most important propositions that they consider in successful AI-ALS design, development, and implementation. They were also asked to rate the 13 preliminary propositions by giving each a score between 1 and 10 (1 = least important, 10 = highly important). The experts did not rate and rank in manner that can allow the identification statistically significant differences, but rather to provide their perspectives and opinions on these propositions. In the course of selecting the five most important propositions for AI-ALS design, development, and implementation, experts often chose personalized and adaptive feedback, learning analytics, automated assessment, students' skill mastery, responsible AI, and recommender and adaptation mechanisms. The results are depicted in Figure 10. These propositions were also frequently chosen by experts when they were asked to rate them according to the rate of importance, as seen in Table 7. Table 7 shows the propositions identified by their grouping rating level of importance and their choice on the most important propositions for a successful AI-ALS design, that is High Rating (highly important) and Low rating (least important). The two sets of reviews helped rank the propositions by importance. Moreover, qualitative justifications, comments on the propositions, and highlighting relevant missing aspects were collected from the experts. I included qualitative questions to obtain expert opinions about the propositions for designing AI-ALS. Three had comments in terms of the formulation and description of the propositions as a bit vague. Specifically, several propositions seemed to be not clear enough to the experts, including sustainable design, actionable information, and affecting learning model. Game-based learning, early warning model, and human in the loop were also considered not as clear as they could be. This may explain why some of the propositions, such as sustainable design and the early warning model were rated as low as they were. The game-based learning proposition led to several questions and comments from the experts: “What kind of games? How long are they played? What do they capture data on, and how is that data used?”; “‘AI systems should have games’ is an empty statement, and it’s impossible to judge.” Another expert noted that “proposition 5 of Games-based learning significantly depends on the age of the students who will use the program.” As to the affecting learning model, I was advised that the proposition could explore the kinds of personalities people have and how students learn. Moreover, it was suggested that the actionable information one be removed or amalgamated into the proposition of learning analytics. One expert stated that “proposition 12 of teacher-AI complementarity is not obligatory; it depends on the purposes. Maybe it’s better to say flexibility of design with the possibility to embed or modify the structure of the program depending on the pedagogical goals.” Additionally, it was suggested that an additional proposition to do with involving students in the co-design of systems should be considered. Two experts also recommended that the proposition focusing on an adaptive, attractive, and user-friendly interface be added or integrated into the sustainable one. Thus, the propositions of early warning model, game-based learning, actionable information, affecting model, human in the loop, teacher-AI complementarity, and sustainable design were recommended for revision so that clear definitions could be provided. Thus, based on the literature review, expert interviews, and the evaluations of the formulated propositions, nine were ultimately considered relevant and important to the successful design and development of AI-ALS and useful in promoting its adoption in educational settings. These propositions are as follows: automated assessment; personalized and adaptive feedback; learning analytics (actionable information was integrated into this proposition); students’ skill mastery; recommender and adaptation mechanisms; ethics and fairness (responsible AI); human-centred AI (instead of Humans in the Loop; teacher-AI complementarity and sustainable design are incorporated into this); early warning (instead of early warning model); and affective domain (instead of affecting learning model). Detailed descriptions of the evaluated propositions, together with their implications, are provided in Chapter 5. 4.6 The Overall Story of the Dissertation To conclude, the published articles and the additional findings constitute the overall research story presented in this dissertation. The papers were built on one another to achieve the desired research goal of narrowing the gap between experimental research and practice by establishing a set of propositions for designing AI-ALS. Each paper made a specific contribution. From a broader perspective, Papers 1 and 2 determined the state of contemporary learning environments, including AI-ALS. The two papers identified research gaps, problems, and concerns in the field of AIEd and TML environments in general. To the best of my knowledge, few, if any, scholars have adopted this kind of reflective perspective to understand the requirements and propositions for designing propositions of successful AI-ALS. Moreover, few studies have investigated how to increase the awareness, adoption, and implementation of AIEd. Addressing the lack of evidence-based propositions for designing AI-ALS in Papers 4 and 5 and the implementation issues of the systems in Paper 3 led the project further towards developing propositions for designing AI-ALS. Papers 1 and 2 are therefore considered the foundation of this study. Paper 3 examines AI-ALS implementation by highlighting its practical benefits and challenges. Various themes and recommendations discussed in Paper 3 emphasize the relevance and value of the propositions formulated in Paper 5 and the need to address the challenges outlined in Papers 1 and 2. In this study, the empirical foundation is laid in Papers 4 and 5, which elicit design requirements by rigorously reviewing the literature (Paper 4) and interviewing experts (Paper 5). Thirteen preliminary propositions were derived from the accumulated findings of the papers. Those preliminary propositions were then evaluated to determine their relevance to and importance for the successful design and development of AI-ALS. Nine propositions were ultimately considered. Figure 11 summarizes the contributions of the published articles. 5 Discussion of Findings This thesis investigates and establishes a set of propositions for designing AI-ALS. In this chapter, I discuss the findings by using Gupta and Bostrom’s model to further understand and provide a comprehensive view of AI-ALS, a mediating tool for adaptive learning, in terms of enhancing the entire teaching–learning process in a real educational setting. 5.1 Propositions for AI-Enabled Adaptive Learning Systems Over the course of this research study, nine propositions for designing AI-ALS, that can ensure high adaptive learning success when using AI-ALS, were identified and evaluated. Overall, the propositions provide design guidance for systematic design, development, and integration of AI-ALS in educational scenarios by identifying the requirements of the entire AI-ALS process. The design requirements include features and functionalities that worked well in AI-ALS, features and functionalities that had issues, and purposes for building AI-ALS (Paper 5). These requirements helped in crafting propositions that comprises design information for adaptive learning delivery by AI-ALS that ensures learning success by exploiting the potential of AI. Thus, the major contribution of this research was to come up with propositions that provide design guidance. The table in the Appendices provide an overview of the identified and validated propositions, together with their specific characteristics and the challenges they address. To sum up, I incorporated findings from both research and practice to design AI-ALS in a more efficient way and increase its learning success. 5.2 Providing Insights Using TML Theoretical Model As highlighted in the first chapters of this dissertation, the critical aspects of the entire teaching–learning process can be overlooked in experimental research. There is a need to have a comprehensive understanding of AI-ALS to enhance the entire teaching–learning process in an actual educational setting. Thus, in this section, I draw on Gupta and Bostrom (2009) to understand AI-ALS as a mediating tool for adaptive learning. The TML model proposed by Gupta and Bostrom was identified as appropriate for understanding the findings of the study around issues like the design of AI-ALS in terms of the components and functionalities and features that should be incorporated in it. I argue in Chapter 2 that the AST theoretical model for TML is most appropriate to the use of contemporary learning environments such as AI-ALS. Other IS theories identified in Chapter 2 are widely used to investigate the design of and user satisfaction with online learning environments (. However, these IS theories largely focus on the use of online learning environments and thus do not capture all the systemic complexities in advanced learning environments such as AI-ALS. Moreover, IS researchers also suggest that to understand a technology-based phenomenon such as TML, theoretical models need to include all the elements of a social-technical system: technology and learning techniques, processes, actors, actions, and outcomes, all of which are crucial elements in AI-ALS. There also has been a lack of empirical focus in both the IS and education literature on the learning process. This is due mainly to the lack of good constructs for examining the learning process from both structural and process perspectives. AST, particularly appropriation, provides these constructs. Thus, I offer insights and understanding using Gupta and Bostrom’s TML model. As detailed in Chapter 2, AI-ALS are more advanced forms of TML. According to Gupta and Bostrom’s model, TML systems are comprised of the following components: learning methods structures, learning processes, and learning outcomes. The present study found that learning through AI-ALS should fit students’ schedules and provide flexibility and adaptive learning process for each learner, thus allowing them to craft their own learning plans. Different strategies that students use to appropriate content and how these strategies change as skills develop are important in the system. With AI-ALS, “the student has the ability to follow the recommendations of the system or move through the course on their own. The system indicates to the student the need to repeat the materials at the right points in time” (excerpts from interviews). As noted above, the system assesses each student’s cognitive state, as well as affective, behavioural, and motivational states based on the student’s interaction with the learning system. Thus, it is crucial that P9 be considered in the design of AI-ALS. Moreover, P3 plays a major role by holding that AI-ALS should describe students’ data, capture logs of students’ activities, analyse, plan, and provide actionable insights, and thus help predict suitable activities for each student. One of our experts’ statements agrees on this point: “It is important to include the ability to store as detailed and granular information of the user experience as possible for the future ways of adapting and doing learning analytics.” Because sensitive data and logs are captured about students and their learning activities, there must be assurances that any data collected will not be used unethically. Thus, P6 is relevant and (should be considered in the design of AI-ALS learning method structures), as it underscores fairness, security, transparency, privacy and explainability. P2 relevant and (should be considered in the design of AI-ALS) learning method structures because it emphasizes that AI-ALS must provide the best way to give feedback to a specific student. One expert highlighted the importance of P2 in the system: Students like that they get personalized feedback, right?... There was this student said [that] the only feedback we get is when we look into a system at the end of the year.... That's the only feedback we ever get... and that's not good.... And so, our idea was to give them feedback throughout the learning process.... And as personalized as possible. One of our experts offered an example of how this worked with their system: “You have students’ written texts, so students are writing texts, essays.... And the idea is then they are supposed to provide peer feedback and, in this process, we also give them individualized feedback on the way they argue.” Thus, I conjectured that P2 (personalized and adaptive feedback), P3 (learning analytics), P6 (ethics and fairness [responsible AI]), and P9 (affective domain) are important to learning method structures, as they can influence the topic that is to be taught. The study also found that learning techniques are usually driven by the learning goal – the knowledge expected to be attained – together with a given level of expertise. So, for instance, AI-ALS may initially provide simple tasks such as multiple-choice questions at a novice stage before moving on to more advanced tasks like real-life projects at the expert stage. The learning techniques are supplemented and enhanced in AI-ALS by curriculum sequencing (also referred to as instructional planning technology). This method provides the student with a sequence of knowledge units to learn that is best suited to his or her learning goals and existing knowledge. It also plans the sequence of learning tasks (examples, questions, problems, etc.) to fill out the sequence. The intent of AI-ALS is to determine what a student really knows and to move students logically and appropriately through a sequential learning path to prescribed learning outcomes and skill mastery. These systems achieve this aim by helping address learning challenges such as varying student learning ability, diverse student backgrounds, and resource limitations. Specific features and functionalities of AI-ALS that have been identified in both the literature and these findings (such as a sequential learning path to prescribed learning outcomes and skill mastery) will lead to better course progression and results. These requirements build up P4, which supports the intention of AI-ALS to support students in acquiring knowledge and skills in a particular domain using various learning techniques. Compared to Gupta and Bostrom’s model, the learning processes in AI-ALS is influenced by process scaffolds combined with students’ individual aptitudes. Scaffolding activities take many forms depending on students’ needs: models, cues, prompts, hints, partial solutions, think-aloud modelling, and direct instruction. These scaffolds help students solve a problem, carry out a task, master a concept, or achieve a goal. Thus, in addition to the propositions linked to the student model, P8 is relevant, as the information obtained from the early warning model (which detects at-risk students) can influence the scaffolding activities that will be provided to a given student. Because scaffolding helps students with solving problems, they will need feedback on their progress, so P2 is considered relevant in AI-ALS, with regard to scaffolding, as it gradually shifts the responsibility for learning from the teacher to student and thus help students be more independent. One of the experts stated how scaffolding is crucial in AI-ALS: We looked at delivering different kinds of cognitive and affective supports to students. So, some students were given worked examples. Some students were given physics animation videos that show the principles behind how these different physics constructs work. We gave some students like Rube Goldberg videos to watch, which don’t really have a ton of physics content but are entertaining. And our preliminary results from that are showing that there are differences in student learning, in terms of which scaffolds you give. So, the first takeaway would be for these adaptive learning systems.... Uhm, the kind of scaffolds that you give to a student matter, and you should pick them intentionally. Though the empirical findings did not highlight much about individual aptitudes, the findings from the systematic literature mapping highlighted the importance of motivation, self-efficacy, learning strategies, and other individual aptitudes as important to AI-ALS design and development. In fact, there has been growing use of AI-enabled learning systems and frameworks to address poor student motivation and enhance the effect of self-efficacy on learning outcomes. In addition to motivation and cognitive abilities, I proposed adding racial, gender, and cultural diversity. Based on the empirical findings from Paper 3, algorithmic bias, racial data bias, and culturally sensitive design issues can be addressed by including the attributes of the affected group (such as race and gender) and should be part of AI-ALS development processes. Gupta and Bostrom (2009) highlighted the importance of the AST perspective to examine other contextual factors that were not captured in their model but might influence learning process appropriation. They identified race, class, cultural diversity, and power as among the variables that could be investigated. Other research also supports the view that culture, experiences, and social context should be added and considered in the background profiles of students in the student model. P3 learning analytics (which focuses on collecting students’ information) and P6 ethics and fairness (which addresses algorithmic bias) are considered relevant for designing AI-ALS as they can influence learning appropriation by including the additional proposed aptitudes. In Gupta and Bostrom’s (2009) model, the circular relationship between learning method structures and learning process components is implemented in AI-ALS by most researchers as the adaptation engine or as adaptation mechanisms in the pedagogical portions of AI-ALS. This relationship acts as a bridge between the student/domain models by combining students’ needs and characteristics with the learning materials. This element determines the next activity the system will provide to the student based on that information exchange between the two components. Thus, the adaptation model is involved in selecting the topic, identifying objectives, sequencing them, and presenting them to meet the student’s needs until the student achieves mastery. As described throughout Chapter 2, AI plays a major role in this part of the system. AI techniques combine learning methods structures and learning processes (the human–machine interface, student, pedagogical, and domain parts of AI-ALS) to identify and recommend the instruction delivered to each student. With AI-ALS, “the student has the ability to follow the recommendations of the system or move through the course on their own. The system indicates to the student the need to repeat the materials at the right points in time” (interview excerpt). In addition, “it is important that adaptiveness, adaptability, and recommendation help the student with his or her knowledge gaps” (interview excerpt). I therefore conjectured that P5 (recommender and adaptation mechanisms) would be relevant to the design of AI-ALS, especially in this part of the system. Thus, P4, P5, P8, P2, P3, and P6 are for designing and developing AI-ALS, especially with learning process component as they influence the entire learning process. As discussed above, AI-ALS are designed to dynamically adjust to the level or type of course content based on an individual student’s abilities or skill attainment in ways that accelerate the learner’s performance with both automated and instructor interventions. New approaches to diagnostic and formative assessment based on AI and adaptive technology are becoming more common in general and are an important element of AI-ALS. P1 (automated assessment) and P2 (personalized and adaptive feedback) emphasizes on the facilitation of learning outcomes in the system. Both propositions showcase ability of AI-ALS to continually assess students’ knowledge, guiding them to progress through a course efficiently and effectively (P1). In addition, the system should provide students with immediate and corrective but encouraging feedback (P2). The study also found that AI-ALS are dynamic, as they continuously integrate the information from students’ interactions in the student models to drive the adaptation. Studies by Ennouamani and Mahani (2018), Martin et al. (2020), and Ofelia San Pedro and Baker (2021) prove this point. This concept of continuous loops of feedback and assessments in AI-ALS is based on students completing their tasks; they receive positive or negative feedback and adjust their actions accordingly. Every time the teacher offers feedback and the learner makes a correction, new feedback is required. This cyclical and recursive process reaches an end when only learning stops. Technology in AI-ALS can facilitate the provision of timely, specific, and ongoing feedback. In addition, most AI-ALS require students to engage in reflection activities as part of the feedback loop. Further, Gupta and Bostrom (2009) describe the singular linear progression of learning that occurs in other TML forms such as blended learning environments; however, those approaches lack the continuous feedback, communications, and assessments that occur during the entire adaptive learning process. The model fails to illustrate the dynamic nature of AI-ALS and the cyclical and recursive feedback process. To address this shortcoming, Figure 12 depicts the TML model that includes the dynamic nature of AI-ALS (i.e., continuous assessments and feedback loops within the system and between students and teachers). Figure 12 also illustrates the positioning of the identified propositions for design on the various AI-ALS elements. As stated in the Background section, with an interpretive perspective, I use the TML model as an approach to explain and understand how to design better AI-ALS as mediating tools for adaptive learning. The study is less concerned with exploring or testing the theoretical relationships among the elements (propositions) in the TML model. The aim rather is to improve how AI-ALS are designed and developed. Thus, the positioning indicates the relevance of these propositions as they depict certain features and functionalities that are important in designing and building AI-ALS for use in real educational settings. For instance, P7 (human-centred AI) is positioned in the learning methods structures and near the IT component because of its influence on the human– machine interaction. Since students learn with the system, P7 ensures that the system continuously learns from human input from both students and teachers and improves as a result, even while providing an effective experience. In addition, since the findings of this study emphasize the role of teachers in AI-ALS to support student learning, P7 advises designers and developers of AI-ALS to involve teachers and students in co-designing such systems, along with co-creating students’ learning goals. The positioning of these propositions is not to replace or to be compare with the propositions in Gupta and Bostrom’s model, since they do not describe relations among elements or concepts in the model. Rather, Figure 12 draws from the work of Gupta and Bostrom (2009), and combined with the findings described in this thesis, that aims to explain how adaptive learning happens. In detail, I show how AI-ALS are employed as a mediating tool to enhance adaptive learning, and how teachers and students interact with such technology. Future research can apply these propositions to design better AI-ALS and also use Gupta and Bostrom's model to test the newly designed systems. The results of both of these studies can contribute to formulating AI-ALS design principles. 6 Contributions and Implications of the Study This chapter discusses the contributions and recommendations of the dissertation. Based on its empirical findings, it offers implications for scientific knowledge, research methods, and practice. All the theoretical and practical contributions are outlined in this chapter. 6.1 Theoretical Contributions First, in addition to using a theoretical model to understand the findings of this study, I have advanced the field by conducting a systematic mapping of the literature on AI-ALS to understand and synthesize the research contributions on that topic. The mapping identified research gaps and provided insights in three main areas. The first is a visualisation of the co-occurrences of authors associated with major research themes highlighted in AI-enabled learning systems. The visualization helped identify prominent themes in the field of AI-enabled learning systems and demonstrate how they are connected to one another. The second area concerns the types of AI-enabled learning interventions and the problems these interventions have and have not addressed. The third area involves analytical methods and their associated techniques and how they are used in AI-enabled learning systems. Based on the research gaps identified, I defined my study’s scope (Paper 1 and 2). This study provides an overview of AI-ALS design, development, and implementation, which will be valuable for the IS research community. Moreover, this study highlights that AI-ALS research is dominated by quantitative methods and experimental research. Researchers have grouped individuals into experiment scenarios, determined their responses to laboratory conditions, and used averaging to extrapolate a general conclusion. The limitation of using this research method is overlooking critical aspects of the entire teaching– learning process. It is essential to have a comprehensive understanding of AI-ALS to enhance that process in real-world educational settings. Thus, I applied an interpretive approach with two different research methods. First, I conducted an exploratory qualitative study using the expert interview technique to explore the implementation status, benefits, challenges, perspectives, experiences, and design requirements for AI-ALS. This, together with findings from the literature, helped shape preliminary propositions for designing AI-ALS. I then conducted a ranking-type evaluation survey to prioritize the most important propositions for successful AI-ALS development. I found this combination appropriate to provide the richest possible understanding of the topic by identifying core themes and their interrelationships. The structured questionnaire also gave experts the opportunity to suggest factors and recommendations that had been missed in the list of preliminary propositions. By using an interpretive approach, I was able to obtain a more comprehensive picture of the requirements and propositions for a successful development and implementation of AI-ALS in educational settings. Research studies could further test and refine these propositions into AI-ALS design principles based on these propositions. 6.2 Contribution and Recommendations for Practice The findings of this study provide a foundation for making practice recommendations. Concerns, issues, and challenges were highlighted in the empirical Papers 3 and 5 (and even the literature reviews papers) that impede the use, adoption, and implementation of AI-ALS. Based on the findings in the papers, the study provides recommendations. for systems developers, educators, and practitioners seeking to implement AI-ALS in education and improve pedagogical processes and outcomes through adaptive learning. First, the empirical findings obtained using expert interviews show the dissatisfaction of most teachers with using AI-ALS and AIEd in general. Most teachers have a sense of being replaced and have negative attitudes towards these contemporary tools. Thus, it is important for AI-ALS designers and developers to partner with teachers in crafting and implementing such systems in their classes. Involving stakeholders like teachers throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. A good example of such a partnership is offered by Holstein et al. (2019); the authors present a detailed case study of the iterative co-design of Lumilo, a wearable, real-time learning analytics tool for teachers working in AI-enhanced K–12 classrooms. The case study illustrates how nontechnical stakeholders can participate meaningfully throughout the design process of a complex tool by giving end-to-end demonstrations and offering methodological recommendations. For instance, designers should centre initial discussions around stakeholder needs rather than specific analytics, visualizations, or other technical considerations. This will help designers to better understand teachers’ values and the nuances of the contexts in which AI and adaptive learning system technologies will be used. This connects with the proposition of human-centred AI, in which co-design is recommended. Another example is ASSISTments, an online math tutor that provides timely student assessments and instruction. ASSISTments is an ecosystem of a few hundred teachers, a platform, and researchers working together. Development professionals help train teachers and encourage teachers to participate in studies. The platform and the teachers help researchers simply by using the content the teacher selects. The platform, hosted by Worcester Polytechnic Institute, allows teachers to write individual ASSISTments (composed of questions with answers and associated hints, solutions, web-based videos, etc.) or to use pre-built ASSISTments, bundle them together in a problem set, and assign them to students. Second, both the empirical findings and recent reviews of AIEd research have consistently emphasized the lack of socio-technical and educational perspectives in AIEd research, development, and implementations. Thus, to further advance AIEd technologies such as AI-ALS, the most important initiative may be to ensure partnerships and collaborations among computer scientists, IS researchers, educational researchers, and psychologists, and to integrate theoretical, conceptual, practical, and empirical support from various disciplines. Interdisciplinary research with IS and educational researchers will more likely result in feasible actionable principles or guidelines. Policy-makers and educational stakeholders should collaborate to analyse the data from multiple AI-ALS deployed in different schools to ensure strategic oversight and provide the kind of real-world feedback that can be integral to success. One expert emphasized during the interviews to open part of the design process to broader sets of disciplines and let people from different academic disciplines work on resolving issues arising in the development and implementation of AI-ALS. In addition, to reach the full potential of AI-ALS in education, collaborative research focusing on AI technology applications that could result in direct or indirect effects on learning outcomes in real educational settings is particularly vital. Thus, one way to facilitate the wider adoption of AI-ALS in education is to facilitate communications and collaborations amongst stakeholders with different areas of expertise (e.g., technological skills vs. learning theories and pedagogies) and from different perspectives (e.g., technology advancement, teaching and learning, administrations of educational systems, educational research) and thus lead to fruitful collaborations in AIEd research, development, implementation, and evaluation. Designers, engineers, and programmers could collaborate and share resources in building AI-ALS; this could help developers use less time creating these systems and support their scaling. A good example of collaborations and partnerships is provided by Dziuban et al. (2018), who describe an adaptive learning partnership involving the University of Central Florida, Colorado Technical University, and the adaptive learning provider Realizeit. Although the institutions are quite different, both use Realizeit as their enterprise solution for adaptive learning, and cooperative research across various institutional demographics has helped inform the study and development of adaptive learning in education. Another example is the Smart Sparrow system, an adaptive e-learning software platform. Smart Sparrow emerged from the University of New South Wales’s Adaptive eLearning Research Group. Intelligent tutoring systems and educational data mining research projects from the Adaptive eLearning Research Group focused on intelligent tutoring systems authoring tools, cognitive load theory, multimedia instructional design, and methods to boost instructors’ pedagogical ownership of intelligent tutoring material. The novel use of Smart Sparrow in the biomedical sciences was piloted in the School of Medical Sciences at the University of New South Wales and was incorporated into the development of resources for the Biomedical Education Skills and Training Network, which is a collaboration among researchers from universities across Australia. Within a few years, Smart Sparrow had been widely adopted in Australia; it was acquired by Pearson in 2020. This study, through Paper 3, has highlighted cases of contemporary learning systems, such as AI-ALS, that were designed for one ethnicity but were used for another. Thus, such systems are sometimes not adopted simply because of cultural design aspects. A lack of insufficient data regarding other ethnic groups not only impedes the provision of quality education for all but also affects AI-ALS implementation. It is important to understand both the context and the purpose of a system where it was originally built and to understand the context where AI-ALS is to be integrated and implemented. From the empirical findings presented in this thesis, a proposed solution is to involve researchers from anthropology. AIEd is already an interdisciplinary research area integrating computer science, learning sciences, psychology, neuroscience, linguistics, and other disciplines. Thus, anthropology would be a good addition, given that its core competency is sensitivity to multiple cultures. It will also be important for personalizing digital technology in the future, because if the system in general and specific learning materials do not resonate with students and instructors in terms of culture, then it is unlikely to succeed at any meaningful level of consistency. For instance, the Squirrel AI learning system that was built for the Chinese market promotes extracurricular provision and exam preparations for students (due to that country’s intense cultural, parental, and competitive pressures to succeed in school); it might not appeal to markets like the United Kingdom that have radically different contexts. Moreover, the findings highlight the importance of resolving the existing algorithmic bias in AI applications. Most experts indicated the need for the right data to be used in algorithms to ensure that they deliver reliable outcomes, both in terms of knowledge tracing and in terms of predicting behavioural, cognitive, and affective outcomes. A common critique of AI and the learning analytics field more generally is that it is predominantly white and male. Thus, a lot of the algorithms that are produced excel at predicting people who are white and male but tend to struggle with other groups. Developers and designers should be more willing to think intentionally about the different kinds of people who are or may be within their systems. This will lead to a better chance of producing positive educational outcomes and make it more likely that AI-ALS will be widely adopted. It is preferable to develop algorithms based on groups of people rather than on individuals. It is more effective to find meaningful statistical differences at the group level and personalize around that information than to personalize down to the individual level. Programming for four or five different groups is much easier and makes it easier to evaluate outcomes. In addition, algorithms should be tested to determine how they affect specific groups of people (such as those defined by race and gender) before deployment. Furthermore, members of communities affected by algorithms should be involved throughout the development process and the use of algorithms in education. As AI-ALS are adopted and integrated in educational environments, it is imperative to balance benefits, ethics, and security. Scholars have already started conversations about AI ethics and the need for responsible AI, but what is really required is developing specific ethical protocols for AIEd. The use of AI-ALS in education can not only promote learning effectiveness and augment the human intelligence during the learning process but may also raise potential ethical issues such as digital hegemony in education, power relationships among learners, teachers, and AI systems, and the digital divide in general. Likewise, privacy is a critical issue that has yet to be fully addressed in the AIEd context. It is unclear in most educational situations who owns the data that are generated by a system. The issues of ownership of data, students’ privacy rights, privacy protection more generally, data processing integrity, system reliability, and security all need to be carefully addressed. A recent semi-systematic evaluation of 22 AI ethics guidelines revealed that they have serious flaws and that several ethical standards that are critical for AI research, development, and implementation are actually missing entirely from such guidelines. The urgent need for AIEd ethics also calls for collaborative efforts from all stakeholders, including educators, administrators, researchers, technology innovators, and even representatives of society to seek possible solutions from a variety of aspects, including technological solutions such as creating a constraint module in AI and policy solutions like establishing principles and ethical codes for the use of AIEd ( Furthermore, the findings show the need to build systems that take account of all players. Designers and developers need to look at education as a complex ecosystem in which there are both human teachers and human learners rather than just thinking about and building a system for learners or a system for teachers. Generally, learner and teacher both operate in a classroom context, and understanding that context is important. Education is above all a social interaction between teachers and learners, so it is essential to see AI-ALS in education not only as a scientific and technological enterprise but also as a complex social interaction. IS research in this field is important because it focuses on socio-technical design, which moves beyond system properties, functionalities, and features to include the human elements. The lack of training of both teachers and students in using AI-ALS was identified as a challenge in this research. The empirical findings revealed a tendency to simply implement learning systems and give them to students without spending much time, if any, on training those students on the uses, benefits, and potential dangers of AI-ALS. Many students do not understand the advantages of adaptive platforms and how they can make the best use of them. It is thus necessary to train the main users (instructors and learners), as adaptive learning involves a very different mindset than traditional learning. In one adaptive learning system partnership example, the Center for Distributed Learning supplied instructors with expertise and training. This centre also has a large number of instructional designers to help out with the implementation and the kind of heavy work that is needed. The belief is that instructors need a lot of training and knowledge on how to navigate and select the right technology for the right students at the right time for the right type of learning. One of the recommendations to address this challenge was that such teacher training could be integrated into the teacher education system; this might help with the negative attitude and resistance towards AI-ALS and other contemporary learning systems. The empirical findings of this study also suggest that HEIs a staff of specialists who are able to work with data and AI algorithms and correctly apply these technologies in helping to manage the educational process. There are no such specialists on the labour market right now; they need to be trained and prepared. For AI-ALS to be fully adopted, integrated, and implemented in education contexts, there must be active buy-in and investment from HEI administrators and responsible governmental authorities. In the example of the partnership between the University of Central Florida and Colorado Technical University, support at the institutional level was provided in terms of time and money to train instructors to use the adaptive learning system. Governmental authorities and HEI administrations should help by supporting students and teachers with the technology through instructional design support. Developing and designing AI-ALS involves substantial investments of time, finances, and human assets. Physical assets are infrastructure shared across HEIs and the applications using that infrastructure. Human assets include the knowledge and skills possessed by human resources and the AI-ALS team. Therefore, it is recommended that administrative and financial support be provided to support AIEd implementation. AI-ALS designers and researchers should not only highlight the benefits to students and teachers but also how these systems are beneficial to the administration, management, and the education sector as a whole. In China, for instance, it is the government that promoted AI and drove the proliferation of the use of AI-ALS in education as a way for the country to progress. Finally, both expert interviews and the literature review revealed that most educational institutions and research still have not moved beyond their first AI experiments and pilot projects. Progress is slow at most institutions because implementing AI depends on both technical and organizational factors – and few resources exist to help leaders plan and strengthen the organizational foundations to prepare for full-fledged AI-ALS implementation. Thus, organizations need the flexibility and time to change how they think and operate to take advantage of AI-ALS and AIEd in general. Experts have advised that HEIs gradually implement AI-ALS until they become mature enough that students, teachers, and other stakeholders actually see the systems as something they cannot do without. It will take time for institutions to attain AI maturity, and there should be collaborative efforts from all stakeholders, including educators, administrators, researchers, technology innovators, and representatives of civil society in coming up with AI maturity frameworks for education. Those frameworks should be designed to help administration, management, and the relevant governmental authorities to understand and prioritize the actions that will have the greatest impact on AI-ALS in their unique contexts. Table 10 summarizes the practical recommendations in this thesis. 7 CONCLUSION This dissertation is one of the few IS studies to investigate AI-ALS design, development, and implementation, this concluding chapter summarizes its main findings. It also notes limitations and challenges before offering recommendations for future research. 7.1 Summary This thesis explores the design and development of AI-ALS and contributes to our understanding of AI-ALS implementation in educational settings. AI-ALS innovations emerged unexpectedly (ahead of most people’s full awareness) and involved high-level, abstract discussions, making it difficult to understand where we are and where we are heading. A variety of reactions have been seen in higher education: some have ignored these technologies, while others have embraced them quickly. Thus, a larger, more inclusive conversation between HEIs, vendors, and other stakeholders is needed to clarify the minimal requirements for AI-ALS applications. In addition, there is increasing market ambiguity and growing concern about the possibility of a “black box” in AI-ALS. This leads to a simple question: what does the next step in truly promising AI-ALS look like? IS research must lay the groundwork for the key design requirements that will enable AI-ALS to achieve their full potential and deliver on their rhetorical promises as more market capital is used to fuel and accelerate their development. This dissertation contributes to this effort by establishing a set of propositions for designing AI-ALS. The main research question that guided this study was as follows: How should AI Adaptive Learning Systems be designed and developed?” To address the main research question, I first reviewed the literature on AI-ALS to understand the research contributions to the topic and define the scope of this study (Papers 1 and 2). Many propositions are derived based on prior research, expert observations, and expert statements. Thus, I developed supportive propositions after identifying issues in a systematic review of the current literature (Paper 4) and then coded and analysed the experts’ interviews to derive meta-requirements (Paper 5) and formulate preliminary propositions for the design of AI-ALS. As I have highlighted in Chapter 4, I developed those preliminary propositions based on the meta-requirements, with each proposition addressing at least one of those requirements. The formulated preliminary propositions were evaluated by experts through a ranking-type evaluation survey to ensure that the propositions were clear in terms of goals, contexts, and mechanisms and based on the actual relationships between AI-ALS elements. In the end, nine propositions were formulated. The propositions for designing AI-ALS and their characteristics will allow researchers and practitioners to design, evaluate, and compare AI-ALS more effectively. Building on these propositions, it is now possible to theorize about how different technological embeddings of the still new field of AI-ALS in education affect student learning outcomes in each pedagogical scenario and task. This study develops propositions, derived from literature and practice, that can facilitate the empirical evaluation and testing of requirements/ recommendations into future new design principles for AI-ALS. Thus, the study contributes to the design of AI-ALSs based on an interpretive approach to ensure a socio-technical perspective on this still-emerging technology. The study provides researchers and practitioners with requirements and propositions to design their own AI-ALS and help them ensure that user manipulations are based on such perspectives. Especially as AI and ML continue to advance, design knowledge about AI-ALS might encourage designers and research towards a more socio-technical design of these novel IS. By employing systematic procedures in this research, I aimed to generate a satisfying design contribution. I also aimed to fill the identified research gaps by providing deeper insights into AI-ALS and presenting to both practitioners and researchers outcomes that can contribute to better user acceptance and experience to be expected from a user-centred design of AI-ALS. I believe that further empirical evaluation and instantiation of the generated design statements will help contribute to the AST model for TML in IS. I therefore hope to encourage designers to focus more on a socio-technical AI-ALS design. 7.2 Reflection on the research The goal of this Section is to reflect upon and share reflections concerning the entire research. These reflections include intangibles that do not necessarily belong to the scope of scholarly conclusions but are nevertheless important to discuss. The personal experiences and opinions are formulated below in a subjective way. I will give an account of some of the limitations of the research and make some theoretical and empirical reflections. 7.2.1 Limitations of the research A few limitations were identified during the research process. One is the lack of a formalized theoretical framework during the initial phase of propositions selection. Differences in experiences and backgrounds of the paper authors may have led to different outcomes. The results from the exploratory interviews are limited in their generalizability. Other experts might have given different answers that would have led to other requirements and propositions. Therefore, propositions alone cannot guarantee success. The AI-ALS propositions were abstracted and derived in order to provide a holistic design perspective. In order to achieve this, a certain level of abstraction was accepted. Individual AI-ALS domains and classes still have to determine how best to apply the propositions to their specific use cases. It would be beneficial for future research to provide empirical insights into the effects of specific propositions and instantiated design features on perceptions of AI-ALS. It is important to be cautious when generalizing to users from different cultures and contexts. During the research process, it was noted that terminologies were understood differently, especially when it came to ranking. For instance, although “early warning model” and “human in the loop” are well known and have been described in literature, some experts indicated confusion about and misunderstanding of these terms. This made the comparison, validation, and screening of the identified propositions more difficult than desired. Based on an extensive and systematic analysis of related literature and interviews, the meta-requirements and propositions were developed in October 2020 and evaluated at the beginning of 2022. In light of this, the experiences presented here may differ for the period following data collection. It is also possible that some important works were missed by limiting the search databases and the selection and combination of keywords. There may be a lack of generalizability of research directions. Some articles may have been discarded unjustifiably based on filtering criteria, so crucial factors may have been overlooked. It would thus be possible to obtain somewhat different results by employing new analysis techniques. As this research was part of my doctoral program, I had to meet predefined deadlines. Because of this, it is possible that I was not able to capture all effects at the right time. Research on AI-ALS may longer to perform a chain of operations that includes literature analysis, theory development, empirical testing, system modelling, requirements analysis, prototyping, evaluation, and system development. It is understood that the number of participants is a limitation of this study. There is a possibility that the sample size selected will not accurately represent all AI-ALS designers, developers, and researchers. Moreover, the number of experts for the ranking-type evaluation may impair the validity of the study. There was a relatively small number of people on the evaluation panel for this study. Experts were difficult to reach, their priorities and willingness were limited, and there was a relative time window for research contributions. Although my supervisors approved the survey, many participants found it lengthy and time-consuming. English content was reported to be difficult for some practitioners. By attracting a larger number of participants, one could examine the topic in greater depth and explore more avenues that might lead to somewhat different outcomes. In addition, the results may have been biased by the experts’ understandings, attitudes, and perceptions. It is possible that the validated results might differ if the panel were extended to other expert groups. Additionally, it is recommended that the proposed propositions be tested empirically, even though they were developed based on a good flow of information. The final limitation is treating AI-ALS in educational institutions as homogenous. It might be possible to obtain more detailed findings by observing institution size, ownership, or sectoral differences with a greater degree of granularity. 7.2.2 Reflections on theoretical and empirical aspects The inspiration for conducting this study was due to the lack of transferable insights on AI-ALS design and development. Thus, this research aimed to codify the existing knowledge on the systems and provide practical design statements that can be applied to better design AI-ALS. I gathered these statements through academic writings and through reflective interviews with experts. The knowledge gained was presented as AI-ALS design principles in papers 4 and 5. Most researchers in the field, when designing information systems, will employ Design Science Research (DSR), Action Design Research (ADR), or Design-Based Research (DBR). This involves the design and development of an IT artifact as a solution to a problem, which is evaluated and through iterations of reflection and learning can lead to the generalization of the solution to other related problems, via the development of design principles. During my work it has become clear that to formulate design principles the design and development of an IT artifact is required. However, this was not the case here. Instead, I conducted a more traditional interpretive study to codify knowledge on the design and development of AI-ALS, and to make it available as propositions. Thus, I used a reflective meta-analysis strategy to formalize knowledge and arrive at a more general level of propositions for designing AI-ALS. For conceptual clarity, the term "propositions" was employed in this thesis (to avoid confusion). This word, I feel, better captures the claims presented as propositions for creating improved AI-ALS, and so addresses the research gap investigated. The derived propositions from literature and practice could perhaps provide some food for thought for future IS designers in the design science field while creating these systems. Moreover, the model in Figure 12 based on the inspiration from Gupta and Bostrom’s TML model was designed to be easily understood and to offer a comprehensive view of AI-ALS in terms of enhancing the entire teaching–learning process in a real educational setting. However, one has to note that there are always challenges and weaknesses in developing such a simplified model. The theoretical representation can give the impression that AI-ALS is that simple, but this is not necessarily the case. The empirical findings of this study have illustrated how an AI-ALS is complex and dynamic in nature. This underlines the necessity of considering these models as a simplified depiction of different choices and not as an absolute representation of "reality". Expressing conceptualizations in such simplified models is useful for communicating findings, but they should also be judged for what they are: my reconstruction of other people’s constructions of different models. 7.3 Future Research The limitations and reflections outlined above all offer opportunities for future research. This section proposes some further potential research avenues through which scholars can concentrate on enhancing and advancing the work in this field. By understanding the important aspects during the design, development, and implementation processes of AI-ALS identified in this thesis, researchers can use it as a foundation for further examinations of AI-ALS adoption in educational settings. The focus of the thesis was to better understand AI-ALS implementation in HEIs. While the potential and importance of such systems are well documented, the actual implementation of AI-ALS and other AI-based learning systems in real-life teaching and learning settings has not lived up to that promise. There is a need for more empirical studies on AI-ALS to provide solid evidence that these advanced systems should be used further in education. Moreover, most educational settings have only used AI-ALS for teaching languages and programming. A further study of the readiness and capabilities of HEIs for AI-ALS is therefore required, as is the application of these technologies to a variety of purposes, such as the teaching of other courses and the support of other technical or IS skill attainment. AI-ALS can be used in different ways according to their capabilities, but in practice there is a discrepancy between them. As the literature suggests, users do not understand how to effectively use such systems or do not overcome complex challenges in practice. The research gap that exists here must be filled if AI-ALS are to reach their full potential in practice. As another avenue for further research, it would be beneficial to evaluate the latest iteration of propositions in terms of usefulness, satisfaction, and efficiency. The ranking presented here was only of a relatively general list of propositions. This latest version incorporates feedback gathered from the survey and supervisor workshops, providing an opportunity to link evaluation with a real-world task for designers to field-test the propositions. Evaluating the propositions in a real educational setting, where students and teachers interact with systems based on the propositions and following their behaviour over time, would provide vital insights into the long-term effects of design decisions. Moreover, future research can apply the proposed propositions to design better AI-ALS that caters for a complex educational setting. In addition, one can use Gupta and Bostrom’s model to test the newly designed systems. Both of these studies' findings can help to develop design principles for AI-ALS. Another opportunity would be conducting and using more data sources (e.g., case studies) to triangulate a more comprehensive look into AI-ALS in educational environments. Future research should also include institutions of various sizes and from different countries to gain insights that incorporate different institutional and cultural perspectives. Apart from this, applications in other sectors might be considered, leading to new findings and potentially refining the results presented here. Finally, the expert interviews study together with the evaluation results offer several future research topics. The identified propositions provide possibilities for quantitative studies to test the relationships between the core issues and other influencing factors and capabilities. A better understanding of the identified propositions could be gained from longitudinal studies that examine adoption processes over time by focusing on the AI-ALS life cycle in education. Appendices. Appendix A: Documentation 1. Table of Propositions. 2. Invitation Letters to Study. 3. The Interview Guide. 4. The Survey. educational progress. User Goals (Students) support working and learning process of students with immediate feedback. Challenges Addressed Slow feedback; inconsistent feedback; poor student motivation. Identified by (Expert Category) Designers Researchers Experts 2, 11, 13, 16, 22, 24, 3. Learning Analytics. No. and Name Functionality of understanding, optimizing learning, predicting, and providing actionable insights. Justification to make informed, data-driven decisions. 115 User Goals it also provides students with better information on how they are progressing. (Teachers & Students) Challenges Addressed Student dropout; Student engagement and motivation issues; Poor performance. Identified by (Expert Category) Designers Researchers Experts 1, 2, 11, 14, 16, 17, 24 Experts 12, 13, 19, 22 4. Students’ Skills Mastery. No. and Name Functionality path and help master more advanced skills (Mastery Learning). Justification accurately and rationally through a sequential learning route to the specified learning goals and skill mastery. information. User Goals Give students who have not grasped the material more time and chances to go over what they missed. 116 Challenges Addressed motivation; engagement issues. 5. Recommendation and Adaptation Mechanisms. No. and Name Functionality of difficulty of questions. Justification User Goals goals, background knowledge, and motivation. 117 7. Human-Centred AI. No. and Name Functionality and AI-ALS. Justification development rather than viewing AI automation as a replacement for human agency and control. a kind of triangle in which each agent informs and is informed by the others. Challenge Addressed Misuse of AI due to algorithm bias and a lack of governance Identified by (Expert Category) Designers Researchers Expert 1, 2, 10, 11, 14, 16, 24 Experts 5, 12, 13, 20, 21, 22, 28, 29 8. Early Warning and Early Detection. No. and Name Functionality AI-ALS should have an early warning model that correctly identifies at-risk students. 119 Justification The datafication of education has made it possible to create automated tools that can find patterns in vast quantities of educational data to infer previously unknowable facts and behaviours about their students. This proposition focuses on developing precise predictive models to recognize students who are in danger of failing or quitting a course or programme. By reducing the time between identification and actual at-risk state, this proposition may lower students’ risk of failure or disengagement. With the help of the proposed prediction models, numerous categorization techniques have been analysed. Some of the techniques used include decision tree, naive Bayes, support vector machine, logistic regression, hierarchical mixed models, k-nearest neighbours, neural network models, and Bayesian additive regressive trees. To make the predictions, these models can use a variety of data types. Numerous variables and features have been investigated, including continuous assessment results, user-generated material, demographic data (age, gender, ethnic origin, marital status, etc.), and AI-ALS data. User Goals Through various dashboards, the key stakeholders (i.e., students and teachers) can analyse the data, and teachers can also provide early feedback as a form of intervention to lessen high-risk circumstances. As a result, it is easier to keep students on pace and support their own learning. Challenges Addressed Students’ risk of failure; lack of enthusiasm; disengagement Identified by (Expert Category) Designers Researchers Expert 10, 11, 22, 24 Expert 12, 19, 20, 22 9. Affective Domain Support within the System. No. and Name Functionality AI-ALS should have an affective model that measures and predicts emotional reactions of students and uses the feedback to improve learning experiences Justification It is believed that students’ emotions, such as enjoyment, boredom, interest, bewilderment, annoyance, satisfaction, disappointment, pride, and worry, impact on their academic performance by influencing their involvement in and attitude 120 2009; Wu et al., 2019). User Goals Challenges Addressed Identified by (Expert Category) Designers Experts 2, 6, 10, 14, 16, 24 Invitation to short interview on AI-enabled learning systems Dear Name, My name is Tumaini Kabudi, and I am a PhD student at the Department of Information Systems, University of Agder, Norway. My supervisors are Prof. Ilias Pappas and Prof. Dag Håkon Olsen. In the context of a research study, I am interviewing lecturers and professors to better understand their perceptions of modern learning systems, such as those enabled by artificial intelligence and machine learning. Your experiences as a lecturer with experience teaching in a technology-rich learning environment will benefit my Ph.D. Thus, I would like to invite you to a short interview (~30 minutes). During the interview, we will discuss your experience and perspective on how we can better design the learning systems of the future. If you have any questions, please do not hesitate to ask. Looking forward to your positive response. Many thanks in advance. Kind regards, Tumaini Mwendile Kabudi, PhD Research Fellow CEDIT-Department of Information Systems University of Agder the linked source Title: Invitation to short interview on AI-enabled learning systems Dear Name, My name is Tumaini Kabudi, and I am a PhD student at the University of Agder, Norway. I very much enjoyed reading your paper in....... journal/conference proceedings while performing a systematic mapping of literature on AI-enabled adaptive learning systems that has been published in Computers and Education: Artificial Intelligence (the linked source). Your work has been very useful to me and my PhD. I would like to invite you to a short interview (~30 minutes) as part of my PhD research study. During the interview, we will discuss your experience and perspective on how we can better design the learning systems of the future. If you have any questions, please do not hesitate to ask. Looking forward to your positive response. Many thanks in advance. Kind regards, Tumaini Mwendile Kabudi, PhD Research Fellow CEDIT- Department of Information Systems University of Agder the linked source The Interview Guide Demographics 1. Tell me about your background (do not forget age and gender). 2. What you do in your work? How long you have been working in this. profession? 3. What is their expertise? a. Different types of expertise. It could be on the design and development of such systems (front- and/or back-end), on the topic (learning), etc. Experience in the Field and with the System 4. What kind of learning systems are you using at your university? 5. Have you used an AI-enabled learning system (adaptive learning system. intelligent tutoring system, recommender system) at your university? a. If yes, describe how you are using that system (for what purpose?) – Their experience of how to use the system b. If not, are you planning to employ such a system? • How likely? • For what purpose – perhaps give an example? c. If not, why are you not using an AI-enabled learning system? 6. What do you think works well with AI-enabled learning system and what. does not? a. In terms of features of the system (What are the most liked features of the system you are using? What are the least useful features of the system?). b. In terms of functionalities. c. What do you like or dislike about the system? d. In what scenarios does the system work well and not? Implementation of an AI-Enabled Learning System 7. If they have used the system. a. Is the system widely available / implemented at a large scale? • If yes, how did you get it to be implemented at full scale, widely available, adopted on a large scale? • If not, what do you think it will take for the university to implement it at full scale, make it widely available, or adopt it at a large scale? b. Did you experience any challenges in the implementation process, and how did you handle those challenges? 8. If they have not used the system. a. What will it take for the university to implement AI-enabled learning systems at full scale, make them widely available, or adopt them on a large scale? b. What do you think are the most significant benefits when implementing and using an AI-enabled adaptive learning system? c. What do you think are most challenging elements of implementing an AI adaptive system? Bonus Questions (If Time Allows) Any recommended improvements for the system? Expert Evaluation Survey of Design Principles for AI-ALS The purpose of this study is to determine the relevance, usefulness, robustness, and importance of the formulated design principles (Propositions) for Al-enabled adaptive learning systems (AI-ALS). AI-ALS are generally digital learning tools enabled by Al that adapt to the learner so that the learning process is optimized and/or student performance improves. Recent AI-ALS include Smart Sparrow, Knewton, LearnSmart, Connect, and DreamBox Learning. The formulated Propositions were obtained from a review of the literature on AI-ALS, together with interviews conducted with almost 40 experts in the field of Al and ALS. Please respond to each question, indicating your level of agreement as to the importance of Propositions to the successful design, development, and implementation of Al-ALS. Each question is followed by a five-point Likert scale ranging from “strongly disagree” to “strongly agree.” You may make comments to justify your rating, question the formulated Propositions, or elaborate on the concept. You may also suggest additional Propositions and identify Propositions that need to be revised and rewritten in the final comment box. The data from this survey will be treated anonymously. The expected time of completion is approximately 15 minutes. You have two weeks to respond to this survey. If you have any additional information that you would like to share, you can use the free-text input field at the end or email me at the email address. Thank you again for participating in this survey. Your input is important. Regards, Tumaini Kabudi PhD Research Fellow Dept. of Information Systems University of Agder Appendix B: Research Publications 1. Paper 1: Kabudi, T., Pappas, I., & Olsen, D. H. (2020). “Systematic Literature Mapping on AI-Enabled Contemporary Learning Systems”. AMCIS 2020 Proceedings. Article 2. Paper 2: Kabudi, T., Pappas, I., & Olsen, D. H. (2021) AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Education: Artificial Intelligence, 2, Article 100017. 3. Paper 3: Kabudi, T. (2022). Artificial Intelligence for Quality Education: Successes and Challenges for AI in Meeting SDG4. In: Zheng, Y., Abbott, P., Robles-Flores, J.A. (eds) Freedom and Social Inclusion in a Connected World. ICT4D 2022. IFIP Advances in Information and Communication Technology, vol 657. Springer, Cham 4. Paper 4: Kabudi, T. (2021). Identifying Design Principles for an AI-enabled Adaptive Learning System. In PACIS 2021 Proceedings, Article 26. 5. Paper 5: Kabudi, T., Pappas, I., & Olsen, D.H. (2022). Deriving Design Principles for AI-Adaptive Learning Systems: Findings from Interviews with Experts. In The Role of Digital Technologies in Shaping the Post-Pandemic World (pp. 82–94). Springer. Contents lists available at ScienceDirect Computers and Education: Artificial Intelligence journal homepage: the linked source AI-enabled adaptive learning systems: A systematic mapping of the literature Tumaini Kabudi, Ilias Pappas, Dag Håkon Olsen University of Agder, Kristiansand, Norway 1. Introduction. Technology has had a significant impact on higher education in-flipped stitutions (HEIs). In fact, virtual reality classrooms and technology-enhanced learning systems have been used in recent years in many HEIs. Technology-enhanced learning uses learning and teaching systems that are technology based, allowing students to develop knowledge and skills with the help of lecturers, tutors, learning support tools and technological resources. The importance of such systems, especially in the times of a pandemic, has been highlighted further due to their ability to assist IT and IS educators, while they rethink and revise the learning design of their courses, in order to offer more meaningful learning experiences to their students. Students also play an active role in the learning process using these technologies. Currently, the most commonly used learning systems include Blackboard, Moodle, Web CT and Canvas. The advantages of utilising such learning systems include constant availability and accessi-bility to course materials, cost savings, collaboration amongst students and lecturers, improved performance, feedback from users and effective communication. Despite these advantages, most learning systems tend to focus on achieving their technical objectives and ignore course requirements and other pedagogical issues related to the whole learning–teaching process. Due to the dominance of the technical aspects of these learning platforms, students and lecturers perceive them as not adaptive to their needs, resulting in their negative attitudes toward these systems. Hence, more advanced learning systems have emerged in recent years. Progress in using new data analytics and artificial intelligence (AI) techniques to develop learning systems has led to the development of more successful learning systems in the education sector. These contemporary learning platforms are ‘systems that strive to incorporate analysis of historical data about the previous users of the system by modelling learning process [es] from the learners’ viewpoint, and, thus, be able to adapt to a rapidly changing environment by providing learners not only accurate and high-quality learning material, but also taking into account the individual learner’s needs’. Increasingly, AI-enabled learning systems are being integrated with new techniques to develop more personalised educational settings. Such systems are gaining traction due to their ability to 2666-920X/© 2021 The Author(s). Published by Elsevier Ltd. deliver learning content and adapt to individual students’ needs. Students are thriving in these digital environments, where current technologies shape their expectations and ‘abilities to access, acquire, manipulate, construct, create and communicate information’. The physical and virtual resources in these learning environments are designed to deliver effective learning by helping students construct their knowledge. Good examples of AI-enabled learning environments include intelligent tutoring systems, adaptive learning systems and recommender systems. An intelligent tutoring system ‘uses techniques of artificial intelligence to model a human tutor in order to improve learning by providing better support for the learner’. Recommender systems are ‘software tools based on machine learning and information retrieval techniques that provide suggestions for potential useful items to someone’s interest’. Adaptive learning systems are personalised learning platforms that adapt to students’ learning strategies, the sequence and difficulty of the task abilities, the time of feedback and students’ preferences. These platforms encourage students to monitor their learning journeys via automated feedback cycles within the systems, allowing them to progress independently of the course instructor. AI-enabled learning systems have been developed based on research on AI (intelligent tutors), learning analytics and educational data mining techniques. The rapid advancement of these systems has been facilitated by the influence of AI in the education field. Indeed, in the education sector AI has helped to provide personalised feedback and support to students through the above-mentioned systems. It is predicted that there will be a growing number of technology-enhanced learning environment studies that will apply AI in education. The application of AI in the educational field has brought new prospects for the design and development of better technology-enhanced learning systems. AI-enabled learning systems offer numerous benefits, including an improved learning experience, time flexibility, the provision of timely feedback, flexibility in managing students’ learning experiences and faster student progression. Due to the capabilities and benefits of these systems and their huge potential to transform the education sector, many companies have begun to invest in AI. It is esti-mated that 1047 billion US dollars were invested in AI-based education from 2008 to 2017. The current literature reviews regarding AI-enabled learning systems concentrates on the existence of these AI-enabled learning systems; technological trends and approaches in adaptive learning; targeted outcomes, such as student performance and identification of personal traits; educational fields and disciplines that are involved in AI-enabled learning systems; and how AI and machine learning techniques are integrated into learning systems. Studies have also examined the potential use of AI techniques to improve existing learning systems; the pedagogical deployment of these AI-enabled systems, such as intelligent tutoring systems; and the technologies being deployed, such as virtual reality (VR). However, these reviews did not examine the implementation status of the AI-enabled learning systems and whether they were fully utilised to address students’ challenges. There are few studies of AI-enabled learning systems implemented in educational settings. Thus, the implementation of these systems in education settings seems to be in the infancy stage. As Verdú et al. (2015) stated, ‘Many of these learning systems as well as Intelligent Tutoring Systems are described in the literature, and their effectiveness has been proven. However, these systems are rarely used in real educational settings practices in ordinary courses.’ The problem remains, and recent studies highlight the lack of successful AI-enabled learning systems, such as adaptive systems, implemented in practice. Thus, in an attempt to better understand the status quo of AI-enabled learning systems, our study maps the recent literature and presents the findings related to the utilisation of these systems. The significance of using systematic mapping analysis instead of other types of literature analysis, such as bibliometric analysis, is its unique characteristic of analysing literature in a wide area. Further, systematic mapping generates new knowledge through meta-analysis of the existing knowledge published in the field. In recent studies, bibliometric analysis has been used to analyse a wide range of research issues with a large-scale dataset. This technique is particularly useful for better understanding ‘what has been investigated in the past and further make predictions about what will happen in the future’. Studies that have used bibliometric analysis have identified the performance of the scientific production of AI in the education field, the evolution of AI in the field, keywords associated with AI-enabled learning research, geographical distributions, the most incident/cited authors in the area and the historical trends. These studies, however, had a notable lack of evidence concerning the potential association between certain problems faced by students and lecturers and AI-enabled learning interventions that solve these problems. This systematic mapping study highlights such an association. In relation to the association, our systematic mapping analysis identifies AI-enabled learning interventions, challenges, and potential future research topics in this field. This study also sheds light on the significance of utilising AI-enabled learning systems in educational settings. We hope that the findings of this research provide practitioners and researchers with insights into AI-enabled learning systems, especially in terms of how they are being utilised to address several challenges faced by the students who use them. The rest of the paper is organised as follows. First, Section 2 introduces the systematic mapping process applied. Section 3 presents the results of the research. This is followed by section 4, which discusses the findings from the retrieved literature. Section 5 highlights the contributions of this study. Finally, the limitations of the study are discussed in Section 6, followed by the conclusion of the paper. 2. Methodology. This study was conducted using the systematic mapping guidelines proposed by Petersen et al. (2015). Systematic mapping is a survey method that is used to ‘give an overview of a research area through classification and counting contributions in relation to the categories of that classification’. Systematic mapping is useful for analysing properties of the research papers in a certain research field. Compared to other types of content-based analysis, such as bibliometric analysis, systematic mapping is unique in creating a map of a wide research field. Bibliometric analysis is a popular literature analysis technique that aims at providing quantitative assessment and evaluation of academic outlets in a particular research area. Systematic mapping is concerned with structuring a research area and identifying gaps in knowledge. Another unique characteristic of systematic mapping is answering general research questions that aim to discover research trends. Systematic mapping studies have been used by many researchers in this field of AI in education. In our case, we employ systematic mapping as the most appropriate method to capture what has been researched in the field of AI adaptive learning systems and to identify knowledge gaps. The systematic mapping process comprises three major phases (i.e., planning the mapping, conducting the mapping and reporting the results of the mapping). The essential steps of a systematic mapping study are defining the research question, conducting a search for relevant papers, keywording, screening of papers, data extraction and mapping. For this process, the researchers utilised EndNote X9, NVivo 11 and Excel spreadsheets to extract publication outlets, find duplicates and organise the information. Planning a well-structured mapping is the first step in conducting any systematic mapping of literature. This step starts with identifying research objectives related to the literature on AI-enabled learning environments. By considering the possible impacts of AI-enabled learning systems, this study proposes three research questions (RQs): ● RQ1: What are the main research motivations and objectives of studies on AI-enabled learning environments? ● RQ2: What are the core research problems and concerns in the field of AI-enabled learning systems and the interventions/solutions proposed to address them? ● RQ3: What are the common AI and data analytics techniques utilised to design the interventions? A protocol was used to guide the overall research method. The study applied both formal and informal searches to identify the above-mentioned research target goals. Previous works published in the past five years were selected to avoid outdated research. After planning the mapping, the next phase involved a systematic mapping of the literature. The first step in conducting the systematic mapping was to formulate the search strategy, which was formulated based on a mapping protocol to reduce research bias. The search strategy was formulated by following and expanding the RQs. Then, the search keywords were identified, and search strings were generated to minimise the number of articles. Synonyms and substitute spellings were also identified. We focused on two main terms of interest to perform database searches: ‘adaptive learning system’ and ‘artificial intelligence’. Two parallel searches were conducted, as the two main terms of interest were sometimes used inter-‘Adaptive ecosystem’, ‘adaptive changeably. learning learning environment’, ‘adaptive learning platform’, ‘adaptive learning setting’ and ‘adaptive learning technology’ were used as synonyms for adaptive learning systems. Further, along with the term ‘Artificial Intelligence’ we included the term ‘machine learning’. These are the two most popular terms when it comes to AI-enabled adaptive learning systems and are typically supersets of other more specific techniques (e.g. data mining, text mining). The Boolean operators OR and AND were used along with these terms. These operators were included to incorporate synonyms and substitute spellings and to connect the keywords and form the final search string, respectively.(see Table 1) This study seeks to capture and map the state of the art in the field, taking into account the vast advancements that have occurred in recent years. To this end, we have limited our search to include articles from 2014 onwards. The search was done on eight databases (i.e. ACM, Web of Science, EBSCO Host, Wiley, SAGE Journals, IEEE Xplore, Scopus and Taylor and Francis). These eight databases were chosen due to their wide selection of relevant and recent articles. The databases included numerous AI-related academic journals, such as Journal of Artificial Intelligence and Soft Computing Research, IEEE Transactions on Pattern Analysis and Machine Intelligence, British Journal of Educational Technology and International Journal of Intelligent Systems. The search was carried out on titles, abstracts, and keywords. A total of 1864 articles were retrieved using the above-mentioned search strategies. To reduce the number of articles, the study underwent further refinement, and several articles were selected based on criteria listed below. This was done to ensure that the selected articles were relevant and answered the RQs. All retrieved documents went through duplicate removal using EndNote software. A total of 1492 articles were retrieved after removing duplicates. All articles that met the inclusion criteria, which considered the title, abstract and keywords, were considered relevant for the study. The inclusion criteria were as follows: A total of 147 papers were included in the study after undergoing the data extraction process. The study selection criteria proposed by Petersen et al. (2015) were adopted to have a standard form to extract data from the chosen articles. EndNote software was used to extract the basic information of the articles, such as the title, authors, year of publication and digital object identifier (DOI). Publication details, such as journal name, publisher, volume, issue, page, abstract and keywords, were also extracted. Then, specific data were extracted from each article for study categorisation. The following data were also extracted: ● Reference type (journal, conference paper, etc.) ● Type of paper based on the research approach classification proposed by Wieringa et al. (2006). ● Common techniques (AI, Machine Learning data mining or soft computing) utilised to design interventions ● Research motivations of these articles ● Type of interventions utilised ● Problems and concerns The required information on whether an article was clearly reported was assigned the value ‘N/A’ in the equivalent cell in the extraction table. The authors created and finalised an Excel spreadsheet after reviewing the primary data extracted (Fig. 1). 3. Research results. This section presents the results based on the analysis of the selected published studies, which were identified as relevant to this study. 3.1. Results overview. In terms of publication channels, 51% of the included papers were scientific journals, and 49% were conference papers published in conference proceedings. The articles were categorised based on the type of research approach used, following Wieringa et al. (2006). The most utilised research approach was evaluation research (43 articles), followed by literature review (32 articles). Validation research and the philosophical approach were third and fourth, with 30 and 22 papers, respectively. The distribution of documents per year is shown in Fig. 2. 3.2. Types of AI-enabled learning interventions. The articles were placed in five categories based on implemented interventions and solutions applied in AI learning environments: systems, frameworks, models, approaches and combinations of interventions. Many of the published documents used a system (adaptive learning system, intelligent mechanism, or adaptive learning platform) as an intervention (61 articles). The other main form of intervention used was adaptive learning frameworks (27 articles). Frameworks are constructs that define concepts, practices, values and assumptions as well as provide a set of guidelines on how to implement the frameworks. Most of the frameworks recommended as solutions in these papers comprised essential elements and features for implementation in learning environments. The proposed items were in the form of AI techniques, user (learner) models and other adaptive techniques. The frameworks highlighted and described the relationships amongst the suggested elements. The coded frameworks provided numerous vital steps and actions for achieving a better adaptive learning experience. Meanwhile, 22 papers utilised models. A model is ‘a pattern of something to be made, a description or an analogy used to visualise and reason about the system to be developed and its likely effects’. The models were either a problem-solving tool, experiment or abstract narrative of a component or system to be designed. Twenty papers explored an adaptive approach as a solution. An approach refers to a set of viewpoints or theoretical concepts applied to understand, explain and solve a problem observed in a particular phenomenon. The distribution of the research papers that utilised the above-mentioned AI-enabled learning interventions, published between 2014 and 2020, is depicted in Fig. 3. The distribution of the articles that used interventions published in conference proceedings and journal articles is illustrated in Fig. 4. 3.3. Types and examples of AI-enabled learning systems. The most identified AI-enabled learning systems in the mapping were Adaptive Learning Systems. Another most identified kind of AI-enabled learning system in the mapping is intelligent tutoring systems. Other categories of learning systems that were identified in this mapping and their examples are highlighted in Fig. 5. The table below highlights the various themes of the designed aims of these AI enabled learning environments. Many of the published papers identified that the AI enabled learning environments were designed to assist with teaching several courses. These courses included mathematics, physics, psychology, nursing, computer literacy and biology. It was also identified that these systems were designed as platforms to teach and learn languages. The identified languages that were taught in these systems include English, German, and Greek. Another category of what AI enabled learning environments were designed to do is improve students’ performance through Personalization of Learning. These systems were designed to act as platforms to provide personalised content based on their level. Also, the AI enabled learning environments are designed to teach and learn programming languages such as SQL and Java. The remaining identified themes are shown in the Table 2. 3.4. AI and data analytics techniques. Various AI and data analytics techniques were identified in our mapping. The graph below shows the frequency of the studies that mentioned or utilised these techniques. The Bayesian networks technique was the most frequently mentioned in these studies. A total of 14 articles proposed, mentioned and utilised this technique in studies involving AI-enabled learning environments. The next most frequently mentioned technique was neural networks (11 studies). Decision trees, genetic algorithms and K-nearest neighbour (KNN) techniques were also identified in this mapping, each with seven studies, followed by Support Vector Machines (SVMs) and Bayesian Knowledge Tracing (BKT) (six studies each). The rest of the identified techniques are presented in the graph below (see Fig. 6). Designed Aims AI enabled learning environments. 4. Discussion of findings. 4.1. Visualisation of the co-occurrences of authors associated with major. research themes in AI-enabled learning studies In this mapping, we identified several major themes, which we grouped according to the purpose and motivations of the research studies (Fig. 7). We visualised the authors’ connections to the main objectives in conducting the selected studies. We chose to visualise the co-occurrences of authors associated with the purpose-related themes to identify the prominent themes and their connections in the field of AI-enabled learning systems. This was done by applying network analysis to a ma-trix of co-occurrences using the corpus analysis platform CorTexT (the linked source). This step allowed the mapping of the papers by clusters (Fig. 7). The papers were numbered and presented as small nodes, while the main themes of purposes were represented by cluster shapes. Clusters of closely associated authors were organised into specific subdomains (groups of highly interconnected nodes), which were instinctively detected by a clustering algorithm and colour-coded accordingly. The clusters provide an indication of topics that were intensely studied by researchers. In Fig. 7, the limits of the clusters are represented by coloured circles, and their surfaces are proportional to the number of small nodes they incorporate. Using the clustering algorithm, all 147 papers were positioned and connected to these themes. As depicted in Fig. 7, the one paper that describes the partnerships between educational institutions in terms of using adaptive learning systems forms cluster PARTN (cluster presented in light green on top right). Then, 22 studies related to redesigning courses to adopt adaptive learning systems or adaptive learning modules form the REDESIGN cluster (green cluster on the right). Next, 61 papers that designed, described, proposed or developed AI-enabled learning systems are connected to the SYSTEM cluster (light orange right below). Twenty papers that aimed to design, develop, identify and propose approaches for AI-enabled learning systems form cluster APPROACH (the dark red in the middle), and 36 studies that proposed and utilised algorithms, mechanisms and AI/ML techniques can be found in the AL-GORITHMS cluster (blue in the lower centre). In addition, 41 studies that presented general or comprehensive literature reviews are grouped in the LITERATURE REVIEW cluster (yellow on the left). Other topic clusters depicted in Fig. 7 can be found in the maroon EVALUATION cluster (29 studies focused on proposing evaluation methods or evaluating AI-enabled learning systems and adaptive courses), FRAMEWORK (orange cluster on the top left with 27 studies that focused on proposing and developing frameworks for adaptive learning and adaptive learning systems) and the MODEL triangle (light green cluster with 22 studies that develop models for AI-enabled learning systems). Interestingly, most of these papers are linked to more than one cluster. For example, SLM32 is connected to the SYSTEM and REDESIGN clusters, indicating that Dziuban et al. (2018) proposed adaptive learning systems and described an institutional partnership between educational institutions involving the use of adaptive learning systems. This is seen in the diagram by cluster overlapping. The proximity between certain nodes and clusters indicates the relatedness and close connections among the identified research themes. Thus, the EVALUATION node is positioned in the APPROACH cluster and close to SYSTEM. This indicates that studies whose main purpose is to evaluate adaptive learning systems are more linked to studies that proposed approaches for AI-enabled learning systems and which designed adaptive learning systems. This is supported by examples of projects and studies in our mapping that have developed AI-enabled learning interventions, such as the PTIME system, Early Recognition System and Yixue Squirrel AI system. More of these should be conducted and published to increase the use of these adaptive learning systems in educational settings. Further, studies that proposed or used algorithms or techniques are closely linked to clusters of studies that aimed to design adaptive approaches and the literature review studies, as seen in Fig. 7 (where ALGORITHMS is between the LITERATURE RE-VIEW and APPROACH clusters). The PARTN node is connected to REDESIGN node only, indicating low relatedness of institutional partnership to the other topics. The largest clusters in our diagram are the SYSTEM, LITERATURE, ALGORITHMS, EVALUATION and FRAMEWORK clusters. This indicates that most AI-enabled learning interventions are systems and frameworks and use algorithms, as shown in Fig. 4. However, some of these designed and proposed systems and frameworks are in their experimental phases and have yet to be used in practice. Hence, these learning systems cannot be easily adopted in real educational settings. If the designed systems or frameworks are not tested, then one cannot understand the consequences of implementing such interventions in terms of their benefits and drawbacks. This factor may have contributed to the fact that these learning systems are not used extensively in real educational settings. The smallest cluster, PARTN, includes only one study on partnerships among institutions to collaborate on using adaptive learning systems. In APPROACH, 20 studies have designed, developed, identified and proposed approaches for AI-enabled learning systems. However, this finding shows that only a few studies have utilised adaptive approaches for AI-enabled learning systems. Another interesting insight from this mapping relates to the number of general literature reviews that have been conducted in the past seven years. However, there are few studies on recent advanced AI-enabled learning systems that have been used as solutions to address more complicated challenges faced by students. Moreover, comprehensive reviews of adaptive learning systems are lacking, especially of those that have utilised modern AI techniques. Several reviews have been conducted, but they are outdated in terms of the application of novel AI techniques. In the current study, we found that AI-enabled learning systems are simply used as platforms for teaching languages and programming courses and for improving performance. However, a few studies on more advanced learning systems have utilised AI to address the design issues of learning systems, evaluation standards or methods for such systems, complex and outdated models of learning systems and personalization issues. In terms of learning systems that are adaptable to the profiles and backgrounds of students, only two systems have been proposed: the LeaPTM system and the Early Recognition System. Appendix A presents several challenges that have few interventions. Most of the authors indicated what these learning interventions (in the form of systems, models, frameworks and even approaches) can do and how they can overcome various complicated challenges found in learning environments. However, several authors in our mapping have shown that most adaptive learning systems in practice are used simply as platforms for teaching languages, programming languages and other courses. Thus, there is a discrepancy between what an AI-enabled learning intervention can do and how it is actually utilised in practice. Arguably, users do not understand how to extensively use such systems, or such systems do not actually overcome complex challenges in practice, as the literature claims. Therefore, this is a research gap that needs to be addressed. The presented topic analysis based on themes is useful for identifying what areas of concentration related to AI-enabled learning systems have and have not been addressed so far. 4.2. Problems and AI-enabled learning interventions. In this mapping, we identified several problems faced by students and lecturers in their respective learning environments, including one of the most common, which is the learning process. Learning process-related challenges include difficulty sharing learning resources, the high redundancy of learning materials, learning isolation and inappropriate information load. Several studies have applied AI-enabled learning interventions to address this concern. One good example is the proposed novel adaptive e-learning model based on big data, which can improve the quality of the learning process by providing the most suitable learning content for each student. This model was designed to address inaccurate and incorrect learning material selection processes in adaptive learning systems. Another example is a personalised adaptive online learning analysis model that analyses the structure of a learning process using big data analysis. In addition, Nihad et al. (2020) proposed a multi-agent adaptive learning system that can collect and detect information describing the learning process of students in a deductive way. This system aims to make real-time decisions and offer students training according to their dynamic learning pace. One study proposed a generic conceptual framework for intelligent adaptive learning systems in order to address the lack of guidance in transferring learning effectiveness to field training when designing such systems. Several concerns, such as poor feedback, have been considered. Bimba et al. (2017) proposed a cognitive knowledge-based framework for adaptive feedback, which combined pedagogical, domain and learner models. Another intelligent model has been proposed, which uses both supervised and unsupervised ML techniques to adaptively select the appropriate learning material for a particular student. Another interesting concern is related to the profiles and backgrounds of students. Existing educational systems utilise standardised teaching methods that do not fit the individual characteristics of each student. This highlights the need to use AI techniques so that learning systems can cater to the distinct backgrounds and characteristics of each student. Several studies have applied AI-enabled learning interventions to address this issue. For example, Tommy et al. (2016) developed an intelligent and adaptive test system to tackle the problem of system inefficiency in capturing student proficiency. Similarly, Hampton et al. (2018) designed a mobile adaptive learning system called the Personal Assistant for Life-Long Learning (PAL3) to prevent knowledge decay. Hssina and Erritali (2019) presented an adaptation approach for their developed adaptive e-learning system. This approach allowed the generation of learning paths that can adapt according to the profiles of students. They used a genetic algorithm to search for optimal learning paths and then tested and evaluated their adaptive learning system. Troussas et al. (2020) proposed and presented a framework that rec-ommends collaborative activities to students, considering their needs and preferences. The authors used Artificial Neural Network (ANN) and the Weighted Sum Model (WSM). Problems related to engagement and motivation are also highlighted in this mapping. On the one hand, high levels of demotivation, passive attitudes, boredom, poor engagement and frustration among the students are specifically identified in this category. Examples of studies that applied AI-enabled learning interventions to mitigate these issues include Maravanyika et al. (2017), who proposed an adaptive recommender system-based framework for personalised teaching on e-learning platforms. An affective tutoring system (ATS) named Tamaxtil was developed to identify when students become frustrated and confused, at which point it offers them the help they needed. On the other hand, some research evaluated existing systems to see how they could be improved. The researchers improved issues with the current systems by adding or utilising intelligent mechanisms, learning analytics, data mining techniques and plugins, such as Smart Adaptive Management for Flipped Learning (SAM-FL). For example, Min Liu, McKelroy, et al. (2017) used Brightspace LeaPTM adaptive technology to create adaptive intervention modules. The main objective of their research was to investigate the impact of adaptive learning on a large research university in the Southwestern United States. Their modules were embedded via the Learning Tools Interoperability integration in the Canvas learning management system. The growing use of systems and frameworks for adaptive learning is in alignment with past studies on using AI-enabled learning systems to address challenges, such as student disengagement and poor student motivation. Thus, as seen above and in the Appendices, various examples of AI-enabled learning interventions have already been applied to address the problems faced by students. However, there are still several problems that have yet to be addressed by AI-enabled learning interventions. One example of an overlooked problem is the use of outdated and highly complex models. Most of the models in the existing ITS, as noted by Dargue and Biddle (2014, p. 1), ‘are quite complex to enable just about any learner to get the optimum tailored experience possible’. Brawner and Gonzalez, 2016, (p. 3) noted that the existing models use ‘generalized data obtained from a large sample of human subjects, which lacks applicability to individuals’. To address the issue of complexity, existing adaptive learning models can be improved by AI techniques building on learning analytics. Further, within complex relations in real life there are also asymmetric relations among variables and their different conditions, which can be captured by employing fuzzy-set Qualitative Comparative Analysis (fsQCA), as exemplified by Pappas and Woodside (2021). Other overlooked problems include personalization issues, designing and assessing adaptive courses, high instructor workload, no specific framework for implementing intelligent agents in the systems and high levels of attention among learners in the execution of the proposed tasks. Our mapping revealed that problems still exist (e.g. difficulty in attaining learners’ skills and issues related to students’ backgrounds and profiles) despite evidence of AI-enabled learning interventions addressing such problems. Xie et al. (2019) noted that, when designing AI-enabled learning systems, designers of adaptive learning systems still give little attention to courses that have practical skills as a prerequisite. Mousavinasab et al. (2018) recommended and identified AI techniques for mitigating difficulties in attaining learners’ skills. For instance, fuzzy-based techniques, condition-action rule-based reasoning, case-based reasoning and intelligent multi-agent and data mining methods are AI techniques that can be used in the field of computer programming. The presented topic analysis based on AI-enabled learning interventions is useful, as it helps identify the problems to which AI-enabled learning interventions have been applied and what problems have yet to be addressed. Appendix A presents this topic analysis. 4.3. Analytics methods and techniques that are utilised in AI-enabled. learning systems The present mapping of the literature shows that 46 papers proposed various techniques (AI and data analytics techniques), which we classified into three basic categories: descriptive, predictive and prescriptive analytics (Appendix B). The most common and utilised method involves predictive analytics, which deals with ‘forecasting and statistical modelling to determine the future possibilities based on supervised, unsupervised, and semi-supervised learning models’. This analytical method is based on statistical methods that seek to reveal patterns and ‘capture relationships in data’. Predictive analytics has been used for detecting and classifying questions that are applied to establish students’ knowledge levels as well as selecting the required items for students. In our mapping, predictive analytics methods and related techniques, specifically naïve Bayes, fuzzy logic, Bayesian networks, neural networks and Bayesian knowledge tracing (BKT) and association rules mining, have been shown to enhance students’ learning performance, personalised learning, motivation and achievements, thus addressing learning process challenges and student disengagement. Based on their capabilities, we recommend the use of predictive analytics to address the complexity of learning systems models and students’ failure to attain target skills. The other type of analytics method identified in this mapping is descriptive analytics. This category is the simplest BDA method that involves ‘the summarization and description of knowledge patterns using simple statistical methods, such as mean, median, mode, standard devi-ation, variance, and frequency measurement of specific events in BD streams’. Usually, descriptive analytics help identify patterns and reveal what has already taken place. These methods and their related techniques identified in Appendix B are utilised to identify deviations in the behaviours of students or lecturers, analyse students’ learning problems and evaluate their mastery and the knowledge they currently possess based on their success and failures. Thus, descriptive analytics techniques have been used to enhance students’ learning performance. They can also be utilised to address issues such as the lack of evaluation standards and methods for AI-enabled learning systems as well as difficulties in finding an efficient way to organise complex information. The least utilised analytics method is prescriptive analytics, which involves ‘optimization and randomized testing’. The prescriptive analytics techniques we identified in the mapping include ant colony optimization and a combination of roulette wheel algorithms and fuzzy logic. These techniques, which select the more suitable solutions to problems, maximise learning path choice and thus establish optimal data. Prescriptive analytics can be used to solve several challenges highlighted in this mapping, such as limitations in adaptive learning systems, the failure to address process-oriented adaptation and difficulties in finding an efficient way to organise complex information. 5. Implications of this study and recommendations. 5.1. Theoretical implications. The study contributes to the previous research (specifically literature reviews and the analysis of studies) by identifying the knowledge gaps in the field of AI adaptive learning systems. We identified research gaps and provided insights in three main areas. The first is a visualisation of the co-occurrences of authors associated with major research themes highlighted in AI-enabled learning systems. We visualised the authors’ connections to the main purposes of the selected studies. We chose to visualise these co-occurrences to identify the prominent themes in the field of AI-enabled learning systems and demonstrate how they are connected to one another. The second area is the types of AI-enabled learning interventions as well as what problems these interventions have and have not addressed. We identified several problems faced by students and lecturers in their respective learning environments. These included challenges in students’ learning process in their learning environments and issues related to their profiles and backgrounds, engagement and motivation as well as how they can be addressed The third area we identified involves the analytics methods, their accompanying techniques and how they are utilised in AI-enabled learning systems. The most utilised methods identified in our mapping were predictive and descriptive analytics. We also identified the different areas in which these two methods have been used, such as enhancing students’ learning performance, motivation and personalised learning; analysing students’ learning problems; and identifying deviations in behaviours among students and lecturers. 5.2. Practical implications. The study provides important insights for practitioners in education settings who are interested in AI-enabled learning systems. The findings of this study indicate that most AI-enabled learning interventions are systems and frameworks. However, some of the systems and frameworks that were designed and proposed were mostly in their experimental phases. Researchers, developers and practitioners can implement these interventions and use them in real educational settings. The frameworks and systems can be tested and evaluated to see how they perform in educational settings. This is supported by Costa et al. (2017), who tested the Drift Adaptive Retain Knowledge (DARK) framework, which deals with challenges of dynamic environments (e.g. adaptive learning environments). These challenges include the inability to easily discern crucial and accurate information. Another example is testing and evaluating an AI-enabled learning system, named Tamaxtil, which detects ‘affective states in students while they are solving mathematic exercises in order to regulate negative emotions’. Moreover, few studies have involved adaptive approaches for AI-enabled learning systems and partnerships among institutions for collaborating in their use. Thus, there should be more collaborations among universities to design and use AI-enabled learning systems, following successful examples in the literature that have presented, experimented and evaluated adaptive approaches in the development of adaptive e-learning platforms. Further, more studies should use adaptive approaches for AI-enabled learning systems, as this could increase the use of AI-enabled learning systems and address students’ challenges. The main aim of the adaptive approach is to ‘allow to generate learning paths adapted to the profiles of the learners and according to the pedagogical objectives fixed by the teacher’... Another practical implication of this study is how to address the issue of outdated and complex models in learning systems. We recommend that existing adaptive learning models should be improved by AI techniques building on learning analytics. Fuzzy-based techniques, along with condition-action rule-based reasoning, case-based reasoning and intelligent multi-agent and data mining methods can be used for issues related to difficulties in attaining learners’ skills. These can be implemented in situations in which skills (e.g. programming skills) need to be attained. Existing systems lack modern techniques or tools for students to practice and master their skills. Thus, based on their capabilities, we recommend that the various techniques (AI and data analytics techniques), which we classified into three basic categories (descriptive, predictive and prescriptive analytics), be used to address the problems related to the complexity of learning systems models, the lack of evaluation standards and methods for AI-enabled learning systems and the difficulties in efficiently organising complex information to support students in skill attainment. In this study, we identified prescriptive analytics as the least frequently used method. It is possible that practitioners and stakeholders are not aware of its capabilities in designing and building AI-enabled learning systems. Prescriptive analytics can be used to address several concerns highlighted in this mapping, such as limitations in adaptive learning systems and difficulties in finding an efficient way to organise complex information. Finally, one of the research gaps identified in our study that needs to be addressed is the discrepancy between what an AI-enabled learning intervention can do and how it is utilised in practice. Arguably, users do not understand how to extensively use such systems. At the same time, such systems—when implemented—have not actually overcome the complex challenges faced by students, as the literature claims. Thus, researchers, developers and designers of these AI-enabled learning systems could promote awareness of the actual potential and benefits of these AI-enabled systems among lecturers and stakeholders who implement systems in educational institutions. Moreover, the study provided examples of AI-enabled learning interventions applied to address students’ problems, as shown in the Discussion section and in the Appendices. Some of these problems have only been addressed by a few AI-enabled learning interventions. Therefore, practitioners and developers could design interventions for problems that have not been extensively addressed, such as in supporting learners’ attainment of skills and complex models in the systems. 6. Conclusion. In this paper, we conducted a systematic mapping of AI-enabled adaptive learning systems presented in the literature using 147 studies published between 2014 and 2020. We found that systems (adaptive learning system, intelligent mechanisms and adaptive learning platform) and frameworks for adaptive learning were the most proposed and utilised interventions for addressing the challenges faced by students and teachers. The importance of such systems has largely increased during the pandemic as they can assist teachers in maintaining high-quality teaching and learning and improving learning design in IT and IS education. However, most of the systems and frameworks that have been designed and proposed are currently in their experimental phases. They have not been tested in practice or adopted in real educational settings. In summary, we find that the use of AI-enabled contemporary learning systems can offer significant benefits. Therefore, we urge HEIs to adopt them where feasible. We contribute to the literature by mapping the recent literature on AI-enabled learning systems. We present the summarised findings of topics related to such systems. The major findings and contributions of this paper include the identification of the types of AI-enabled learning interventions used, a visualisation of the co-occurrences of authors associated with major research themes in AI-enabled learning systems and an analysis of the most utilised BDA methods and accompanying techniques used in AI-enabled learning systems. Such mapping is needed, as research on AI-enabled learning systems is on the rise, and it is expected to continue with great potential for higher education institutions. Our mapping can aid in identifying and selecting the right kind of AI-enabled learning intervention to address a specific challenge. The findings on AI-enabled learning systems presented in this paper contribute to a better understanding of learning systems. Future research can address the above-mentioned overlooked problems by applying AI-enabled learning interventions. Moreover, studies should be conducted on the limited usage of AI-enabled learning systems in education and how this problem can be overcome. Specifically, the issue of designing and assessing courses that utilise AI-enabled learning systems should be given attention in order to increase the usage of these systems in real educational settings. Another significant recommendation is that future research should attempt to bridge the gap between pedagogy and emerging AI techniques. More studies are needed to address this gap and align technology platforms with course content, students’ expectations and lecturers’ needs. In sum, in future research, more systems, frameworks and models should be put in practice and tested so that researchers can determine whether they can provide solutions for over-coming the learning challenges faced by students. This study has several limitations due to the nature of the research. Although the recommendations of Petersen et al. (2015) were followed to ensure a systematic literature mapping, the search words, strings and databases may have limited the mapping. The key strings were limited to adaptive learning systems, while AI-related terms were limited to AI and ML. This was done because these terms are the most popular. Indeed, the results identify papers that may deal with other AI more specific techniques (such as data mining or text mining). As AI-enabled learning systems evolve, future research should keep a close eye on the developments with regard to the inclusion of more advanced techniques, such as deep learning and natural language processing. Moreover, the selected databases, inclusion criteria and exclusion criteria may, by their nature, have excluded some research. Finally, questions like the methodological approaches applied and the purposes for which AI has been used in learning systems were not reviewed. These issues can be addressed in future research. Statements on ethics This material is the authors’ own original work, which has not been previously published elsewhere. The paper reflects the authors’ own research and analysis in a truthful and complete manner. The results are appropriately placed in the context of prior and existing research. All authors have been personally and actively involved in substantial work leading to the paper and will take public responsibility for its content. Credit author statement Tumaini Kabudi: Conceptualization, Methodology, Validation, Investigation, Data Curation, Writing – Original Draft Ilias Pappas: Conceptualization, Methodology, Validation, Writing – Review and Editing, Supervision. Dag Hakon Olsen: Conceptualization, Supervision. Source of support This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Declaration of competing interest interests or personal relationships that could have appeared to influence the work reported in this paper. The authors declare that they have no known competing financial Appendix A. Supplementary data Supplementary data to this article can be found online at the linked source.