Latest / 1 More Paper / Advancing 21st-Century Professional Competencies with Learning Analytics in the Age of Generative AI — Barthakur et al. (2026)
Transcript
- You’re listening to “Advancing 21st-Century Professional Competencies with Learning Analytics in the Age of Generative AI,” by A. Barthakur and colleagues. Published in 2026. Abstract. The rise of generative artificial intelligence (GenAI) and accelerated globalization have necessitated a fundamental recalibration of higher education to prioritize domain-agnostic, 21st-century professional competencies. While institutional commitment to these skills is high, their systematic integration into the curriculum and evaluation remains fragmented, highlighting a critical gap between traditional academic success metrics and demonstrated workforce readiness. This special issue presents five complementary studies that investigate how the intersection of learning analytics (LA) and GenAI can bridge the gap between institutional rhetoric and demonstrated professional readiness. The contributions collectively advance a research agenda across four dimensions: 1) benchmarking large language models (LLMs) for curricular-competency alignment using reasoning-based prompting, 2) the iterative design of Socratic-style GenAI chatbots to scaffold self-regulated learning, 3) the application of psychometric modelling and Latent Profile Analysis to quantify 21st-century professional competencies, and 4) institutional governance and adoption of curriculum analytics. Collectively, these studies advocate for an epistemological shift toward process- sensitive assessments that move beyond static, episodic indicators toward dynamic, longitudinal representations of learner capability. We conclude by outlining the sociotechnical infrastructure, including robust governance and interdisciplinary collaboration, required to responsibly transition these AI-driven innovations from research prototypes to sustainable enterprise infrastructure, ensuring that analytics serve the evolving needs of students, educators, and professional bodies. Notes for Practice • Conventional metrics of academic success often obscure the nuanced, non-linear development of 21st century professional competencies. • The synergy GenAI and LA offers transformative ways to capture student skills development, but their application must be grounded in human-centred design, ethical governance, and robust data-privacy protections. • Transitioning LA from isolated research prototypes to enterprise-wide infrastructure requires robust governance aligned with professional accreditation standards and a sustained focus on building sociotechnical capacity among educators, administrators, and learners. 1. Introduction and Motivation. The transition into the third decade of the 21st century has been marked by a structural shift in the global labour market. This transformation is driven by the dual forces of accelerated globalization and rapid technological advancement, most recently exemplified by the emergence of generative artificial intelligence (GenAI; Li, 2022; Salas et al., 2008; Sarala et al., 2025; Tushar & Sooraksa, 2023). These shifts are reshaping what knowledge is valued, how knowledge work is produced, and how competence is demonstrated in practice. GenAI systems increasingly perform tasks traditionally associated with human cognition, including drafting, ideating, summarizing, coding, and analyzing. As a result, both the knowledge we value and the competencies we seek to cultivate in schools and reward in the labour market are being recalibrated. The ability to recall and reproduce disciplinary knowledge, long central to higher education assessment, is no longer seen as guaranteeing professional readiness in environments where information is ubiquitously accessible and algorithmically generated. Instead, employers and policymakers emphasize a broader set of domain-agnostic “21st-century” competencies, such as applied problem-solving, collaboration, critical and computational thinking, adaptability, digital and media literacy, and, increasingly, AI and privacy literacy. In this context, higher education faces a dual challenge. First, institutions must meaningfully integrate 21st-century competencies into curricula that have historically been organized around disciplinary silos and content mastery. This requires more than articulating professional competencies as aspirational goals; it demands constructive alignment between intended learning outcomes, pedagogical activities, and assessment evidence so that competencies are demonstrably enacted (Barthakur, Kovanović, et al., 2024). Second, institutions must develop scalable, interpretable, and ultimately defensible approaches for assessing whether learners are actually acquiring these competencies. This remains difficult because such competencies are inherently complex, context-dependent, and often manifested through situated performance, judgement, and adaptive expertise rather than discrete, easily measurable behaviours. More challenging yet, the performance involved in successfully demonstrating these competencies can often be mimicked by using GenAI. Despite widespread recognition of their importance, the systematic integration and evaluation of these competencies remain uneven, fragmented, and frequently under-instrumented across higher education systems (Barthakur, Jovanovic, et al., 2024; Foster & Piacentini, 2023). Competencies are often framed in abstract or aspirational terms, complicating their operationalization across disciplinary contexts. Furthermore, conventional curriculum review processes tend to be labour-intensive, episodic, and poorly suited for continuous monitoring, limiting institutions’ capacity to generate timely evidence about competency development at scale (Barthakur, Kovanović, et al., 2024). The result is a persistent misalignment between institutional learning outcomes, accreditation expectations, and the forms of evidence increasingly sought by professional bodies, policymakers, and industry stakeholders. The learning analytics (LA) community has long engaged with questions surrounding 21st-century competencies and professional capabilities. Work six years ago emphasized moving “beyond cognitive ability” toward richer representations of learner development. Buckingham Shum and Crick (2016) argued for LA frameworks that foreground not only measurable outcomes but also the complex capabilities required for learning and work. Subsequent research has explored curriculum analytics, competency profiling, and the alignment between learning design and assessment evidence (Barthakur, Kovanović, et al., 2024; Jovanović et al., 2025; Pardos & Nam, 2020), highlighting both the promise and the methodological challenges of such approaches. The rapid diffusion of GenAI has prompted renewed interest in the assessment of professional competencies within the LA community. Researchers increasingly highlight the potential of conversational GenAI chatbots to enhance student ability to communicate their reasoning, demonstrate creativity, and work through complex problems. At the same time, there are growing concerns that students who use chatbots supplant their own creativity with standardized ideation. Recent studies indicate a strong uptake of GenAI in higher education, suggesting that GenAI-mediated learning has become a consequential part of student study practices rather than a fringe activity. This presents an untapped opportunity to evidence competency development by leveraging LA to capture and analyze the process data generated during student interactions with GenAI chatbots. Editorial discussions in the Journal of Learning Analytics have further underscored that while GenAI expands the technical frontier of analytics, it must be guided by principles of human-centred design, transparency, and responsible innovation. There is a broad consensus that GenAI-informed analytics should not only scale insight into competency development but also ensure interpretability, ethical accountability, and methodological robustness, particularly where competency claims may have high-stakes implications for learners’ academic and professional futures. This special issue builds on these ongoing conversations. It situates LA at the intersection of competency-based education and GenAI, asking how analytics can both measure and meaningfully support the development of professional competencies in a rapidly evolving technological ecosystem. In doing so, the issue recognizes that competency development is neither linear nor uniform; it is shaped by contextual, cognitive, and technological factors that require multi-layered analytical approaches. By bringing together scholarship on curriculum analytics, competency profiling, GenAI-supported reflection, and the evidencing of professional standards attainment, this special issue’s articles advance a coherent and timely agenda: moving beyond aspirational rhetoric toward empirically grounded, scalable, and ethically defensible models of competency development in the age of GenAI. Rather than treating competencies as abstract policy constructs, the contributions collectively examine how they can be operationalized, traced, and validated through learning analytics methodologies. In doing so, the issue positions LA not merely as a measurement tool, but as a critical infrastructure for aligning curriculum design, assessment practice, and professional accountability in ways that are transparent, interpretable, and responsive to emerging AI-mediated learning environments. 2. Overview of the Special Issue Papers. The five papers in this special issue offer diverse perspectives on advancing 21st-century professional competencies using GenAI and LA. The study by Xu et al. (2026) shows that existing curriculum documents constitute a valuable but underutilized resource for competency-oriented analytics. Their study shows that large language models (LLMs) can produce curriculum–competency alignments. More importantly, it also shows that the evidentiary value of those alignments depends on the characteristics of the underlying documents and the inferential demands of the task. By benchmarking models against 38 competencies, 200 courses, and five curriculum document types, the authors demonstrate that the concept of “curriculum” is not a neutral variable. The granularity of the documents, particularly their instructional activity descriptions, significantly influences the inferences a model can make. Furthermore, the findings indicate that while both open and proprietary LLMs exhibit commendable performance in coarse-grained classifications, none achieve the precision of human judgment when tasks require fine-grained pedagogical interpretation. The study emphasises that the full potential of this approach does not rely on technical advances but also on the quality of curriculum data infrastructures and established shared benchmark datasets. Chichekian et al. (2026) argue that competency assessment should move beyond single scores toward interpretable patterns that can guide differentiated support, especially in professional preparation contexts where readiness is consequential. They posit that competency development is heterogeneous and often non-linear, and assessment systems that assume uniform trajectories can unintentionally reproduce inequities. Focusing on computational thinking (CT) as a high-priority competency for preservice teachers, they combine multimodal data with latent profile analysis (n = 128) to identify three CT profiles (Novice, Developing, Proficient). They show that metacognitive strategies and prior coding experience predict profile membership. Different behavioural indicators from a problem-solving task further differentiate the profiles in terms of efficiency and perceived difficulty. The paper also places a meaningful constraint on how profiling approaches are used, emphasizing that profiles can become labels unless they are tied to actionable pedagogical responses (e.g., targeted scaffolding, adaptive pathways) and designed to make structural opportunity gaps legible rather than attributing differences to individual deficits. Their suggestion of dashboard designs grounded in profile and behaviour metrics points toward a concrete way learning analytics can support both personalization and equity in competency development. A complementary perspective is offered by Barthakur and colleagues (2026), who address a structural issue in professionally accredited higher education programs. Accreditation bodies often require the curriculum to be mapped to professional competencies, but institutions rarely have credible, program-level evidence of student attainment of those competencies beyond self-report or aggregate GPA. This study’s contribution is a curriculum-analytics approach that maps assessment grades to professional competencies and then applies psychometrics and LA to identify distinct patterns (profiles) of competencies acquisition across an Initial Teacher Education program. The paper challenges the sector’s default reporting regime (transcripts, GPA) and proposes an alternative representational infrastructure. This alternative approach presents standards-based profiles that can reveal misalignments between “academic success” and “professional readiness.” At the same time, the paper productively surfaces a validity tension that must become central in competency assessment debates: if we do not change assessment tasks, then competency attainment will be inferred through the construct space of existing grading practices. The strength of the approach, scalability without redesign, also becomes its main challenge: under what conditions do grades provide warranted evidence of the standard, rather than only institutional signals of performance? Barthakur and colleagues offer the field a concrete, program-level method that forces this question into the open, making it harder for institutions to hide behind maps and accreditation narratives without learner-level evidence. Hernández-Campos et al. (2026) address a related challenge and the persistent gap in the transition from research prototypes to enterprise-wide infrastructure. Even when learning outcomes (LOs) are defined and measured, universities struggle to translate assessment data into program-level improvement, particularly when academic performance is used as a proxy for LO development. In a comparative two-case study of universities in Chile and Costa Rica, the authors investigate how curriculum analytics (CA) tools support direct assessment of LOs aligned with 21st-century competencies and can generate actionable insights that prompt more authentic and reflective teaching practices. They note that competency assessment at scale is less a measurement problem than a translation problem. Their identification of cross-case themes highlights that institutions need shared constructs, robust metrics, and governance structures to prevent CA from becoming another compliance layer. The move from “new measures” to discussing “new institutional capacities” for acting on competency evidence is highlighted as a key step for the work in this domain. Finally, Hilliger and colleagues (2026) shift the discussion from AI in assessment to AI in learning support, probing what it means to design an AI system that scaffolds competency development rather than merely evaluates it. Their design-based research develops a GenAI chatbot intended to scaffold self-regulated learning through structured self-reflection, iteratively refined across three design cycles involving 276 students in ten courses. This chatbot is not positioned as a surrogate teacher or productivity shortcut, but as a course-integrated dialogic scaffold that can cultivate (or erode) learner agency depending on how it is designed and governed. Their findings are especially relevant to emerging directions in conversational assessment, where students actively seek human-like tutoring but also require explicit pedagogical guidance to engage meaningfully with AI, and the design’s emphasis on Socratic prompting signals a pathway toward eliciting evidence of reflective competence rather than supplying content. The paper helps reposition GenAI interaction traces as a potential assessment substrate for professional competencies while simultaneously warning that without educator involvement and a clear division of responsibilities, chatbots create a risk of overreliance and superficial engagement. 3. Critical Reflection and Future Directions. The special issue highlights the enduring risk of metric substitution. When competencies become policy imperatives, there is a risk that what is easiest to count becomes conflated with what is most important to learn. As 21st-century competencies are embedded within accreditation standards, employability frameworks, and institutional dashboards, the pressure to establish “new” metrics intensifies. LA, when coupled with GenAI, can expand our capacity to generate high-quality and meaningful competency indicators at scale. The inclusion of GenAI affords opportunities for automated interpretation of multimodal data, the classification of textual and trace data, the mapping of curriculum artefacts to professional standards, and the modelling of learner profiles across programs and cohorts. Tasks that previously required substantial human labour and expert judgement can now be conducted quickly at scale. However, scale does not confer validity. If anything, it increases the danger that partial proxies become institutionalized as authoritative signals of capability. We are at an inflection point in both research and practice. GenAI is accelerating methodological innovation, but it is also reshaping the epistemological basis on which competency claims are made and validated. For decades, assessment has operated on a relatively stable inferential logic. The use of observable and reportable artefacts acted as proxies for individual capability under set conditions where authorship and contribution were assumed to be authentic. GenAI completely removes this base assumption. When systems can generate essays, code, analyses, and plausible reasoning, the relationship between product and capability is less interpretable. In many cases, assessment and learning artefacts increasingly reflect sociotechnical interactions rather than individual mastery (cf. Hutchins, 1995). This raises many questions about academic integrity alongside questions of what now counts as defensible evidence of capability. In this special issue, early responses to this instability are visible. Several contributions move beyond single-score representations toward profiles and developmental patterns, better reflecting the heterogeneity and non-linearity of competency development. Others interrogate what infrastructural and governance conditions are required for competency claims to be meaningful at the program and institutional levels. This is a particularly acute concern given the many contexts where curriculum mapping and standards alignment have historically been treated as compliance exercises rather than evidentiary practices. In complementing these approaches, other works in this collection have positioned GenAI as a dialogic scaffold that can elicit metacognitive and reflective processes. This shifts the intention of learning from polished outputs towards more developmental learning processes. The outcomes and themes from the papers in this special issue suggest that LA must move from measuring episodic performance toward modelling capability development as a dynamic, situated, and relational process. This shift carries important implications for the field of LA and education systems. First, evidence of learning in AI-mediated contexts must become process-sensitive (an argument prominent in literature on finer-grained assessment — e.g., Mislevy et al., 2003). Final artefacts alone are insufficient proxies for capability. The use of multiple data sources, such as engagement traces, revision histories, and reflective dialogue, may provide more defensible indicators of development over time. Second, validity must precede scale. GenAI-powered analytics can accelerate the classification and inference of competency. However, this should be undertaken with some caution. Scaling misaligned constructs risks the need to wind back what can be long-term institutionalized distortions. Third, competency analytics is not simply a measurement problem. It represents a significant and complex sociotechnical challenge that demands new models of leadership and institutional readiness. Shared definitions, data infrastructures, governance arrangements, educator capability, and ethical safeguards are necessary conditions for responsible use. We offer the final cross-cutting message of this special issue. GenAI does not only add new tools to existing competency frameworks; it places significant pressure on the field to revisit our long-held foundational assumptions about evidence, attribution, and judgment. The opportunity is substantial. LA can help the field move from aspirational statements to more interpretable, actionable, and development-oriented evidence. The risk is equally significant. Without careful construct work and governance, analytics may harden proxies into new regimes of legitimacy that obscure rather than illuminate professional capability. As the popularity of GenAI-based assessment and evaluation systems grows, research–practice partnerships will play a vital role in generating externally valid evidence for the ways in which these systems could yield robust measures of 21st-century professional competencies. Research must strengthen the theoretical and validity foundations of competency inference in AI-mediated contexts, while practice must build the institutional capacities required to act on evidence responsibly, ethically, equitably, and in ways that genuinely support learner development in the age of GenAI.