Latest / 1 More Paper / Enhancing transport innovation knowledge management workflows using artificial intelligence: a TRIMIS database case study — Cheimariotis et al. (2026)

Transcript

Enhancing transport innovation knowledge management workflows using artificial intelligence: a TRIMIS database case study — Cheimariotis et al. (2026) · 1 More Paper

• You’re listening to “Enhancing transport innovation knowledge management workflows using artificial intelligence: a TRIMIS database case study,” by Ilias Cheimariotis and colleagues. Published in European Transport Research Review on October 26, 2026. Abstract. The Transport Research and Innovation Monitoring and Information System (TRIMIS) tracks EU-funded transport research and innovation projects to identify trends and inform policymaking. A key function of TRIMIS is maintaining a comprehensive and accurate database of transport projects. Selecting and categorising relevant projects is a manual and time-consuming process. This study explores the use of artificial intelligence (AI), specifically large language models (LLMs), to support and accelerate this task. A workflow was developed using an LLM platform (GENESIS, the JRC internal, secure, general-purpose LLM platform), iteratively refining prompts to match expert human judgment in identifying relevant projects from CORDIS. Comparing LLM-generated outputs to the existing TRIMIS database, projects were assessed for inclusion/ exclusion and classified by transport mode and sector, and technology thematic area. The classifications allow for filtering and consequent analysis and visualisation by these categories. A further application investigated was the assessment of projects for their contribution to the competitiveness of the EU Automotive Industry. Prompt engineering iterations and structured output analysis improved accuracy and consistency and reduced the risk of unsupported or fabricated classifications by requiring every positive decision to be grounded in a verbatim quotation from the project text. Using the ensemble configuration, we analysed 52,923 Horizon 2020 and Horizon Europe projects (2014–2026), identifying 6,986 relevant for inclusion in the TRIMIS database, with reporting in this paper focused on the most recent 19,025 Horizon Europe projects, of which 2,184 (11.5%) were assessed as relevant. The analysis of the full corpus required about two days of serial model time for the two deployed configurations (Annex 5), reducing manual work and identifying matches in previously overlooked funding sources. Outcomes also helped to re-evaluate existing taxonomies. The results demonstrate the potential of AI-assisted workflows to reduce manual workload, improve accuracy, and scale project curation for the TRIMIS database, with broader implications for using AI-assisted methods in public-sector knowledge management. Future work will explore hybrid human-AI review pipelines and domain-specific fine-tuning. Full list of author information is available at the end of the article ​1 Introduction The role of the Joint Research Centre (JRC) is pivotal in ensuring that the implementation of all European Union (EU) policies is grounded in robust scientific research. Regarding Research and Innovation (R&I) policy, to ensure that public funding is effectively targeted and that policy decisions remain evidence-based, it is critical to maintain a clear, comprehensive and continuous overview of the vast R&I landscape. TRIMIS was established to meet this need, serving as a unique policy support tool for the European Commission in the domain of transport R&I. It provides a detailed overview of the state of the art and current development of transport technologies, helping to identify emerging trends, thereby supporting strategic decision-making. TRIMIS maintains an extensive database of EU-funded transport-related R&I projects, which are systematically categorised to facilitate access and analysis. Given the scale of the database, the process of populating it poses operational challenges, namely the efficient and accurate curation of projects from diverse and heterogeneous sources. Among sources for TRIMIS is the broader database of the Community Research and Development Information Service (CORDIS), from where relevant projects are identified and subsequently classified according to the TRIMIS framework. Although manual curation has generated a comprehensive and well-targeted database, the process has potential for improvement. It is time-consuming and resource-intensive but can also lead to inconsistencies, such as the potential oversight of relevant projects with less obvious backgrounds, for instance, funded through bottom-up schemes not associated with transport research. To address these challenges, this study explores the application of AI, specifically LLMs, to support and accelerate the database curation workflow, with additional investigations into the automation of parallel workflows. The work draws on a growing body of work examining AI-assisted workflow automation and human-AI collaboration. However, the deployment of LLMs in this context is non-trivial. The classification of research and innovation projects is complex due to the presence of domain-specific terminology, varying and evolving taxonomies and conventions and the limited availability of consistently labelled training data. This study therefore makes the following contributions. First, it presents and evaluates an LLM-assisted curation workflow for a live, policy-linked R&I database, providing an empirical case study of LLM deployment in a public-sector knowledge management context. Second, it conducts a structured comparative evaluation of multiple prompt engineering strategies, including zero-shot, chain-of-thought, signposting approaches, assessed against human-curated ground truth across classification accuracy, thematic consistency and coverage. Third, it proposes and evaluates a hybrid human-AI workflow design in which LLMs are used for an initial screen while human experts focus on validation and resolution of margin cases, contributing to the literature on human-AI collaboration, particularly in institutional settings. Fourth, it investigates whether LLM-assisted curation can improve project discovery coverage relative to manual processes, with particular attention to projects from bottom-up instruments such as the ERC, MSCA and EIC, which may otherwise be systematically underrepresented in manually curated databases. Rather than replacing expert judgement, the approach aims to augment knowledge curation processes by redistributing effort from routine screening tasks toward higher-level validation and interpretation. Positioned within the operational environment of TRIMIS, the study serves both to enhance internal knowledge management practices and to provide an empirical case study illustrating how large language models can support public-sector research intelligence workflows. This contribution is particularly relevant in public administration contexts characterised by growing information volume, limited expert resources and increasing demand for transparent and evidence-based innovation monitoring. 2 Related work The workflow proposed in this study sits at the intersection of several research areas. This section moves from the general methodological foundations, the use of LLMs for text classification and the engineering of prompts to steer them, towards the specific setting that motivates the work: the classification and curation of research and policy text, and ultimately the deployment of such methods within public-sector institutions. For the two domain-specific strands it traces the shift from earlier, pre-LLM approaches to current, LLM-based ones. The section closes by identifying the gap that the present work addresses. 2.1 Foundations: natural language processing (NLP) and large language models for text classification Automated text classification has progressed from keyword-matching and feature-based machine learning towards transfer learning with pre-trained language models. The Transformer architecture and bidirectional pre-training established the basis for contextual language representations, and domain-adapted variants such as SciBERT improved performance on scholarly corpora. The subsequent emergence of generative LLMs introduced few-shot and zero-shot capabilities, enabling classification without task-specific retraining and without large, labelled datasets, which is decisive in settings where such datasets are unavailable. A growing body of empirical work has examined how reliably these models reproduce human judgement: ChatGPT and open-source LLMs have matched or exceeded crowd-sourced annotators on relevance, stance, topic and frame detection and on political-science text classification; zero-shot, few-shot, fine-tuned and instruction-tuned approaches have been compared systematically; and data-efficient alternatives based on entailment and transfer learning have been proposed. Reviews and methodological guidance nonetheless caution that the reliability of LLM-generated labels in operational, specialised settings remains task-dependent and must be validated rather than assumed. 2.2 Steering general-purpose models: prompt engineering Where fine-tuning is impractical, prompt engineering has become the principal mechanism for aligning general-purpose LLMs with domain-specific objectives, with formalised strategies including chain-of-thought prompting, instruction tuning and automated prompt optimisation. The same literature documents the fragility of prompted behaviour — sensitivity to prompt format, example choice and ordering — and the unreliable effect of persona framing on objective tasks, which justifies treating prompt design as an empirical object of evaluation rather than as a purely intuitive craft. Comparative evaluation of prompt strategies within real operational knowledge-management workflows, particularly in public-sector settings, nevertheless remains scarce. 2.3 From traditional to AI-assisted classification and curation of research and policy text The organisation and curation of research and policy text predate the current generation of LLMs. Earlier approaches reduced manual effort through automated content analysis of policy and political text, while stressing the continued need for validation, through bibliometric and scientometric mapping of research landscapes and through unsupervised topic modelling, from Latent Dirichlet Allocation to embedding-based approaches such as BERTopic. Comparative work on LDA, NMF, Top2Vec and BERTopic illustrates the continued relevance of topic-modelling choices for empirical social research. Supervised taxonomy classification with BERT-family models applied to scholarly corpora extended this line of work to assigning research texts to predefined categories. These methods established the feasibility of automated organisation but typically targeted generic disciplinary structures or self-derived topics rather than externally defined, policy-linked schemes. The classification of EU-funded projects is an especially relevant adjacent case. Machine-learning support for categorising EU funding projects has been examined previously, showing that project descriptions can be used to assign projects to funding categories. However, this work remains closer to fiscal and administrative categorisation than to the operational classification of transport R&I projects against a policy-defined taxonomy. CORDIS itself has also introduced semi-automatic classification of EU-funded research projects against the EuroSciVoc taxonomy, using NLP and machine-learning techniques combined with human validation. This demonstrates that EU project classification is already partly automated, but the taxonomy used by CORDIS is a general science taxonomy rather than the TRIMIS transport-policy framework. More recent work applies LLMs to related annotation and curation problems. Generative AI has been proposed as a tool for automated annotation, but this literature stresses that LLM outputs must be validated against human labels on a task-by-task basis and that humans should remain in the loop throughout. This connects to the broader field of human-in-the-loop machine learning, which encompasses active learning, interactive machine learning and machine teaching, and whose most direct operational precedent is the active-learning screening of large textual datasets into relevant and non-relevant records, with the human acting as oracle. Comparisons of manual and computer-assisted coding complete this hybrid framing. 2.4 Generative AI in the public sector and policy These methods are increasingly relevant to public administration. Overviews of AI applications across the public sector and empirical studies of AI adoption in public organisations show that deployment must be analysed as an implementation and governance process rather than only a technical adoption decision, and recent evaluations of LLM-based tools for policy work conclude that specialised models can support policy writing and synthesis while still requiring human evaluation. This literature supports a hybrid framing in which AI output is used to augment rather than replace expert policy work. This is the setting in which TRIMIS operates. TRIMIS is an open-access transport research and innovation information system designed to support EU and Member State transport-policy implementation, monitor the Strategic Transport Research and Innovation Agenda, and analyse technology trends and R&I capacities in the transport sector. TRIMIS has also been described as an integrated policy-support tool for sustainable transport governance and decision-making. In operational terms, TRIMIS is characterised by growing information volumes, limited expert resources and demand for transparent, evidence-based innovation monitoring. 2.5 Research gap Taken together, this literature establishes that LLMs can classify and annotate text competitively, that prompt engineering can steer general-purpose models without fine-tuning, that research corpora can be organised automatically, that human-in-the-loop designs are appropriate where reliability is uncertain, and that public-sector deployment is both feasible and increasingly studied. This body of work, however, is domain-general: it addresses policy drafting, classification and deployment outcomes in the abstract, without engaging any specific policy domain in depth. Transport research and innovation policy — with its programme-level portfolio monitoring, its evolving thematic taxonomies, and its portfolio-level competitiveness questions — has not been one of the domains examined. We consider this gap worthy of dedicated investigation: specifically, the automated classification of EU-funded transport R&I projects against a policy-defined, operationally evolving taxonomy such as the TRIMIS framework has not previously been addressed. Adjacent work has examined automated classification of EU-funded projects, and CORDIS itself uses semi-automatic NLP/ML classification against EuroSciVoc, but these approaches concern general scientific-field or funding-category classification rather than alignment with a transport-policy monitoring framework. Prompt-engineered interactions with general-purpose LLMs therefore represent a practical approach grounded in existing theory but not yet evaluated in this operational setting. This distinguishes the present study from generic classification tasks: the TRIMIS taxonomy is not a static benchmark, but a dynamic scheme tied to EU transport-policy objectives, which makes alignment a more consequential and moving target. Furthermore, although CORDIS is frequently used as a data source, its systematic use for the extraction of transport-specific R&I intelligence, and the associated curation challenges, has not been examined from an AI-assisted workflow perspective. The choice to focus on prompt engineering is motivated by the absence of large, labelled training datasets for TRIMIS-specific classification, as well as the need for an approach that can be maintained and adjusted by domain experts without machine-learning expertise. Nor, to our knowledge, has the comparative performance of prompt-engineering strategies been evaluated with deployment into an operational public-sector curation workflow as the explicit objective; comparative studies against policy-relevant taxonomies exist as one-off benchmark exercises (e.g., legislative-text classification against a fixed coding scheme), rather than as the validation basis for an institutional system moving into production. The contributions set out in the introduction respond directly to these gaps. 3 Methodology This study designed and evaluated a series of AI-assisted workflows to automate distinct project curation tasks for the TRIMIS database. While separate workflows were developed for each use case, they followed a common methodological foundation in their development. This section details that core process, focusing on the two foundational workflows: relevance analysis and classification. The three workflows — relevance screening, classification and competitiveness assessment — share a single AI-assisted method, summarised in Fig. 1: a task-specific prompt is developed and validated against a human reference before being applied at scale, and only the prompt-construction strategy differs between workflows. This section describes the shared method (3.2) and then each workflow’s specifics (3.2.1–3.2.3); the datasets and metrics used to evaluate it are given in 3.3. 3.1 Data source and scope The primary data source for this study was the CORDIS database. The base dataset comprises 52,923 research and innovation projects funded under the Horizon 2020 and Horizon Europe programmes (CORDIS, to 1 March 2026). Configuration comparison and selection were carried out on a random sample of 250 projects (Sect. 3.3), which kept the factorial comparison of prompt styles, scoring definitions and models tractable. Once the deployed configuration had been selected, it was applied to the full corpus. Reporting in this paper focuses on the 19,025 Horizon Europe projects, being the most recent and currently active programme. For each project, key textual data was collected and provided to the model as input, including the project title, its description and objectives, depending on availability of information for each project. The combination of chosen information was the primary basis for the model to determine a project’s relevance, classification or to execute any other task detailed in the prompt. This data allowed the model to capture some more nuance, for example, by distinguishing projects that contained “transport” in the title but not in the sense used in TRIMIS, e.g. projects relating to fluid dynamics and transport phenomena. Conversely, projects that did not have transport applications clearly indicated in the title could be identified as TRIMIS-relevant via the details provided in their description. 3.2 AI-assisted workflow A key aspect of the methodology was an iterative process of prompt engineering applied to tailor prompts to each distinct workflow. This process involved four steps: 1. Initial Prompt Design: A specific prompt was. engineered for a single task (e.g., determining relevance), using an established template. 2. API Execution: The prompt was used to process the. project data via the LLM platform, executing the task described by the prompt. 3. Human Comparison: The LLM’s output was. systematically compared against human processing that had been carried out previously or in parallel (without any exposure to model results). 4. Analysis and Refinement: Discrepancies were. analysed to understand their sources, considering both the possibility of human error and model hallucination, and the prompt was refined accordingly. This cycle was repeated to progressively enhance performance. These four steps are the prompt design and examination loop (stages 2–3 of Fig. 1). The prompt-construction strategy itself differs by workflow: a controlled factorial design for relevance screening (3.2.1), and iterative refinement to the expert reference for classification (3.2.2) and the competitiveness assessment (3.2.3). Model and settings. All workflows ran on GENESIS, an internal, secure, general-purpose LLM platform hosted by the JRC. Four open-weight engines were benchmarked — GPT-OSS-120B, Llama-3.3-70B, MiniMax-M2 and Mistral-24B, served as gpt-oss-120b, llama-3.3-70b-instruct, minimax-m2 and mistral-small-3.2-24b on the JRC inference infrastructure, so that the contribution of the engine, alongside the prompt factors and the associated processing cost, could be estimated. These four engines were benchmarked because they are the open-weight models deployed on the JRC infrastructure at the time of the study; the comparison is therefore a selection among the locally available options, not a survey of the model landscape. All results reported in this paper refer to these specific models as served on the JRC’s local infrastructure; serving configuration can affect output behaviour and processing cost, so the reported figures characterise the deployed configurations. Three levels of reproducibility should be distinguished. The prompts, scoring definitions and output contract are reproduced in full (Annexes 1 and 7). The engines are downloadable open-weight models, run at a zero sampling temperature, so the model-level configuration can be reproduced on any host, although generation may differ across serving implementations. End-to-end computational reproducibility is not claimed: the models were served on GENESIS, the JRC’s internal platform, and the CORDIS extraction, orchestration and adjudication code is not released. The reported results characterise the deployed system rather than a reference implementation. Deployment. Once the deployed configuration had been selected it was applied to the full corpus. Because automated screening is intended to support rather than replace expert curation, projects on which the automated judgement is least certain are routed to a compact expert-review queue while the confident majority is curated automatically; the routing signals are detailed in 3.2.1. 3.2.1 Workflow 1: Relevance analysis 3.2.1.1 Relevance screening The first prompt was designed to identify a project’s relevance for inclusion in the TRIMIS database. The prompt instructed the model to assign each project a relevance score on a three-point scale, together with a justification and a verbatim supporting quotation from the project text. The scale distinguishes the two ways in which a project can belong in TRIMIS from those that do not belong at all: a score of 2 (TRIMIS core) denotes a project whose primary subject is transport research or innovation; a score of 1 (TRIMIS extended) denotes a project that is substantively relevant to transport but where transport is one contributing dimension among others, or an enabling technology with a clear but not exclusive transport application; and a score of 0 (exclude) denotes a project with no substantive transport relevance. Both scores 1 and 2 therefore correspond to inclusion in the database — the extended tier captures genuinely relevant projects that a narrower, keyword-oriented search would miss — while the exclude tier is filtered out. The prompt provides the context of TRIMIS and its scope, along with the operational scoring guidance described below. Requiring an explicit justification and a supporting quotation for every non-zero score forces a deliberate analytical process and provides an auditable basis for each decision. This design is also what keeps the classification grounded in the source material rather than in unsupported model output. Every positive score must be accompanied by one or two quotations that are exact substrings of the project text; a decision whose supporting quotation cannot be located verbatim in the input is, by construction, detectable and can be rejected, which directly constrains the fabrication of evidence. The fixed JSON schema further restricts the model to the permitted score values and fields, and decoding was configured for low, near-deterministic variability (a zero-sampling temperature) so that the same project yields stable scores across runs. Together, verbatim-evidence grounding, a constrained output schema and low-temperature decoding are the mechanisms behind the reduction in unsupported or fabricated classifications. 3.2.1.2 Prompt calibration approach The prompts were organised as a factorial design crossing two factors, as presented in Fig. 2. The first factor, the scoring definition, distinguishes a basic-criteria formulation, which states the three relevance levels in general terms, from a detailed-criteria formulation, which additionally encodes the operational scoring rules derived from the expert calibration (the principle that relevance is judged by the strength and dominance of the transport element, together with explicit tie-break rules for recurring boundary cases such as transport fuels, energy carriers, batteries and dual-use components). The second factor, the prompt style, varies the formulation and reasoning technique while holding the scoring definition fixed. The six styles — plain baseline, chain-of-thought, role framing, explicit criteria, signposting and Fig. 2 – Factorial design for the relevance workflow confidence rating — represent established families in the prompt-engineering literature and are described in Annex 1. Crossing the six styles with the two scoring definitions yields twelve prompts; a thirteenth, combining several techniques on the detailed-criteria definition, was included as an upper-reference configuration. The full text of all prompts is provided in Annex 1. The factorial design was evaluated on a dataset of 250 random CORDIS projects and across four open-weight models, so that the relative contribution of scoring definition, prompt style and model could be estimated, together with the associated processing cost. The datasets and evaluation procedure are described in the following subsections. 3.2.1.3 Targeted manual review Automated relevance screening is intended to support, not replace, expert curation, so a mechanism is needed to direct expert attention to the projects where it is most valuable. Review routing is based on uncertainty, not on the relevance tier. The TRIMIS-extended tier is a relevance judgement rather than a flag of uncertainty, and it covers a substantial share of the relevant population, so routing every extended-tier project to manual review would reproduce much of the workload the screening is meant to reduce. Two such signals were examined, both of which locate the same small set of genuinely ambiguous projects. The first is the model’s own self-reported confidence (Annex 6): when prompted to accompany each score with a confidence value, the model reports systematically lower confidence on the boundary cases than on clearly relevant or clearly irrelevant projects, so that low-confidence items form a compact, review-worthy subset. The second is disagreement between two complementary screening configurations run in parallel — one precision-leaning and one recall-leaning — where the projects on which the two configurations disagree are, again, concentrated on the ambiguous boundary. The deployed rule combines them: a project is escalated to an expert when the two configurations assign different scores on the three-class scale and the recall-oriented configuration’s self-reported confidence falls below a threshold set on the reference sample (Sect. 4.1); all other projects are curated automatically. The rule thus uses confidence operationally; its validation, reported in the Results on the same reference sample, remains preliminary. 3.2.2 Workflow 2: Project classification For relevant projects, a separate workflow was developed for classification. Within the TRIMIS database, projects are classified by transport mode (Airborne, Rail, Road, Waterborne, or Multimodal), transport sector (Active Mobility, Micromobility, Passenger Transport, or Freight Transport) and transport technology thematic area (Connected and Automated Transport, Transport Electrification, Low-Emission Alternative Energy for Transport, Smart Mobility and Services, Transport Infrastructure, Network and Traffic Management Systems, Vehicle Design and Manufacturing, Urban Mobility, Other). The prompt for this task instructed the model to assign these labels to each project from a set of potential labels that was provided. These labels are based on the existing taxonomy used in TRIMIS, with one of the potential outcomes of this experiment being to re-evaluate this taxonomy. Classification was performed by an ensemble of two open-weight models with complementary dispositions: GPT-OSS-120B in a precision-oriented configuration and Llama-3.3-70B in a recall-oriented one. Each model labelled every project independently across the three taxonomies. Projects on which the two models agreed were accepted directly; those on which they disagreed were treated as contested and resolved by an adjudication step, in which GPT-OSS-120B re-examined the case against the prompt criteria and the two candidate label sets. This applies, at the level of individual labels, the same precision–recall disagreement principle used in the relevance workflow: agreement isolates the confident cases and disagreement concentrates the genuinely ambiguous ones, where a second, focused pass is most valuable. During prompt calibration these contested cases were adjudicated and reviewed by hand to refine the prompt; at scale the adjudication is carried out by the model, with the disagreement set retained for optional expert inspection. 3.2.3 Workflow 3: Competitiveness assessment The developed approach was subsequently tested on a specific application based on an actual TRIMIS use case. The task was to assess a set of projects on their contribution towards the competitiveness of the European automotive sector. The choice of the automotive sector as the competitiveness case study is motivated directly by the current EU policy agenda. The Commission’s Competitiveness Compass for the EU places competitiveness at the centre of the 2024–2029 mandate and, building on the Draghi report, organises it around three imperatives: closing the innovation gap, a joint roadmap for decarbonisation and competitiveness, and reducing excessive dependencies while increasing security. The Compass identifies the automotive sector as a priority and launches the strategic dialogue that feeds into a dedicated Industrial Action Plan for the European Automotive Sector, whose actions span innovation and digitalisation, clean mobility, supply-chain resilience (notably batteries and raw materials), and workforce and skills — the same dimensions along which this workflow assesses the R&I portfolio. Mapping where the EU’s automotive R&I projects do and do not provide evidence on these competitiveness dimensions is therefore a timely, policy-relevant test of the workflow, and structures the theme grouping used in the results. For this purpose, 11 relevant competitiveness dimensions were identified, including consumer price, user features, societal readiness, interoperability, operational efficiency, technical novelty, EU leadership, production scaling, raw materials, circularity, and skills development. The dimensions were grouped into five broader analytical clusters: • Market adoption: consumer price, user features, societal readiness. • Systems and operations: interoperability, operational efficiency. • Technology and industry: technical novelty, EU leadership, production scaling. • Resources and circularity: raw materials, circularity. • Skills and capabilities: skills development. The dimensions, together with their definitions, were incorporated in a prompt, along with project data from a sample of randomly chosen automotive-oriented projects. The prompt assigned scores 0, 1, or 2, depending on whether a particular dimension was addressed by the project (0 for the dimensions not addressed at all, non-zero otherwise), and to what extent (1 or 2, depending on whether concrete actions and results were described or not). In parallel, the same scoring exercise was carried out manually and independently by two transport experts, subsequently leading to a common, agreed human consensus version of the assessment. To evaluate the assessment, two transport experts scored the same calibration projects independently and reconciled their divergences into a single consensus, formed without sight of any model output. Criterion-by-criterion comparison of the model scores with the consensus located the recurring boundary cases, which were used to sharpen the criterion definitions into their final, action-grounded form; the reconciliation process and an ex-post review of the model’s extracted evidence are documented in Annex 7. The refined prompt was then applied, unchanged, to a second held-out set of ten projects scored by the same two experts under the same protocol, providing an out-of-sample check on both the expert agreement and the model–expert agreement before the full projection. After the successful validation, the final version of the prompt was used to generate a projection on approximately 150 EU automotive R&I projects. These 150 projects had been previously identified, through earlier TRIMIS analytical work, as relevant to the competitiveness of the EU automotive sector. This ensured that the projection was carried out on a curated and thematically coherent set of projects, consistent with the scope of the assessment. 3.3 Evaluation The performance of each workflow was benchmarked either against the existing TRIMIS database or against expert reference labels produced for a reduced but carefully constructed evaluation set. For the relevance workflow, a single annotated dataset was used. Performance was measured on a random set of 250 projects, drawn from the corpus without reference to any model output so as to reflect the true population prevalence of transport-relevant work (for exclude / TRIMIS-extended / TRIMIS-core 217 / 23 / 10, i.e. 86.8 / 9.2 / 4.0%, so that roughly seven of every eight projects are not transport-relevant). It measures whether screening is reliable on a representative, heavily imbalanced population. Since the same sample served both to compare configurations and to report their performance, the effect of selection on the reported figures was quantified by selection-aware resampling (Annex 11). Workflow outputs were evaluated against a human reference by standard information-retrieval and agreement metrics: precision, recall, macro-averaged F1, and Cohen’s κ. Each metric is reported under two labels, stated wherever a figure is given. The three-class labelling scores each project on the full 0/1/2 scale (exclude / TRIMIS-extended / TRIMIS-core) and measures the finer core-versus-extended distinction. The binary (collapsed) labelling maps the task onto the operational inclusion decision, grouping the core and extended tiers together as “include in TRIMIS database” against “exclude”; because both tiers lead to inclusion, this binary view is the deployment-relevant measure. All precision, recall, F1 and κ values in this paper are labelled “three-class” or “binary” accordingly, and within any single comparison all metrics share the same labelling. Cohen’s κ is quadratic-weighted on the three-class scale. Full definitions and formulas are given in Annex 10. The reference labels were produced by members of the TRIMIS team — transport-research analysts at the Joint Research Centre responsible for curating the database — who are domain experts in EU transport research and innovation. Each annotator scored projects from the project text alone, without access to any model output, applying the same detailed scoring definition (Annex 1.2); three reviewers scored all projects independently and agreed unanimously on 85%; chance-corrected agreement was moderate-to-substantial (Fleiss κ = 0.60; ordinal Krippendorff α = 0.71, the more appropriate statistic given that roughly 87% of projects are not transport-relevant), and a fourth reviewer adjudicated the 37 contested items to produce the reference, with an audit trail of every contested decision retained rather than resolved by discussion to consensus. Full agreement statistics are reported in Annex 4. For the competitiveness workflow, the same evaluation philosophy was applied to a harder, multi-dimensional task: two expert-referenced sets of ten projects each (calibration and validation), nested within the 150-project projection corpus, were double-scored on eleven criteria, giving 110 expert judgements per rater per set. Because competitiveness scoring is interpretive, all comparisons are reported as inter-rater agreement — between the two experts, between the model and the expert consensus, and between independent models — rather than as accuracy against a ground truth, and on both the three-level scale and the binary evidence-present/absent collapse. The collapse is reported because, for experts and models alike, most residual disagreement concerns the strength of evidence (score 1 versus 2) rather than its presence, so the binary view isolates the decision the workflow supports most reliably. Full statistics are reported in Annex 7. 4 Results 4.1 Relevance workflow 4.1.1 Relevance evaluation factorial outcome and deployment choice To compare settings for the relevance analysis task, a detailed experiment was conducted on the random 250-project sample using the factorial presented, as detailed in 3.2.1. Table 1 presents the outcome for four selected prompts for each engine, selected for their performance or diagnostic value, on the random 250 project dataset: The detailed factorial outcome is presented in Annex 2. Selection-aware resampling (Annex 11) places the optimism of the Table 1 maxima at 0.02–0.04 binary macro-F1 under argmax selection and below 0.01 under the procedure actually followed, in which only the scoring definition was chosen from the data and the prompt style was fixed on operational grounds; the corrected estimate for the deployed GPT-OSS-120B configuration is 0.92 under a per-engine argmax and 0.94 under the procedure used, and the engine ranking is stable under resampling. While GPT-OSS-120B recorded the highest binary macro-F1 in the factorial evaluation, the two-configuration ensemble — GPT-OSS-120B and Llama-3.3-70B run in parallel, reconciled by agreement and confidence-based routing — was adopted for production deployment. Running two configurations in parallel is a robustness measure: it substantially reduces the manual review burden without depending on a single model’s judgement, and it provides the disagreement signal that targets expert review at the genuinely uncertain cases. Figure 3 presents the effect of the more detailed scoring definition on the agreement level with the reviewer assessment. The benefit of the detailed scoring definition (encoding the degree-of-centrality principle and the grey-zone tie-break rules) is not uniform across models: it scales inversely with baseline model capability. The smaller models gain the most from the detailed definition, while the larger models gain comparatively less — indicating that for smaller models a greater share of the task knowledge must be supplied explicitly in the prompt. The practical consequence differs by scale: for Llama-3.3-70B the detailed definition closes the gap to the larger engines, bringing it to comparable agreement performance (Table 1), whereas for Mistral-24B the improvement, although the largest of the four engines, still leaves agreement below the level adopted for deployment on this specific task. Engine choice involves a cost-performance trade-off, not only an accuracy ranking (Fig. 4). On the random 250-project sample, GPT-OSS-120B combines the highest binary macro-F1 (0.94; three-class 0.90) with the lowest estimated serial processing time of the engines at that accuracy level (Annex 5). Llama-3.3-70B trades a few points of accuracy for the fewest tokens per call of the four engines, which motivates its use as the second, recall-oriented member of the deployment Random 250-project sample. “3-class” denotes three-class scoring (0/1/2) and “Binary” the collapsed include/exclude scoring. wκ is the quadratic-weighted Cohen’s κ; CI is the 95% bootstrap interval on macro-F1; latency is the per-project mean for the configuration shown. Precision and recall are macro-averaged over the include and exclude classes; for the include class alone, the deployed configurations reach precision 0.97 and recall 0.85 (GPT-OSS-120B) and 0.72 and 0.94 (Llama-3.3-70B), and together recover 32 of the 33 relevant projects (97%) at a false-positive rate of 5.5%. For three of the four engines the configuration shown is the one with the highest binary macro-F1 on this sample; for Llama-3.3-70B the configuration reported is statistically indistinguishable from the highest-scoring one for that engine and is preferred on operational grounds (Annex 11). These values are in-sample maxima, and selection-aware estimates are given in Annex 11. The choice of prompt style within each engine’s statistically indistinguishable configurations is explained in Annex 11 ensemble rather than as a standalone choice. MiniMax-M2 achieves comparable accuracy to GPT-OSS-120B but at roughly seven times the estimated runtime for the same workload (Annex 5). Figure 5 presents the deployment behaviour on the 250-project reference sample, which includes the ensemble run of the two deployed configurations, and the routing of their qualifying disagreements to expert review. 4.1.2 Horizon portfolio outcome The relevance workflow was applied to the full set of Horizon Europe projects recorded in CORDIS up to 1 March 2026 — 19,025 projects. Of these, 2,184 (11.5%) were assessed as relevant to TRIMIS: 977 as TRIMIS-core (transport as the primary subject) and 1,207 as TRIMIS-extended (substantively transport-relevant as one contributing dimension). The relevant share is distributed unevenly across the portfolio’s institutional and thematic structure (Table 2). The thematic Clusters carry the largest number of relevant projects (823, 23.9% of Cluster-funded projects), concentrated in Cluster 5 (Climate, Energy and Mobility). The Joint Undertakings and partnerships show the highest relevant share (44.0%), driven by the transport-sector JUs. Investigator- and innovation-driven instruments — the ERC, Marie Skłodowska-Curie Actions and the European Innovation Council — contribute large project counts but markedly lower relevant shares (4.6–11.3%), consistent with their non-thematic scope; they are nonetheless numerically important, since their scale means they still account for a substantial absolute number of relevant projects (948 combined). A more detailed view of the relevance distribution specifically within the HORIZON CL5 - Climate, Energy and Mobility subgroups can be seen in Fig. 6. Within Cluster 5 (Climate, Energy and Mobility), relevance concentrates sharply in the two transport destinations: projects under destinations D5 (clean and competitive solutions for all transport modes) and D6 (safe, resilient transport and smart mobility services) are essentially fully relevant, whilst the presence of relevant projects in less expected Destinations such as D1 and D4 highlight the advantage of using LLMs for a more comprehensive analysis. The same pattern is visible at the level of the Joint Undertakings (Fig. 7): the transport-sector partnerships — SESAR 3, Clean Aviation and Europe’s Rail — are almost entirely relevant, whereas the health-oriented partnerships contain no relevant projects, and hydrogen and digital partnerships fall in between according to their transport proximity. Because these destination and partnership labels play no part in the screening — which reads only project text — this concentration provides internal face-validity: the workflow places inclusions where transport research sits. 4.1.3 Cross-examination against manual curation To assess whether the workflow reproduces expert curation, its output was compared against the manually curated TRIMIS record for the same programmes and period (H2020 and Horizon, from 1/1/2014 to 1/10/2024, 2,448 curated projects matched by CORDIS identifier). The workflow independently flagged 95.0% of the curated projects as relevant — direct evidence that it reproduces expert relevance judgement without access to the curators’ decisions. The comparison also surfaced a substantial body of transport-core projects (score 2) that the workflow judged relevant but that are absent from the curated record (Fig. 8). These fall into two groups. A first group sits in thematic lines that curation does cover — the Climate/Energy/Mobility and Digital clusters and the transport Joint Undertakings — and represents candidate omissions for curator review. A much larger group (roughly three in five) sits in non-thematic instruments that call-based curation does not systematically scan — the ERC, Marie Skłodowska-Curie Actions, the European Innovation Council and the SME Instrument — where transport-relevant work is funded without any transport-specific call to filter on. Recovering this second group is the workflow’s distinctive contribution: it extends portfolio coverage into funding lines that a call-based manual process cannot practically reach. 4.2 Categorisation workflow 4.2.1 Categorisation outcome Figure 9 summarises how the classified projects distribute across the three taxonomies, using the deployment model on all projects found relevant. Because a project can carry several labels in a dimension, each bar is split into the share where the label is the project’s only one (darker) and the share where it is combined with others (lighter). Road and airborne are the most common modes, at about a quarter of projects each, ahead of waterborne and rail; multimodal is rare. On sector, passenger and freight dominate and frequently co-occur. Among thematic areas, vehicle design and manufacturing is the largest single class (32%), but the residual “Other” class is nearly as large (28%, highlighted) — a first indication that Note: bolded/highlighted rows (Cluster 5 and Joint Undertakings & Partnerships) are the main thematic contributors to TRIMIS-relevant projects the thematic taxonomy does not yet cover the portfolio fully, discussed in Sect. 4.2.2. A large share of projects carry no specific mode or sector; these counts are given in the caption rather than shown as bars, since “unspecified” is not a category used in TRIMIS. 4.2.2 Workflow analysis and refinement The categorisation workflow focused on ascribing labels to relevant projects within three distinct taxonomies: transport mode, transport sector, and transport technology thematic area. The LLM was provided with a taxonomy based on existing labels in the TRIMIS database and tasked with assigning the most appropriate labels to each project. For this workflow, the prompt was refined iteratively across successive small, hand-adjudicated batches. Agreement with the curated TRIMIS labels was then assessed on the held-out sample described above, per dimension and per category, treating neither source as ground truth. Because the curated TRIMIS labels were assigned non-systematically and do not constitute a validated ground truth, the categorisation is assessed as agreement between two independent processes rather than as accuracy against a reference. On a random sample of 200 projects classified with the final prompt, agreement was strongest for transport mode (70–81%) and for the well-defined thematic areas, where the technically specific categories — alternative energy, vehicle design, connected and automated transport, network and traffic management, electrification and smart mobility — reached Cohen’s κ between 0.53 and 0.69. Agreement was lowest for transport sector, and the divergence was directional: the curated record pairs passenger and freight in roughly two-thirds of projects, whereas the prompt assigns freight only on specific textual evidence. As in the relevance workflow, performance varied little across prompt formulations, indicating that the residual human–AI differences arise from the definition and application of the taxonomies rather than from prompt style. This analysis helped identify shortcomings of both the AI’s interpretation and the existing labelling system, providing insights for improving the classification framework. Expert review of the divergences found the automated labelling more consistent with the stated definition of ‘multimodal’ than the curated record; found that sector labels required a prompt revision to admit implicit mentions; and located the main residual human–AI difference in overlapping thematic areas such as connected and automated transport and smart mobility and services. Finally, the workflow highlights the projects it cannot place. About one in five classified projects share a residual signature: no specific mode, no specific sector, and the “Other” theme. These are of two kinds. A minority are genuine transport projects that the current taxonomy cannot express — for instance transport policy and governance, mobility behaviour, skills and training, coordination and support actions, transport safety and resilience, or cross-cutting enabling technologies — indicating thematic areas the taxonomy could later add. The majority, about 13% of all classified projects, were admitted on a weak or peripheral transport link, as the relevance step’s own justifications show (for example, “transport is mentioned only as one of several applications”). We keep these projects in the corpus but treat the signature as a built-in detection signal: it points both to candidate taxonomy extensions and to inclusions that merit a second look, and is the object of ongoing investigation. 4.3 Competitiveness workflow 4.3.1 Synthesis of the task and qualitative evidence The competitiveness assessment covered 150 EU automotive R&I projects, spanning the EU Framework Programmes H2020 and Horizon Europe, organised under the following themes: Connected, Cooperative and Automated Mobility, Software-Defined Vehicle, Batteries for electric vehicles, ZEV Components, ZEV infrastructure, Vehicle deployment, Advanced Materials and Manufacturing, and Workforce transformation. The analysis combines the scoring framework with evidence extraction. Table 3 lists the eleven dimensions by cluster; their role for competitiveness, assessment criteria and extracted evidence examples are given in Annex 7 (Table A7.4). 4.3.2 Evaluation The evaluation procedure for the competitiveness workflow follows the methodology described in § 3.3 (two expert-referenced calibration and validation sets, agreement reported on both the three-level scale and the binary evidence-present/absent collapse; full statistics in Annex 7). Figure 10 positions each dimension by how strongly the two strict engines reproduce one another and how well the workflow agrees with the expert reference. Most dimensions fall in the reproducible-and-aligned region, and their theme-level values can be read directly. EU leadership stands apart: the two engines agree almost perfectly with each other (≈ 95%) yet with the expert reference only ≈ 18%. This combination is a diagnostic rather than model noise: revisiting the extracted evidence against the deliberately narrow criterion definition showed that in eight of the ten calibration projects the evidence concerned advanced technology or raw-material substitution rather than an explicitly stated European capability, and the definition was tightened accordingly (worked example in Annex 7). All reported agreement figures use the experts’ recorded scores as they stand. Repeatability is not uniform across dimensions; the portfolio-level view (Fig. 11) marks, dimension by dimension, which trends carry reproducible absolute levels and which support only comparison across dimensions. 4.3.3 Main quantitative patterns across themes Figure 11 shows, for each competitiveness dimension, the share of the 150 deployed projects with evidence present according to each engine, with the median marked as a robust coverage anchor. Where the three engines cluster tightly the portfolio level is reproducible and can be read directly — circularity (≈ 31% of projects), interoperability (≈ 44%), raw materials (≈ 23%) and skills development (≈ 19%) — while technical novelty is claimed across almost the entire portfolio and so acts as a ceiling rather than a discriminator. Where the engines spread widely, only relative comparison across dimensions is safe (operational efficiency, consumer price, societal readiness, user features), because the absolute level moves with rater strictness. Two dimensions have the widest spread — EU leadership and production scaling — but the spread is driven entirely by the single lenient engine; the two strict engines agree closely, and a verbatim-evidence check confirms their reading, so both are reported from the strict engines. Explicit EU-leadership evidence — specific first-of-a-kind European capabilities, such as first-of-a-kind large-scale demonstrations or capabilities that reduce external dependencies — is present in roughly a fifth to a third of projects, and genuine manufacturing scale-up in about a sixth; the lenient engine over-credits European policy-alignment and fleet deployment respectively. Because the pipeline reads the macroscopic project summary rather than the underlying deliverables, these coverage figures are best read as lower bounds. The theme-level heatmap (Fig. 12) shows that competitiveness contributions are unevenly distributed across thematic areas and that the portfolio is characterised by differentiated profiles rather than a single dominant model of competitiveness. The trends below are read in line with these reliability tiers: levels are interpreted on the high-confidence dimensions, and cross-theme comparisons on the remainder. A first robust pattern is the prominence of technical novelty across much of the portfolio. It is particularly strong in software-defined vehicle, EV components, advanced materials and manufacturing, and batteries for electric vehicles. This suggests that many projects contribute through technically specific advances that are explicitly framed as going beyond current practice. At the same time, these high novelty scores are not mirrored equally by production scaling, which remains low across almost all themes. This indicates that the portfolio is rich in technical innovation, but that explicit manufacturing-scale evidence is much rarer. A second pattern is the concentration of interoperability and operational efficiency in infrastructure and deployment-related themes. Charging infrastructure, ZEV infrastructure, vehicle deployment, and connected, cooperative and automated mobility show especially strong scores in these dimensions. This is consistent with projects focused on interfaces, communication protocols, energy management, traffic integration, operational performance, and complex multi-actor system environments. 4.3.4 Evolution over time The normalised time chart in Fig. 13 shows how the relative composition of competitiveness contributions changes over active project years. The chart is normalised, and it should be read as a change in portfolio profile over time rather than as a change in raw output volume. Annual contributions are aggregated from the Fig. 11 EU automotive R&I portfolio: share of the 150 projects with evidence present on each competitiveness dimension (three engines; median coverage marked; tight cluster = reproducible level, widespread = strictness-dependent) per-project median score across the three engines and smoothed with a 3-year centred moving average to suppress year-to-year sampling variation; the series runs to 2027 — the final year of Horizon Europe, before the next Framework Programme (2028–2034) — and the earliest years rest on few active projects and should be read as indicative. The most visible trends are: • The increasing weight of technical novelty and interoperability, • The introduction of EU leadership, manufacturing, raw materials, and circularity starting from 2020 onwards, • The decreasing weight of operational efficiency, skills development, and societal readiness-oriented actions, • A relatively constant trend with regards to consumer price and features-oriented actions, which however are main enablers of competitiveness. The time-based profile therefore adds an important dynamic reading to the thematic analysis as the project portfolio does not only vary across themes but also changes in composition over time. 5 Discussion This paper proposes and assesses a new approach for knowledge management using LLMs for the TRIMIS database, developed, tested and used across several use cases with promising results. The project portfolio mapping exercise across transport modes, sectors, thematic areas, and finally competitiveness dimensions can provide valuable insight into trends and potential areas that require additional focus. For example, in our analysis of the competitiveness trends, the relatively lower weight of actions supporting reduced consumer price, and increased production scaling, may point to future focus areas for R&I. On the relevance screening task, analysis of the residual disagreement points to a consistent issue of the task: the difficult part of relevance screening is not detecting transport content but grading its strength. Reviewers reliably agree on whether a transport dimension is present and diverge mainly on how dominant it is. Consistent with this, no disagreement spanned the extreme classes — a project judged strongly relevant by one reviewer was never judged irrelevant by another — and agreement was within a single scale step in essentially every case. The contested items concentrate on a small group of projects — enabling technologies and adjacent-domain projects — on which human scoring tended to over-score. These cases fall into a small number of recurring categories, which the detailed scoring definition resolves through explicit tie-break rules: fuels and energy carriers, scored by end-use; batteries and e-mobility components; dual-use components such as motors, magnets and fuel cells; the built environment; incidental application mentions; and space systems (rules in Annex 1.2). These are the categories on which disagreement concentrated in the annotated sample. Because these boundaries are genuinely contestable rather than a matter of model error, they are best understood as a property of the curation task itself rather than a failure mode specific to large language models. This is why the workflow does not attempt to resolve them automatically: the detailed scoring definition fixes the reproducible decisions, while the residual, inherently ambiguous cases are precisely those escalated to expert review through the confidence and ensemble-disagreement routing described in the Methodology. Relevance screening and classification (mode, sector, thematic area) are moving toward operational, human-in-the-loop deployment within TRIMIS’s curation process, with this validation serving as its evidentiary basis; competitiveness assessment is an ad-hoc synthesis case study that branches from the same pipeline. It is important also to discuss the limitations of the approach, given the behavioural characteristics of LLMs and the nature and availability of the input data. Limitations of the relevance workflow. The relevance-screening workflow is subject to limitations. Performance depends on the formulation of the scoring definition, and the smaller models showed greater sensitivity to changes in prompt wording — equivalently, greater responsiveness to prompt engineering — which means that a configuration validated at one point may not transfer unchanged to another model or prompt. More generally, the reported results are valid for the specific benchmark performed, namely transport-relevance screening of project descriptions, on the specific open-weight models deployed on the JRC infrastructure at the time of the study, under its local serving configuration. The same models may perform differently on other tasks and other infrastructure, and other model grades and types from the same developers were not tested. The approach also inherits the general behavioural characteristics of large language models: outputs can be influenced by surface wording and by broad semantic associations, and the open-weight models used here are periodically updated, so that identical prompts may yield slightly different behaviour over time. This includes model bias — systematic over- or under-association of domains or terminologies with transport relevance inherited from training data — which the workflow constrains: verbatim-evidence grounding ties each decision to the project text, and validation against independent expert references makes systematic deviation measurable. As set out above, the residual disagreement is concentrated on the boundary categories and reflects the difficulty of the curation task rather than model error. Finally, the evaluation rests on a sample of limited size in the extended and core classes, so the corresponding figures should be read as indicative. These considerations reinforce the case for retaining expert oversight, particularly for the boundary cases and for the periodic revalidation of the configuration as the underlying models evolve. Competitiveness assessment. The competitiveness application illustrates what the workflow is for: extracting portfolio-level trends that inform transport R&I programming, not certifying individual project scores. This distinction matters statistically as well as conceptually — estimating how a portfolio distributes across competitiveness dimensions is a more forgiving target than classifying every project correctly, because individual errors in opposite directions partly cancel at the aggregate level. The results carry direct policy signals: actions on consumer price and production scaling are comparatively scarce — genuine manufacturing scale-up appears in roughly a sixth of projects on the strict reading — pointing to candidate focus areas for future R&I calls; circularity is the most strongly represented sustainability-related dimension; and explicit EU-leadership evidence, in the form of specific first-of-a-kind European capabilities, is present in roughly a fifth to a third of projects, pointing to candidate focus areas for future R&I calls. The limitations are real but managed structurally. Expert agreement on this task is moderate because the task is interpretive; the automated assessment approaches the expert consensus (in line with evidence that large language models can match human inter-coder agreement on text assessment ), while adding what a time-pressured expert panel cannot: identical application of the criteria across 150 projects, a verbatim evidence quote behind every non-zero score, and measured run-to-run stability. Residual uncertainty is handled by the hybrid design: where independent models disagree — roughly a third of criterion-level judgements (29% of present/absent decisions and 37% of full-scale scores on the 150 deployed projects, Annex 8), concentrated on the same boundary cases that divide human experts — the case is flagged and presented to a reviewer with the competing evidence quotes side by side, a pattern consistent with validated human-in-the-loop annotation practice. The reviewer adjudicates on evidence. The assessment has been validated at project level on 20 double-scored projects; the framework is presented as a portfolio-level screening aid, and broader independent validation would be required before its scores are used for individual-project assessment or formal policy evaluation. Finally, the instrument generalises: the criterion set is the only topic-specific component, so the same summarise–score–calibrate–validate–project pipeline can be re-targeted from competitiveness to, for example, sustainability, resilience or digitalisation dimensions of the transport R&I portfolio, at the marginal cost of one expert calibration round — turning a one-off assessment into a reusable knowledge-management capability for TRIMIS and comparable observatories. The division of labour differs by task. Relevance screening is a judgement an expert can make quickly from an abstract, so the model’s contribution there is mainly scale. Competitiveness assessment is more demanding for a human — each project carries far more input, across eleven dimensions — so the model’s contribution is scale and consistency: one definition applied identically to every project, with a verbatim quotation behind every non-zero score. In both cases the experts remain the authority; they specify the dimensions, calibrate the definitions, review the cases the workflow flags, and synthesise the portfolio reading. The workflow is designed to concentrate expert time on those steps rather than on scoring every cell. Applied to the full set, the ensemble scored all 1,650 dimension-level judgements for the 150 projects in about two hours of serial model time, each non-zero score carrying a verbatim supporting quotation; the three engines agreed on 71% of the present/absent decisions, and the contested remainder is concentrated on the most interpretive dimensions (Annex 8), so review effort is directed to the cases that require it. The results have also exposed the need for strategic updates to the TRIMIS database. For example, in the categorisation workflow much of the divergence arose from the fact that in the TRIMIS database the projects were placed under a single category, whereas the AI classification often attributed more than one category given that they are not mutually exclusive. Such updates to the TRIMIS database contribute to its completeness and accuracy and are in themselves a useful output of this research. 6 Conclusion This study developed and evaluated an AI-assisted workflow using an ensemble of two large language models to address the time-consuming and resource-intensive process of curating the TRIMIS transport research database. By employing a rigorous, iterative prompt engineering methodology, it was demonstrated that a well-designed AI pipeline can enhance the manual curation process. On the random 250-project reference sample and the include/exclude decision, the deployed GPT-OSS-120B configuration reached a macro-F1 of 0.94 (κ = 0.89) with include-class precision of 0.97 and recall of 0.85 (Table 1); the deployed pair together recovered 97% of relevant projects at a false-positive rate of 5.5%, with 5.2% of projects routed to expert review (Fig. 5). The full set of 52,923 projects was processed in about 48 h of serial model time for the two configurations at their measured per-call latencies (Table 1), or about 25 h with the two engines run concurrently, excluding queueing and other system overheads — a task that would typically require several person-months. The implications of these findings extend beyond efficiency improvement but are important as a case study that AI-assisted approaches can serve as valuable analytical tools. By identifying relevant transport projects in bottom-up funding calls, such as from the ERC, EIC, and MSCA calls, that could have previously been overlooked the AI-enhanced workflow can help to increase the comprehensiveness of the database. The study culminated in a targeted use case assessing the contribution of some 150 EU R&I projects towards the competitiveness of the EU automotive sector, demonstrating that the workflow can deliver reproducible, evidence-grounded portfolio trends. Because only the criterion set is topic-specific, the same instrument extends directly to other policy lenses on the transport R&I portfolio. While the results are promising, it is important to acknowledge the limitations of this approach. The performance is intrinsically linked to the clarity of the taxonomies, and a significant level of human expertise remains indispensable, particularly for assessing ambiguous cases and for the strategic evolution of the classification framework, necessitating a hybrid human-AI review pipeline, where the AI performs the initial large-scale filtering and categorisation, flagging borderline cases for expert review. Two further limitations should be stated. No untouched held-out sample exists for the relevance workflow: the random sample served to select the deployed configuration, to set the routing threshold and to report performance, and although the selection-aware estimates (Annex 11) indicate the optimism is small, periodic re-validation on incremental CORDIS batches is the operational safeguard. The routing rule uses self-reported confidence operationally while its validation remains preliminary. Because it is triggered by disagreement between the two configurations, it cannot surface projects on which both are confidently wrong in the same direction; the reference sample contains one such case (Annex 11). A practical refinement of the human-in-the-loop design follows from the observation that the residual uncertainty of the screening is concentrated on these same boundary cases rather than spread evenly across the portfolio. Two low-cost signals were found to flag them: the model’s own self-reported confidence, which was systematically lower on the partially relevant class and on the contested grey-zone projects, and the disagreement between two complementary screening configurations. The deployed rule combines the two (3.2.1) to route the small set of genuinely ambiguous projects to expert review automatically, concentrating curator effort where transport relevance is contestable while the large, unambiguous majority is cleared without manual intervention. A preliminary assessment of this strategy on the present samples is encouraging; its validation across further samples is left to future work. In conclusion, this study provides an evaluated case study of the integration of LLMs into public-sector knowledge management, with its performance and limitations quantified on a reference sample. The AI-assisted workflow presented here offers a scalable, accurate, and efficient model for augmenting the capabilities of human experts, ensuring that vital policy support tools like TRIMIS can remain comprehensive and responsive in an ever-expanding landscape of research and innovation. Abbreviations Supplementary Information The online version contains supplementary material available at ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​ g​/​1​0​.​1​1​8​6​/​s​1​2​5​4​4​-​0​2​6​-​0​0​8​5​4​-​w​.​ Supplementary Material 1 Acknowledgements. The authors thank Lorenzo Maineri for the initial IT infrastructure supporting this work. The work was funded by the institutional funds of the Joint Research Centre of the European Commission. Ilias Cheimariotis: Conceptualization (lead), Data curation (lead), Formal analysis (equal), Investigation (supporting), Methodology (lead), Software (lead), Visualization (lead), Writing – original draft (equal). Joao Costa: Conceptualization (supporting), Investigation (equal), Formal analysis (equal), Methodology (supporting), Writing – original draft (equal). Francesca Piazza: Formal analysis (supporting), Methodology (supporting), Investigation (equal), Writing – original draft (supporting). Chiara Lodi: Validation (equal), Investigation (equal), Data curation (supporting), Writing – original draft (supporting). Michal Rataj: Validation (equal), Investigation (equal), Data curation (supporting), Writing – original draft (supporting). Jakub Zawieska: Investigation (equal), Writing – original draft (supporting). Marcin Stepniak: Methodology (supporting), Writing – review & editing (supporting). Monica Grosso: Supervision (lead), Writing – review & editing (supporting). Funding This article has been published open access with the support of the TRA2026 project funded by the European Union. Data availability. Prompts and scoring definitions used for all workflows are provided in full in the Appendix. The validation dataset for the relevance-screening workflow — the per-project outputs of all 52 model and prompt configurations for the random 250-project sample, with the expert reference labels — is publicly available in the JRC Data Catalogue ​(​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​2​9​0​5​/​J​R​C​.​X​4​1​W​8​5​ 8​)​, together with the prompts and the analysis code that reproduces the reported performance statistics, tables and figures. The underlying CORDIS project data are publicly available. TRIMIS curation outputs and the full projection results are institutional products of an operational monitoring system; these are available from the authors on reasonable request, subject to the JRC’s data-governance and release procedures. Declarations Ethics approval and consent to participate This study did not involve human participants, human data or tissue, or animals, therefore, the Ethics statement is not applicable. The reference annotations used for evaluation were produced by members of the research team in the course of their professional duties. Disclaimer The information and views set out are those of the authors and do not necessarily reflect the official opinion of the European Commission. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein. Mention of trade or commercial products does not constitute endorsement or recommendation by the authors or the European Commission. Competing interests Author details 1Joint Research Centre, Ispra, Italy 2Piksel (Italy), Milan, Italy Received: 15 April 2026 / Accepted: 7 October 2026 1. Gkoumas, K., Marques Dos Santos, F. L., Stepniak, M., & Pekár, F. (Dec. 2021). Research and innovation supporting the european sustainable and smart mobility strategy: A technology perspective from recent european union projects. Applied Science, 11, 11981. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​3​3​9​0​/​a​p​p​1​1​2​4​1​1​9​8​ 1​ 2. Grosso, M., Cheimariotis, I., Stepniak, M., Lodi, C., & Marotta, A. (2025). In C. Mobility, P. McNally, B. Carroll, B. Martinez-Pastor, M. Ghosh, Efthymiou, & N. Valantasis-Kanellos (Eds.), The Role of trimis as a policy support tool’, in transport transitions: advancing sustainable and inclusive (pp. 758–764). Cham: Springer Nature Switzerland. in lecture notes in mobility ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​0​7​/​9​7​ 8​-​3​-​0​3​1​-​8​5​5​7​8​-​8​​1​0​3​ 3. Tsakalidis, A., Gkoumas, K., Grosso, M., & Pekár, F. (2020). Dec., TRIMIS: ‘Modular development of an integrated policy-support tool for forward-oriented transport research and innovation analysis’, sustainability, 12, 23, p. 10194, ​h​t​t​ p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​3​3​9​0​/​s​u​1​2​2​3​1​0​1​9​4​ 4. Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023). ‘Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration’, J. Inf. Technol. Case Appl. Res., vol. 25, no. 3, pp. 277–304, Jul. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​8​ 0​/​1​5​2​2​8​0​5​3​.​2​0​2​3​.​2​2​3​3​8​1​4​ 5. Rabby, G., Auer, S., D’Souza, J., & Oelen, A. ‘Fine-tuning and prompt engineering with cognitive knowledge graphs for scholarly knowledge organization’, Sep. 10, 2024, arXiv: arXiv:2409.06433. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​4​8​5​5​0​/​a​r​X​i​v​.​2​4​0​9​.​0​6​ 4​3​3​ 6. Wirtz, B. W., Weyerer, J. C., & Geyer, C.. Artificial intelligence and the public sector—applications and challenges. International Journal of Public Administration, 42, 596–615. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​8​0​/​0​1​9​0​0​6​9​2​.​2​0​1​8​.​1​4​9​8​1​ 0​3​ 7. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is All you need, in advances in neural information processing systems, vol. 30. 8. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding, in proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: Human Language Technologies, 4171–4186. https:/​/doi.or​g/10.18​653/​v1/N19-1423 9. Beltagy, I., Lo, K., & Cohan, A. (2019). SciBERT: A pretrained language model for scientific text, in proceedings of the conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, 3615–3620, 2019. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​8​6​5​3​/​v​1​/​ D​1​9​-​1​3​7​1​ 10. Brown, T. B. (2020). Language models are few-shot learners, in advances in neural information processing systems,33 1877–1901. 11. Gilardi, F., Alizadeh, M., & Kubli, M. (2023). ChatGPT outperforms crowd 12. Alizadeh, M., Kubli, M., Samei, Z., Dehghani, S., Zahedivafa, M., Bermeo, J. D., Korobeynikova, M., & Gilardi, F. (2025). Open-source LLMs for text annotation: a practical guide for model setting and fine-tuning. Journal of Computational Social Science, 8. https:/​/doi.or​g/10.10​07/s​42001-024-00345-9 13. Chae, Y., & Davidson, T. (2026). Large language models for text classification: From Zero-shot learning to instruction-tuning. Sociological Methods & Research, 55, 501–567. https:/​/doi.or​g/10.11​77/0​0491241251325243 14. Heseltine, M., Clemm, B., & von Hohenberg (2024). Large language models as 15. Yin, W., Hay, J., & Roth, D. (2019). Benchmarking Zero-shot text classification: Datasets, evaluation and entailment approach, in proceedings of the conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, 3914–3923, 2019. 16. Laurer, M., van Atteveldt, W., Casas, A., & Welbers, K. (2024). Less annotating, more classifying: addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI. Political Analysis, 32, 84–100. https:/​/doi.or​g/10.10​17/p​an.2023.20 17. Wankmüller, S. (2024). Introduction to Neural transfer learning with transformers for social science text analysis. Sociological Methods & Research, 53, 1676–1752. https:/​/doi.or​g/10.11​77/0​0491241221134527 18. Barberá, P., Boydstun, A. E., Linn, S., McMahon, R., & Nagler, J. (2021). Automated text classification of news articles: A practical guide, Political Analysis, 29 19–42. https:/​/doi.or​g/10.10​17/p​an.2020.8 Author contributions The authors declare no competing interests. 19. Tan, Z., Li, D., Wang, S., Beigi, A., Jiang, B., Bhattacharjee, A., Karami, M., Li, J.. Cheng, L., & Liu, H. (2024). Large language models for data annotation and synthesis: a survey, in proceedings of the conference on empirical methods in natural language processing, 930–957, 2024. 20. Törnberg, P. (2024). Best practices for text annotation with large language 21. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, prompt, and predict: A Systematic survey of prompting methods in natural language processing, acm computing surveys, 55 195. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​ 1​4​5​/​3​5​6​0​8​1​5​ 22. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models, in advances in neural information processing systems, 35, 24824–24837. 23. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners, in advances in neural information processing systems, 35, 22199–22213. 24. Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2022). Finetuned language models are zero-shot learners, in proceedings of the international conference on learning representations. 25. Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2023). Large language models are human-level prompt engineers, in proceedings of the international conference on learning representations. 26. Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate before use: Improving few-shot performance of language models, in proceedings of the 38th international conference on machine learning, 12697–12706. 27. Zheng, M., Pei, J., Logeswaran, L., Lee, M., & Jurgens, D. (2024). When ‘A helpful assistant’ is not really helpful: personas in system prompts do not improve performances of large language models, in findings of the association for computational linguistics: EMNLP 2024. 28. Grimmer, J., & Stewart, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21, 267–297. https:/​/doi.or​g/10.10​93/p​an/mps028 29. van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84, 523–538. ​h​t​t​p​s​:​/​/​d​o​i​.​ o​r​g​/​1​0​.​1​0​0​7​/​s​1​1​1​9​2​-​0​0​9​-​0​1​4​6​-​3​ 30. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022. 31. Grootendorst, M. (2022) BERTopic: Neural topic modeling with a class-based TF-IDF procedure, arXiv:2203.05794. 32. Egger, R., Yu, J., & Topic Modeling Comparison Between, A. (2022) Top2Vec, and BERTopic to Demystify Twitter Posts. Frontiers in Sociology, 7(886498). https:/​/doi.or​g/10.33​89/f​soc.2022.886498 33. Zamazal, O., & Machine Learning support for eu funding project categorization. (2019) The Computer Journal, 62, 11, 1684–1694, ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​9​3​ /​c​o​m​j​n​l​/​b​x​z​0​2​1​.​ 34. CORDIS / European Commission Semi-automatic project classification and EuroSciVoc documentation, n.d., accessed 2 July 2026. 35. Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023). Human-in-the-loop machine learning: a state of 36. van de Schoot, R., et al. (2021) An open source machine learning framework for efficient and transparent systematic reviews. Nature Machine Intelligence, 3, 125–133. https:/​/doi.or​g/10.10​38/s​42256-020-00287-7 37. Miller, B., Linder, F., & Mebane, W R. (2020). Active Learning Approaches for Labeling Text: Review and Assessment of the Performance of Active Learning Approaches. Political Analysis, 28, 532–551. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​1​7​/​p​a​n​.​2​0​2​ 0​.​4​ 38. Nelson, L K., Burk, D., Knudsen, M., & McCall, L. (2021). The Future of Coding: A Comparison of Hand-Coding and Three Types of Computer-Assisted Text Analysis Methods. Sociological Methods & Research, 50, 202–237. ​h​t​t​p​s​:​/​/​d​o​i​.​ o​r​g​/​1​0​.​1​1​7​7​/​0​0​4​9​1​2​4​1​1​8​7​6​9​1​1​4​ 39. Zuiderwijk, A., Chen, Y C., & Salem, F. (2021). Implications of the use of artificial intelligence in public governance: A systematic literature review and a research agenda. Government Information Quarterly, 38(3, article 101577,). https:/​/doi.or​g/10.10​16/j​.giq.2021.101577 40. Chen, T., Gascó-Hernández, M., & Esteve, M. (2024) The Adoption and Implementation of Artificial Intelligence Chatbots in Public Organizations: Evidence from U.S. State Governments. American Review of Public Administration, 54, 255–270. https:/​/doi.or​g/10.11​77/0​2750740231200522 41. Nzobonimpa, S., Savard, J F., & Lawarée, J. (2026). Generative AI in public administration: evaluating a fine-tuned large language model for policy briefing notes. Science and Public Policy article scag026. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​9​3​/​s​c​i​ p​o​l​/​s​c​a​g​0​2​6​ 42. European Commission Transport Research and Innovation Monitoring and Information System (TRIMIS), n.d., accessed 2 July 2026. ​h​t​t​p​s​:​/​/​t​r​i​m​i​s​.​e​c​.​e​u​r​o​ p​a​.​e​u​ 43. A Competitiveness Compass for the EU European Commission, & Communication (2025). COM(2025) 30 final, Brussels, 29 January 2025. ​h​t​t​p​s​:​/​/​e​u​r​-​l​e​x​.​e​u​r​o​ p​a​.​e​u​/​l​e​g​a​l​-​c​o​n​t​e​n​t​/​E​N​/​T​X​T​/​?​u​r​i​=​c​e​l​e​x​:​5​2​0​2​5​D​C​0​0​3​0​ 44. Industrial Action Plan for the European Automotive Sector European Commission, & Communication (2025). COM(2025) 95 final, Brussels, 5 March 2025. https:/​/eur-le​x.europ​a.eu​/legal​-conten​t/EN/TX​T/?u​ri=celex:52025DC0095 45. Forman, G. (2008) Quantifying counts and costs via classification. Data Mining and Knowledge Discovery, 17, 164–206. ​h​t​t​p​s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​0​0​7​/​s​1​0​6​1​ 8​-​0​0​8​-​0​0​9​7​-​y​.​ 46. Hopkins, D J., & King, G. (2010). A method of automated nonparametric content analysis for social science. American Journal of Political Science, 54, 229–247. https:/​/doi.or​g/10.11​11/j​.1540-5907.2009.00428.x 47. Pangakis, N., & Wolken, S. (2025) Keeping humans in the loop: Human-centered automated annotation with generative AI. Proceedings of the International AAAI Conference on Web and Social Media, 19, 1471–1492. ​h​t​t​p​ s​:​/​/​d​o​i​.​o​r​g​/​1​0​.​1​6​0​9​/​i​c​w​s​m​.​v​1​9​i​1​.​3​5​8​8​3​ Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.