Latest / 1 More Paper / Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews — Sadek & Mougenot (2024)

Transcript

Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews — Sadek & Mougenot (2024) · 1 More Paper

• You’re listening to “Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews,” by Malak Sadek and Celine Mougenot. Published in 2024. ISSN: 1044-7318 (Print) 1532-7590 (Online) Journal homepage: the linked source Malak Sadek & Celine Mougenot To cite this article: Malak Sadek & Celine Mougenot (2025) Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews, International Journal of Human–Computer Interaction, 41:17, 10877-10894, DOI: 10.1080/10447318.2024.2439021 © 2024 The Author(s). Published with license by Taylor & Francis Group, LLC. View supplementary material Published online: 18 Dec 2024. Submit your article to this journal Article views: 3613 View related articles View Crossmark data Citing articles: 14 View citing articles Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews Malak Sadek and Celine Mougenot Dyson School of Design Engineering, Imperial College London, London, UK 1. Introduction. The pervasive adoption of AI-based systems across diverse domains underscores the critical need to address design considerations. This study delves into a pressing concern within this context: the lack of sensitivity to stakeholders’ values during system development. In this context, values are taken to mean aspects or concepts that are important to people in their lives, such as privacy, autonomy, and connection. Building on concepts from the framework of Value-Sensitive Design (VSD), recent studies have found two areas where existing design processes and design practices for building AI-based systems are lacking in terms of supporting value-sensitivity: (i) intentionally eliciting and understanding stakeholders’ values (“value elicitation”) and (ii) operationalising those values to ensure that design outcomes embody and respect them (“value embodiment”). AI design processes were found to lack practical guidance and comprehensive support for practices, steps or activities that enable practitioners to identify and understand stakeholders’ values, as well as embed and verify those values in the AI systems they create. Meanwhile, collaborative design activities that aimed to engage and include stakeholders when designing AI-based technologies were also found to have no focus on stakeholders’ values and vary widely in the extent of stakeholder participation they enabled. In light of these findings, the aim of this study is to understand how the lack of methodological support for value elicitation and value embodiment translates into practical barriers faced by AI practitioners, leading to two research questions:  RQ1: What challenges do AI practitioners encounter when eliciting stakeholders’ values? RQ2: How do AI practitioners incorporate both their  own values and stakeholders’ values into the systems they develop? We interviewed a diverse set of 30 participants all of whom work with AI-based systems in different capacities. These include technical practitioners such as data scientists and developers, as well as non-technical practitioners such as UX designers, conversation designers, policy makers, and those in managerial or executive positions. Our findings show that practitioners struggle to identify, engage, and work with stakeholders. This is due to a variety of issues, including practitioners being unsure who the users of their systems are, practitioners not being able to access representative stakeholders easily, and stakeholders using AI as a buzzword or treating it as magic. We also found that practitioners struggle to work with values in practice due to the limitations of support available to them, the multiplicity of ways that values can be interpreted and incorporated, and unhelpful mindsets and perceptions among practitioners. As an increasing number of practitioners in the human-computer interaction (HCI) community begin to encounter and design AI-based systems in their work, this study is of relevance to a number of community members as they endeavor to build more human-centred AI (HCAI). By conducting a rigorous interview study (refer to Section 3) we explore the barriers AI practitioners face, which hinder the processes of value elicitation and value embodiment (refer to Section 4). We then structure this study’s findings and previous work into a concept map consisting of four main practitioner-based barriers to building value-sensitive AI and provide recommendations for overcoming these barriers (refer to Section 5). The goal is to raise awareness of newly-identified barriers and renew the urgency to address them, while structuring both sets of barriers into a concept map to allow future work to explore them systematically and in relation to each other. 2. Background. 2.1. Value-sensitivity and working with values. 2.1.1. Value-Sensitive design. The concept of value-sensitivity originates from the field of Value-Sensitive Design (VSD), which offers methodologies and techniques for understanding stakeholders’ values and embedding them into technologies. VSD focuses on eliciting values from stakeholders, users, and communities who are (or will be) directly or indirectly affected by a given technology, and foregrounding those values during the technology’s design process. The methodology of the design field includes three investigations: conceptual, empirical, and technical. These can be carried out several times and in any order. Conceptual investigations revolve around identifying stakeholders and understanding their contexts, empirical investigations focus on eliciting stakeholders’ values and experiences, and technical investigations work on reflecting on how a given technology can embody and affect those values. There are several practical methods and tools that can be used to identify stakeholders, elicit their values, highlight and address value tensions, consider and embed values in designs, and reflect on the extent to which the technology embodies the chosen values. 2.1.2. Value elicitation and value embodiment. When working with values and engaging in VSD, it is possible to use elicited, context-specific values from relevant stakeholders, or to use pre-defined lists of values of which there are several (for example those provided by the IEEE- 7000 (IEEE 7000-2021, 2021)). This would involve selecting a set of values (such as privacy, security, transparency, and so on) at the beginning of a project and embedding those values into the system’s design through the decisions taken and features introduced. Instead, experts have stated that using elicited values through “value discovery” (i.e., a bottom-up approach of eliciting dynamic stakeholder values) or using a combination of elicited and pre-defined values is preferable over solely “favoring known values” (i.e., a top-down approach using static pre-defined values). This approach involves beginning with stakeholder engagement to understand which values are relevant and important in a given context, and working with those values (potentially alongside other pre-determined values). The benefits of this approach is that stakeholder values and perspectives are captured, avoiding relying solely on practitioners’ assumptions of what matters to stakeholders. Another challenge when working with values is that the values that technology creators intend to embed or respect may not necessarily be reflected in the final system. Additionally, the system’s realised values cannot be evaluated until it is completed and has been in use for a significant amount of time (van de Poel, 2020). Instead, there needs to be a focus on embodied values, which are the values present in intermediate artefacts and prototypes created throughout the design process, to make sure that intended values are being propagated along the design process successfully (van de Poel, 2020). Accordingly, conscious decisions to respect relevant values within design decisions being made and features being built, as well as consistent prototyping and value verification need to take place throughout the AI design process. As mentioned earlier, recent reviews of socio-technical design processes for AI-based systems and collaborative design practices for building AI-powered technologies have found that most processes and design practices are lacking in terms of supporting value elicitation and value embodiment. Accordingly, this study focuses on these two aspects to examine whether they create difficulties for practitioners and act as barriers to creating value-sensitive AI. 2.2. Considering values in AI-based systems. 2.2.1. Value-sensitive AI. Given rising public concern over a number of factors regarding how AI-based systems are designed, built, and used, there have been numerous recent attempts to increase the accessibility, transparency, and inclusiveness of these processes. Several digital applications, design-based toolkits, and methodologies have surfaced to lower the barrier-to-entry to participate in creating these systems, with little-to-no technical background required. Despite the increase in methods and tools for collaboratively designing (co-designing) AI-based systems, there is still a limited focus on value-sensitivity in most approaches. The space of value-sensitive AI is an emerging one, as opposed to the also new, but more developed spaces of human-centred AI (HCAI) and participatory AI. Capel and Brereton (2023) have recently classed work revolving around “values embedded in AI” as falling under the umbrella of HCAI. Interventions in this space mostly take the form of design processes and design approaches (Sadek, Calvo, et al., 2024). The overarching goal of these interventions is to encourage and enable practitioners to design AI systems with stakeholders’ values in mind. Therefore, design processes and frameworks such as as the IEEE 7000 (IEEE 7000-2021, 2021), Requirements Specification for Machine Learned Components, Value Sensitive Algorithm Design, and Value Sensitive Design for Social Goals (Umbrello & Van de Poel, 2021) were created. These processes focus on reorienting different activities within the traditional AI design process towards incorporating and considering stakeholder values. Similarly, design approaches such as design fictions, scenario-based storytelling, and toolkits have been used to engage different stakeholders in the design process and allow them to express and discuss their values with practitioners. 2.2.2. Empirical research on value-sensitive AI. Previous interview studies have looked at AI practitioners’ engagement with Responsible AI and Ethical AI, as well as their use of socio-technical tools, and collaboration with domain experts. Focusing on values, interviews have also been used to elicit discuss specific values with different AI stakeholders and AI users. In terms of combining both areas, this study also centres around practitioners’ burdens, but explores a new facet of AI practitioners’ experiences in relation to creating value-sensitive AI. Specifically, while this study also employs interviews, the aim is to explore the barriers that a wide range of AI industry practitioners face in relation to (i) eliciting values from stakeholders and (ii) achieving value embodiment in the AI systems they create in non-research-based settings. 3. Methods. To understand the challenges AI practitioners face regarding value elicitation and embodiment, semi-structured interviews were conducted with professionals in these fields. 3.1. Participants. Thirty professionals from various backgrounds participated in the study—see Table 1, including: Data Scientists, AI Engineers, Conversation Designers, UX Designers, Ethicists, Researchers, Policy-makers, and Executives. This study adopts an approach that involves engineers, designers and ethicists, among others, as “their disciplinary boundaries are becoming less distinct” p.3. This allows for a more comprehensive understanding of the challenges encountered by diverse practitioners. 3.1.1. Diversity of representation. Participants hailed from diverse industries, encompassing healthcare, retail, finance, design, real estate, automotive, and technology. They represented various company profiles, ranging from startups to multinational corporations and research institutes. Participants possessed varied experience with AI-based systems, spanning from 1 to 30 years (M 1⁄4 6.9, SD 1⁄4 7.6). This deliberate diversity aimed to capture a broad spectrum of challenges without introducing context-specific biases. While predominantly male, with only three female participants, the participant pool reflects the prevailing gender disparity in these fields. 3.1.2. Recruitment process and inclusion criteria. Recruitment occurred through professional networks and social media channels via emails and direct messages. Participants were not pre-screened for experience with values or value-sensitive AI. Only six participants had explicit experience working with human values, while others referenced ethical principles or regulations like GDPR when discussing human values. Recruitment continued until data saturation was reached, indicating no new challenges or barriers were identified during the interview process. 3.1.3. Ethics. The study was conducted with ethical approval from the Science Engineering Technology Research Ethics Committee at Imperial College London under the SETREC reference 21IC7361, and participants provided informed consent. While no compensation was provided for participation, all participants expressed willingness to contribute to the discussion due to the importance of the topic. 3.2. Data collection: Semi-structured interviews. 3.2.1. Interview format. The first author conducted the interviews virtually using platforms such as Microsoft Teams, Zoom, or Google Meet. On average, interviews lasted 36 min, with durations ranging from 16 to 55 min, depending on the interviewee’s role. It is worth noting that the majority of interviews ranged from 30 to 55 min and included all planned questions, with only a few lasting less than 30 min due to participants’ time constraints. In the three cases where interviews were shorter and questions had to be cut, participants were asked the introduction questions, the first four questions on value elicitation, and the last three questions on value embodiment (available in Appendix A). 3.2.2. Interview script. The interview process began with a pilot interview conducted with an AI practitioner to evaluate the clarity of questions and reduce repetition. Based on the feedback received, adjustments were made to the interview questions. Participants were guided through a structured interview script (see Appendix A). Initially, they were asked about their job title, domain, company, and years of experience. Following this, participants responded to general questions about AI-based systems. They then delved into discussions regarding their engagement with stakeholders outside their teams, including end-users or domain experts, where they detailed challenges faced and decisions made, particularly related to value elicitation. Participants also shared insights into the values they prioritised when working and how they integrated different types of values throughout their projects, relating to value embodiment. Additionally, participants provided recommendations or discussed the support or improvements needed to address identified difficulties. Finally, non-technical AI practitioners were presented with additional questions concerning their perceptions of existing guidelines, interventions, and design tools they have used or created, depending on their expertise. 3.3. Data analysis: Thematic analysis. Recordings of the interviews were automatically transcribed and then reviewed for errors. Transcripts were cleaned based on the audio recordings, then coded using thematic analysis on NVivo (v12 for Mac) following the guidelines outlined by Braun and Clarke (2012). The first author coded the interviews, after which the three researchers convened to discuss emerging themes and codes, refining them until consensus was reached. The coding process employed both top-down and bottom-up approaches. Initially, high-level codes were established based on the research questions, encompassing “challenges related to value elicitation or working with stakeholders” and “challenges related to value embodiment or working with values.” Subsequently, these themes were further broken down into emergent sub-themes derived directly from the data. The Findings section provides a detailed exploration of these high-level themes and their corresponding sub-themes. 4. Findings. Results reported are divided based on the two investigated dimensions: value elicitation and value embodiment. Value elicitation looks at practitioners’ challenges when working with stakeholders, including identifying them and communicating with them. Value embodiment examines practitioners’ experiences when working with values in practice and the challenges relating to that. As mentioned earlier in Section 3, when asked about stakeholder values, several practitioners instead referred to ethics, regulations (GDPR), and so on. While this finding is in itself telling of practitioners’ lack of familiarity with values and value-sensitivity, it is highlighted here to explain why terms relating to ethics and values are sometimes used interchangeably in the upcoming sections. 4.1. Value elicitation—working with stakeholders. Value elicitation requires practitioners to work with various stakeholders in order to extract, understand and work with their values. To that end, this section focuses on challenges faced by interviewees when working with stakeholders, ranging from identifying relevant stakeholders, to educating them and communicating with them. 4.1.1. Identifying and involving stakeholders. The first step towards eliciting values from stakeholders is to identify potential stakeholders to engage with. Starting at this preliminary step, practitioners already report struggling to identify stakeholders. 4.1.1.1. Clients confused with users. Given the fact that many AI-based systems are used internally within companies, it can be difficult to identify who end-users are, as practitioners’ clients are often business managers from other companies, and not the direct system users themselves: “I think the problem is that you work for a bank that needs a chatbot cause they have too much pressure on their contact center, so you’re designing it for the bank, not designing it for the individuals that need credit.” (A6). This indirect chain of communication makes it difficult to identify who the true system users are, as well as other direct and indirect stakeholders – especially as the general public might be an indirect stakeholder in several cases. As a result, several practitioners do not interact with the end-users of their systems, which one practitioner has said leads to “a diluted understanding of what’s required,” making it easy to “overlook something that’s very important to a minority or even a large group of the users” (T1) as there is a “disconnection” between client needs and user needs (A6). This diluted understanding and tendency to overlook details when it comes to the true end-users of a system extends to their values as well. 4.1.1.2. Lack of representative stakeholders. Aside from identifying stakeholders, sometimes practitioners are only able to engage with co-workers and “internal staff” (C8) within their company as “Guinea pigs” (A4) or “mock” users, without involving the “wider public” (T10), as “there’s still kind of a tendency to not want to bother the user and not get the actual user feedback, direct feedback.” (C10). This is done for a variety of reasons, the most obvious being the gains in time and costs, and the extra control over employees (A4, C7, C8). Participants have also complained about not being able to recruit a diverse sample of stakeholders (T2) and settling for the “next best thing” while hoping that the sample of people is representative enough (T10, A6). Others reported feeling that it is either “not [the end-users’] concern” (A3) or that they “already know what end customers need” (A5). One participant explained that sometimes employees are believed to have sufficient user knowledge to replace end-users’ presence in meetings and activities (C2). Nevertheless, some participants were aware of the drawbacks of “cheating” (A6) and not consulting “true” stakeholders (T10, C7): “You don’t know what you don’t know... things like when the when Apple put out their health app, it was a team of primarily male developers, and they didn’t have a period tracker in. It is the kind of thing where when you’re missing a representative voice that will be impacted by the technology, it’s very easy to overlook something that’s very important to a minority or even a large group of the users if they’re not represented in the background that has been working.” (T1). Designers and non-technical practitioners especially struggled to make technical practitioners understand the importance of diverse representation: “It would be a process of like taking people’s hands at times and making them understand that there are other people who are users of this system, product innovation people of companies or whoever is actually the consumer of your product or let’s say conversational AI, who have certain requirements, who might think different, who will have a job to do.” (C7). 4.1.2. Engaging with stakeholders. Assuming appropriate stakeholders have been identified and approached, practitioners still report difficulties in engaging with them during the project. These difficulties pose a barrier to value elicitation as they prevent practitioners from meaningfully engaging with stakeholders and involving them more deeply in the design process. Practitioners reported challenges regarding stakeholders’ expectations for projects and miscommunications or tensions that arise. They were also divided in their opinions on whether some base education is needed for stakeholders to effectively participate. 4.1.2.1. Unrealistic expectations. Several participants mentioned stakeholders having unrealistic expectations regarding various aspects, such as: The skills, time, and effort needed to build AI sys tems (C5),  AI’s capabilities (T3, C7) or “what’s achievable technically” (C10), The role of humans in the process and that AI is not  fully automated (A5), The fact that a lot of software engineering is actually  involved (C4),  The fact that AI is not a robot or like science fiction (C1, C8),  The power and quality of data (T8), The limitations of AI and machine learning (ML) mod els (A6). More impactfully, practitioners reported that many clients, and high-level managers and executives, treat AI either as “magic” or as a “buzzword.” When it comes to “the magic wand [they] call AI” (A8), several companies simply want to use AI as a selling point or because of the fact that it is “in fashion” (T5), either by “AI wash[ing]” (T6) without actually truly using AI (A3), or without actually assessing if there’s a need or benefit from it (A1, A7, A9, T9). Several practitioners have described this as clients or executives just wanting to “throw AI at the problem” (A5, T11) or “put data into an AI and see what happens” (A8) as if it were a “magic wand” (A6) or a “powerful thing that can do anything they want” (A2). They also mentioned that clients or companies are almost “blinded by the benefits” (T11) of AI without having a concrete business goal tied to it (C4, A7) or considering its drawbacks (C4): “Because I think everybody is speaking about AI and machine learning, everybody has heard these buzzwords... you know SMBs [Small-Medium Businesses] and smaller companies by now as well, they try to almost beat themselves in a fight of who was the best buzzwords.” (C7). Practitioners noted that these perceptions of AI can lead to “a lot of projects could be built almost for like no reason [due to] misalignment between like the data team and the business” (A4) and that “there’s no like overarching direction to it” (T9) because clients “don’t really know what they want” (C5) (echoed by A8). They also mentioned that sometimes “finding an appropriate use case... [that’s] actually useful for the end user” (C10) was a challenge for them. A lack of alignment between user needs and the AI system also extends to a misalignment between users’ values and the values embedded within the AI, as it was not genuinely designed to suit the context in which it will be used. 4.1.2.2. Educating stakeholders. Several practitioners felt that stakeholders did not need a basic understanding or education in AI-based systems to meaningfully participate. Twelve practitioners shared this sentiment that “they’re really just a user of the of the product at that point, so they don’t need any knowledge.” (C6) and “that they can just use it for what they need to do, what they need to use it for, and don’t worry about what’s going on in there.” (C4). Practitioners’ views that education is not needed were also sometimes rooted in a preconception that end-users can only participate in interface and usability decisions or the “creative bits” (A4) and not in technical decisions, in which case “all they need to do is know what they want” (A3). Participants also explained that stakeholders’ feedback “doesn’t really affect the models” (A3) and that they did not show the AI-related parts to stakeholders, only the front-end (A4, A9). Some practitioners actually preferred that stakeholders have no education in this space. These practitioners felt that it better represents stakeholder diversity (T1, T3). Other practitioners held opposing views, stating numerous benefits to stakeholder education, which are summarised in Table 2. When it comes to educating stakeholders themselves, practitioners employed a number of different strategies or techniques to bridge the knowledge gap, such as offering initial consultations (C5) to highlight common problems (C1) and provide background knowledge (T4, C4) and different options (A1), and making use of user friendly solutions and tools for nontechnical users (A8) as well as metaphors to explain technical concepts to non-technical stakeholders (A2). Both cases can pose challenges to value elicitation as providing stakeholder education makes it more difficult and burdensome for practitioners to engage with stakeholders, while not providing stakeholder education can lead to shallower and less meaningful interactions with stakeholders. 4.1.2.3. Miscommunications and tensions. Eleven practitioners complained about miscommunications that arise between them and clients, and between them and other team members or colleagues. Practitioners struggled with the fact that clients don’t always tell them “the whole story... from the beginning” (A3). These issues create the need for several “back and forth[s]” (A3, T8) and people’s needs and values can then “tend to get blurred out” (A5). Participants mentioned that stakeholders do not know how to “translate” (A7) their needs and requirements “into the machine learning project” (A7) and that it was difficult to translate back problems in a way stakeholders understood (T9), as they do not “speak the same language” (C7). Several practitioners mentioned that these problems were further exacerbated by the lack of interdisciplinarity and diversity across those involved in projects. Practitioners wanted interdisciplinary teams that include more than just data scientists and engineers (A6, T6, C6, T8), speaking of “clashes” (T8) between machine learning practitioners, who wanted all information upfront and those doing user research, who were used to uncovering information iteratively. One technical practitioner also recounted this clash from their perspective, wanting all the information upfront: One of the things that really surprises me is you can’t seem to have a simple conversation up front with someone about what would be a magic solution for any AI, if you ask the clinician, ‘what do you want’, I don’t think they give great answers. It’s a really difficult one to address because I think even having worked with some of them for multiple years, I still feel like why does it take so much time for relevant information to dribble out... It felt to me like that should be easy to define up front, when do you need to know the answer to this, and what actually can be pulled in order to respond to those answers. And yet it didn’t matter how directly when I ask that, no matter how many times I ask that, I don’t think you ever get a completely straight answer... it just seems like this extraordinarily messy process. (T8). Having solely technical teams can make identifying and working with stakeholders values more challenging. However, even when interdisciplinary teams were present, other structural biases also hindered value-based considerations: “I always have this problem that I don’t feel supported that way, the customer may actually agree with me, but I will be pretty much almost always challenged by the developers who, as you may think, are mainly guys, mainly white guys, young, also older I suppose, so yeah white males. I mean, maybe it’s my foreign accent that immediately disqualifies me in their eyes, or whether it’s because I’m a woman. I’ve got everything stacked against me, I’m a woman and I have an accent.” (C2). 4.2. Value embodiment—working with values. This section on value embodiment explores the challenges interviewees face when working with values in practice. These challenges include: the multiplicity of ways that values can be incorporated into practitioners’ work, unhelpful mindsets and perceptions that some practitioners have which could hinder value-related work, and the limitations of various theoretical guidelines and practical tools and processes available. 4.2.1. Multiplicity of approaches to considering values. One identified challenge regarding value embodiment was the number of different approaches practitioners could take in relation to value-based considerations. There were three main patterns regarding how practitioners address and work with values. The patterns range from not working with values at all, to centralizing work around values. Some practitioners did not consider values or did not work with them (A4, A6, A7): “I think in the one I’m currently working on, I haven’t done much, but I’ve had an introduction on ethical AI and there is some like mandatory training on the pillars of AI and these are a bit like guidelines that you apply when working and I guess that there isn’t that many repercussions if not followed, but it’s badly seen. So there’s enough like, I guess, just social and professional pressure to have them.” (A6). Practitioners also mentioned struggling due to the variance in values involved and their importance which differed based on: projects (A3, A6), application domains (C5, A2, A8), phases of the design process (T3), and users’ preferences and motivations (C10). Other practitioners referred the responsibility to third parties, or simply followed existing guidelines and regulations (T3, T4). Several practitioners were unclear as to who the responsibility of conducting value-related considerations should be assigned to. Their remarks highlighted the fact that it was unclear to several practitioners whether responsibilities relating to user values and ethics in general belong to clients, the practitioners’ company, co-workers, or the practitioners themselves: “Ethics and values are often seen as somebody else’s problem. So it might be that so very rarely do the coders take any responsibility, normally it’s like ‘ohh ethics isn’t that legal’s department? Or isn’t it something to do with marketing or human resources, or, you know, governance’ or essentially everybody else sees it as somebody else’s problem and which is why we end up building these systems which contain so many flaws and problems.” (T5). Rarely did any practitioner assign responsibility of handling values and ethical aspects to themselves. Instead, these issues were designated as the responsibility of: anyone but data scientists and technical practitioners (T10, A7), designers (T3), and platform owners (C9). Finally, few practitioners actually placed significant importance onto values and reported that they interact with them in a variety of ways in their work. Participants described working with values as a “delicate process” (T4) of “getting user voices” (T11) and “understanding which parts affect which types of values that are important” (T4). They also described working with values as a “deliberation process” (T9) that can allow practitioners to “unpack what those kind of key challenges are” (T8) and to “try and mitigate those within design choices” (T10), while also “investigat[ing] their own work at more of a philosophical level” (T6). Interestingly, some practitioners placed values on a makeshift scale or hierarchy. Some values were defined as constant or core values that do not change, while other values below them were subject to change and evolve: “I think it’s at least plausible to me that there’s like a few fundamental values that are very, very important things like you know, equity and aggregate welfare, and that maybe it and then there’s kind of like mid-level values like privacy, transparency and stuff. I think there’s a lot of like case dependence or context dependence on what kind of mid-level values come up” (T9). This hierarchy was also described by participants working with conversational AI: “The founding principles of every conversational AI project should be, you know, empathy, inclusion, and human centricity. Those are kind of like the three fundamental principles... And then as your corporate grows, you start also adding dialogues about, you know, when people ask about security or when people are being triggered by something that the assistant says, is there a protocol for that in place?” (C6). In response to these scales and hierarchies, the participants who mentioned them seemed to then focus on what they deemed as “core values” in their work, while treating “mid-level” values as future problems to consider later: “I think first some of the things like trust and transparency that always come up, because they’re sort of are pretty core and they’re relatively universal.” (T11). 4.2.2. Unhelpful mindsets and perceptions. Participants described two mindsets or perceptions regarding value-related considerations that could hinder value embodiment as they make practitioners more reluctant to work with stakeholders’ values. 4.2.2.1. Maturity required. Practitioners attributed handling values and ethical aspects in general to a sort of “maturity” in the company or team where members become “digitally ethically mature” (T10), referring to the teams’ experience with AI. Other practitioners suggested this maturity is an attribute of the AI-based systems and their associated projects themselves (T8) in the sense that they are better developed and complete, or an attribute of the AI target application domain (e.g., finance, retail, etc.) (T3) in the sense that there are established protocols and processes in place. Participant T10 speaks of this maturity and how their team have not quite reached it yet: “That’s a long-term maturity that we can aim to get towards and have gotten in some places already, but right now it’s more from their own personality, their own professional experience, their own interest in, ethics, morals, values-based design or any of these things, but it isn’t necessarily us going out and educating everyone.” 4.2.2.2. Compliance mindset. When discussing values, many practitioners spoke of regulations and the need to comply with certain standards and requirements. Several practitioners mentioned compliance to the GDPR, and especially in relation to the value of privacy, when asked about values in their work (C5, T3, C8). Other participants also mentioned that certifications and regulations are sometimes only sought out for “reputational benefits” (T7) and that socio-technical considerations are made in order to avoid bad publicity: “all people wanna avoid PR [Public Relations] blunders” (T11) as “there’s an immense pressure not to fck up, not to have like a horrible PR destroying thing, so that’s a bit of a driver that keeps you alert to make sure that things remain ethical.” (A6). Practitioners who worked with policy and ethics discussed trying to address these mindsets among their peers (T5, T9, T10): “What we don’t want to do, which I think has happened a bit with data protection is create a separate track of work that then feels like it is bureaucratic, that it’s red tape, that it’s going and asking permission of someone to move forward with a step... We want it to be so it’s seen less as an add on at the end, but more defined as the practice by design, rather than creating something separate that they need to log into to then do the ethics stuff, and then they go back to their, you know, core work or real work or any of those things.” (T10). 4.2.3. Struggles with support available. This section explores the struggles that participants mentioned with regards to support offered by theoretical interventions (such as guidelines and recommendations) and practical interventions (such as tools and design processes). 4.2.3.1. Theoretical support—guidelines & recommendations. When asked about user values, many practitioners referred to guidelines, checklists and recommendations for “ethical” or “responsible” AI instead. In this study, most practitioners saw drawbacks to using these guidelines, but some practitioners described a few potential benefits in certain situations. Benefits and drawbacks mentioned are summarised in Table 3. 4.2.3.2. Practical support—tools & design processes. In terms of process and tool support, it is useful to distinguish between practitioners. Technical practitioners tended to have a more established or mature development process but little support for socio-technical steps and factors such as understanding the needs and values of stakeholders. For example, when asked which values are prioritised, participants mentioned that “no one decides really. If it’s just between the scientists, you know what you want to optimize.” (A3). When asked about how they communicate with stakeholders, they explained that “It’s formal, but there’s no process. It’s a one-to-one meeting between teams and you try to get the information... [we] follow a bit with our gut feeling I guess I have to say” (A1). Finally, when asked about working with values, participants mentioned that “it’s a lot more interpersonal and professional advice driven rather than process driven.” (T10) Non-technical practitioners had more user-centered activities and were more experienced in talking to stakeholders and understanding their needs. However, they lacked a specific overall journey or development process, with each practitioner describing a different process, and little tool support for socio-technical and value-based activities (C7, T7, C9): “There’s none of that kind of journey to fixing a problem... You know, people think they know what they’re doing, or they just don’t have a clue so they launch something and they try and learn as they go.” (C9). Participants (C2) and (T3) especially describe the lack traceability and connectedness across the different tools they use throughout their design processes: “I think that we move between different tools in the design process, so formulating concepts in any kind of tool could be PowerPoint, then designing initial dialogues or dialog flows in for example a Miro board, and then setting up these chatbot interactions in some kind of chatbot platform where the main importance is that it’s fast and easy to implement and then maybe after this implemented and maintained in the chatbot platform for which it is eventually intended... One sort of thing that is missing across this is a way to sort of connect or make sense or to keep traceability across these different platforms.” (T3). Finally, non-technical also practitioners commented on technical practitioners’ work, maintaining that working with values and socio-technical aspects, when present, is currently an embedded practice and not a deep understanding (T10). They maintain that there is a narrow-sighted focus on compliance and that user involvement is shaky, with a lack of a practical guiding process beyond assessments and checklists: “People just think that [technical practitioners] can go for it and with your best intentions you get to best results and I don’t think that’s sufficient, you need to actually be experts and you need the methodologies” (T4). 4.3. Summary of barriers. The findings of this study were synthesised into high-level barriers to value elicitation and value embodiment. These barriers are summarised below: 1. Barriers to Value Elicitation:. Reliance on Stakeholder Proxies  Lack of stakeholder identification due to confus sion between clients and users Lack of involvement of representative s stakeholders Lack of Meaningful Stakeholder Engagement  Unrealistic stakeholder expectations make s involvement difficult Managerial and executive stakeholders treat AI s as magic or as a buzzword, leading to AI-based systems that do not solve a need or problem and therefore make engaging with values less impactful Non-technical stakeholders may need education s to participate meaningfully Miscommunications and tensions between stakes holders can arise A lack of interdisciplinary expertise across s teams makes working with values difficult 2. Barriers to Value Embodiment:. Lack of Value Consideration  Practitioners vary widely in their consideration, s categorisation, and use of values when creating AI-based systems A lack of clarity regarding the practitioners s responsible for value considerations lowers accountability and incentive to engage with values Unhelpful mindsets and perceptions among techs nical practitioners in relation to working with values and other socio-technical aspects can hinder their consideration Struggles with Support Available  Conceptual support such as guidelines and recs ommendations have several drawbacks and depend heavily on practitioners’ usage Practical support such as frameworks and toolkits s are difficult to find and adapt 5. Discussion. 5.1. Concept map of practical barriers to creating value-. sensitive AI Based on the data collected from 30 semi-structured interviews with technical and non-technical AI practitioners, and the subsequent analysis of emergent themes and topics, a concept map of the practical barriers to creating value-sensitive AI through the hindrance of value elicitation and value embodiment practices is presented below in Figure 1. As mentioned previously, the aim of this paper is to identify and understand these barriers, with the goal of inspiring future work to begin exploring solutions. Accordingly, we explore the relation between the seven barriers presented in this framework and other existing findings and literature, and then discuss and reflect on their impact on value elicitation and value embodiment in the following sections. 5.2. Reflecting on barriers’ impact on value elicitation. and embodiment practices 5.2.1. Reliance on stakeholder proxies. Identifying relevant users, onboarding them, and communicating with them are well documented problems in the space of AI design and creation. The term “client chain” is used to describe the situation where clients have their own clients, making end-user identification difficult Kross (2022). Yildirim et al. (2022) also describe a similar situation, where clients are classified as either “enterprise” or “consumer” clients, with enterprise clients having the potential to cause a client chain. In all cases, client chains create a vacuum or distance between practitioners and end-users which hinders practitioners’ ability to elicit and understand users’ values, among other limitations. While confirming that these aspects still pose significant challenges, this study also highlights the impact of these difficulties on eliciting stakeholders’ values. Our findings indicate that these challenges are clearly problematic as the most vulnerable or marginalised stakeholders can often be the most difficult to access. This can lead to biases and a lack of consideration of diverse values. Recent work has shown that different groups of people assign and prioritise different values to the same situation or AI-based system, often very differently to AI developers. Accordingly, it is critical to capture these different perspectives to ensure holistic value elicitation. 5.2.2. Lack of meaningful stakeholder engagement. The second synthesised barrier is a lack of meaningful stakeholder engagement. Even if relevant stakeholders are identified and involved, there are a number of reasons that could render their participation ineffective. As meaningful participation is critical for effective value elicitation, this poses a significant potential barrier. Practitioners in this study highlighted that stakeholders’ unrealistic expectations can make it difficult to involve them deeply in projects. Practitioners in previous studies also reported struggling with stakeholder expectations. However, this study extends these findings to highlight that unrealistic stakeholder expectations can also pose a threat to value elicitation as practitioners might be reluctant to engage with stakeholders more deeply to elicit and understand their values. Practitioners additionally mentioned that several clients, managers, and executives tend to approach AI as either “magic” or a “buzzword,” without understanding its true capabilities and contexts. Academics have touched on a similar problem where not enough product owners ask, “should we?” before starting AI-based projects and stressing the importance of doing so. These insights all point to a necessity of ensuring that AI-based projects have a clear value or need and that stakeholders are made aware of AI capabilities and limitations. Our findings add an additional facet to this necessity. Without meaningful considerations and reflections on whether an AI-based systems will indeed be useful and impactful, any value-related work revolving around value elicitation and embodiment will not yield significant impact. This is because the use-case may not be needed or desired by stakeholders in the first place. Ideally, these considerations need to go hand-in-hand in order to understand whether an AI-based system would enable or violate any critical values and whether alternative approaches may be more suitable. Additionally, in the context of value elicitation, the need for stakeholder education should be considered on a case-by-case, contextual basis. Interestingly, with regards to value elicitation, education is not only constrained to education on how an AI-based system functions or is built, but can also mean education on specific values (including their definitions, methods for operationalisation, and potential impacts), which might be also needed by both technical and non-technical stakeholders in various roles. The need to educate stakeholders, both on technical aspects and on value-related aspects as necessary, also poses an extra burden that might discourage practitioners. Our results highlight that it is therefore crucial to develop methods and tools to facilitate these aspects, therefore facilitating stakeholder value elicitation mechanisms and practitioners’ willingness to engage with them. Additionally, miscommunications can make it difficult or costly to understand stakeholders’ values. These difficulties are synonymous to the challenges of finding a common language or common ground, which is common in multidisciplinary working environments. These miscommunications can impact working with values specifically. Designers and UX practitioners, who are typically most familiar and experienced in working with stakeholders and their values, are more effective when involved in an AI project early on. Developers have also been found to benefit from this collaboration by understanding the “experiential value” of their creation and acting as facilitators for framing or aligning between developers’ work and users’ needs. While more diversity and interdisciplinarity is commonly advocated for in the space of AI design, our findings show that (i) sufficient levels have not yet been achieved, and (ii) they pose a negative impact on value elicitation practices as technical practitioners are largely unequipped and unwilling to do so. In particular, this study involved only three practitioners who identified as females. While this might be perceived as not being representative of diverse practitioners experiences, it in fact reflects the current lack of gender diversity in the field of AI. As highlighted by two participants (T1, A6), this lack of gender diversity directly leads to a misalignment between AI systems and diverse users’ values and experiences. 5.2.3. Lack of value consideration. A large number of participants were unfamiliar with working with values and several others described various challenges when considering values while creating AI-based systems. There was a wide range of reported value-related practices across participants. Some participants said they did not consider values at all in their work. It is interesting here to note that sometimes practitioners have been found to implicitly consider values in their work, but do not explicitly recognise or identify the underlying values guiding their decisions. They thus need guidance and support to surface these values and foster an awareness towards them. Among these practitioners, there was an overall lack of clarity regarding whose responsibility it is to elicit and consider values. Similarly to this study, data scientists in other studies also did not assign responsibility to themselves, neither did researchers or those in a managerial level, and practitioners found it difficult to assign responsibility in many cases. This ambiguity can make it difficult for any practitioner to be held accountable for value-related considerations, especially in teams with no designers. Practitioners who did consider values in their work often described a “value hierarchy” of universal or fundamental values which they often framed as values that no one can disagree on, and mid-level values which can be debated and change across projects, people or domains. Such a hierarchical view of values bears similarities to the distinction that Dindler et al. (2022) make between “macro ethics” considerations made at the beginning of a project (akin to more universal values) and “micro ethics” that are dealt with throughout the project through discussions and negotiations with different stakeholders (akin to more contextual, project-specific values). Practitioners tended to focus on macro ethics and values they perceived as universal, while overlooking or delaying considerations around micro ethics, or contextual values that would require elicitation from stakeholders. The large range in attitudes and approaches towards working with values can be problematic. This points to a larger issue relating to the lack of standardised processes and frameworks for dealing with human values in AI systems. While few exist (Umbrello & Van de Poel, 2021), there is little to no evidence of their use or impact as of yet. Our results show that structuring of values in hierarchies or taxonomies might prove to be a useful technique for helping AI practitioners work through relevant values in a less abstract and more structured approach, which is one avenue we encourage future work to explore. With regards to unhelpful mindsets and perceptions which can prevent or discourage practitioners from working with values, previous works have talked at length about the need to change organisational cultures with respect to values and ethics. In this study, some practitioners felt that a certain level of maturity was needed before value-based aspects can be considered. Similarly, recent work has shown that since beginning their education, computer scientists conceptualise ethics and its importance to becoming “good” computer scientists varies widely and that the majority do not consider ethical aspects sufficiently and approach them as a burden. Overall, it seems that working with values is treated as something achieved when more AI maturity has been reached, instead of being an initial focus. Just as “responsibility shifting” takes place, it might be the case that practitioners hold off thinking about values by framing them as a “future problem,” just as they might frame them as “someone else’s problem.” Additionally, several practitioners from this study focused on compliance with regulations, which is commonplace when designing AI. In the context of working with stakeholders’ values, this focus on compliance creates two problems. On one hand, practitioners are content with “satisficing” (T9) and being good enough, instead of engaging meaningfully with values and stakeholders. On the other hand, regulations and compliance imply rigidity, which can then further demotivate practitioners from engaging with socio-technical considerations perpetuating a harmful cycle. 5.2.4. Struggles with support available. Practitioners interviewed in this study and in related works tended to either ignore guidelines if they did not fit their work or were overly simplified/lacked practical translation, or they used a mishmash of them to suit their needs. When asked about more practical forms of support such as tools and design processes, different practitioners struggled to find these types of support. In general, practitioners of various types have also reported a lack of methods, frameworks and tools for more socio-technical and human-centered aspects of their work, with one describing it as “the wild west”. Many practitioners working on AI are also having to adapt existing methods and techniques from different fields to suit their socio-technical needs. Tying back to our reflections in the previous section, there is a lack of standardisation across the support available to AI practitioners when engaging with human values in their work. Ideally, there should be a combination of different forms of support and interventions available for AI practitioners to be able to select the most suitable one. In all cases, it is important to investigate the impact of existing and future interventions for embedding of human values in AI systems, which there is very little work on at the moment. 5.3. Overcoming AI practitioner barriers to value. elicitation and embodiment Based on the barriers highlighted by AI practitioners during this study and additional insights from practitioners interviewed in related work, we conclude with some strategies and recommendations for overcoming these barriers. It is worth noting that, while existing tools, frameworks, and guidelines have specific limitations in terms of supporting practitioners when engaging with values, a significant part of the challenge is the emergent and abstract nature of values themselves. Several practitioners reported struggling with these aspects as values emerged, evolved, or changed entirely over time. Theoretical and practical forms of support can make working with values more practical and systematic, however, as mentioned by several practitioners, the role of (i) involving diverse and interdisciplinary practitioners who might have more suitable training, and (ii) addressing unhelpful practitioner mindsets that lead to a reluctance and unwillingness to engage with values, should not be overlooked. Accordingly, these aspects are the focus of our recommendations. 5.3.1. Improving value elicitation practices. Practitioners struggled with identifying stakeholders, engaging them, and understanding their values. One approach for facilitating these practices is the use of tools and methodologies for value elicitation. The field of Value Sensitive Design has been criticised for not offering much practical guidance around how to elicit stakeholders’ values. However, a variety of techniques have been effectively used, such as: using surveys, analysing datasets and codes of conduct, and collaborative design activities. Creating tools that allow practitioners to utilise these techniques can make value elicitation more practical. Such tools can also incorporate strategies for identifying stakeholders, such as Critical Systems Heuristics, which identifies and describes a number of direct and indirect stakeholders for AI systems. Additionally, when recruiting representative stakeholders is not possible for any number of reasons, alternative approaches have been found to bring diverse values and voices into the design process. One example is the crowd-sourcing of relevant communities’ values through polling and similar mechanisms. Such approaches allow for an understanding of experiences and values that can be closer to target stakeholders then relying solely on co-workers and team members. 5.3.2. Facilitating more meaningful stakeholder. engagement Another barrier that practitioners faced was engaging stakeholders meaningfully in order to understand their values. The extent to which stakeholders need to be educated on AI-related aspects in order to both manage their expectations and allow them to participate is debatable. In all cases, Responsible AI (RAI) toolkits can be utilised here to effectively improve stakeholders’ AI literacy. Several RAI toolkits have been developed specifically for stakeholder education and provide numerous examples and case studies to aid with understanding different concepts. They also offer a friendlier approach where practitioners can use them in a collaborative or participatory setting with stakeholders to invite more meaningful participation. While increasing diversity and interdisciplinarity within the field of AI is an ongoing commitment and a longer-term goal that begins with educational systems, current practitioners can be trained to improve their empathy and open-mindedness. Leveraging principles and recommendations from frameworks such as Culture Sensitive Design (van Boeijen & Zijlstra, 2020) and Value Sensitive Design can help practitioners who work in smaller, homogeneous teams to elicit, understand and incorporate a range of perspectives more effectively. Whether the responsibility of learning these frameworks lies with individual practitioners, companies who should offer training, or educational institutions is a topic for future discussions. Finally, moving past the “AI hype” is crucial for delivering systems that align with stakeholders’ values meaningfully. The current “AI race” occurring across companies and countries stands as a barrier for responsible and ethical AI design practices and fuels unhelpful, compliance-focused mindsets (Sadek, Kallina, et al., 2024). The use of tools such as the Human Centered AI canvas, Analytics Use Case canvas, and Data Landscape canvas, can encourage more critical thinking around AI use-cases and whether other forms of technology are more suitable. 5.3.3. Understanding the impact of stakeholder values. An underlying catalyst for a number of barriers revolves around practitioners’ willingness to engage with stakeholder values in any capacity. Several of our recommendations revolve around using existing tools and methodologies. Yet, practitioners in this study reported either not knowing about them or not wanting to use them. A major underlying recommendation is to therefore understand why practitioners are unable, and more critically, unwilling to use some of these tools and frameworks. Understanding and improving practitioners’ attitudes and perceptions towards socio-technical considerations and aspects within AI design is critical (Sadek, Constantinides, et al., 2024). A large body of work has found technical practitioners to be unwilling and reluctant to engage with value and ethics-related aspects during AI design, treating them as a burden or extra work. Training and tools that highlight the impact of considering stakeholders’ values, or the dangers of not doing so, can help practitioners move from passive compliance towards more self-initiated and active considerations (Sadek, Kallina, et al., 2024). Additionally, bridging the gap between technical workflows and the outcomes of socio-technical collaborations and considerations, by making these outcomes more technically useful for example, can make practitioners more willing and able to engage in these practices. Creators of interventions such as toolkits and frameworks should therefore include the concept of “technical utility” for practitioners as one of their success metrics (Sadek, Constantinides, et al., 2024). 5.4. Limitations and future work. This study aims to push the space of value-sensitive AI forward by highlighting the practical barriers practitioners face when engaging in value-related work. While several previous works have touched on different aspects and findings of this study, the aim is to synthesise these findings into practical barriers that hinder value elicitation and value embodiment specifically, and to frame them as such. By highlighting these barriers and providing a conceptual framework, the goal is to raise awareness regarding these barriers and their impact on creating value-sensitive AI more widely. This work is a first step towards addressing some of the practical challenges faced by practitioners when developing value-sensitive AI-based systems, by encouraging future research within the HCI community to explore solutions for these barriers. The overarching aim is to make it more practical and desirable for the increasing number of practitioners working with AI-based systems to consider and embed their stakeholders’ values into their creations. Future work will also need to explore each challenge in depth to understand the complex network of factors affecting them, evaluate how effective different strategies can be in addressing them, and determine the extent to which they hinder or undermine the creation of value-sensitive AI-based systems. The scope of this research was around AI technologies and all interviewees were working in the field of AI, making these results indicative of the state of working with values during AI design. Nevertheless, several of the findings and barriers from this study are applicable to value elicitation and value embodiment practices within other technologies. While future work can explore the extent of this generalisability, we believe that these barriers are especially challenging in the context of AI. This is because of the complex, non-deterministic, embedded, and black-box nature of AI, which makes stakeholder involvement and embedding values even more difficult than in traditional, deterministic software with graphical user interfaces. There is also much more interest and “hype” around AI technologies presently, which can lead to more meaningless and shallow engagement for the sake of pushing out AI products quickly. AI systems are therefore more likely to exhibit a lack of proper value elicitation and embodiment, as opposed to other forms of technologies. Finally, AI-based systems often have critical and far-reaching implications for users and communities, and are often created solely by technical practitioners who are unequipped for the necessary socio-technical considerations. Accordingly, several of the recommendations discussed focus on the specifities of AI-based systems. There are two main limitations of the study that need to be highlighted. Firstly, it is difficult to assess whether the opinions and perspectives of the 30 practitioners interviewed generalize to all practitioners working in the space of AI-based systems. While referring to similarities in the results of other studies was used as a technique to increase generalisability, future studies with larger sample sizes are needed. Secondly, given the emphasis on interviewing a wide range of practitioner roles, there are only a few designers included in the study. Designers are likely to be the most familiar with values and value-sensitive design, and so the small number of designers included may have biased the results to be less representative of the full community of AI practitioners. An important question to consider here is whether the responsibility of being aware of stakeholders’ values, and making appropriate considerations accordingly, should fall on other AI practitioners beyond those in design-related positions. Echoing Wong (2021)’s findings: there is “the need for tools, practices, and structures that place responsibilities for values in collectives beyond the individual UX professional” [p. 23]. 6. Conclusion. This study has reported on the practical challenges faced by practitioners working with AI-based systems in terms of value elicitation and value embodiment. 30 practitioners working on technical and non-technical aspects of AI-based systems were interviewed to understand their struggles and opinions. Findings included identifying barriers hindering value elicitation, such as identifying and engaging with stakeholders. They also included difficulties relating to working with values, which hinder value embodiment, such as unhelpful practitioner mindsets and the lack of discoverability of helpful tools and processes. These challenges were summarised and synthesised into a concept map of four practical barriers hindering the creation of value-sensitive AI. Preliminary strategies and recommendations for overcoming these barriers were also discussed. The aim is to identify these barriers and encourage future work to explore solutions, in order to facilitate the creation of value-sensitive AI-based systems and make it more practical and desirable for practitioners to do so. Disclosure statement No potential conflict of interest was reported by the author(s). Funding This work was supported by the Leverhulme Trust through the Leverhulme Centre for the Future of Intelligence under Grant RC-2015-067.