Stakeholder engagement in European research and innovation: An investigation into how and why EU R&I projects develop engagement tools
Luka Gudek, Madhura Rao, Jacqueline Broerse, Angelo Leogrande, Krzysztof Goniewicz, Johannes Starkbaum

TL;DR
This study explores how European research projects develop tools to engage stakeholders, identifying two distinct approaches influenced by EU rules and project teams' priorities.
Contribution
The paper introduces a framework of four phases (Purpose, Prototyping, Praxis, Post-project continuity) and identifies two distinct engagement approaches: project-focused and user-focused.
Findings
Stakeholder engagement tools in EU R&I projects evolve through four distinct phases with unique challenges and opportunities.
Two contrasting approaches to tool development—project-focused and user-focused—emerged based on EU guidelines and project team priorities.
The study provides actionable recommendations for improving stakeholder engagement in EU-funded research projects.
Abstract
The European Union’s research and innovation (R&I) efforts have increasingly prioritised collaboration, co-creation, and stakeholder engagement to address complex systemic challenges in recent decades. However, while stakeholder engagement has become a cornerstone of EU innovation policy in this area, there has been limited research into how tools supporting stakeholder engagement are developed, deployed, and sustained within R&I projects. To address this gap, this article explores factors influencing the development of stakeholder engagement tools in European R&I projects and their relation to the broader European R&I trends. This study adopts a qualitative approach, with conducting semi-structured interviews with 22 participants representing 14 Horizon Europe projects on topics of agri-food, bioeconomy, and sustainability. Data were collected, coded, and analysed concurrently and the…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Figure 1| Project | Engagement tool(s) |
|---|---|
|
| |
| AGROBRIDGES |
|
| CULTiVATE |
|
| ECO-READY |
|
| FIT4FOOD2030 |
|
| FOODITY |
|
| FoodSafety4EU |
|
| FoodShift |
|
| FOX |
|
| FUSILLI |
|
|
| |
| BlueRev |
|
| ALLthings. BioPRO |
|
| FER-PLAY |
|
| UNaLAB |
|
| Sector-agnostic project | |
| RRI tools |
|
| Identified challenges | Strategies (proposed or applied) to overcome
| |
|---|---|---|
| Purpose | -Engagement tools developed by projects
| -Including potential users of tools as early as the
|
|
| ||
| Prototyping | -The input of stakeholders is valued
| -Creating tailored engagement formats that fit the
|
|
| ||
| Praxis | -Limited timeframes of projects make it
| -Outlining monitoring and evaluation frameworks
|
|
| ||
| Post-project continuity | -No incentive to keep the tools active and up
| -Designing follow-up projects and incorporating
|
|
|
| Phase | Project-focused approach | User-focused approach |
|---|---|---|
| Purpose | Narrow tool perception | Open tool perception |
| Prototyping | Moderated feedback | Open input and co-creation |
| Praxis | End-of-project dissemination | Ongoing mobilisation |
| Post-project continuity | Limited effort | Active search for alternative funding |
- —Horizon Europe Framework Programme
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSustainability and Climate Change Governance · Bioeconomy and Sustainability Development · CRISPR and Genetic Engineering
Introduction
In recent decades, Europe ^ 1 ^ has grappled with several challenges, ranging from food security and the climate crisis to maintaining global competitiveness. These grand challenges encompass both environmental and societal aspects and addressing them requires directing societal transitions through ambitious innovation policy ( Mazzucato, 2016). To achieve this, the European Union (EU) employs wide-ranging Research and Innovation (R&I) activities, increasingly turning to models of innovation such as Transformative Innovation Policy and Mission-oriented Innovation Policy which explicitly attempt to turn ambitions of resolving these challenges into actionable policy agendas ( Bergek et al., 2023). However, given the elusive nature of societal transitions, the EU must adopt a distinctly novel and reflexive approach to R&I policy – one that not only acknowledges the complexity of existing system dynamics but also actively navigates and reshapes them to drive meaningful transformation ( Borrás & Edler, 2020; Mazzucato, 2016).
Additionally, R&I funders, such as the EU, increasingly recognise that innovation for these grand challenges must include expansive and dynamic constellations of actors working together ( Howoldt & Borrás, 2023). In the recent years, the EC has therefore introduced various efforts to increase the collaboration and engagement aspects of its R&I policy, to enhance understanding of system dynamics, and to increase legitimacy and facilitate impact of R&I results ( Michali & Eleftherakis, 2022). These have included increased co-creation and inclusion with society under Responsible Research and Innovation ( Burget et al., 2017) and Open Science ( Armeni et al., 2021), and more recently a focus on aligning various societal actors around measurable innovation targets in Mission-Oriented Innovation Policy ( Robinson et al., 2021). Prioritisation of these engagement instruments in R&I could support innovation efforts in various ways, including ensuring the relevance, longevity, and application of R&I processes ( Mazzucato, 2019).
Engagement within European R&I processes takes many different forms. An interesting example is that of various engagement tools developed within R&I projects. Within the context of European R&I, the term 'engagement tool' encompasses a range of interpretations; however, for the purposes of this study we define it as actionable guidelines and practical processes on how to engage with various stakeholders on a specific topic in the context of R&I. While recent research has explored the overall success factors of European R&I projects (e.g. Tenhunen-Lunkka & Honkanen, 2024), no studies specifically focus on the tools developed and deployed within these projects, despite their importance for enhancing stakeholder engagement. Investigating this gap is particularly important as recent research points to the failures of the wide collection of engagement tools to support the implementation of RRI in Horizon 2020 ( Tabarés et al., 2022). Therefore, this study aims to explore factors that influence the development of stakeholder engagement tools in European R&I projects and their relation to the broader European R&I trends. The paper proceeds with an overview of the development of the current European R&I system and the role of food systems R&I within it. This is followed by an empirical analysis of tool development processes within R&I projects focusing mostly, but not exclusively, on engagement in food system transformation. Finally, two distinct approaches to development of these tools are outlined.
European R&I for food system transformation
Ever since the beginning of the European Community, Europe has had a role in coordinating and stimulating R&I activities. This role expanded from bounded technological research – for example on coal and nuclear energy – to the subsequent introduction of transnational cooperation in R&I and the use of Framework Programmes (FPs), which outline the scope and priorities of R&I while also serving as financial instruments ( European Parliament DG for Parliamentary Research Services, 2017).
The role of engagement in R&I has evolved significantly since the adoption of the first Framework Programme (FP). By the time FP6 was adopted in 2000, there was an established focus on the interaction between science, and society and citizens, including a thematic area on 'citizens and governance in a knowledge-based society' ( European Parliament DG for Parliamentary Research Services, 2017). Building on this, the Horizon 2020 programme (the precursor to Horizon Europe) introduced key elements of Responsible Research and Innovation (RRI), including “public engagement, policy deliberation, participatory research agenda-setting, and citizen science” ( Robinson et al., 2021, p. 210). The current FP, Horizon Europe (2021 to 2027) aims to address climate change, achieve Sustainable Development Goals (SDGs), and enhance EU’s competitiveness ( European Commission - DG R&I, 2024b). It introduces specific instruments such as EU missions. These are a novel R&I instrument aimed at aligning innovation with systemic challenges faced by the EU through collaboration and engagement across various levels of governance and diverse societal stakeholders ( European Commission - DG R&I, 2023; Mazzucato, 2019). Additionally, European partnerships support bringing together private and public partners to work on grand challenges through R&I. While Horizon Europe presents innovative R&I approaches (e.g. explicit normative orientation and new methods for engagement with society), further policy focus on implementation of innovation policy with society is needed ( Haddad et al., 2022; Kastrinos & Weber, 2020).
Even though the Horizon Europe introduces various novel engagement instruments to European R&I, such as missions and partnerships, most R&I activities take place through a project-based approach. This is achieved by publishing calls for project proposals, aiming to build international consortia around a certain topic, with a specific scope and envisaged outcomes ( European Commission - REA, 2024). Within Horizon Europe, food systems are an important focus. Food systems bring together a wide array of actors, from small-scale farmers and food processors to policymakers, consumers, and multinational corporations and face a wide variety of urgent complex environmental, social and health challenges. In 2016, Food 2030 was launched as a “research and innovation policy framework supporting the transition towards sustainable, healthy and inclusive food systems, that respect planetary boundaries” ( European Commission - DG R&I, 2024c). This framework operationalizes the challenge of food system transition by outlining thematic priorities and specific pathways to achieving sustainable, healthy and just food systems. Food systems as a special area of R&I are also highlighted in Horizon Europe through Partnerships with for example FutureFoodS – European Partnership for Sustainable Food Systems ( European Commission - DG R&I, 2024a); and missions with for example an EU mission: A Soil Deal for Europe ( European Commission - DG R&I, 2024c). This highlights the complexity of the European R&I environment, while at the same time emphasizing the importance of engagement in R&I activities.
The increased emphasis on meaningful engagement in R&I programs is also visible in projects themselves, with European R&I projects institutionalising RRI, through the development of methods to engage with the public, as well as making those freely available ( Owen et al., 2021). While the project-based work allows the EC to bring R&I closer to citizens, the project logic of European R&I itself might structurally fence off certain actors, asserting specific manners of work, and reproducing employment uncertainty, thereby limiting the impact of engagement instruments (e.g. living labs) through projectification ( Büttner & Leopold, 2016; Erisman et al., 2024). Recent studies indicate that achieving meaningful engagement and transformative ambitions of projects could be hampered by the wider project logic of Horizon Europe, which often leads to depoliticization and inability to address deep injustices ( Luger et al., 2023). In contrast, other studies point to project management as a potential solution to ‘managing the unmanageable’ challenges of EU R&I projects ( Tenhunen-Lunkka & Honkanen, 2024). This makes it important to gain more of an understanding of the process by which the projects implement engagement activities.
Many projects attempt to connect more directly with society through their outcomes. Recently, there has been a trend toward operationalizing these outcomes as ‘tools’ that can be used to engage stakeholders. These tools are designed to engage different users with the core themes of a project. However, their development processes, objectives, methods for reaching stakeholders, and post-project applications remain understudied. The development logics of these tools in R&I projects likely reflect generalisable approaches to tool development. For example, design thinking is deployed as an iterative and intentionally non-linear approach to identifying and solving problems in the private sector and academia and comprises of a range of methods ( Luchs, 2015). Such non-linear approaches attempt to create societally robust knowledge through inclusion of diverse stakeholder perspectives and types of knowledge ( Regeer et al., 2009). Indeed, stakeholder engagement has been increasingly a part of research on sustainability transitions field, with one of the objectives being creation of more robust and practical knowledge ( Huttunen et al., 2022; Schmidt et al., 2020).
On the other hand, the context of EU R&I environment asserts restrictions and specific resource strains on organisations taking part in R&I processes (e.g. Lages et al., 2023), which may restrict what kind of approaches to tool development are feasible. This might encourage more linear-oriented tool development approaches. This study therefore delves deeper into the empirics of tools development within the EU R&I environment in order to understand what kind of logics govern their design.
Methods
Data collection
For this study, we conducted semi-structured interviews with 22 participants representing 14 Horizon Europe projects. Out of these, nine projects focus on food and agricultural systems, four on the bioeconomy, and one sector agnostic project. Between one and three project staff members were interviewed per project, depending on their level of involvement in the tool development process. If a participant had extensive knowledge of this process, no further interviews were conducted for that project. However, if an interviewee had only partial knowledge, additional participants were interviewed to ensure a comprehensive account of the development process. The study focuses on a subset of Horizon Europe projects, namely the projects under the Food 2030 umbrella, attempting to address the challenges of food system transformation. Potential projects were identified via a desk study involving the analysis of all ongoing and recently concluded Horizon Europe food and agriculture projects. Projects websites were scanned for engagement tools. Subsequently, coordinators of all projects that had developed or were in the process of developing such tools were contacted by the first and the second authors. Table 1 presents an overview of the included projects, and the tools developed by them. Initially, interviews were conducted with representatives of 17 projects, but three projects were excluded from the analysis due to the tools not being sufficiently focused on stakeholder engagement.
We selected interviewees through a theoretical sampling strategy ( Glaser & Strauss, 2017). As per Glaser and Strauss, while applying this strategy, the process of data collection is controlled by the emerging theory. The initial decisions pertaining to data collection are based on the general problem area and not on a preconceived theoretical framework. Data were collected, coded, and analysed concurrently and the emerging results guided which group or sub-group to turn to next ( Glaser & Strauss, 2017). As a result of this, although we started with projects with a strong focus on agri-food systems, midway through the data collection process, we decided to reach out to, and interview participants engaged in five other projects – four projects that had developed engagement tools focused on the bioeconomy and one sector-agnostic project focused on stakeholder engagement in research and innovation in general. By including participants who worked on engagement tools outside the food systems context, we aimed to understand whether the divergence in their focus might reveal alternative perspectives or challenge the assumptions we had developed. This approach was intended to uncover any inconsistencies or unique factors that might not have been evident within the food systems projects alone, thereby strengthening the robustness of our findings. The study participants were informed in advance about the purpose of the study and were promised anonymity. All interviews lasted around 60 minutes and were conducted online in English. Excerpts of the interviews are used in the results and quoted verbatim, except when adjusted to improve readability. Descriptions of interviewees have been deliberately withheld to ensure their anonymity. Written and verbal informed consent to record the interviews, analyse the resulting data, and publish the findings, including anonymised excerpts, was requested and obtained from all participants.
Data analysis
Qualitative data analysis software Atlas.ti was used for analysing the interview transcripts. Atlas.ti is proprietary qualitative data analysis software, which the authors accessed through the licence of their institution. Open-source alternatives to Atlas.ti are available, for example, QualCoder. The first step of our analysis was open coding ( Saldaña, 2021). The first and second authors undertook independent, line-by-line analysis of the text and assigned codes describing insights identified in the data. This was an iterative process and involved constant comparison between various pieces of text within the same transcript as well as with text in other interview transcripts. The researchers referred to the video recordings of the interviews from time to time during this stage of analysis to ensure that data were coded in relation to the contexts in which they were shared. Data saturation was reached at 10 interviews for the first author and 11 interviews for the second author, indicating that no new codes could be assigned to the data. After this, the first and second authors compared their codes and combined similar codes. Dissimilar codes were discussed, and most were eventually integrated into the analysis. This resulted in 46 codes assigned to 257 pieces of text.
At this point, it became clear that participants had described their activities chronologically, discussing the steps undertaken as well as the challenges faced and addressed per stage of tool development. Based on this, we identified four phases of the tool development and dissemination process, as described in the consequent section of this paper. The 46 codes that were created during the previous analysis step were then assigned to one or more of the stages.
Results
This section describes the four stages of the engagement tool development and dissemination process, as identified from the data: (1) purpose, (2) prototyping, (3) praxis and (4) post-project continuity. It illustrates the tasks undertaken by project stakeholders in every stage as well as the challenges they faced and strategies they employed to overcome them. Table 2 at the end of this section provides an overview of the results.
Purpose: Assessing the need for engagement tools
The first step towards developing a stakeholder engagement tool entailed reflecting on the purpose of such a tool, the form it would take, and the stakeholders it would serve. This subsection describes how participants define stakeholder engagement tools, the various approaches they take towards their conceptualisation, and the increasing focus in EU-funded food projects on considering the needs of marginalised stakeholders while developing engagement tools.
Interviewees indicated that their consortia developed stakeholder engagement tools to operationalise the knowledge created during the project and better involve relevant actors in achieving the project’s goals. Broadly, the aim was to create a tangible, well-structured output that could be used by stakeholders to undertake action to improve food systems. While this overarching aim appeared to be a shared one, interviewees expressed diverging views regarding what constituted a stakeholder engagement tool.
Some believed that " anything can be a tool" – an inspirational video, a workshop script, a policy brief, a set of best practices, a digital library, or a training programme for setting up open-innovation ecosystems such as Living Labs. These interviewees emphasised the flexibility and broad applicability of a tool in engaging stakeholders. They argued that tools should not be limited to conventional formats but should encompass various media and methods that inspire and educate stakeholders. Others were more cautious in their imagination of a tool and believed that a tool was something that could be ‘used’ by stakeholders as opposed to something that one could merely gain knowledge from. The projects of these interviewees developed tools such as games, mobile applications, databases, and decision-support tools. These tools were designed to be interactive and practicable, providing stakeholders with concrete resources to support their decision-making processes and actions. This view emphasised the importance of usability and functionality in tool development, ensuring that the tools have a direct and tangible impact on stakeholders' involvement in food systems transformation. Both these views could be considered top-down approaches to tool development wherein project partners determined which formats suited their goals best. In these cases, the expertise and vision of the consortium played a significant role in shaping the tools, leveraging their knowledge to create effective engagement strategies.
A few interviewees took a more user-centric approach wherein the needs of potential tool users were assessed first, and tools were developed accordingly. Discussing their perspective, a participant engaged in the Eco-Ready project shared:
“ A tool needs to have well-defined users. When we talk about developing a tool, we need to ask who the potential users are and what benefits they will gain. A tool can be a methodology, a report, a set of policy recommendations, a contingency plan, or a roadmap that guides the user in making better-informed decisions. We can have a vast spectrum of tools in terms of format and complexity, as long as there is a potential user group that benefits from utilising the deliverable.”
This approach prioritises the needs and preferences of the stakeholders, ensuring that the tools are relevant and beneficial to the intended users. By conducting baseline studies, stakeholder mapping exercises, and surveys, many projects aimed to assess the needs of various stakeholders, their interest in the project's topic, and their connectedness to other stakeholders in the food system. The outcomes of such exercises, along with project goals, determined what kind of tools were to be developed and why. The availability of resources and project partners' skills also influenced the feasibility and scope of tool development, guiding the selection of appropriate formats and methods to achieve the desired outcomes.
Better involving marginalised or overlooked actors in their projects was also an important aim for many interviewees when developing an engagement tool. Some were keen on involving groups that are not traditionally considered to be stakeholders in the food system. For instance, one project focused strongly on children and young people because its consortium believed that the new generation is keenly interested in the sustainability movement but frequently overlooked. Other projects focused on specific aspects of engagement, such as improving farmers' digital connectivity or enhancing the skills or knowledge of multiple stakeholder groups from a particular region on a specific topic. The inclusion of migrant workers and consumers with lower socio-economic position by creating tools in languages understood by them is another example of projects' efforts to include marginalised groups. Participants shared that a noteworthy challenge to including such stakeholders was convincing them that their opinion was relevant and needed for the tool prototyping. At an early stage, project partners often found it difficult to promise tangible outcomes and therefore struggled to convince marginalised groups regarding what they might gain from participating in the tool prototyping. Some projects addressed this challenge by offering the participants compensation in the form of food products for their time. This incentive has proven effective in engaging underrepresented participants by providing immediate and tangible benefits for their involvement. However, this is not the case for most projects, and they continue to struggle to involve marginalised participants despite recognising it as a critical area for improvement.
Prototyping: Developing engagement tools in line with project aims and stakeholder needs
Once the purpose of developing a stakeholder engagement tool was established and there is an agreement about the form such a tool would take, the second stage – the prototyping of the tool – takes place. This subsection sheds light on characteristics of this stage such as frequent interaction with target user groups, increased cooperation between project partners, and bottlenecks connected to users as well as some project partners’ lack of familiarity with technology. Additionally, we will describe the frequently reported disconnect between the project proposal and the ground reality of tool development as described by interviewees.
Most participants described the tool prototyping stage as iterative and co-creative. Initially, project partners responsible for the tools convened to develop a prototype. Once such a version was ready, the tools were piloted with members of the target user groups, often in the form of co-creative workshops. Such pilot sessions and workshops facilitated dialogue between tool developers and potential users – be it facilitators or implementers – integrating practical insights with subject expertise. Interviewees emphasised the importance of involving end-users from the outset to ensure the tools were functional and user-friendly. However, different user groups’ opinions appeared to be evaluated differently. Interviewees sometimes exhibited bias towards the validity of the feedback that came from specialist groups such as policymakers, researchers and food systems professionals. On the other hand, user groups such as citizens were perceived as less knowledgeable, particularly about the feasibility of implementing certain changes. A participant working on the AllThings.Bio project described their experience with citizens as follows:
“ If you ask citizens for their opinion and let them be creative, they come up with all kinds of crazy ideas. But then you always have the developer with technical expertise who needs to see what can be done with the capacity and money we have.”
The observation that power inequalities have direct consequences on which stakeholders get to engage in co-production in European R&I projects is increasingly highlighted in recent research (e.g. Cronin et al., 2024). It was also noted that certain groups were better represented at co-creative sessions compared to others. Which groups these were depended on the project partners’ professional networks and the region where the sessions were held. However, stakeholders with steady employment and a personal interest in food systems transformation were found to be more represented in co-creative processes. In contrast, both the least powerful and the most powerful stakeholders were found to be harder to reach, because of lack of interest. An interviewee shared an example of this from a pilot session with potential end users of their tool:
“ They [project partners] have really struggled to reach people who suffer from food insecurity because these people do not have the time nor enough faith in the system to go to such events. Then you also have stakeholders that are really powerful actors such as high earning farmers or supermarket executives and they are notoriously bad at attending any kind of co-creation events because they don’t see the benefit in it for them.”
This often led to the same kind of stakeholders, or at times the exact same set of people, attending feedback or co-creative sessions, causing their preferences to be taken into account more prominently when refining engagement tools.
Interviewees described the tool development phase as requiring frequent cooperation and clear communication among project partners, which posed several challenges. Partners with IT expertise such as coders and web developers often had different views on the feasibility of implementing user feedback compared to project coordinators, or research partners, often leading to discord. Additionally, academic partners were often described as “ speaking a different language” compared to other partners. These communication challenges sometimes resulted in project partners working in silos, contrary to the inter- and transdisciplinary nature of the project.
Accessibility was a key consideration in developing engagement tools, most of which were digital. Users’ affinity for digital technology often became central to the prototyping. Due to Europe’s well-developed digital infrastructure, the high ambitions of many projects, and increased move to digital work from the COVID-19 pandemic, many projects aimed to develop advanced digital tools. While these were well-received by some user groups, such as young people and educators, others were less keen on online engagement. An interviewee engaged in the AgroBridges project shared an example of this:
“ We had one tool that was really not accepted by farmers – an online matchmaking tool. When we tested it, it was almost unanimously rejected by all farmers in all regions. They were annoyed by having to make their profiles, book meetings, and so on. They considered it very complex.”
Besides some users’ lack of familiarity with digital technology, some project partners were also reported to lack the capacity to carry out tasks like updating a website or application. Consequently, partners with a better grasp of such processes had to take on these tasks, leading to an additional burden for them. Some interviewees shared that such challenges were the result of dissonance between what was promised in the project proposal and what was feasible considering time and resource restrictions. Those responsible for executing the project were often not involved in writing the project proposal and felt taxed by the pressure to deliver what was promised. An interviewee shared their thoughts about this as follows:
“The project proposal was already written by my supervisors when I was hired. In the proposals, you can be as creative as you want. But when it comes to putting the proposed ideas into action, it's much more complicated.”
The intense competition to acquire funding was described as a possible reason for overpromising. Further elaborating on how this was a systemic issue in EU-funded projects, they shared:
“ I don't mean to sound arrogant, but the challenges we have faced in developing the tool have nothing to do with my own capacity or the capacity of the partners. It has everything to do with the fact that we're working on an EU-funded project and people who work on these projects are always structurally lacking time and resources to work together. And because of this, collaboration between the different partners has been really, really difficult.”
Praxis: Implementing and disseminating developed tools
Once the prototype for tools has been developed, participants reported an increased focus on piloting and implementing the engagement tools. In this subsection, we describe challenges such as the pressure to popularise their tools within a short period of time, managing translation, and evaluate the success and impact of the developed tools.
The implementation and dissemination of engagement tools involves several strategic steps designed to maximise their impact and ensure that they are effectively utilised by target stakeholders. As per interviewees, these steps often include publicising the tools at in-person events, leveraging their social media networks, enlisting the help of influential intermediaries, translating tool content to improve usability, and making adjustments to accommodate regional differences.
Most interviewees shared that they reached or expected to reach the implementation stage only towards the end of their projects. As a result, they faced considerable pressure to organise publicity campaigns and popularise their tools within a short period of time. For tools developed for professionals, LinkedIn was the most frequently used social media platform. For tools where citizens were seen as the target users, platforms like Instagram and TikTok were used. Although this had not yet been implemented for any of the projects in this study’s sample, some interviewees discussed the possibility of recruiting social media influencers to better publicise developed tools. In offline contexts, interviewees discussed popularising their tools via actors who could be considered ‘multipliers’. These could be consultants, advisors, educators, or anyone with a large network in the food industry. However, the budget for such endeavours was often limited. Regarding this, an interviewee shared:
“ For a communication partner, it is very frustrating when there are so many excellent outputs that we would like to disseminate, but there's no money for dissemination. There is no money for infographics, social media campaigns, publicity events, or anything else.”
A considerable amount of time and resources also need to be spent on translation at this stage. Many interviewees, especially those working on projects spanning several countries, shared that the effort required to translate tools was more than they had expected. Once the translation was underway or completed, making adjustments to the tools also became a major challenge. Often, changes could be made to the original tool, which was usually in English, but there was a lack of resources to implement the changes in translated versions. Stakeholders from countries such as the Netherlands, Belgium and Finland, where English is not the native language but is widely spoken, were also found to be at an advantage since they could use original versions of the tools. However, most interviewees accepted translation and its management as an inherent part of European projects and saw intercultural collaboration as an opportunity to exchange knowledge and learn from different communities. Regarding this, one interviewee shared:
“ This is the beauty of working in European projects, right? You get to bring together and learn from people from very different geographies, cultural backgrounds, contexts, food and farming practices, et cetera.”
The implementation stage was also seen as a conducive period to evaluate the success and impact of the developed tools. Many interviewees monitored website visits and downloads as a way to measure success. For tools that required users to create accounts, it was possible to collect additional data about tool use. However, signing up or creating an account was also mentioned as a barrier to a tool’s uptake and therefore many interviewees preferred not to have it as a requirement. When asked about how they assessed the impact of their engagement tools, all interviewees shared that they did not think it was possible to do this within the timeframe of the project. They admitted that the metrics they assessed merely indicated the success of their publicity campaigns and were not indicative of a tool’s actual use or impact. Even over a longer period, assessing systemic change was thought of as difficult and at times, impossible to achieve. Additionally, since the projects often came to an end soon after the implementation stage, interviewees thought that there was no incentive to evaluate the success or impact of engagement tools. Reflecting on this, an interviewee shared:
“ For some reason, the way we've organised research and innovation in Europe, only short-term accountability is emphasised. During the project, we have to write these huge reports for the European Commission where we indicate how impactful the projects have been. But once the project is over, we stop measuring its impact and we stop drawing on its value.”
Post-project continuity: What happens once funding runs out?
The last stage of the tool development and disseminations process entails ensuring the continued functioning and relevance of the tools. This subsection describes the various strategies such as the development of new related projects, seeking out private and public investors, and turning tools into marketable services that projects undertake to prolong the usability of their tools.
In most cases, the development, maintenance and promotion of engagement tools were feasible only while project funding persisted. This issue was frequently mentioned by interviewees spontaneously, but they were also specifically asked to comment on the future of the tools after their respective projects concluded. Having invested several years and significant resources into the tools, all interviewees concurred that these tools should remain available for public use for several years post-project. Typically, one of the project partners assumes the responsibility of maintaining the project website, where tools could be accessed for three to five years after the project’s completion. However, this usually entailed covering website subscription fees and little more. Other maintenance activities, such as updating a tool or responding to user queries, required additional resources and were deemed unfeasible without further funding. Often, the burden of maintaining tools fell on the project coordinators and their institutions. However, the limited resources and capacity within these organisations made it challenging to ensure continued functionality and relevance of the tools. Commenting on the pressure of such expectations, an interviewee shared:
“ Post-project usability and helping new projects use existing tools are often seen as afterthoughts. Despite being very important pieces of the project, they're managed on a shoestring budget and on the goodwill of project partners. Often, this responsibility is pushed onto the project coordinator. However, it should also be the responsibility of the funders. Funding organisations need to seriously consider how the work that they have funded can be sustained in the long term and lead to transformative change.”
One proposed solution to this dilemma was the establishment of follow-up funding mechanisms specifically targeting the sustainability of promising tools. Such mechanisms could involve a dedicated fund to support the continuation and refinement of tools deemed valuable. This would prevent these tools from falling into a " project graveyard" where they are no longer actively maintained or utilised. Additionally, incorporating long-term planning into the initial project proposal could help address these challenges by allocating resources for post-project maintenance and updates from the outset. For example, when tools developed within a project are relevant to them, external public or civil society stakeholders such as national agencies, ministries, municipalities, and NGOs take over the tools and invest in their maintenance. This strategy often entails modifying tools to match the context in which these stakeholders wish to use them. However, this ensures that the tools remained free to access and useful to the public.
Another suggestion coming from the interviewees was to invest project resources of new projects to improve and build upon existing tools (and toolkits) instead of creating new products from scratch. Regarding this, an interviewee engaged in the FOX project reflected:
“ There are so many great tools, toolkits, apps, and websites out there. Instead of enhancing these existing resources, we are wasting money creating something new just to have our project logo on it. It doesn't matter if the project logo is there, as long as the tool is useful and has a real impact.”
Improved cooperation with other EU-funded projects with adjacent goals (often referred to as ‘sister projects’) and co-development of tools was suggested as yet another way to address the lack of sustainability currently experienced by projects. This could allow projects to save on resources in the tool prototyping in order to reserve funds for tool maintenance once the projects have concluded. Another strategy to build a new project around the developed tools. This strategy involves leveraging promising tools and expertise within the consortium to write proposals to acquire funding for new projects. By doing this, project partners ensured that their tools continued to evolve and remained relevant. This was the most popular continuation strategy discussed by interviewees. The Horizon Europe project ‘CLEVERFOOD’, which seeks to develop an engagement toolkit, was frequently mentioned as a viable solution to prolong the life of the developed tools by establishing a toolkit hosting tools already developed by other projects.
Finally, some projects pursued potential commercialisation opportunities by transforming tools into marketable products or services; which could then be sold by a project partner. This approach requires developing a business model that can sustain the tool financially. Projects in this study explored various revenue streams, such as subscription models, licensing fees, or consultancy services, to support the tools’ maintenance and development. Commercialisation not only ensures the tool's survival but also its continuous improvement and adaptation to market needs. Some projects explored hybrid approaches, combining elements from all three categories to maximise their chances of sustainability. For example, they might integrate their tool into new projects while simultaneously seeking partnerships and exploring commercial opportunities. This approach also requires a critical perspective to a tool development to create a tool perceived as useful by potential users.
Overall, the insights shared by the participants allowed us to identify best practices and challenges associated with all phases of the tool development process. Table 2 presents an overview of this.
Discussion
This study aimed to explore the factors influencing the development of stakeholder engagement tools in European R&I projects and their relation to the broader European R&I trends. As presented in the results section, tools for engagement developed as a part of European R&I projects support social innovation efforts of various projects. The development of these tools can be observed in distinct phases, relating to defining their purpose, the process of developing or designing them, implementing them in praxis, and their post-project continuity. This phased analysis of the tool development process allowed for a more structured examination of the different approaches that projects take to tool development. In addition, the results highlight the growing importance of a distinct approach centring end-users in the development process of stakeholder engagement tools. After discussing the two approaches to tool development, and a need to transition to socially robust tools, this section ends with a reflection on the limitations of the study.
Project-focused and user-focused approaches to tool development
Based on the tool development trajectories of the different projects, two approaches were observed in the study. We demarcate these approaches based on the way they conceptualised engagement. The most common approach – which we termed the ‘project-focused approach’ – observed in our findings was rather narrowly focused on project objectives. This approach is characterised by development of tools according to predetermined project guidelines with a focus on fulfilling the project milestones and deliverables according to the principles of good project governance. In the second approach – the so-called ‘user-focused approach’ – engagement with the relevant stakeholder group(s) is central to the development of the tool. The observed characteristics of both approaches are outlined in Table 3.
Characteristics of the project-focused approach are visible in each distinct phase of the tool development process. In the early purpose-defining stage, projects applying this approach seem to already define the tool format and user benefit. As shown in the previous section, this could be a result of attempting to present value of the tools that are to be developed to the stakeholders who will eventually use it and should be engaged in its development. As this approach seeks to clearly define the tool early in the project, the stakeholders are engaged on the basis of strictly defined guidelines and roles. The kinds of stakeholders engaged by such projects are often limited. Our results suggest that this may be driven by concerns regarding the feasibility of citizens' proposals and perceptions of their limited technical expertise. In the project-focused approach, the dissemination of the developed tools often takes place towards the end of the project. Our analysis indicates that most projects do not dedicate significant budgets to dissemination and largely rely on the existing channels of dissemination such as LinkedIn. With regard to post-project support and tool longevity, the project-focused approach involves following the obligatory guidelines imposed by funders, such as ensuring the project deliverables are available online for at least five years after the project ends. The results also show that some projects develop proposals for new projects by leveraging tools that have been particularly successful.
In our study, we also observe a distinctly different approach to tool development – the user-focused approach. In the early purpose-defining stages of projects, this approach entails involving various stakeholders, especially vulnerable or marginalised groups in conceptualising the tools. Projects employ monetary as well as non-monetary resources to ensure sufficient stakeholder participation. Further on, in the prototyping phase, this approach leaves ample space for an iterative process where user input is not limited to a single feedback session and may even allow for abandoning the initial tool formats and developing new formats that resonate with the received feedback. This is evident in the example we highlight on reworking a digital engagement tool developed for farmers within the AGROBRIDGES project. In the praxis phase, projects employing this approach implement communication plans and outreach strategies with the support of their local partners, based on trust-building and long-term approaches. However, the short project timelines that are characteristic of the Horizon Europe framework often prevent projects from fully realising this. As seen in the results section, interviewees frequently expressed their frustration with short timelines for implementing and disseminating their engagement tools and measuring the tools’ impact. Finally, when it comes to the post-project continuity, the user-focused approach was visible in the creative strategies applied to ensure sustained impact of the tools. These included identifying funding bodies to maintain and further improve the tools developed as a part of project, as well as developing the tools into marketable solutions.
The two approaches represent distinct ways of developing engagement tools in European R&I projects. Each of these approaches presents a group of coherent characteristics. Namely, if a tool is narrowly specified in the first phase (project-focused approach), it will likely follow project-oriented approach characteristics such as limiting the feedback options. However, the results also point that these approaches are not always isolated from each other. Project managers may for example organise an inclusive tool co-creation session, while later in the project be restrained (regarding time, budget, and staff capacity) in how they approach dissemination. Therefore, the two approaches are distinct in their logics across development phases – with the project-focused approach prioritising predefined structures and objectives, and the user-focused approach emphasising adaptability and iterative development based on user input. However, this does not necessarily mean that the different approaches predetermine a development path of a tool.
Transitioning towards socially robust engagement tools
The majority of the projects examined in this study applied the project-focused approach in their work, wherein tools are developed as deliverables within a predetermined project logic. This approach can be regarded as maintaining the status quo as it represents the dominant mode of operation. At the same time, some projects employ a user-focused approach in at least some aspects of tool development. In this approach, tools serve as a means to mobilise stakeholders around a specific topic, with an emphasis on their robustness. As demonstrated in the findings, such projects prioritise coordination strategies that include the early and continual involvement of users in tool conceptualisation.
However, even among projects that predominantly apply the project-focused approach, this choice is not only driven by a lack of interest in or knowledge about user engagement, the choice regarding the applied approach is also driven by structural constraints and requirements from the funder. The competitive nature of European R&I funding often pushes consortia to focus on project-focused aspects such as research excellence and deliverables in their project applications, rather than RRI aspects such as inclusion of vulnerable stakeholders, as they feel the former will be valued more highly ( Tabarés et al., 2022). As a result, there is often insufficient time, funding, and institutional support to adopt a fully user-focused approach. Additionally, the tight timelines of EU projects, combined with the precarious employment conditions of many researchers and project staff, further reinforce a project-focused approach.
Since securing future funding is a constant concern, teams often prioritise meeting deliverables deadlines over engaging in iterative, user-focused tool development. This is compounded by the fact that the individuals who draft grant proposals are not always the same as those who later execute the projects. As a result, commitments to stakeholder engagement made at the proposal stage may prove unrealistic in practice, particularly when organisational resources are stretched thin. Projects might want to include stakeholders for various aims, including the normative belief that it is the right thing to do to include the users of a tool in its design, to the substantive belief that including perspectives of a range of stakeholders will increase the quality of the knowledge fed into developing the tools (these and other aims of stakeholder inclusion were recently summarised by Schmidt et al. (2020). Therefore, not incorporating stakeholder engagement in tool development could result in less robust tools.
Despite these constraints, our findings indicate that there is a growing interest in working in a more user-focused way. While fully embedding this approach remains challenging, we observed instances where user-centredness was integrated into specific stages of tool development, often in an ad hoc manner or where project teams found ways to work around structural barriers. This suggests that there is momentum towards more engaged approaches, but for them to become the norm rather than the exception, changes are needed at multiple levels – ranging from how proposals are evaluated and funded to how projects are managed and supported throughout their lifecycle. Figure 1 presents a graphic overview of this momentum towards a user-focused approach and the changes needed to achieve it.
The transition to a user-focused approach to stakeholder engagement.
As shown in the figure, changes in both the EU R&I structural level as well as the project coordination are critical for moving towards a user-focused approach in developing and implementing stakeholder engagement tools. From our analysis, it is clear that solutions for creating more robust engagement tools come from both venues.
The growing interest in the user-focused approach is important as it emphasises the inclusion of diverse perspectives and knowledge sources to make tools more robust, while also considering the long-term viability of tools beyond a project’s lifespan. By prioritising broad stakeholder involvement, this approach aligns with the evolving European R&I landscape, which has shifted from a narrow focus on technological advancement to a more inclusive model of innovation that actively engages citizens. This transition, as outlined in the introduction, reflects broader trends in EU research policy, where engagement and co-creation have become key pillars of innovation. Furthermore, the user-focused approach resonates with the European Commission’s mission-oriented innovation principles, particularly those emphasised by Mazzucato (2018): broad participation, the co-production of robust knowledge, and interdisciplinary collaboration. However, despite policy-level commitments to RRI and Open Science, their implementation within EU-funded projects has been patchy ( Tabarés et al., 2022). This may explain why the user-focused approach to tool development is emerging only in a partial or fragmented manner rather than as a widespread standard.
Nevertheless, the project-focused approach also has strengths that should not be overlooked. While it may be less oriented towards reflexivity and flexibility, it reinforces key governance principles essential for large-scale collaborative research. The structured nature of EU projects – defined by specific timelines, deliverables, and reporting procedures – ensures accountability, providing a clear framework for producing required knowledge and outputs within agreed parameters. These procedural elements, though sometimes perceived as restrictive, serve an important function in maintaining transparency and reliability within the research process. Moreover, what may appear as bureaucratic constraints within Horizon Europe could, in some cases, signal a need for greater skill development among project consortia. As Tenhunen-Lunkka and Honkanen (2024) suggest, the capacity to navigate project structures effectively – balancing compliance with flexibility – may be an area where additional training and institutional support could facilitate a more integrated approach. Rather than viewing engagement and structured project management as opposing forces, a more nuanced perspective might recognise the potential for their coexistence, leveraging the strengths of both approaches to enhance the impact and sustainability of R&I efforts.
Limitations
While this study provides valuable insights into how tools for engagement reflect wider engagement efforts in European Research and Innovation processes, certain limitations should be highlighted regarding its reach. Firstly, due to the scope of the study, the focus was solely on the tool development process, without examining how different development approaches influence the usability of the tools. Future research could address this gap by investigating the practical implications of these approaches, as well as the long-term impact of engagement tools on stakeholder participation and decision-making in European R&I initiatives. Secondly, we recognise that this study has a specific thematic focus, with the studied projects mostly concerning agri-food themes, the bioeconomy and sustainable development. This is relevant as the agri-food system transformation efforts might entail specific types of participation as well as engagement institutionalised through living labs, food policy councils, and various collectives ( Garcia-Gonzalez & Eakin, 2019; Wiarda et al., 2023). In order to ascertain whether the patterns identified through this study are visible in other areas of European R&I, research in additional fields of R&I should be further explored.
Additionally, the non-food specific projects that were analysed in this study did not diverge from the findings observed in the food-related projects. While this lack of apparent divergence is noteworthy, the emphasis on co-creation and sustainability central to both projects might make them more aligned with food system transformation efforts due to their higher participative potential, as explained earlier. To confirm this, further research is needed into the trends of engagement in R&I projects related to co-creation.
Conclusion
In this study we explored how tools for stakeholder engagement are developed by Horizon Europe R&I projects. We did so by analysing 14 Horizon Europe projects and identifying their best practices as well as the challenges they faced in the creation of stakeholder engagement tools. These insights allowed for a development of wider approaches to development of tools for stakeholder engagement in European R&I projects. Illustrating how these approaches are implemented in the context of R&I projects allowed for broader findings on a mismatch of a rhetoric on open science practices and collaboration in EU R&I on the one hand, and the high administrative burden and inflexibility of the EU-level public funding on the other.
Using engagement tool development as a case highlighting the European R&I shift towards more RRI and Open Science principles helps underscore the multiscalarity of the issue (which has been discussed in the context of geographical perspective by Wiarda & Doorn, 2023) , with differing views on whether it should be addressed with the structural reform of the R&I system or project management reform. The case discussed in this study is also indicative of wider challenges the EU faces in engaging with the European public ( Gudek et al., 2024). We specifically contribute to the literature by highlighting how a growing interest in a user-focused approach to stakeholder engagement in EU R&I projects is compatible with the wider structural shift towards RRI and Open science principles in Horizon Europe.
Additionally, while previous studies highlight the need for either a project management reform (for example Tenhunen-Lunkka & Honkanen, 2024), or stress the structural reproduction of injustice ( Luger et al., 2023), this study indicates how both the project management and structural dynamics influence EU R&I implementation. It appears that the project-level as well as structural elements relate to the nature of how R&I is done, both supporting the project-focused approach and allowing for the emergence of a user-focused approach. Therefore, this study highlights how the elements of both the structural R&I level and the project coordination level both produce distinct approaches to implementing R&I projects in Europe. Finally, these considerations point to the importance of further exploring the dynamics of how the EU engages with citizens through R&I in the future to facilitate more responsive, robust, and accountable innovation.
Ethical approval
Ethical approval was not required.
Consent to participate
Written informed consent to record the interviews, analyse the resulting data, and publish the findings, including anonymised excerpts, was requested and obtained from all participants. Additionally, the rights of the participants were validated at the beginning of each interview, with the explicit right to retract their consent.
Consent to publish
Written consent to publish the interview data, including anonymized text excerpts and generalised professional roles of the interview participants was gathered. The form of these data was briefly discussed at the beginning of each interview.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Armeni K Brinkman L Carlsson R : Towards wide-scale adoption of open science practices: the role of open science communities. Science and Public Policy. 2021;48(5):605–611. 10.1093/scipol/scab 039 · doi ↗
- 2Bergek A Hellsmark H Karltorp K : Directionality challenges for transformative innovation policy: lessons from implementing climate goals in the process industry. Industry and Innovation. 2023;30(8):1110–1139. 10.1080/13662716.2022.2163882 · doi ↗
- 3Borrás S Edler J : The roles of the state in the governance of socio-technical systems’ transformation. Research Policy. 2020;49(5): 103971. 10.1016/j.respol.2020.103971 · doi ↗
- 4Burget M Bardone E Pedaste M : Definitions and conceptual dimensions of responsible research and innovation: a literature review. Sci Eng Ethics. 2017;23(1):1–19. 10.1007/s 11948-016-9782-1 27090147 · doi ↗ · pubmed ↗
- 5Büttner SM Leopold LM : A ‘new spirit’ of public policy? The project world of EU funding. Eur J Cult Polit Sociol. 2016;3(1):41–71. 10.1080/23254823.2016.1183503 · doi ↗
- 6Cronin E Block T Fosselle S : Whose knowledge counts? Power dynamics in the co-production of knowledge and innovation in agri-food systems. Science and Public Policy. 2024;51(6):1117–1132. 10.1093/scipol/scae 050 · doi ↗
- 7Erisman JC Feenstra LD Broerse JEW : Labbing for sustainability transformations: learning about challenges and strategies for impact. GAIA - Ecological Perspectives for Science and Society. 2024;33(1):64–71. 10.14512/gaia.33.S 1.10 · doi ↗
- 8European Commission - DG R&I: Horizon Europe work programme 2023-2025. 2023. Reference Source
