Contexts logo

the systemic design journal

Citylab X: Towards a lens on the urban living lab as driver of systemic innovation

Anja Overdiek

Anja Overdiek’s exploration of the ISLE model as a lens for systemic innovation in urban living labs (ULLs) involves the study of a municipal ULL in the Netherlands (Citylab X). She observes that European municipalities increasingly use ULLs for complex situations, demanding a paradigm shift from service design to systemic innovation and a transition approach. The ISLE model is proposed as a process that might support co-creation, provide a heuristic process framework, and mobilise knowledge across ULLs and related networks. The findings include considerations for systemic design practice in ULLs.

#V2002 - 308 Downloads

Volume 2 | 2023

© 2023 Author, published by the Systemic Design Association
Open Access article published under the CC-BY-NC-ND 4.0 International License

Citation (APA)

Overdiek, A. (2024). Citylab X: Towards a lens on the urban living lab as driver of systemic innovation. Contexts—The Systemic Design Journal, 2.


Diverse European municipalities are using the urban living lab (ULL) approach to conceive and conduct innovation experiments that might enable the mobilisation of sustainable new energy solutions and policies for local and regional energy transitions. These municipality-enabled labs differ from social labs that originate from the direct concerns of citizens and communities. In both such lab contexts, designers of services or technology solutions are involved, but in different ways. The municipality-enabled ULLs are rarely facilitated and co-designed by systemic designers with an eye on their systemic or transitional impact. Citylab X is one of these labs, currently active in a larger city in the Netherlands. We followed this lab as a case study for one year to unpack specific challenges and opportunities in the realm of stakeholder co-creation. Based on the literature on systemic innovation labs and a newly developed lab process model, we reveal particular lessons for this type of government-enabled living lab. Thus, we propose a paradigmatic case study of a ULL for energy transition that further adapts a systemic design process model and delivers rich material for theory development. We conclude with dilemmas the different stakeholders encounter, which serve as valuable pointers for reflection that similar starting or ongoing lab initiatives can use.

Keywords: energy transition, multi-stakeholder co-creation, municipality-enabled lab, ISLE model, systemic innovation lab, urban living lab

Image created with DALL-E.

hands reaching together with a bright cheerful light


Municipalities, universities, technology companies and citizens in Western countries have employed place-based innovation labs since the beginning of the millennium. Initially guided by ideas of open innovation towards economic growth (Leminen et al., 2012; Westerlund & Leminen, 2011) or citizen and user involvement in urban planning and solution finding (Ballon & Schuurman, 2015; Steen & Van Bueren, 2017), urban living labs (ULL) are a special type of innovation lab, rapidly growing in number in Europe and North America. ULLs are spaces of real-life experimentation in urban areas where citizens are actively engaged in developing and creating solutions for urban sustainability challenges (Brons et al., 2022; Bulkeley et al., 2016; Steen & Van Bueren, 2017; Voytenko et al., 2016).

ULLs are a subcategory of a broader “living lab” approach. The goals of living labs (LL) are manifold, but methods typically include tackling a complex or wicked problem (Rittel & Webber, 1973) through the co-creation of knowledge and solutions, a triple- or quadruple-helix collaboration (Carayannis & Campbell, 2009; Schütz et al., 2019), and experimentation with new products, services, or ways of working together with users in a real-life context (Maas et al., 2017; Malmberg & Vaittinen, 2017). The idea of an equal and creative collaboration between different stakeholder groups qualifies their co-creation and is central to the LL approach (Franz, Tausz & Thiel, 2015; Mulder, 2012; 2018; Puerari et al., 2018; Siscode, 2021). LLs can be considered both as arenas (i.e. geographically or institutionally bounded spaces) and as an approach for intentional, collaborative experimentation between researchers, citizens, companies, and local governments (Schliwa, 2013).

Formal design discipline is not a defining characteristic of a LL. Often, technology designers contribute to LLs in the user testing phase of innovative products or services (Alavi et al., 2019; Ballon et al., 2018). Sometimes, designers also facilitate citizen/user participation in an earlier stage of the lab process. However, design thinking and design processes have profoundly influenced the ideal LL process. Models of the LL process (Dell’Era & Landoni, 2014; Ståhlbröst & Holst, 2013; Yasuoka et al., 2018) follow the innovation design process from challenge definition to implementation. The typical outcome of a LL is described as a product or service “solution”—an innovation. This solutionist and technology-centred focus in practice results in the fact that ULLs are rarely facilitated and co-designed by systemic designers with an eye on the lab’s broader societal impact. Systemic design scholars also observe that LLs employ a service design focus when a more systemic design perspective, though nascent, is needed (Jones, 2014; Kjøde, 2022; Mulder, 2018; Zivkovic, 2018a).

Recently, the focus of LL stakeholders has been shifting from an innovation to a transition paradigm, accelerated by the urgency of the social and technological transitions in energy, healthcare, and food systems (Puerari et al., 2018). For this reason, experimentation as a defining characteristic of LLs is increasingly seen in the light of its contribution to transitions, dubbed by transition theory as “niche” experimentation (e.g., Geels 2022, 2011). In Geels’ multi-level perspective (MLP) theory of societal transitions, niche experimentation plays an essential role in creating a realm where change initialises, dies out again, or synergises to changes on other societal manifestations such as that of existing institutions (regime level) and overarching system paradigms (part of the landscape level).

Geels developed this model based on historical material, but there is less theory available that defines the quality or process of niche experimentation, how it relates to the meso and macro levels, and how synergising could work. Transition arenas “as a specific network of frontrunners” (Loorbach & Rotmans, 2010, p. 237) with guiding transition principles have been proposed to fill in this gap, but they have been criticised by LL and systemic design scholars for not participating with users in their experiment and lacking the involvement of designers (Zivkovic, 2018a). The ULL could be a more suitable approach here, focusing on co-creation as the equal and creative collaboration between different stakeholder groups and commitment to user involvement. Systemic design authors recently addressed the need for systemic analysis and deployment of LLs (Jones, 2020; Zivkovic, 2018a). A systemic design lens on LLs is explored here toward an initial and deeper understanding of the ULL as a driver of systemic innovation.

Governments, municipalities in particular, are increasingly initiating ULLs (Bulkeley & Castán Broto, 2013; Bulkeley et al., 2016; Kronsell & Mukhtar-Landgren, 2018). The political responsibility for some domains, for example, food and energy transitions, is shifting to municipal jurisdictions in some geographical areas (Brons et al., 2022). Thus, ULLs have been revisited as methodology platforms that could enable reflexive agency and engage stakeholders in reshaping the future (Brons et al., 2022). As the enabling partner hugely determines the dynamics in a LL (Kronsell & Mukhtar-Landgren, 2018; Leminen et al., 2012; Leminen et al., 2016), it is interesting to have a closer look at these municipality-enabled labs while developing a systemic design lens on LL collaborations. The first question then needs to be: What co-creation challenges and opportunities in a municipality-enabled ULL could a systemic design lens unveil?

There is consensus among researchers that the process facilitated by the LL is key to the systemic impact of the lab (Williams & Robinson, 2020). However, there is still a knowledge gap and practical challenges when it comes to determining the quality of the lab process (Holmén et al., 2022) and finding and disseminating models for reflection and evaluation in LLs and between different local lab initiatives. A lack of knowledge sharing also applies across living lab contexts (i.e. energy, health care, or educational system transitions) as evidenced by a Horizon 2020 study into 40 LLs (Siscode, 2021). As a consequence, knowledge and new ways of working from local LL practice are seen as scattered and not synergising towards systemic changes (Ballon et al., 2018; Schuurman et al., 2016; von Wirth et al., 2019; Williams & Robinson, 2020). Thus, the second question picked up in this research is: How can a systemic design lens help ULL stakeholders reflect on the quality of their co-creation process and share related knowledge with other LLs?

A systemic lens on the urban living lab as systems innovation

From a systemic design perspective, different types of labs have been put forward through the years to explore contributions to urban innovation and transition in labs. “Street level think tank” (Flanagan et al., 2022), “change lab” (Westley et al., 2012; Hsu & Pacini, 2019), “public sector innovation lab” (Cole, 2021), “social innovation lab” (Westley et al., 2015), and “innovation lab” (Campbell & Geobey, 2020) concepts were employed to frame and interpret systemic design interventions in multi-stakeholder processes towards systemic change. These research contributions are reflections of one or several systemic design interventions in innovation labs. Their possible relation to an overarching model or lens on urban living labs as drivers of systemic change has not yet been explored.

Predominantly, the lab has been approached in this literature with a focus not on technical but on social innovation—dubbed “social (innovation) lab” (Hassan, 2014). A social innovation lab, in general, can be defined as “a process, one that is intended to support multi-stakeholder groups in addressing a complex social problem” (Westley et al., 2015, p. 1). Social labs are “container[s] for social experimentation, with a team, a process and space to support social innovation on a systemic level” (Kieboom, 2014, p. 9). They draw on participatory design, design thinking, and systems thinking, as well as create space for dialogue and sensemaking while “stewarding a group of diverse stakeholders through a systemic design process” (Tiesinga & Berkhout, 2014, p. 17). As evidenced by these quotes, this perspective on labs focuses on social innovation and experiments guided by systemic design, whereas the systemic lens we are looking for would frame the ULL as a driver of systemic innovation. This would include the ULL’s focus on technical experimentation and the possible absence of systemic designers leading or influencing the lab.

Several authors analysed labs from a systemic design perspective and implicitly or explicitly used the ULL approach. While they recognised community-driven ULLs (Mulder, 2012; Mulder, 2018; Lake et al., 2022) and the rich contributions of participatory process in labs, they did not focus on the particular situation of the growing number of municipality-enabled ULLs. Thus, the first research question remains unaddressed.

As a theoretical contribution to systemic design using labs, Zivkovic (2018a) proposed the “systemic innovation lab,” which relates and distinguishes between the different lab types used by systemic design practice. Drawing on systemic design principles (Jones, 2014), among others, she presents seven characteristics of an ideal systemic innovation lab that could be integrated into a systemic ULL process model. Such a process model would allow for the alignment of different systemic interventions in labs towards a more integrated approach to the ULL as a driver of systemic innovation. Thus, it could provide answers to the second research question on how a systemic lens can help to reflect on and share knowledge about the quality of co-creation in ULLs to work towards systemic change.

Systemic innovation lab

For the model of a systemic innovation lab, Zivkovic (2018a, 2018b) uses a complex systems approach to address societal challenges. Design activity is centrally situated in this lab model and focuses on the development of initiatives that assist system transitions and not on “delivering solutions.” Using this design focus and theory approach, Zivkovic offers an alternative to the focus on service design solution that is common in urban living labs. According to the author, complex and wicked problems call for self-organising governance networks (Meuleman, 2008, p. 104) to enable the conditions for the emergence of self-organisation, adaptive capability (McKelvey & Lichtenstein, 2007), as well as systemic innovation and transitions to new, improved states (Goldstein et al., 2010). She suggests that systemic innovation labs can be used as a mechanism to enable these conditions and highlights seven key features from other lab approaches that are recommended for addressing complex problems. Next to addressing complex problems, systemic innovation labs take a place-based and transition approach, enable coherent action by diverse actors, involve users as co-creators, support a networked governance approach, and recognise government as an enabler of change (Zivkovic, 2018). Zivkovic embedded a range of principles from solution ecosystems and systemic innovation into this model, as well as the Jones (2014) principles proposed for systemic design in social systems. The model of the systemic innovation lab could add important elements to a ULL process model and thus provide a framework within which stakeholders can learn, interact, and adapt to develop their capacity for solving complex problems. Hence, a newly developed LL process model that aims to guide multi-stakeholder co-creation in ULLs towards driving systemic change should embrace and be nuanced by the systemic innovation lab features.

A process model for systemic innovation through living labs

Co-creation dynamics and scaling were a focus applied by the author in earlier living lab action design research. From this research done across more than 30 living labs in different contexts, a lab process model (De Lille & Overdiek, 2021) was developed, which, to our knowledge, is the only model that captures a multi-stakeholder living lab process from a systemic design perspective. The process spans from system mapping to the implementation of new products and ways of working in society.

With time on the x-axis, the model distinguishes between three basic phases:

  1. Understanding the current system
  2. Engaging in local experimentation
  3. Working towards a preferred future system

The model was found by inductively building on case studies but can be underpinned by the multi-level perspective on transitions (Geels, 2022, 2011) and “system shifting design” (Design Council, 2021; Drew et al., 2021; Leadbeater & Winhall, 2020), which introduces different levels of action and possible influence via the vertical dimension. The resulting curve represents the multi-stakeholder co-design group navigating between concrete, local action and experiment (low, micro level), more abstract levels of regional/­institutional/societal narratives (meso level), and national policy and overarching purpose (high, macro level). The framework shows three important activities in phases one and three and suggests iterative testing in local real-life environments in phase two. The authors of the model explain elsewhere (Bergema & De Lille, 2022; De Lille & Overdiek, 2021) how design principles like distributed governance have helped researchers and systemic designers to “navigate the curve” in the different phases. Thus, they “deliberately and generatively” created systemic change opportunities “for others to fill in and build on” (Drew et al., 2021, pp. 3, 12). The model has so far been validated in three workshops with systemic design practitioners and researchers who recognised their own process in the model and added interventions (Overdiek & Bergema, 2022).

The ISLE curve embraces the seven key features of a systemic design lab formulated by Zivkovic (2018a):

  • It focuses on addressing complex problems with a transition approach
  • Takes a place-based approach and involves users as co-creators (the “Engaging in local labs” phase)
  • Enables coherent action by diverse actors, supports a networked governance approach, and recognises government as an enabler of change.

As opposed to the solution focus of current LL process models (see Introduction section), the ISLE process focuses on muti-stakeholder co-creation outcomes over time. We propose this lens to unveil challenges in a municipality-enabled ULL process and further develop the (co-design model by applying it to the ULL as a driver of systemic innovation.

ISLE model

Figure 1. Impacting Systems with Local Experimentation (ISLE) model (De Lille & Overdiek, 2021).


The overall methodology of this research was an action case study (Halecker, 2015) informed by multiple interviews, observation, and action learning processes in situ. After five years of research developing the ISLE model by reflecting on our own design interventions, we wanted to observe how an urban living lab co-creation is organised in practice and how the ISLE model can be used for critical systemic reflection (with respect to answering the second research question).

Regarding the first research question that we formulated, “What co-creation opportunities and challenges in a municipality-enabled ULL could a systemic design lens unveil?” case study methods can be employed to gain a deeper understanding of an issue like co-creation over time using the case as a specific illustration. A case study is a good approach when the inquirer has clear, identifiable cases with boundaries and seeks to provide an in-depth understanding of a particular phenomenon (Creswell & Poth, 2017; Yin, 2015). The unit of analysis is the organisation of the ULL, and in this case, specifically, the municipality-enabled ULL.

An ethnographic approach to the fieldwork was taken, using structured observation and interviews with representatives of all stakeholder groups associated with the lab. A literature review and review of primary sources (publications of the ULL) were done to interpret observations and to allow for triangulation in the analysis (Denzin, 1978). The ethnographic researcher chose the role of an observer-as-participant between March 2021 and June 2022. This role, as described by Pearsall (1970), involves more observation than participation. Researchers who adopt an observer role typically advance slowly in their involvement with the insiders. While still mostly involved in observing, they may conduct short interviews on-site. Pearsall (1970) described two advantages to this role. First, insiders may be more willing to reveal issues to “attentive strangers” than they would to people with whom they are more entangled. Second, there is less “temptation either for the observer to go native or for the natives to try to include” them permanently in their practice (Pearsall, 1970. p. 342). In our case, other researchers with a governance and action research background were already involved as stakeholders in the lab. So, the more distanced observer-as-participant role allowed us to collect data with minimal interference.

Next to the ISLE model, a thematic template (King, 2004) was employed to guide the observation and interviews, but we also allowed for new themes to emerge from the data. The themes of the template replicate the findings of the Siscode research into stakeholder co-creation (Eckhardt et al., 2021; Siscode, 2021), which is the most representative and in-depth study of co-creation across living labs to date. The template also provides the thematic codes for data analysis performed in the Atlas TI software.

The unit of analysis was the organisation, represented by the particular case of the Citylab X ULL. Four LL researchers and an expert group of five LL practitioners were involved in two half-day workshops to conduct the final analysis of the case data. In this light, we propose the case study of Citylab X as paradigmatic of a ULL in the energy transition. Paradigmatic cases are “cases that highlight more general characteristics of the societies in question” (Flyvbjerg, 2006, p. 16) and, therefore, allow for some degree of generalisation.

The Citylab X case study

The urban living lab researched for this case study is a collaboration between a large municipality in the Netherlands (more than 500,000 inhabitants) and a research group focused on governance from the Hague University of Applied Sciences. In 2019, the municipality decided to pursue a transition towards a sustainable and carbon-free energy supply with a so-called “city energy plan.” The ambition of this plan was to develop—together with local stakeholders—approaches for more than 150 different districts of the city (neighbourhood energy plans). Some districts, known for their diversity and active citizens, were chosen by the municipality (without citizen consultation) as “frontrunner” districts. These were called Groene Energiewijken, or regions where activities would be tested as a basis for other districts to learn from.

The urban living lab context

The case is located in a neighbourhood where an urban living lab approach has been used since 2020 to collaborate on the district energy plan. The stakeholders of this lab asked the case to be identified by a pseudonym, referred to as “Citylab X” in this paper. Citylab X was not conceived initially for energy transition projects. In 2019, lab activities proceeded in this neighbourhood with a dialogue between the municipality and citizens and between different groups of citizens through growing grassroots activities. District X’s population was around 15,000, inhabiting more than 8,500 houses, with 80% built between 1950 and 1970 and 60% being rentals. About 55% of the citizens in this district have a migration background, 20% are Western, and 35% are non-Western.

The traditional ecological knowledge framework is depicted by two layered spheres, one is local knowledge of land and fauna and the other is affective knowledge. Cosmotechnic attunement is a dark circle connecting all spheres of affective knowledge.

Figure 2. Timeline of key events in Citylab X.  Double line marks the start of the author’s research.

From March 2020 onwards, the observed researchers collaborated with the district manager, a representative of the district’s citizen council, and a manager from the largest housing cooperative in the district. They formed a core lab team together and decided to start the lab project towards energy transition through online plenary meetings with interested citizens, including 50 interviews with diverse citizens about their “desired energy futures.” However, due to the COVID-19 pandemic and a change in lab coordinator (unrelated to the research), the lab activities paused in late 2020, and were later restarted in March 2021. The lab project was initially received with enthusiasm by active citizens, as a former lab project around grassroots activities for low-income groups had been perceived as a success. Influenced by this enthusiasm, good communication, and accessible meetings (people did not have to travel or meet in an unknown space), organising lab meetings online broadened rather than hampered the lab. More diverse citizens from the neighbourhood joined these meetings.

In June 2021, the first physical meeting took place with around 30 citizens and representatives of the municipality’s “energy expert team.” This meeting was considered controversial, as multiple clashes of interest occurred between house owners, tenants, and energy transition experts in favour of different solutions. A meeting with feedback on the interview results followed. In January 2022, the lab coordinator changed once again, and three months were taken to plan and organise a new meeting with citizens, the housing cooperative, and energy experts. During this meeting, the lab worked with a futures method to express various prospective ideas from the different stakeholders. In June 2022, the case study research resumed, and data was analysed. The following “storyline” of this ULL emerged by using the collected data and the phase description of the ISLE model.

Citylab X’s Co-Creation Dynamics along the ISLE Curve

Phase one—understanding system stakeholder dynamics

Between March and December 2021, the main activity of the urban living lab was organising dialogue sessions between the partners and seeking to engage diverse citizens from the district. The lab’s core team and the steering group were comprised of the lead researcher, the neighbourhood manager (employed by the municipality), neighbourhood council members, and a representative of the largest housing cooperative in the municipality. This team met every two weeks to discuss developments in the district, form ideas for lab sessions, and organise the dialogues. A strategy group also met regularly. A representative of the municipality’s energy expert team and the lead researcher came together once a month to evaluate activities and connect them to a longer-term strategy towards engaging citizens in the neighbourhood energy plan. Since the declared goal of the municipality (according to the lead researcher) was to “explore the desired citizen participation in the energy plans,” the involvement of businesses or energy providers was considered potentially too complicated, so the living lab concept was kept to a triple helix relationship (municipality, citizens, research team).

The municipality’s energy expert team had been a client of the University research group since late 2019; through the city, it financed the lab’s energy transition activities in the framework of the research project. As such, Citylab X was entirely funded by the municipality and through research. We saw three important repercussions. Firstly, the lab researcher doubled as a lab coordinator and was dependent on the enabling party of the lab—the municipality. The observed researcher experienced this as a challenge from the beginning, as they felt the role of lab coordinator should be independent of the particular interests of one stakeholder group. Secondly, as project funding did not account for housing or materials, the ULL had no physical location but was a virtual context for collaboration in and across projects. Thirdly, due to this lack of capacity and time, evaluations and results from prior lab activities (before the energy transition project) were not systematically consulted and used.

So, from the perspective of the researcher and the municipality’s energy expert team, Citylab X was established in tandem with this same project, whereas most citizens and the neighbourhood manager experienced it as a follow-up to earlier lab activities. Therefore, upon reflection, we observed that there was an increased citizen expectation for participation and impact, of which the energy team was unaware. All interviewed parties, in some way, commented on the negative effects of this situation, indicating a lack of shared framing of the lab for all participants, as well as what seemed like a lack of continuity in the lab’s activities and outcomes.

Of course, you would prefer someone from the neighbourhood who also coordinates such a citylab. We would have much preferred that. (Lead researcher/coordinator, Citylab X)

If we could really set up a Citylab with a physical place where we could work on the neighbourhood development together, then there would be more trust. Then you get to know each other better. (Community manager, District X)

Due to the second COVID-19 lockdown in the Netherlands, the lab’s stakeholder engagements moved to online sessions from March 2020 onwards. Supported by active communication from the research team, together with the neighbourhood council and the district manager, the online stakeholder sessions were well-visited by diverse citizens. While the core team had developed a good working relationship, the need to change resources twice for the role of lead researcher/lab coordinator (first in late 2020 and later in 2021) led to much discontinuity, which resulted in disappointment from the neighbourhood stakeholders. Trust needed to be rebuilt after every change, which sheds light on the importance of sustainable personal relationships for the ‘core’ of the ULL to work.

While there were active citizens in the neighbourhood, mostly house owners with strong opinions about the ‘right’ sustainable energy solutions for the district, most tenants were not outspoken in the lab conversations. Still, research interviews show that they were worried about the cost of alternative energy solutions in their flats and the possible inconvenience from construction work in and around their homes. In the dialogues organised by Citylab X, the house owners sometimes directly opposed suggestions from the municipality’s energy expert team. Tenants were mostly represented by the housing cooperative, which was reluctant to discuss particular energy solutions as they did not want to create false expectations with their tenants. All in all, there were various entanglements and conflicts (see Figure 3) in this lab, and without the energy corporations or new energy providers as participants, parts of the relevant system were missing in the dialogues. This lack of requisite social variety (Ashby, 1956) had a negative impact on lab dynamics. An energy provider or SME from the neighbourhood might have brought new perspectives and shifted the primary conflict line between the citizens and the municipality. The described dynamics were not addressed, which could have paved the way to a feeling of “picking up a shared task” among the different stakeholders.

Challenges to co-creation based on equality

Before developing the urban living lab approach in early 2020, the municipality had started the process by organising information evenings around the neighbourhood’s position regarding the energy transition. They also presented the plan for a district heating network, which created the perception of a “decision already made” among citizens who would have liked to explore options together. In this situation, the energy expert team had turned to the research group to help them with a different approach—to find out how the citizens of District X would like to be involved.

From the start, the researchers’ focus was on equal co-creation between all ULL stakeholders. For them, engaged citizens should be able to exert influence on neighbourhood-level plans, decisions, and activities. The municipality, on the other hand, was divided on this equal co-creation principle. Whereas the district manager supported it, the energy experts expressed a top-down approach to information sharing. Consciously or not, their aim in engaging with the ULL was not to structurally empower citizens. Instead, they wanted to share their knowledge and plans with the citizens and, ideally, raise enthusiasm for the plans.

CityLab actor map

Figure 3. Actor map of Citylab X used in the research. Flash icon marking major conflict lines.

As described in the section above, this approach clashed, in particular, with the values and preferences of active citizens. This systemic conflict of values was also present in the conflicting attitudes of the different municipality representatives, which created confusion on the side of the citizens. Citizens reported that it was not transparent to them which level of participation was granted in this lab: were they just consulted, or were they offered some level of partnership and decision-making power?

This challenge of co-creation as partners was addressed in Citylab X’s strategy group. The lead researcher openly questioned the role of the municipality as a “client of the Citylab” and asked its representative to assume the role of a collaborating and equal stakeholder in the lab context. The participating researchers presented themselves not as “collaborating stakeholders” but as neutral observers. Yet, they steered the lab agenda, had a clear interest in certain knowledge questions, and promoted actions following a tight project timeline. It was not until the end of the observation process, when triggered by the reflective interviews held with them that the observed researchers realised and reclaimed their role as stakeholders of the ULL and pronounced these stakes openly. In this way, there was a significant exchange of learning in the collaboration between the municipality and the researchers in the strategy group. These insights, however, stayed within the strategy group and were not shared with the core team or during dialogue sessions of Citylab X, and thus were not converted into lab learnings. The ambitious nature of and inherent challenges with co-creating as equals in the lab were not discussed between all stakeholders. This was, however, something that citizens were aware of:

A co-creation process in which roles and responsibilities are moving is vulnerable. Recognising this and starting from equality is important. This lab process feels unequal due to the imbalance of power between municipality/ research and citizens and due to the failure to recognise shifts. (Citizen, Citylab X)

It can be qualified as a missed opportunity that the “rules” of co-creation were not transparently established together with all stakeholders in the first phase of the living lab.

Steering toward the future without trust

The value conflicts observed between citizens and energy experts revealed differences in expectations within the citizen group. Active citizens were expected to be involved and play a role in concrete energy transition experiments (micro level) in their district. Interviewed citizens (mostly tenants who did not participate in the dialogue) were more concerned that the least costly solution was chosen and that implementation would not be too burdensome for them.

In order to explore future prospects together, including other ways of collaboration between different citizen groups and between the municipality and citizens, the citizens and the community manager expressed that they needed a chance to work together to experiment with concrete solutions. The community manager thought that low-threshold citizen experimentation within the context of broader sustainability goals would bring more positive energy and trust into the lab. The idea, however, was not heard. This foregrounds the goal of co-creation in community development (social innovation), which would precede co-creation on more complex technical issues, such as the district’s energy transition.

A milestone in the urban living lab process is that the lab researchers managed to conduct more than 50 individual interviews with diverse citizens from District X on their ideas about the energy transition and their wishes for participation. The results of these interviews were fed back to the ULL in a session in the winter of 2021 to bring more perspectives into the dialogue. But there was still a lot of mistrust. A planned session about desired futures derived from the interviews and suggested as a co-creative workshop was withheld and then organised as a “method trial without consequence for the lab project” (District manager, Citylab X). While the lead researcher thought the workshop could be helpful for the lab process, the municipality, housing cooperative, and citizens were sceptical. A “futures thinking” workshop was finally held in March 2022. While citizens participated actively, municipality participants were unfortunately reluctant to share information, which resulted in more confusion for the citizens. Overall, the lab seemed stuck. With the city energy plan itself postponed (until late 2023), there was no felt urgency for the municipality to make significant steps in Citylab X’s process. At the same time, the citizens expressed an increased sense of urgency towards changing their neighbourhood to a more sustainable place (an outcome of the interviews), and some groups of active citizens had concrete plans for experimentation (e.g., insulating old houses). These plans were not taken up in the ULL, and no other shared plans for experimentation were developed.

Phase two—experimenting together

According to the lead researcher and the district’s community manager, the complexity and high investment cost of the energy transition did not allow for experimentation to explore different carbon-free or carbon-reduced energy opportunities. These technologies include smart grids fuelled by solar collectors or the insulation of old houses in combination with heat pumps. Citizens believed they were not provided with enough knowledge to make choices and were not listened to when they suggested possible energy solutions, like sun collectors or heat pumps. Some of them also thought that the municipality did not have sufficient expertise in alternative energy opportunities for the district. Ironically, it is the fact that the energy expert team did not want to propose a ‘best’ solution from the start that triggered this response from the citizens. The community manager left the dialogue because she thought the experimental activities that came from the neighbourhood, like the idea of building a butterfly path or a community garden, would be necessary common activities that could help connect and build trust among different citizens, the municipality, and the district. However, the willingness to undertake concrete experiments in this lab was unclear from the beginning. For the observed researchers and the energy expert team, the experiment was suggested to learn how the citizens wanted to be involved. Yet, that was considered too abstract of an experiment for the different citizen groups to take part in. This not only hindered the development of Citylab X but also contributed to confusion among stakeholders about the meaning of the lab’s approach.

Currently, the term lab is being used as an argument by different stakeholders with different meanings. (Lead researcher, Citylab X)

When we zoom in on this, we see that the municipality’s energy expert team consciously wanted to go into the neighbourhood first, in a learning process about how best to inform residents and talk to them, even though it was not yet clear how the energy transition would take shape in X. (Lab researchers in reflective publication, Ruiter & van der Veen, 2021)

The municipality wanted to know which plan has the most citizen support. (Internal communication publication, Citylab X, 26 March 2021)

Preferred scenario choices are not helpful. Parallel paths will have to be followed in the energy transition. (Citizen, District x)

The quotes above reflect very different ideas about the willingness to explore and the scope of experimentation. While the homeowners, in particular, had concrete ideas for experiments, the researchers and municipality did not see any need or opportunity to experiment with different technical approaches for the energy transition. At the moment of documenting this case study, it can be said that the stakeholders still have very different interpretations of how the ULL should proceed. The lead researcher is even considering removing the designation of the ULL for the time being—to relax expectations and improve the opportunity for open dialogue toward a future research agenda with the district.


Toward a systemic design lens on the urban living lab process

The stakeholders of Citylab X were interested in making progress on a local energy transition project and, ideally, sharing successful practices with other districts. But where are the stakeholders situated in the change process overall? What learning can they reflect on and share? By using the Innovating Labs with Local Experiment (ISLE) process model (Figure 2) to interpret this case, we can situate the urban living lab’s co-creation challenges on the model and interpret what could be reflected on to “make the next step.”

Citylab X has identified and engaged stakeholders (the first phase of the lab process) but has yet to develop the shared tasks for experimentation. It struggles with a trust challenge, which stems from the lack of an independent and stable lab coordinator, access to resources, and unclear lab participation rules. Moreover, the requisite variety of stakeholders is insufficient in the lab projects, and there is confusion due to different ideas about what a concrete micro-level experiment in this lab could be. This situation hindered the multi-stakeholder group in Citylab X from moving forward on futuring activities.

With this analysis, Citylab X’s co-creation so far can be firmly situated on the lab curve, as indicated by the blue triangle in Figure 4. We propose to call this the “maturity” of the lab. Like this, the ISLE model can be used as a systemic design lens to help ULL stakeholders reflect on the quality of their co-creation. The model could also constitute a “heuristic process framework” (Schwaninger, 1997, p. 113) used to share knowledge with other living labs. Of course, the above findings are only the very first step towards a lens on systemic innovation through ULLs. The ISLE model might be explored further across all phases, using different contexts of ULLs. Puerari et al. (2018) have shown that places and place narratives have a positive influence on co-creation in ULLs. Concilio & Molinari (2014) have suggested that the concrete experience of doing experiments in ULLs can trigger reflections that have a long-term influence on individual and collective mental models and behaviour. Other research cases also need to confirm if “establishing an independent lab coordinator and core resources,” “transparency about co-creation rules,” “taking care of requisite variety,” and “working towards concrete experiments on a micro level” should be added as steps in terms of co-creation goals to the first phase of the model. These could possibly replace the more activity-related step description, which is visualised in the current model.

ISLE maturity figure

Figure 4. Maturity of Citylab X indicated as a triangle in the ISLE model.

It is remarkable how competing value systems were not openly addressed in Citylab X. One reason for this lies in the fact that the different stakeholder groups all struggled with their own dilemmas while co-creating the municipality-enabled ULL. These dilemmas have been compared to challenges in living labs enabled by other stakeholder groups (i.e. citizen or industry-enabled living labs), where they were not found. They were also discussed and recognised by our expert group (see methods chapter) for the specific case of municipality-driven ULLs. Thus, they can be qualified and presented as paradigmatic dilemmas (Flyvbjerg, 2006, p. 16) for municipality-enabled ULLs in the following section.

Paradigmatic dilemmas of stakeholders in municipality-enabled ULLs


The municipality, on the one hand, follows a pragmatic, scalable, “majority” consultation approach towards citizen participation. In particular, the energy experts team pursues this path and wants to know what level of participation is necessary to implement a certain energy solution. On the other hand, there are civil servants, such as the district manager and the community manager, who are more engaged in the district and see opportunities for small-scale citizen experimentation with diverse energy solutions. They embrace the urban living lab approach and envision more of a partnership between engaged citizens and the municipality. There is, thus, the governance and accountability role of the municipality on the one hand and the commitment needed as a co-creating partner in a ULL, which could put that role at risk. This dilemma is neither addressed between the municipality actors nor discussed among the lab’s core team. This leads to a lack of clarity around the role of the municipality actors and the level of envisioned citizen participation in the lab.


The role of citizens also discloses a dilemma. Active house owners or more wealthy citizens who have the time, money, and ideas to invest in experimentation can more easily embrace different standpoints in the dialogue. Whereas tenants—an under-represented citizen group in the lab—are solely preoccupied with the impact of time and cost that alternative energy solutions might entail. This tension between short-term investment and long-term goals is another dilemma that is not addressed in the lab dialogues. This leads to a double standard in citizen expectations towards the municipality (“be our partner” versus “find the best and most cost-effective solution for us”). This also creates a lot of frustration as both citizen groups do not feel heard in the lab.


Thirdly, researchers are stakeholders in the ULL. The researchers are motivated to produce knowledge within a predetermined timeframe (due to funding) and tend to frame themselves as “neutral” observers of the ULL, independently of the lab situation. They are actively engaged in the lab and are strongly supportive of a high level of citizen participation, which, given the dilemmas of the other stakeholder groups, is a significant challenge and takes time. This dilemma plays between practical confinements of research activity in terms of budget and time and theoretical ideals of an equitable co-creation of all stakeholders in the ULL.

Balancing social and technical innovation

All three stakeholder groups seem caught in a balancing act between social and technical innovation that the lab pursues at the same time. The case suggests that for a municipality-enabled ULL as a particular type of living lab, interventions towards social innovation need to precede technical innovation experiments. As a municipality, it is important to build lab activities on transparent information flows about the level of participation granted to citizens and on a clear mandate for municipality representatives to share information and engage in lab experimentation. With an informed and open dialogue about co-creation “rules” among all stakeholders, the lab can be used to empower active citizens and municipalities to conceive and experiment with different local solutions. For this to happen, transparency about interests, structural empowerment (Sinteur, 2018), and an open co-creation attitude on the part of all stakeholders are necessary.

As argued in the methodology chapter, we think that these findings can be generalised across municipality-enabled ULLs for the energy transition in the Netherlands and also probably in culturally similar countries across Europe. This case study shows a deep mistrust between citizens and local governments. Findings might not apply to countries or places where this mistrust is not found. It could be interesting to replicate the study in those countries and places to find out which findings apply and which need to be modified.


The study of Citylab X narrates and reflects on the case of a municipality-enabled urban living lab in the Netherlands dedicated to local energy transition. It argues that living labs, particularly ULLs, are increasingly used by European municipalities with a transition paradigm, as opposed to an earlier focus on innovation and solution. This complex situation calls for a systemic design lens on using the living lab approach for transitions. The ISLE process model was proposed as a lens for systemic innovation through living labs. This lens not only supports the analysis of co-creation challenges and opportunities of the ULL but it can also be proposed as a heuristic process framework to help ULL stakeholders in the future to reflect on the quality of their co-creation and to share related knowledge with other labs.

The Citylab X case study allows for four important conclusions:

  1. Trust between the different stakeholders needs to be actively established before the co-creation process so that it can move “down the curve” to more concrete alignment around shared, desirable futures.
  2. The establishment of a working core team (successful in Citylab X), an independent and stable lab coordinator, and a (temporary) physical place for the lab (failed in Citylab X) all seem to be very important for successful co-creation among stakeholders.
  3. It is not sufficient that rules for co-creation are only addressed in the strategy group of a ULL. Desired rules of co-creation between the stakeholders (i.e. situations when shared decision-making is desired and/or when informing or consulting citizens is sufficient) must be transparent in the entire lab community.
  4. The ISLE model proposes that a ULL steers towards local technical experiments (down the curve) and invites the needed requisite variety to do this in the lab group. The case demonstrates that by failing to do this in Citylab X, all stakeholders were left with confusion about the purpose of the lab.

The case study reveals that in the co-creation between the municipality, citizens, and researchers, the specific dilemmas that occurred within each stakeholder group are paradigmatic of municipality-enabled ULLs. In addition to this, all stakeholders were caught in a balancing act between technical and social innovation, which suggests that foregrounding one or the other in particular phases of the lab process might be preferable instead of striving for both at the same time.

The findings have interesting implications for theory-building. The systemic innovation and co-creation process from stakeholder engagement to implementation indicated in the proposed ISLE model might deliver important elements for a more general systemic design lens on facilitating transitions with living labs. Such a process model would allow the alignment of different systemic interventions in labs towards a more integrated approach to the living lab. This is while keeping in mind that the linear-looking ISLE lab process is a “frozen” model of dynamic and iterative activities.

Key characteristics of the systemic innovation lab (Zivkovic, 2018a) are implemented into the ISLE model. The findings of the case study show nuanced characteristics and deliver valuable pointers for future systemic design practice in ULLs. The three nuanced features are:

  • Enable coherent action by diverse actors
  • Involve all stakeholders as co-creators
  • Recognise government as an enabler of change

To clarify the first claim, the enabling of coherent action in the case suggests that different types of action are necessary in the phases of a living lab systemic innovation process. In the first phase of “understanding the current system,” transparent rules for co-creation for the different lab stakeholders are necessary. The case demonstrates how the understanding phase calls for transparency and decision-making within the stakeholder groups first and then for an open conversation between the different groups.

Secondly, in order to involve people as co-creators, experimentation is necessary in collaboration with the different systemic stakeholders and users of innovations and new ways of working (in this case, for new energy systems) in a lab. However, the interaction dynamics in Citylab X illustrate that such experiments ought to be concrete and tangible. Basing an innovation experiment on “the way citizens want to be involved” resulted in high ambiguity for the citizens. These observations reveal important consequences for designing and facilitating ULLs. Designers in living lab contexts could work on different levels of abstraction—as service designers conceiving and testing different energy solutions (our case) and as systemic designers working towards greater adaptability of the system as a whole. Combining user-centred service design and systemic design interventions geared at transforming the current system is also suggested by a recent study about transitioning the food system using ULL (Brons et al., 2022).

Thirdly, the claim to “recognise government as an enabler of change” requires that government actors become reflexive about the possibilities and challenges of different roles in lab co-creation. Kronsell & Mukhtar-Landgren (2018) suggest that municipality actors can distinguish between different roles in a ULL, which entail different attitudes and capacities to co-create with citizens and other lab stakeholders and must be transparently communicated. Thus, systemic design of a ULL system can not only recognise government as enablers but might also support government actors in the reflection to enhance or add new possible productive roles in ULLs.

These three implications are propositional claims that could be evaluated by further research into living labs and the adaptation of systemic practices such as the ISLE process. They mark the first steps towards a lens on the ULL as a driver of systemic innovation that needs to be further worked out in the future.


This study is indebted to data collection and analysis by the Future-Proof Labs research group (J. Sluijs, B.G. Bustamante, F. van Doorn, A. Schut, M. Genova, J. Ferrer I Pico, and Prof. C. De Lille) at the Hague University of Applied Sciences, and the project partners Van Waarde and Livework studio. It was financed with the PPP grant by CLICKNL. CLICKNL is the top consortium for Knowledge and Innovation (TKI) of the Dutch Creative Industry


  1. Alavi, H. S., Lalanne, D., & Rogers, Y. (2019). The five strands of living lab: A literature study of the evolution of living lab concepts in HCI. ACM Transactions on Computer-Human Interaction, 2 (2): 1-26.
  2. Ballon, P. & Schuurman, D. (2015). Living labs: Concepts, tools and cases. Info, 17 (4).
  3. Ballon, P., Van Hoed, M., & Schuurman, D. (2018). The effectiveness of involving users in digital innovation: Measuring the impact of living labs. Telematics Informatics, 35 (5): 1201–1214.
  4. Bergema, K. & De Lille, C.S.H. (2022). Scaling Up: From labs to systemic change. Proceedings of Relating Systems Thinking and Design, RSD11.
  5. Brons, A., van Der Gaast, K., Awuh, H., Jansma, J. E., Segreto, C., & Wertheim-Heck, S. (2022). A tale of two labs: Rethinking urban living labs for advancing citizen engagement in food system transformations. Cities, 123, 103552.
  6. Bulkeley, H., Coenen, L., Frantzeskaki, N., Hartmann, C., Kronsell, A., Mai, L., Marvin, S., McCormick, K., van Steenbergen, F., & Palgan, Y. V. (2016). Urban living labs: Governing urban sustainability transitions. Current Opinion in Environmental Sustainability, 22, 13-17.
  7. Bulkeley, H., & Castán Broto, V. (2013). Government by experiment? Global cities and the governing of climate change. Transactions of the Institute of British Geographers, 38 (3): 361-375.
  8. Campbell, T. & Geobey, S. (2020). Rethinking innovation labs for complex adaptive systems going through release and reorganisation. Proceedings of Relating Systems Thinking and Design, RSD9.
  9. Carayannis, E. G. & Campbell, D. F. (2009). ‘Mode 3’ and ’Quadruple helix’: Toward a 21st century fractal innovation ecosystem. International Journal of Technology Management, 46 (3-4), 201-234.
  10. Cole, L. (2021). Moving toward paradigms and patterns of transformative innovation in public sector labs. Proceedings of Relating Systems Thinking and Design, RSD10.
  11. Concilio, G, & Molinari, F. (2014). Urban living labs: Learning environments for collective behavioural change. Knowledge and Management Models for Sustainable Growth, 746-763.
  12. Creswell, J. W. & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches. London: Sage.
  13. Dell’Era, C. & Landoni, P. (2014). Living Lab: A methodology between user centred design and participatory design. Creativity and Innovation Management, 23 (20): 137-154.
  14. De Lille, C.S.H. & Overdiek, A. (2021). From system to local to system: Design principles to scale for a system in transition. Proceedings of Relating Systems Thinking and Design, RSD10.
  15. Denzin, N.K.  (1978). The research act (2d ed.). New York: McGraw-Hill.
  16. Design Council (2021). System-shifting design. An emerging practice explored.
  17. Drew, C., Robinson, C., & Winhall, J. (2021). System-shifting design: An emerging practice explored. Proceedings of Relating Systems Thinking and Design Symposium, RDS11.
  18. Eckhardt, J., Kaletka, C., Krüger, D., Maldonado-Mariscal, K., & Schulz, A. C. (2021). Ecosystems of co-creation. Frontiers in Sociology, 6.
  19. Flanagan, T., Christakis, A. N., & Kakoulaki, A. (2022). Institutionalising new civic learning traditions: Nurturing design practice in planning cultures. Proceedings of Relating Systems Thinking and Design, RSD11.
  20. Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12 (2): 219-245.
  21. Franz, Y., Tausz, K., & Thiel, S. K. (2015). Contextuality and co-creation matter: A qualitative case study comparison of living lab concepts in urban research. Technology Innovation Management Review, 5 (12): 48-55.
  22. Geels, F. W. (2002). Technological transitions as evolutionary reconfiguration processes: A multi-level perspective and a case-study. Research Policy, 31 (8-9): 1257-1274.
  23. Geels, F. W. (2011). The multi-level perspective on sustainability transitions: Responses to seven criticisms. Environmental innovation and Societal Transitions, 1 (1): 24-40.
  24. Goldstein, J., Hazy, J. K., & Silberstang, J. (2010). A complexity science model of social innovation in social enterprise. Journal of Social Entrepreneurship, 1 (1), 101-125.
  25. Halecker, B. (2015). Action case study—A research strategy based on abduction for relevant and rigorous management research. International Journal of Business Research, 15 (4): 23-32.
  26. Hassan, Z. (2014). The social labs revolution: A new approach to solving our most complex challenges. San Francisco: Berrett-Koehler Publishers.
  27. Holmén, J., Williams, S., & Holmberg, J. (2022). Comparing sustainability transition labs across process, effects and impacts: Insights from Canada and Sweden. Energy Research & Social Science, 89: 102522
  28. Hsu, C. & Pacini A. (2019). Designing systems outcomes as desirable side effects: Reflections from the CMHC solutions labs. Proceedings of Relating Systems Thinking and Design, RSD8.
  29. Jones, P. H. (2014), Systemic design principles for complex social systems. In G. Metcalf (Ed.), Social Systems and Design, Volume 1 of the Translational Systems Science Series (pp. 91-128). Tokyo: Springer.
  30. Jones, P. (2020). Systemic design: Design for complex, social, and sociotechnical systems. Handbook of Systems Sciences (pp. 787-811). Singapore: Springer.
  31. Kieboom, M. (2014). Lab matters: Challenging the practice of social innovation laboratories. Amsterdam, NL: Kennisland.
  32. King, N. (2004). Using templates in the thematic analysis of text. G. Symon & C. Cassell (Eds.). Qualitative Methods and Analysis In Organisational Research (pp. 256–270). London: Sage.
  33. Kjøde, S. G. (2022). Navigating designerly systemic approaches for sustainability transitions. Proceedings of Relating Systems Thinking and Design, RSD11.
  34. Kronsell, A., & Mukhtar-Landgren, D. (2018). Experimental governance: The role of municipalities in urban living labs. European Planning Studies, 26 (5): 988-1007.
  35. Lake, D., Marshall, D.J., Carducci, R., & Thurnes, T. (2022). The power and place collaborative: Participatory strategies for scaling. Contexts—The Systemic Design Journal, 1.
  36. Leminen, S., Westerlund, M., & Nyström, A. G. (2012). Living labs as open-innovation network. Technology Innovation Management Review, 2 (9): 6-11.
  37. Leminen, S., Nyström, A. G., Westerlund, M., & Kortelainen, M. J. (2016). The effect of network structure on radical innovation in living labs. Journal of Business & Industrial Marketing, 31 (6): 743-757.
  38. Leadbeater, C. & Winhall, J. (2020). Building better systems: A green paper on system innovation. Copenhagen: Rockwool Foundation.
  39. Loorbach, D. & Rotmans, J. (2010). The practice of transition management: Examples and lessons from four distinct cases. Futures, 42 (3): 237-246.
  40. Maas, T., van den Broek, J., & Deuten, J. (2017). Living Labs in Nederland. Van open testfaciliteit tot levend lab [From open test facility to living lab]. Den Haag, NL: Rathenau Instituut.
  41. Malmberg, K. & Vaittinen, I. (Eds.) (2017).  Living lab methodology handbook. European Commission.
  42. McKelvey, B. & Lichtenstein, B. (2007). Leadership in the four stages of emergence. J. Hazy, J. Goldstein, & B. Lichtenstein (Eds.). Complex Systems Leadership Theory (pp. 94-107). Mansfield, MA: ISCE Press.
  43. Meuleman, L. (2008). Public management and the metagovernance of hierarchies, networks and markets: The feasibility of designing and managing governance style combinations. Springer Science & Business Media.
  44. Mulder, I. (2012). Living labbing the Rotterdam way: Co-creation as an enabler for urban innovation. Technology Innovation Management Review, 2 (9), 39-43.
  45. Mulder, I. (2018). Co-creative partnerships as catalysts for social change. Strategic Design Research Journal, 11 (3), 178-185.
  46. Overdiek, A. & Bergema, K. (2022). Framing a process model for design impact with a systemic lens. Proceedings of Relating Systems Thinking and Design, RSD11.
  47. Pearsall, M. (1970). Participant observation as role and method in behavioral research. W. J. Filstead (Ed.), Qualitative Methodology: Firsthand Involvement with the Social World (pp. 340–352). Chicago: Markham.
  48. Puerari, E., De Koning, J. I., Von Wirth, T., Karré, P. M., Mulder, I. J., & Loorbach, D. A. (2018). Co-creation dynamics in urban living labs. Sustainability, 10 (6), 1893.
  49. Rittel, H. & Webber, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.
  50. Ruiter, C. & van der Veen, H. (2021). Het Ontwortelen van Wantrouwen [Uprooting mistrust]. Helleman, G., Majoor, S., Peek, G. J., & van der Veen, H. (Eds.). Participatiemoe (d). Publicatie Platform Stad en Wijk. Amsterdam: Hogeschool van Amsterdam.
  51. Schliwa, G. (2013). Exploring living labs through transition management-challenges and opportunities for sustainable urban transitions [Master thesis, IIIEE, Lund University, Sweden].
  52. Schütz, F., Heidingsfelder, M. L., & Schraudner, M. (2019). Co-shaping the future in quadruple helix innovation systems: Uncovering public preferences toward participatory research and innovation. She Ji: The Journal of Design, Economics, and Innovation, 5 (2), 128-146.
  53. Schuurman, D., De Marez, L., & Ballon, P (2016). The impact of living lab methodology on open innovation contributions and outcomes. Technology Innovation Management Review 6, 7–16.
  54. Schwaninger, M. (1997). Integrative systems methodology: Heuristic for requisite variety. International Transactions in Operational Research, 4 (2), 109-123.
  55. Sinteur, I. (2018). Autonomous groups between power and empowerment [Doctoral dissertation, Vrije Universiteit Amsterdam, Netherlands].
  56. Siscode (2021). Co-design for society in innovation and science. Resource D5.1 Models of Co-creation Ecosystems.
  57. Ståhlbröst, A. & Holst, M. (2013). The living lab methodology handbook. Luleå, Sweden: Vinnova.!/file/LivingLabsMethodologyBook_web.pdf
  58. Steen, K. & Van Bueren, E. (2017). The defining characteristics of urban living labs. Technology Innovation Management Review, 7 (7), 21-33.
  59. Tiesinga, H. & Berkhout, R. (Eds.). (2014). Labcraft: How social labs cultivate change through innovation and collaboration. London: Labcraft Publishing.
  60. Von Wirth, T., Fuenfschilling, L., Frantzeskaki, N., & Coenen, L. (2019). Impacts of urban living labs on sustainability transitions: Mechanisms and strategies for systemic change through experimentation. European Planning Studies, 27 (2), 229-257.
  61. Voytenko, Y., McCormick, K., Evans, J., & Schliwa, G. (2016). Urban living labs for sustainability and low carbon cities in Europe: Towards a research agenda. Journal of Cleaner Production, 123, 45-54.
  62. Westerlund, M., & Leminen, S. (2011). Managing the challenges of becoming an open innovation company: experiences from Living Labs. Technology Innovation Management Review, 1 (1), 19-25.
  63. Westley, F., Goebey, S., & Robinson, K. (2012). Change Lab / Design lab for social innovation. White paper, Waterloo Institute for Social Innovation and Resilience.
  64. Westley, F., Laban, S., Rose, C., McGowan, K., Robinson, K., Tjornbo, O., & Tovey, M. (2015). Social innovation lab guide. Waterloo Institute for Social Innovation and Resilience.
  65. Williams, S., & Robinson, J. (2020). Measuring sustainability: An evaluation framework for sustainability transition experiments. Environmental Science & Policy, (103), 58-66.
  66. Yin, R. K. (2015). Qualitative research from start to finish. New York: Guilford Publications.
  67. Yasuoka, M., Akasaka, F., Kimura, A., & Ihara, M. (2018). Living labs as a methodology for service design: An analysis based on cases and discussions from a systems approach viewpoint. DS 92: Proceedings of the DESIGN 2018 15th International Design Conference (pp. 127-136).
  68. Zivkovic, S. (2018a). Systemic innovation labs: A lab for wicked problems, Social Enterprise Journal, 14 (3), 348-366.
  69. Zivkovic, S. (2018b). The early stage analysis of a systemic innovation lab. Proceedings of Relating Systems Thinking and Design, RSD7.
Anja Overdiek

Anja Overdiek | PhD | Professor Cybersocial Design, Rotterdam University of Applied Sciences | Associate Professor Designing Value Networks, The Hague University of Applied Sciences in the Netherlands | Founding Partner, Dutch Expertise Network Systemic Co-Design

The nexus between theory and practice in social movements and systemic change is a red thread in Anja’s work. Her current focus is multi-stakeholder co-creation and its opportunities and challenges in transitions.

Page views