Web's Largest listing of crowdsourcing and crowdfunding events

Events

Advertise

Web's Largest Directory of Sites

2,338 crowdsourcing and crowdfunding sites

Implementing Open Innovation Beyond the Pilot Stage: Barriers and Organizational Interventions

document Open Innovation

Implementing Open Innovation Beyond the Pilot Stage: Barriers and Organizational Interventions

RWTH-TIM Working Paper, October 2012

Dirk Lüttgens, David Antons, Patrick Pollok, and Frank Piller RWTH Aachen, Technology and Innovation Management Group Templergraben 55, 52056 Aachen, Germany tim.rwth-aachen.de luettgens | antons | pollok | piller @tim.rwth-aachen.de

Abstract: Open Innovation has been demonstrated to be an effective strategy to enhance the efficiency of a firm's innovation process. In this paper, we focus on broadcast search (also called tournament-based crowdsourcing), a method in the later stages of an innovation project directed to solve technical problems in form of an open call. Based on a design science approach and a longitudinal study of six companies engaged in piloting of open innovation, we identify barriers and sources of resistance that hinder its implementation in firms. Our paper contributes to open innovation research by analyzing crowdsourcing on the level of pilot projects, hence providing a workflow perspective that considers the creation of dedicated processes and operations of crowdsourcing. This project level analysis of crowdsourcing enables the identification of specific challenges managers face when implementing crowdsourcing within an established R&D organization. Following a design science approach, we also derive suggestions for organizational interventions to overcome these barriers. We find that dedicated promotor roles strongly contribute to a successful implementation of crowdsourcing, turning pilot projects into an organizational routine. Keywords: Tournament-based Crowdsourcing, Broadcast Search, Open Innovation, Promotor, Barriers to Innovation

1

Electronic copy available at: http://ssrn.com/abstract=2161264

1

Introduction

As knowledge is widely distributed across the external environment, many companies have opened up their innovation process to gain access to the expertise of external parties and existing knowledge from different domains (Laursen/Salter 2006). Past literature has used the term open innovation to characterize an innovation process that operates as an open search and solution process between several agents beyond conventional organizational and technical boundaries (Chesbrough 2006; Dahlander/Gann 2010). As a management approach, open innovation offers a set of different methods and practices which support innovating companies to identify and integrate relevant external knowledge. The idea is to enable new forms of distributed, crowdsourcing-based problem solving beyond conventional arrangements such as innovation alliances or contract research (Reichwald/Piller 2009). While earlier literature has demonstrated positive performance contributions of open innovation (Almirall/Casadesus-Masanell 2010; Fey/Birkinshaw 2005; Laursen/Salter 2006; Leiponen/Helfat 2010; Vega-Jurado et al. 2009), more recent literature has emphasized that firms need to build dedicated processes and internal capabilities to effectively utilize this opportunity (Bianchi et al. 2011; Dahlander/Gann 2010; Foss et al. 2011). As Gupta and Govindarajan (2000, p. 10) state, a firm's internal organizational structure determines "the information processing potential between its various subunits and with the environment." A few studies have looked upon the different structural dimensions through which firms open up their boundaries to identify and utilize knowledge from external sources (Siggelkow/ Levinthal 2003; Zhang/Baden-Fuller 2010). Yet, two key aspects in the context of open innovation have been neglected in the past literature: First, prior research has not differentiated between the different approaches of open innovation (Lichtenthaler 2011), like ideation contests, co-creation toolkits, developer communities, or tournament-based crowdsourcing. Even though most open innovation practices are based on the crowdsourcing principle, they differ significantly in terms of the type of knowledge exchanged as well as with regard to the stakeholders getting involved. Ideation contests, for example, require only limited disclosure of sensitive information by the focal firm and are primarily used for the acquisition of need information during the first stages of the innovation process, often organized by the marketing department (Piller/Walcher 2006). Tournament-based crowdsourcing in the context of technical problem solving, on the contrary, requires not only the participation of other internal organizational units (research & development), but also seeks a different type of knowledge, solution information in a later stage of the innovation process. In this paper, we focus on the latter method for open innovation, tournament-based crowdsourcing, also called broadcast search (Jeppesen/Lakhani 2010). Here, a technical challenge of a "seeker" is announced broadly to a group of external "solvers" in form of an open call (Afuah/Tucci 2012; Spradlin 2012). Potential participants screen the challenge and decide whether to invest in solving the challenge and submitting a solution proposal. The seeker then acquires the winning solution, i.e. those that best meet pre-defined performance criteria. In most cases, the problem broadcasting and solution transfer is facilitated by an intermediary. Established players in this domain include NineSigma, InnoCentive, and Yet2 (Diener/Piller 2010). Following such a procedure for technical problem solving is, in most instances, a radical departure from a firm's established routines of problem solving during R&D (Sieg et al. 2010; Jeppesen/Lakhani 2010). For example, during the cooperation with the intermediary, seeker companies have to disclose technical problem information and might reveal sensitive information about the firm's future development projects – in particular areas where this firm lacks problem solving capacity or where it failed in the past. In addition, the company's R&D staff has to acknowledge that "outside people are smarter than us" (Spradlin

2

Electronic copy available at: http://ssrn.com/abstract=2161264

2012, p. 86). Commonly, such perceived risks invoke internal opposition to pilot crowdsourcing or later lead to the rejection of identified external knowledge by the seeker company. The objective of this paper is to investigate the organizational processes and structures that overcome this inertia and support the successful implementation of open innovation. Implementing open innovation (or any method of crowdsourcing) is typically carried out by means of pilot projects. Most firms strive to gather experience by testing crowdsourcing first on the level of pilot projects before implementing it (if ever) within the entire company. This follows a common practice of implementing organizational or managerial innovation (Lechner/Floyd 2007; Turner 2005; Witte 1997). Previous open innovation research, however, has studied aspects of the organizational structure at the company level only (Dahlander/Gann 2010; Foss et al. 2011), but neither has focused on the project level nor has it taken specific aspects into account which follow from the first implementation of crowdsourcing within an established organization (Lechner/Floyd 2007). In this paper, we want to close this gap by investigating the implementation process of broadcast search (tournament-based crowdsourcing) on the level of pilot projects. Our objective is to identify appropriate organizational procedures and practices. In particular, we investigate how tournament-based crowdsourcing can be effectively implemented as an innovation management practice in a (seeker) organization. Following a design science approach (Hevner et al. 2004; Pfeffers et al. 2007; von Aken 2005), our intended contribution is twofold: We want to identify both critical incidents which may occur during the implementation of crowdsourcing in the innovation process and also want to derive suggestions for organizational interventions to overcome these barriers. By doing so, we extend the recent state of literature in several ways. First, by analyzing the implementation process of crowdsourcing on a project level, we offer a deeper understanding of the general project flow of crowdsourcing projects and their critical activities. Secondly, we identify and describe specific management challenges companies face when implementing tournament-based crowdsourcing as a new method during their innovation process. Thirdly, by matching the identified challenges and hurdles to barriers to innovation already described in the literature, we find that dedicated promotor roles strongly support resolving the challenges. We provide a detailed assessment how these promotor roles enable the implementation of crowdsourcing during the innovation process. The remainder of our paper is organized as follows. Section 2 outlines the conceptual background of this research and briefly reviews the literature on crowdsourcing in the context of new product development and innovation. The section concludes with a detailed description of the general project flow of tournament-based crowdsourcing on the project level. This structure serves as the conceptual framework for our empirical study. Section 3 describes our research setting and the two research phases that structured our investigation. Section 4 reports the results of our research on an aggregated level as well as from a more detailed process perspective. Finally Section 5 provides the discussion of our findings, comments on the limitations, and derives suggestions for further research.

2 2.1

Background: Literature Review and a Process Model Literature Review: Crowdsourcing for Technical Problem Solving

The rationale behind systematic boundary-spanning searches in the sense of open innovation is to overcome the problems of local search and industry blindness (Brunswicker/Hutschek 2010; Stuart/Podolny 1996; Rosenkopf/Nerkar 2001). While local, contextually bounded searches turn out to be advantageous when current problems are similar to old problems, e.g. in the case of improvements of existing products and processes, such search routines often do 3

not lead to radical advancements (Rosenkopf/Almeida 2003). When focusing on a limited solution space, companies only apply the most obvious instead of the most efficient of all solutions in order to solve an innovation problem. Tapping instead into the knowledge of a variety of external agents has been shown to overcome the negative biases of purely local search-based problem solving approaches (Jeppesen/Lakhani 2010). One way to efficiently access solution information beyond a firm's boundaries is an open call directed at a heterogeneous network of external experts. "Open call" here refers to a problem statement that is publicly announced (also called a "request for proposals", or RFP). The idea behind this "broadcasting of problems" is to spread the problem statement as widely as possible, allowing even unknown outsiders to contribute to its solution. Potential solution providers (solvers) decide via self-selection whether they want to participate in the process of finding a solution to the respective technical problem or not. The seeker, i.e. the entity issuing the call, then selects the best submitted solutions and either awards a pre-defined incentive to the winning solver or engages in collaboration with the identified solution provider. Jeppesen and Lakhani (2010) refer to this process as broadcast search, creating a synonym for the crowdsourcing neologism coined by Howe (2006). Other authors use the term "tournamentbased crowdsourcing" (Afuah/Tucci 2012) to describe the search for solution opportunities related to actual development tasks in the form of innovation tournaments. This process is facilitated by a number of specialized intermediaries who apply the broadcast search method as a service for other organizations to bridge the gap between solution seeking companies and external solvers (Feitler et al. 2012). Examples of these intermediaries include NineSigma, InnoCentive, YourEncore, Atizio, or Yet2.com. The success of these intermediaries is greatly dependent on their ability to achieve a connection between seekers and solvers while establishing it via an interface of their own web platform (Jeppesen/Lakhani 2010). In addition, these intermediaries support their clients with expertise how to draft a good problem statement (RFP) or engage in a pre-selection of fitting solutions. The role of the intermediary also is to monitor fair play and to prevent the exploitation of proposals by a seeker that has not acquired the corresponding intellectual property rights (Diener/Piller 2010). Prior research on intermediary mediated crowdsourcing has demonstrated the effectiveness of this approach in the context of technical problem solving. Afuah and Tucci (2012), for example, show that depending on the nature of the innovation task, tournament-based crowdsourcing can be a more efficient and effective problem solving mechanism than internal sourcing or designated contracting. As part of an empirical study of 166 crowdsourcing projects, Jeppesen and Lakhani (2010) observe that InnoCentive could facilitate answers for 30 percent of technical problems which previously could not be solved by their originators, all large global pharmaceuticals or chemical firms. With respect to the solver side, Jeppesen and Lakhani (2010) find that technical marginality, i.e. the distance between a solver's field of expertise and the problem domain, is associated with a higher likelihood of success from the solver's perspective. These studies demonstrate the power of tournament-based crowdsourcing for solving innovation challenges and for identifying new collaboration partners to trigger knowledge transfer across different domains. Other research has focused on the intermediary side of the process. Some authors provide classification schemes in an attempt to add structure to the constantly growing arena of innovation intermediaries (e.g. Howells 2006; Lopez-Vega 2009; Diener/Piller 2010). Other studies have focused on the optimal design and award structure of innovation contests in general and crowdsourcing tournaments in particular, hence investigating an important part of an intermediary's business model (Terwiesch/Xu 2008; Boudreau et al. 2011). Terwiesch and Xu (2008), for example, discuss the effect of solver community size on the outcome of an innovation contest and find that an increase in solver pool size results in a trade-off situation between the overall diversity of solutions (associated with higher submission quality) and 4

each solver's problem-solving effort. However, surprisingly little research has dealt with the seeker side of the crowdsourcing process. With the notable exception of Sieg et al. (2010), who identify critical activities in crowdsourcing projects, and some very recent work by von Krogh et al. (2012) who add to a better understanding of the problem formulation stage of crowdsourcing tournaments, research on the seeker perspective of problem broadcasting is lacking. Evidence on organizational aspects such as internal processes or governance structures supporting the implementation of broadcast search from the perspective of a seeker organization does not yet exist. Earlier research on innovation management has also emphasized the need to differentiate between the adoption and the implementation of organizational innovation (Klein/Sorra 1996; Klein/Knight 2005; Sproull/Hofmeister 1986). In contrast to implementation, which is defined as a process with dichotomous outcomes (successful implementation and implementation failure), innovation adoption is described as an activity that refers to "an organization's decision to install an innovation" (Klein/Sorra 1996, p. 1057). The need to differentiate between adoption and implementation is supported by the fact that organizations often adopt innovations but later fail to implement them successfully (Klein/Knight 2005). Consequently, successful implementation is anything but an obligatory sequel to the decision to adopt (Sproull/Hofmeister 1986). Rather, it has to be interpreted as the "transition period" or "critical gateway" between the decision to adopt the innovation and its routine use (Klein/Sorra 1996, p. 1057). Engaging in "broadcast search" and crowdsourcing in the innovation process can be regarded as an (process) innovation of radical nature from the perspective of the seeker organization (Spradlin 2012). Hence, the initial decision to adopt crowdsourcing as a management practice by a seeker firm will result in an implementation process by means of pilot projects which will either result in an institutionalized use of crowdsourcing later (when successfully implemented) or in the rejection of the method (when the pilot is considered to be a failure). To our knowledge, earlier research has not distinguished between crowdsourcing pilot projects and the reapplication or routinized use of crowdsourcing as an alternative approach to internal problem solving. In summary, we have a rather limited understanding of how companies should organize in order to effectively apply and implement broadcast search as a management practice. 2.2 Crowdsourcing for Innovation: A Conceptual Framework

When investigating the implementation of tournament-based crowdsourcing from the perspective of a seeker organization, different stages of such a process can be distinguished (Diener/Piller 2010; Jeppesen/Lakhani 2010; Sieg et al. 2010). Sieg et al. (2010), for example, identify critical activities in crowdsourcing projects and describe a typical process using the example of InnoCentive. We will broaden this view and extend the description of the general project flow by two additional stages (initiation and contract negotiation) in order to also cover the adoption and implementation stage of crowdsourcing. This extended description of tournament-based crowdsourcing is built on our experiences generated during a longitudinal case study which will be described in Section 3 in larger detail. This structure will later serve as a framework for the analysis of the pilot projects included in this study. Stage I: Initiation. Starting point of the initiation stage is the decision to adopt a crowdsourcing approach in the seeker company. A pilot project is initiated. Central during this stage is to communicate the introduction of the approach within the company and to educate employees about it. A second activity in this stage is the identification of an intermediary. This should include a thorough analysis of the intermediary marketplace. Intermediaries differ with regard to their solver community, ways of broadcasting the open call, their IP model, and the level of control that seeker companies can obtain during the crowdsourcing

5

project. Hence, it is important to identify an intermediary that provides the best “fit” to the seeker company and its technical problems (Diener/Piller 2010). Stage II: Contract negotiation. Once the intermediary has been selected, a contract is completed establishing the legal aspects in terms of the solutions to be generated, the IP terms, as well as the financial conditions (e.g. platform usage fees, success fees, incentive structures). Various contractual agreements are possible, which also can be attributed to the different business models of the intermediaries. Stage III: Problem formulation. Next, the crowdsourcing process starts with crafting the problem statement. The objective is to create a so-called "request for proposals" (RFP) document. The RFP document describes the technical problem to be solved and highlights the performance criteria that a winning solution has to meet. Moreover, the RFP informs potential solvers about targeted and possible partnership models (e.g. development contracts, licensing, consulting, etc.) in the course of the respective request. At this point of the process, the seeker also decides whether its company name should be disclosed or whether the RFP should be posted anonymously. Writing a RFP is no trivial task and constitutes one of the most important activities of the process (it hence is a core activity that is supported by experienced personnel of the intermediary): First, since the RFP will be broadly distributed among potential solvers, it has to be regarded as a public document and must only contain nonconfidential information. Secondly, the problem statement should be written in such a manner that it is clear enough for potential solvers from other fields to understand the core technical issue. At the same time, the technical problem has to be defined with high specificity to keep the scope of interpretation as narrow as possible for potential solvers. Stage IV: Open call. Once completed, the RFP document is broadcasted to the intermediary's community of potential solution providers (refer to Feitler et al. (2012) for a more detailed description of this stage). At this point, broadcast search takes the form of a tournament where potential solvers self-select themselves to participate and submit solutions for evaluation (Jeppesen/Lakhani 2010). Stage V: Evaluation of responses. After the submission deadline has passed, all solutions submitted by potential solvers are being evaluated. The intermediary supports the evaluation process by rating the level of solution competence indicated by each respondent. Based on this pre-evaluation, the client company decides which proposals are interesting enough to engage in a more detailed interaction with the solver (e.g., by means of sample requests, site visits, or telephone conferences). Stage VI: Reintegration. After detailed review and evaluation, the seeker will start to build a working relationship with interesting solution providers. Different types of contractual agreements such as licensing agreements, material transfer agreements, or development contracts are possible. Once a contract between the seeker company and a solution provider has been established, the cooperation with the intermediary has reached its conclusion (for this particular RFP project). The crowdsourcing process ends, and a conventional cooperation or contracting process starts. Based on the experience and perceived success of the reintegration stage, a company will decide whether to engage after the pilot projects in additional crowdsourcing activities or not. Hence, the perception of the outcome of this stage determines whether crowdsourcing for technical problem solving will be implemented as a standard practice in the R&D process.

6

3 3.1

Research Setting Methodological Approach

Our research is of exploratory nature, as no previous investigation of the implementation of crowdsourcing for technical problem solving on the project level exists. But as the description of the six stages of implementing such a process may already have indicated, multiple factors with different properties may influence the outcome of such an adaption process. To get a better understanding of these underlying factors, we followed a qualitative case study approach, which is suitable when the proposed research addresses a contemporary phenomenon and is largely exploratory in nature (Darke et al. 1998; Yin 2008). Typically, multiple case studies based on a cross-case analysis are used to get a deeper understanding of such factors and to increase the methodological rigor of the investigation (Eisenhardt 1991; Numagami 1998). Multiple case studies are believed to be particularly appropriate in the context of the implementation of new methods and practices (Miles/Huberman 1994; Shakir 2002). Two types of multiple case studies can be distinguished (van Aken 2005): In developing case studies, the researcher collaborates with the individuals of the organization studies in some kind of action research. In extracting case studies, best practices to solve managerial problems are analyzed and rules are inducted, tested, and refined by adding more cases. We applied the latter type during our study. From a methodological point of view, our research follows the paradigm of design science research (Hevner et al. 2004; Pfeffers et al. 2007). Design science intends to build, evaluate and, in a second step, improve research objects which are applied in a specific business context. In the design science connotation, the research object is called the artifact. We consider the implementation process of crowdsourcing as our artifact and conduct a twostep research approach (called the design cycle, Takeda et al. 1990). In Research Phase 1 we followed four companies during their piloting of tournament-based crowdsourcing in R&D in form of longitudinal case studies. We studied how the process of broadcast search was perceived and run through by these companies. Based on the analysis of Phase 1 we derived a set of recommendations to improve the implementation process. During the second research phase we applied these suggestions for improvement in form of organizational interventions within two additional companies who followed a modified piloting process, educated by the research team based on the experiences from the first research phase (Pfeffers et al. 2007). We then compared the outcomes of both processes (a similar two-step design science approach has been followed by Heinrich et al. (2009a, 2009b), investigating the piloting and implementation of customer relationship management software). 3.2 Empirical Setting, Data Collection, and Analytical Approach

In order to identify common patterns across cases in Research Phase 1 and to transfer the derived insights to our cases in Phase 2, we applied a homogeneous sampling strategy (Curtis et al. 2000). The rationale behind this sampling was to improve the comparability of piloting companies and to reduce potential influences of factors other than our modifications of the design artifact. Therefore, we selected cases with sufficient similarities to allow for a meaningful identification of differentiating factors with regard to the research questions: First, we exclusively focused on pilot projects to avoid biased results due to different levels of experience within our sample. In addition, all of the piloting companies are headquartered in Germany and belong to the domain of mechanical engineering. This excludes the potential influence of cultural and industry specific differences from our study. Firms were selected via cooperation with VDMA ("Verein Deutscher Maschinen- und Anlagenbauer"), a large German industry association in the field of engineering. Secondly, all case companies 7

engaged with the same intermediary, NineSigma. While it would have been interesting to also study differences between pilots with different intermediaries (e.g., InnoCentive, YourEncore, Atizio, or Yet2.com), we purposefully decided to focus this study on one intermediary. This should help us to better identify differences in internal project structures of the seeker organizations, as all pilots followed the same project structure given by the intermediary. An overview of the selected crowdsourcing pilot projects can be found in Table 1. For reasons of confidentiality, company names have been removed. Table 1: Overview of cases

Case

Number of employees (in 2011) < 1500 10.000 – 15.000 >50.000 10.000 – 15.000 >50.000 > 50.000

Revenue in Euro (in 2011) <200 million < 10 billion >10 billion < 10 billion > 10 billion >10 billion

Industry Aerospace; energy; automotive Food; clothing; infrastructure; energy Automotive; construction vehicles; drives industry Automation; renewable energy; railroad systems Automotive; renewable energy; agriculture; railroad systems Automotive; military; sports; industrial engineering

Nature of innovation problem Product Product Process Process Process Product

Crowdsourced task (content of RFP) Development of materials for a heavy-duty manufacturing Development frictionless surfaces in mass production Development of chemical process technology Development of a filter technology Development of a testing procedure Development of a laser technology

Proposals received 26 35 10 7 17 37

Number of follow-up projects 0 0 0 0 3 8

1 2 3 4 5 6

Our case study research spanned over a period of three years. Data were collected between 2008 and 2011 as part of a longitudinal research performed in real time, meaning that we actively accompaniment the implementation of the observed broadcast search pilot projects during the two research phases. During this time, our research group was able to obtain data from observations, interviews, meetings, documents such as e-mails and internal reports: We participated in a large number of informal conversations as well as in a total of 15 all-day project meetings with 6 to 25 participants from all company levels. Important decisions were made at some of these meetings concerning the further execution of the different projects. In addition, 20 semi-structured interviews were conducted with project participants, including managers and technical experts at the seeker companies, as well as with program managers from the intermediary. To ensure the reliability of our results, the in-depth interviews, which lasted from 1 to 2 hours, were recorded, transcribed, and analyzed by three different members of our research team independently in terms of problems and hurdles as well as process delays during the pilot projects. These interviews were kept broad in scope in order to address a wide range of topics due to the exploratory character of our research in Phase 1. This approach provided deep insight into the overall satisfaction of the employees with the project and the cooperation process, their motivation to participate in the tournament-based crowdsourcing project, the process of evaluating solution proposals, critical decision-making situations, the competency and authority of different team members, and difficulties observed during the individual stages of the broadcast search process.

8

4

Results

Following the typical cycle of design science, we will report the findings of Research Phase 1 first, whereby we distinguish between results on the aggregate level of all pilot projects of this phase and specific observations on the level of each individual project and the individual pilot stages. The observations in Phase 1 allowed us to identify typical problems and hurdles that evolve during the broadcast search process. We will discuss these factors and derive suggestions for improvements, based on our own observations and recommendations from the literature. Following the design cycle, we used these suggestions as interventions in two further projects during Phase 2 of our research, as described in Section 4.2. We will discuss how this organizational treatment affected the piloting process and the implementation of the method in the case companies. 4.1 Findings of Research Phase 1

Analysis: Aggregated perspective In this section, we will look at the overall outcomes of the projects in the four case companies observed during Research Phase 1. The companies received between 7 and 35 proposals to their RFP. Analyzing the proposals, we found that broadcast search in principal is well suited to provide solutions for technical problems from the mechanical engineering domain. Based on the evaluation by employees with technical expertise in the respective domain from the different case companies, the proposals provided access to technical knowledge, solutions, and ideas judged to be unfamiliar and unknown compared to the state of the art, but still being "highly interesting" and "worth further processing". For any of the individual RFP project, more than 60 percent of the submitted proposals were (radically) new to the organizations. Moreover, the companies were able to identify 53 new potential cooperation partners in total. Solutions were primarily submitted by institutions throughout Europe and North America, which resembles the general case at NineSigma. 38 proposals were submitted by companies, 26 by universities, and 14 by other organizations like research centers or research councils. Also noteworthy, the evaluating experts stated that every submission showed a high level of elaboration and detail. While this enabled a rigid evaluation due to sufficient information, the evaluation stage also lasted considerably longer than expected, as much more information had to be processed. A surprising observation was the huge difference between the time required for company internal processes (Stages I-III and V) compared to the crowdsourcing activity (Stage IV). In average, it took less than 40 days to identify the novel solutions via an open call within the network of NineSigma (Stage IV), while the time to get this activity started and to evaluate the incoming proposals spanned over a period of 110 to 145 days. The four case studies demonstrated us that on the one side broadcast search has the ability to deliver novel technical solution approaches and to identify promising new cooperation partners. But on the other side, a number of hurdles and barriers within the seeker companies seem to influence the progression of the pilots substantially. Our more granular analysis of the six stages of a RFP project will investigate these barriers in more details in the following. Analysis of critical instances during the stages of piloting broadcast search In the following section, we will elaborate on critical instances observed during the six stages of crowdsourcing for technical solutions in R&D, as outlined in Section 2.2. As an overview,

9

Table 2 provides exemplary quotes from the interview studies to illustrate these hurdles and sources of resistance. Stage I: Initiation. The initial stage of each project challenged companies with creating a realistic awareness towards the new method. Because of its novelty, the method was not welcomed in the beginning – many employees from R&D had a rather critical attitude towards the new approach, based on fear of potentially losing their job because of outsourcing R&D activities. This skepticism led to the fact that technical problems with a lack of relevance to the current situation were chosen to test the instrument of the RFP. In addition, often problems with the character of a "holy grail" in the industry were suggested – problems long known as highly relevant but complex and "unsolvable" to the entire industry. In addition we found that managers asked to "submit problems" to the RFP were reluctant to disclose their incapability not having found a solution to a specific problem. For example, in Cases 1 and 3, R&D managers chose not to post a RFP for a problem regarding quality management as they feared negative consequences for their company, even though the RFP was posted anonymously. This situation was further increased in cases where the decision to sponsor such an initiative had been made on a lower management level. In sum, these problems led to a rather long duration of the first stage as a result of the necessity of multiple iterations and consultations between departments. Also noteworthy, we found that companies selected the intermediary based on rather subjective criteria such as the first impression or an evaluation solely based on best practices. Furthermore, it later turned out that in Cases 3 and 4 problems were chosen which were not in the current technological focus of the respective companies. Hence, these two case companies lacked any incentive to reintegrate external solutions in Stage VI. Stage II: Contract negotiation. Hurdles in this stage were based on difficulties to integrate the intermediary's contractual conditions into the companies' contracts (terms of conditions). The collaboration model with an open innovation intermediary was an unfamiliar format for the firms' legal departments so that a new framework of rules had to be found in order to proceed with the project. In particular, issues concerning intellectual property rights were a challenge, e.g. defining a model for property rights on solutions or ideas or the protection of future claims. Often, working out the contract conditions was prioritized low within the legal departments due to their piloting characteristics. But we also observed that the legal counsels had rather little experience in the specific demands of the legal governance structure of an open innovation project. This again resulted in a large delay because tasks were deferred. An assessment of e-mails written during this stage indicates delays of 4 weeks and more due to conflicts with the legal department. Stage III: RFP Formulation. Formulating a problem statement meeting the requirements outlined in Section 2 definitely is a core factor of success. This task not only requires the ability of abstraction but also to define ex ante suitable performance criteria for evaluation. In general, the intermediary provides plenty of support in this stage. Still, capacities within the seeker firm are necessary to formulate a proper RFP. Although R&D managers and engineers in the industry of our study in general are trained to verbalize problems, this stage took significantly longer than anticipated. A first challenge was to find staff able to formulate the problem. Here, the firms missed the opportunity to assign competences clearly. The farther the problem deviated from current topics, the less employees were interested in investing hours of work. Furthermore, to our surprise, even trained R&D managers were not truly able to characterize the problem detached from the context of their firm. Firms also had considerable troubles in formulating suitable evaluation criteria. Two firms (Cases 1 and 2) came up with criteria during a first iteration which would have made a solution technically impossible. Firms 3 and 4 even refused to define criteria for evaluation at all. But without evaluation criteria it is very hard for potential solvers to match their potential solution proposal with the seeker's expectations. Altogether, it was apparent that firms faced signifi10

cant challenges formulating an appropriate RFP. This not only led to a considerable delay of the process but also already to frustration of the projects' internal supporters and a high dissatisfaction with the open innovation approach in general and tournament-based crowdsourcing in particular. Stage IV: Open call. This stage is essentially handled through the intermediary. Hence, there were no major problems concerning our project firms. Nevertheless, it became apparent that the firms did assign responsibilities only limitedly as inquiries from the intermediary during this stage were answered rather slowly. Stage V: Evaluation of responses. The evaluation of the submitted solutions was accompanied by several difficulties. Initially, the main problem was to identify suitable technical experts who were capable to evaluate the individual proposals. Due to the fact that the solutions often originated from domains unfamiliar to the firm, employees did not feel comfortable to evaluate solutions due to lack of competencies. Furthermore, it became obvious that the duration of this stage was in all cases beyond the expected time necessary for evaluation (keeping in mind that solution providers also expect a feedback on their proposal within an adequate timeframe). Although the firms were able to apply existing valuation methods and practices, they still needed to customize them according to the respective evaluation task. Additionally, in Case 3 some employees felt that the submitted proposals were not useful for their own work so that they showed little commitment to evaluate thoroughly. Overall, it became evident that the selection of evaluators is a critical activity. But in the course of the projects, evaluators or experts were often rather chosen for reasons of availability and capacity than with regard to their qualification. Stage VI: Reintegration. Despite the fact that qualified technical solutions were submitted for all RFPs, at the present time not one of the seeker firms acquired an external solution or engaged in further cooperation with a solver. From a pure efficiency perspective, hence all four pilots can be regarded as a failure: not one of the broadcasted problems has been solved. Apparently, all critical incidents described before added up, especially the problems of "not invented here" and the lack of resources for a follow up. From an innovation management perspective, however, a RFP that does not lead to any in-licensing activity or development contract with a solution provider still can be considered a success. By scanning all submitted proposals, the seeker firm gets a better overview of the state of the art in a field (technology scanning), is able to combine ideas from the proposals for an internal solution of the problem (given that there are no IP restrictions), or even learns that its original problem statement perhaps is "unsolvable" and needs to be reconsidered. From our observations it were exactly these arguments that were used by the responsible managers to turn the pilots into a success, masking typical behavior of "not invented here" or the sheer unwillingness to engage in further interactions with a new cooperation partner. Our analysis of these four cases indicates that firms face major problems in utilizing tournament-based crowdsourcing in a piloting situation. Hurdles and obstacles occurred in almost every stage of the process. After following the cases in detail, we tried to map the problems within the broad existing literature concerning barriers and obstacles in the innovation process (for an overview see, e.g., Bond/Houston 2003; Mierow et al. 2007). We found that many of the challenges identified in our cases match earlier findings concerning resistances in (open) innovation processes (Van de Vrande et al. 2009; Chesbrough/Crowther 2006; Keupp/Gassmann 2009; Sieg et al. 2010). Chesbrough and Crowther (2006), for example, mention an inadequate management support, the NIH-syndrome, barriers in corporate culture, and insufficient resource endowment as key challenges of implementing open innovation. Van de Vrande et al. (2009) find potential problems arising from administrative barriers and a lack of predefined property rights. To perform this matching, three members of our research team analyzed the transcribed interviews for recognizable similarities, repeated patterns, and any notable differences with regard to barriers and causes of 11

delays of the pilot projects. This content-analytical evaluation was conducted with the help of Atlas.ti, a software application for the analysis of qualitative data. To assess the reliability of the resulting coding scheme, we applied Holsti's inter-coder reliability formula (Holsti 1969). With all of the indicators, the average inter-coder agreement resulted in a satisfactory value of CR = .862. In sum, we were able to identify 11 different types of barriers and obstacles in the literature which matched the hurdles occurring in the four case studies. Table 2 illustrates the results of this comparison. Here, the identified types of barriers, a description of the specific barrier as well as an exemplary statement made by an interviewee are given. Finally, the table outlines in which process stage the barrier could be identified. Looking at this table, it becomes evident that barriers exist in every stage of a crowdsourcing project, but initiation and RFP formulation seem to be particularly challenging. In these two stages, nearly every barrier occurred. Besides barriers like lacking communication, cultural aspects, and insufficient resources, particularly striking barriers within the evaluation stage were inflated expectations and the NIH-syndrome. This led to the fact that no firm started to seriously reintegrate the external input identified during the RFP. But without knowledge reintegration, the piloting and implementation of the method failed in the end. Table 2: Mapping of barriers observed in the pilot cases
Stage Barrier and description (B1) Workflow rigidity: Describes a situation in which workflows and internal processes are always done in a specific way that is not adapted. In our projects, this was seen primarily in the way certain company departments were strongly opposed to adopt the new technique of crowdsourcing. (B2) NIH (not-invented-here) syndrome: The suggestions of "externals" are disregarded and considered to be the wrong independed from their content. Internal processes are done in a way that new knowledge is first and foremost generated "inhouse". In the projects at hand the suggested solution approaches were not reintegrated into the development cycle, although the proposals had been positevely evaluated. (B3) Lack of internal commitment: This barrier describes the lack of interest of the R&D department in the pilot process in general. This barrier resulted in a rather slow commitment to identify potential problems for the pilot RFP and to provide feedback on the draft RFP. (B4) Bottom-up management: Pilots are being implemented as bootleg projects, bypassing official decision trees. In some of our cases, the projects had the feel of almost being subversive. Management only became aware of them in hindsight. Upper management levels then stepped in or even stopped the projects because they felt that someone had “gone over their heads.” (B5) Insuffiecent resources: The pilot projects are not equipped with sufficient resources. For instance, not enough personnel were assigned to formulate the problem and to determine what the right solution would be. This was basically due to unrealistic expectations about the project and/or methods. (B6) Allocating wrong task to pilot: Employees act opportunistically when the problem for the pilot RFP is being selected, suggesting unrealistic tasks that they hope are unsolvable anyway to demonstrate the inefficiency of the new method. In the four pilot cases, frequently "holy grail" problems were supposed, i.e. problems that to date have not yet been solved in the industry, but are of importance to Examplary quote from expert interviews 1 “I was surprised at how people built such strong internal barriers that they simply wouldn't allow to be torn down, and their overall reluctant attitude. (…) It was a real eyeopener to see this in so many of our company's departments.” (head of marketing, Case 4) “We see the not-invented-here problem a lot. R&D says: Why should we find an external solution when we can do it ourselves? If we do this, we're really only showing how bad we are.” (innovation manager, Case 2) 2 3 4 5 6

x

x

x

x

x

x

x

“The people in R&D weren't particularly interested.” (innovation manager, Case 1)

x

x

x

x

“There was just a lack of authority.” (engineer, Case 3)

x

x

x

x

“[...], so of course I met with our company's experts once the project was finished to talk about what we could have maybe done better if they would have helped. If they had provided more of their time, the open call would have been better and more detailed.” (innovation manager, Case 1) “[...] that was the feedback coming from the colleagues that did this RFP [...] they had the impression that the open call wasn't formulated properly. Next time they would present it differently in a way that communicates it from a perspective of their own technical problemsolving approach, and not in a ‘cutting-edge

x

x

x

x

x

x

x

12

Stage Barrier and description everyone involved in it. In addition, opponents suggests minor problems without internal commitment and interest in their resolution. (B7) Insufficient top management support: People in charge of the pilots receive insufficient support from top management. In seeker companies where the top managers were not actively involved in the our pilots, there was often a lack of support for the project by company leadership. This often resulted in a lack of sufficient resources. (B8) Unrealisitc expectation: Success stories of open innovation and broadcast search in the general press generates an unrealistic set of expectations on the level of some managers. In our pilots, this could be seen in the expectation that even "holy grail" problems in the industry could now be solved using the new method, and this even at very low cost. As a result, the effort required for the pilot was underestimated, i.e. there was the impression that the solution could be found on its own. (B9) Legal barriers: The legal departments of the pilot companies lack experience in IP related issues of open innovation. In our pilots, legal counsels often expressed a general concern about the terms of conditions of the intermediary and the IP structure underlying tournamentbased crowdsourcing. Worries by people objecting to contract process were a key reason for delays in the contract stage of the pilots. (B10) Organizational / administrative barriers: Those in charge of the project are generally not authorized to circumvent improper standard processes in the company and/or acclimate to the requirements of the broadcast search project. In our pilots, frequently organizational (work) routines and opponents insisting that the project stick to these led to significant delays. Examplary quote from expert interviews 1 research´ fashion.” (innovation manager, Case 4) “I can't tell you what it's like to stand in front of conservative managers whose leadership has not been able to solve the problem for years, and then you suggest solving the problem, but now with the help of external parties.” (engineer, Case 3) “I think my colleagues' expectations were very high or downright unrealistic [...]. I remember how everyone's eyes lit up when we talked about Goldcorp and Netflix and such. Everyone was thinking ‘We're going to hit the jackpot with this one and watch the money pile up.' I think the project showed us that these kinds of expectations are simply unrealistic.” (head of R&D, Case 2) 2 3 4 5 6

x

x

x

x

x

x

“After a lot of back and forth in the legal department, we finally had to admit that our business conditions didn't properly capture the relationship between us and the intermediary.” (head of innovation management, Case 4)

x

“[...] when you obtain a [solution], well, someone needs to have put the money aside to pay for it. But then when you don't find a [solution], the money is there, and might just lie around and start to gather dust. These mechanisms are typical of our annual budget planning. […] A lot of things converged [in the administration] that, instead of helping the project, did their part to make it a little harder.” (head of innovation management, Case 1) “[...] I always thought that the [department heads] would let their employees know what was going on. Unfortunately, that's exactly what they didn't do.” (engineer, Case 2)

x

x

x

x

(B11) Communication barriers: This barrier describes communication problems between the different departments and among the employee hierarchies. These barriers often led to misunderstandings and delays within the pilot cases.

x

x

x

x

x

Design recommendations from Phase 1 Our previous analysis indicates that piloting broadcast search projects faces many challenges and has to overcome severe barriers. This, however, is a common situation in R&D projects in general and pilots of process innovation in particular. Previous research has distinguished between three larger classes of barriers to innovation (Witte 1973; Hauschildt 1999; Gemünden et al. 2007): barriers related to will, barriers related to ability, and bureaucratic or administrative barriers.  The barrier of will describes resistance against innovation and change in general. The cause of this resistance is a rejection based on less justifiable reasons such as individual personal career opportunities, e.g. the motive to increase the own power position, or ideological and ethical motives.  The barrier of ability results from an actual or supposed lack of knowledge in the field of a specific domain. In the case of supposedly missing knowledge, someone knows about a fact or a technical problem or a method, but cannot apply this knowledge in a given situation.

13



Bureaucratic and administrative barriers are created by organizational or hierarchical circumstances. Employees willing and able to contribute to an innovation project do not have the permission to do so due to existing internal rules or limited capacity and resources.

To overcome barriers to innovation, already early innovation management literature recognized the importance of key individuals (Schumpeter 1911). Current literature suggests the usage of role models as means for describing tasks of individuals to overcome inertia in innovation projects. Role models such as the mono-personal champion concept (Schon 1963; Howell et al. 2005) or the multi-role model of different promotors (Gemünden et al. 2007) offer essential, empirically-founded explanatory models for overcoming obstacles in the innovation processes (Fichter 2009; Rost et al. 2007). These individuals are able to create conditions to overcome organizational inertia and opposition (Howell et al. 2005). While the earlier literature has focused on mono-personal models, the recent literature has stressed multi-personal models. The rationale behind the latter is that an innovation process regularly is complex and involves different persons, departments, and disciplines. Hence, multipersonal role models are seen as particularly effective in providing conditions to overcome inertia in complex innovation projects (Hauschildt/Kirchmann 2001). The multi-personal model established best in the literature originates from the German innovation management literature. It distinguishes three kinds of promotors, differentiating the type of barriers they help to overcome (Gemünden et al. 2007; Hauschildt 1999; Hauschildt/Kirchmann 2001). In addition, promotors can be differentiated according to their base of power on which their influence is grounded: The power promotor is a person who has the hierarchical power to drive a project, to provide necessary resources, and to help to overcome many obstacles concerning will and bureaucracy that might arise during the course of a project. The role of the expert promotor describes a person who has the specific technical knowledge for the innovation problem at hand. The process promotor derives her influence from organizational know-how and intra-organizational social networks. The process promotor establishes and maintains the connection between the power promotor, the expert promotor, and other project members (Gemünden et al. 2007). In addition, research has shown that promotor roles are especially affecting the success of an innovation project if they appear jointly (Hauschildt/Kirchmann 2001). Different persons have to be identified who take over the roles and corresponding tasks (one individual, however, can also hold more than one role). In summary, Table 3 illustrates the characteristics of the three promotor roles. By identifying promotors and enhancing their specific competencies and responsibilities, organizations increase the positive outcomes of their innovation projects (Gemünden et al. 2007). Following the design science logic of our research (Pfeffers et al. 2007), the promotor model also appeared to us as a promising idea to overcome the barriers we observed in the pilot cases of tournament-based crowdsourcing. In the following section, describing Phase 2 of our research model, we applied exactly this thinking to two further organizations (Cases 5 & 6, as outlined in Table 1 above). The last column of Table 3 already indicates the expected contribution of a promotor role in a crowdsourcing pilot in overcoming the internal barriers and resistance to this new method. This positive influenced, derived from the earlier empirical literature, also motivated our decision to focus on promotors as the suggested design intervention for Phase 2 of our research. We will discuss alternative interventions in the Conclusions section of this paper.

14

Table 3: Promotor roles according to Gemünden et al. (2007) and Hauschildt (1999)
Role Type of barrier addressed Source of power Typical contributions Providing information and topical input; development of alternatives; concept evaluation Goal definition; resource allocation; protection against opponents; process control Uniting and connecting; conflict management; goal-oriented communication; process management and coordination Expected contribution in crowdsourcing pilot Overcoming barriers B6, B8

Expert promotor

Barriers of ability

Object or problem specific know-how

Power promotor

Barriers of will; bureaucratic and administrative barriers Bureaucratic and administrative barriers; barriers of will

Hierarchical position

Overcoming barriers B1, B2, B3, B4, B5, B9

Process promotor

Communication and organizational skills, internal social network

Overcoming barriers B5, B7, B8, B9, B10, B11

4.2

Findings of Research Phase 2

The objective of the second research phase in the design science approach is to validate the findings from the first phase and to provide evidence whether the derived solution is suitable or not. Reflecting on the outcomes of Research Phase 1, it appeared appropriate to us to adjust the implementation and piloting process of crowdsourcing by fostering the contributions of proper persons who could be able to act as a promotor in order to overcome the observed hurdles. Hence, we actively accompanied and analyzed two additional companies (Cases 5 & 6 in Table 1) where we advised managers in charge of the crowdsourcing pilot to actively identify key individuals able to fulfill the promotor roles. Analysis: Aggregated perspective In response to their RFPs, case companies 5 and 6 received 17 and 37 solution proposals, respectively. Of these 54 proposals received in total, 25 proposals originated from industry, 21 from university institutions, and 8 from non-profit organizations. Solution providers again originated primarily from European and North American institutions. With regard to novelty and quality, the proposals did not differ from Cases 1 to 4. However, the overall duration of the process was 120 and 132 days from start to finalizing the evaluation stage in Case 5 and 6, respectively (in comparison to an average of 172 days in Cases 1 to 4). This constitutes a remarkable acceleration of the process of about 30 percent. Even more, we could observe a broader implementation of the broadcast search method: In contrast to Cases 1 to 4, both case companies continued with the method after completing the pilot. Company 5 immediately signed a contract for three, and Company 6 for eight follow-up projects with the intermediary. While it is too early to state that tournamentbased crowdsourcing has become a routine in these companies, this is a striking difference to the earlier cases (especially given that the actual outcomes and performance of all six pilot RFPs showed no large difference). We will explain this difference in the following, demonstrating how the organizational treatment we identified in Phase 1 – identifying key personal who can serve as promotors during the pilot stage – had a positive effect on the outcome. Validation of design recommendation: Intervention of promotors According to the design science approach, we actively coached the companies from Research Phase 2 by motivating and helping them to identify key individuals who could serve as promotors to overcome the barriers identified in Phase 1. Our main intervention was a 15

training and coaching activity that educated the responsible managers about the potential challenges and facilitated the development of means to overcome these barriers. First, we recommended to the companies to place the responsibility and ownership of the broadcast search method in the innovation management function of these firms. A central innovation management function often is a natural process promotor and has the given responsibility to create methodological knowhow for the organization. During the initial project kickoff, our team reported learnings from Research Phase 1 and educated the innovation management and its head of the potential positive role of promotors in crowdsourcing projects (and innovation in general). Building on the procedure to identify promotors proposed by Hauschildt and Kirchmann (2001), we asked a series of central questions aiming at the typical characteristics of promotors. In the following, we will report the effect of this intervention on the six process stages of tournament-based crowdsourcing via broadcast search. Stage I: Initiation. In both cases, an innovation manager identified himself as a potential process promotor after reading and thinking about the central questions. These managers, who had a wide in-house network at their disposal, took over the role of the process promotor. Through his own network, the process promotor addressed head of departments known to him in order to introduce the method: “To some extent, I had to take the role of salesman, a role in that one cannot decide but has to promote something to others so that they decide for it”(innovation manager from Case Company 5). The motivation of this intervention was to start the entire piloting process with a "problem owner", i.e. an individual who would provide a technical challenge for the RFP that would originate from her or his actual work. Instead of asking departments to "donate" problems, the process promotor actively engaged in search for a head of department who wanted to pilot the method in order to solve a given technical problem (and not for the pilot's sake), and hence would also allocate own capacities and resources. Furthermore, we encouraged the process promotors to identify power and expert promotors in a discussion with department heads by adopting the set of questions suggested by Hauschildt and Kirchmann (2001). Again, some heads of department self-identified themselves and took up the role of the power promotor within the projects. The persuasiveness and motivation of the power promotor made it easier to locate a problem suited for broadcasting and to identify an appropriate and qualified expert promotor, a R&D engineer within the division of the power promotor willing to post a problem and later engage in the evaluation and exploitation. The willingness of the individual in question to engage in the process was also taken into account during the selection of an eligible problem. Thus, individuals with the attributes of the entire troika of promotor types were present in the Cases 5 and 6. Stage II: Contract negotiation. The legal department got involved in the cooperation with the intermediary at an early stage. Both the power and the process promotor took large care to engage the legal department already in early meetings with the intermediary. In Case 6, the legal department wanted to postpone the final evaluation of the contract with the intermediary due to more urgent tasks in their core business. Here, the process promotor was able to accelerate the process through his contacts in the corporate headquarters. Stage III: Problem formulation. During the stage of the actual formulation of the RFP, the respective expert promotor of Cases 5 and 6 was in close touch with the intermediary and worked on the RFP drafting, providing his or her professional competence regarding the technical problem. The intermediary helped them in order to abstract the problem statement. In case that additional competencies were needed, the process promotor assisted by passing on contact details. When the evaluation criteria for the solutions had to be specified, the power promotor in Case 6 supported the technical expert to identify other experts who could provide input for the criteria definition. The final RFP document had to be cleared by the firms' legal department. At this point, the process promotors again ensured the prompt execution of this task. 16

Stage IV: Open call. As in Cases 1-4, the intermediary handled the majority of work during this stage. But as the contact already had been established, the process and expert promotors were available for inquiries of the intermediary and could provide swift feedback on any issues that came up during this stage. Stage V: Evaluation of responses. Differences between the two firms became apparent in this stage: In Case 5, the expert promotor took over the evaluation of incoming solutions, while in Case 6, the expert promotor noted that he was not able to evaluate the proposals on his own because they originated from very different areas of expertise. Hence, other experts were consulted to ensure objectivity and achieve higher quality of the evaluations. The process promotor was present as a moderator during the meetings in both cases. He could intercede if he felt that an idea was evaluated negatively due to a lack of comprehension of the proposed solution approach or any (unjustified) objections towards the institution of the solution provider. Stage VI: Reintegration. In Case 5, test objects were sent to the solution pro

Summary
Open Innovation has been demonstrated to be an effective strategy to enhance the efficiency of a firm's innovation process.

Description
In this paper, they focus on broadcast search (also called tournament-based crowdsourcing), a method in the later stages of an innovation project directed to solve technical problems in form of an open call. Based on a design science approach and a longitudinal study of six companies engaged in piloting of open innovation, they identify barriers and sources of resistance that hinder its implementation in firms. The paper contributes to open innovation research by analyzing crowdsourcing on the level of pilot projects, hence providing a workflow perspective that considers the creation of dedicated processes and operations of crowdsourcing.

SOURCE:http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2161264

Tags: Adoption of Innovation, Barriers to Innovation, Broadcast Search, Crowdsourcing, Implementation, Open innovation, Promotor, Tournament based Crowdsourcing

Posted by Benjamin Fossel

Jan 27, 2013 04:49 am GMT Viewed 701 times

Author: Dirk Lüttgens , David Antons , Patrick Pollok , Frank T. Piller Source

Contact us if you are owner of this content and want to make changes.

Download Flag This

Events

Implementing Open Innovation Beyond the Pilot Stage: Barriers and Organizational Interventions

Comments

Crowdsourcing.org

Categories

Content

Crowdsourcing.org, an initiative by massolution®

Events

Related Content

Implementing Open Innovation Beyond the Pilot Stage: Barriers and Organizational Interventions

Comments

Crowdsourcing.org

Categories

Content

Crowdsourcing.org, an initiative by massolution®