The evaluation of ideas is the final phase of the front end of innovation. The ideas that emerge from this phase turn into development projects and ultimately may become new products or services.
As part of our Idea Engineering research we are developing rules for the design of the idea evaluation phase. Here is a „Top 10“ list compiled from our experience.
- Start with clearly presented, concrete innovation goals. Many evaluation errors and tedious discussions stem from the lack of clearly stated goals. Are you looking for incremental, radical, platform, process, business model or new market innovations?
- Divide the process into phases. Idea Evaluation needs a stage-gate-type process in order to be efficient. Simple ideas may only need two phases, complex ideas will need up to four.
- Idea Evaluation consists of three tasks: Refinement, Evaluation and Selection. Refinement means adding information and comments to ideas in order to make them better understandable and evaluable (see Rule #5). Evaluation means assigning a judgement to an idea. This might vary from a simple „yes/no/maybe“ to a comprehensive system of scores. Selection means keeping some ideas and removing others from the process.
- Formulate refinement questions and evaluation criteria that serve the innovation goal. Every item of work that is invested in an idea should be geared towards answering the question „Will this idea help us meet our innovation goals?“ Too often, superficial criteria (such the PMI method) are used. This Rule presupposes that Rule #1 has been implemented.
- Selection must always be preceded by Evaluation and Evaluation must always be preceded by Refinement. It is inadvisable to remove or promote an idea without having first assigned some estimate of value to it, and it is dangerous to evaluate an idea before it has been described sufficiently thoroughly.
- Never process an idea expensively that could have been eliminated cheaply. This follows from the need for efficiency.
- The amount of time spent on an idea should be inversely proportional to the number of ideas under consideration. This also follows fairly obviously from the need for efficiency.
- Never compare apples and oranges. Different types of idea (in particular ideas that serve different goals, see Rule #1) must be treated differently. They are subject to different evaluation criteria and serve different goals. Ideally, each class of idea would have its own evaluation process.
- Never let anyone single-handedly kill or promote an idea who has a vested interest in it. Many participants in idea evaluation have a large degree of self-interest and may often try to kill or promote an idea based on how it affects them personally rather than on its benefit to the organisation.
- Know the psychology of idea evaluation. There are several effects which can affect the evaluation of ideas that have been studied by psychologists (one of them is described in Rule #9). Others are Evaluation Apprehension, Groupthink and Social Loafing. Design your evaluation workshop accordingly.
Thank you very much for this list! It is pretty much complete and also corresponds to my experiences.
I totally agree to rule #8 (Never compare apples and oranges) but want to add that from a portfolio point of view we constantly have to compare all kind of ideas and project options. This is because at the and of the day we have to fund those projects and all ideas compete for the same money.
Thanks for your comment!
I understand that an innovation portfolio may contain projects of different types, but I don’t understand the necessity of comparing them.