Prototypes and pilots can certainly stretch for months and sometimes even years. These almost always require payment as well due to the time commitment.

A PoC of 1-2 days, on the other hand, seems more of a litmus test than a real chance to learn whether a solution is usable.Given the size and [potential] scope of a CMS solution, this may only be enough time to install the base CMS software!

Also, consider whether you will learn anything in 2 days usage. Perhaps you will, but you must compare this to other software you use daily. After 2 days of using MS Word, could you use all of its features? Odds are high that even after many years of using MS Word, your knowledge is STILL not complete! Such is the nature of a CMS solution. You will have a few features you use a lot, and many more features that will require time to use and understand.

If you want to keep your cycle short, you MUST have your team ready to experiment with the solution–and keep them focused–once it is set up. This means that considerations must be made for their normal workload! This marks the single largest obstacle most organizations face in a PoC, prototype, or pilot — personnel simply don’t have time to try out a solution and give a full assessment of whether it fits their needs (let alone their desires)!

Plan at least a week or two to try out your new software. 2 Days is completely unrealistic.

1. During the selection process
You should definitely have all vendors you’ve short-listed come in and show off their products, addressing not only the issues you’ve raised but also showing off their unique selling points, suggesting things that perhaps you haven’t thought of. I t think clients should positively encourage vendors to sell to them.

2. Once you have a preferred solution
This is the time to do a chargeable Proof of Concept. Get the vendor to come in and install their system and run it with a project group, ideally with stakeholders from across your organisation. The PoC must prove something that you think should be easy to do, perhaps that you do well already, and something you suspect will be difficult, such as single sign-on via your user directory. That way you will have proved that the riskier user-focussed and technical aspects of your implementation will be smoother to roll out.

3. The proof is in someone else’s implementation too
Finally, it’s a good idea to see someone else’s implementation before you sign the contract. Go and visit the vendor reference and ask them what they found tricky in their implementation. Put some caveats into your contract schedules around those issue.

We have just had an interesting discussion within my current organisation on what are the definitions of Proof of Concept, versus Prototype, versus ‘Pilot’ – our use of these labels may change depending on our organization and our context.

However, I still say my example quoted above is truly a Proof of Concept, because after the work was undertaken, we went back to the CIO and the steering committee, who looked at the results and said “In our opinion this organization is not actually mature enough to make the most of the portal software” – so in fact our work could be said to have dis-proved the concept, and stopped us going further and wasting more money on it.

Having said that, I agree with Peter’s definition of ‘scoping exercise’ above, its what you do once you have chosen your supplier and technology platform, so in my previous example we would have perhaps kept the PoC setup and worked with it and the vendor to fully scope the complete implementation (i.e. is connecting the ECMS to the portal in or out of scope ?)

I guess the main thing is: caveat emptor, or buyer beware! Ensure you go through some process, however it is labeled to ensure the resulting deployed system actually meets your requirements.