Proof of ConceptWhen selecting a new CMS you definitely want to see the system in action before you sign the contract. A “Proof of Concept” (PoC) is often used as an opportunity to get a detailed demo, but there are many different opinions about what a PoC actually is. Let me clear up some of the confusion.

In our report, Best Practices for Selecting a CMS, we define a PoC in this way and argue that allowing 1 or 2 days should be enough:

A proof of concept is the stage in the process at which you invite a selected vendor to demonstrate the validity of the concept they have outlined in their proposal. Up to this point in the process, most has been on paper. Now it is time to see things in action. This is also an ideal opportunity to meet the vendor team and verify whether the chemistry is right.

Here are some other definitions of a web content management PoC to show that the term can be used in different ways:

  • CMS Watch (from the Web CMS Report): “(…) ask your list of vendor finalists to come in to your shop, install their packages on your hardware, and develop some sample templates and workflows that make sense to your team. (…) Note that, whether the proofs can be done concurrently, a proof-of-concept step could add a month or more to your selection cycle.”
  • Jed Cawthorne (industry-expert and past J. Boye conference speaker in his feedback when ‘beta testing’ our report): “… neither a one or even a two day visit is what I would refer to as a “Proof of Concept”. (…) I would consider a PoC to be something like what we got involved in at [Organisation]. We got [Vendor] to come in and build a portal environment for us, we went on some training courses and we evaluated it for 3 months (we paid for all this of course!) after which the [Organisation] decision was to not go down the portal route – that to me is a true Proof of Concept.”
  • Seth Gottlieb (vendor-neutral analyst in the article POC, Prototype, or Pilot? When and Why): “A Proof of Concept is a small exercise to test a discrete design idea or assumption.”

The PoC is an important step in the selection phase. Nevertheless, it is our opinion that you shouldn’t spend too much time on it. You simply need to find out whether the system and the implementation team live up to your main requirements.

The process outlined by Jed resembles what we refer to as a scoping exercise – which is the next step. A scoping exercise usually takes 1-4 weeks and is carried out before the actual full-blown implementation with one vendor only. This vendor will have been carefully selected on the basis of their answers to the request for proposal and the (short) PoC. As a buyer you pay for the scoping exercise. One important advantage of going through this phase is that it is still possible to pull the plug if the vendor doesn’t meet your expectations. This reduces the risks involved significantly.

This is a reminder that as a buyer, you have to work with – and stick to – a strict and transparent terminology when engaging vendors. They might not define the concepts in the same way you do, so in order to avoid misunderstandings, you should provide a short, concise glossary of the terms you use.

Thanks to @adambindslev, @cmswatch, @jedpc, @markmorrell, @sigurdmagnusson & sggottlieb for valuable input.