Latest IT law, outsourcing and e-commerce legal updates
Success by Planning for IT Project Failure (Part 2)
Posted 22nd February 2012 by Jagvinder Kang, Director
In the previous edition of the Technology Column, we started to consider how to deal with some of the ‘what if’ scenarios in IT projects, that need to be planned for. We covered ‘Scope & Scope Creep’, as well as the ‘Failure of a Customer’s Obligations’. In this edition, we continue the theme, by considering certain other prudent steps one needs to take in the context of IT projects, with our focus being on the topic of ‘Acceptance Testing’.
Legal and Business Considerations
As should be apparent from the previous Technology Column, a useful strategy for attempting to mitigate against IT project failure, is to put in place a structure which allows problems to be identified early on, so that the project can be ‘steered back on track,’ with appropriate resources and involvement of both the customer and the supplier.
‘Acceptance testing’ is another important mechanism for controlling risk, yet it is surprising, how in the rush to get the contract signed and the project moving, that the acceptance testing arrangements are also often inadequately drafted by the respective parties’ technical teams.
Clarity is once again a key requirement, to ensure that successful acceptance testing acts as a proper gateway to moving the project forward. At the very least, key requirements for success need to be defined, together with the process and testing strategy. This is to avoid everything being left as an ‘agreement to agree’ (with the associated legal risks to the actual validity of the contract itself) for a stage which might occur quite late in the project, after considerable time and cost has been expended, and possibly after strains in the parties’ relationship have started to materialise - not a scenario which would be conducive to mutual agreement.
It is important though, that the ‘acceptance testing’ provisions in the contract are not made unnecessarily onerous just for the sake of appeasing the lawyers rather than the clients! So what do we mean by this? Well, acceptance testing can conceptually have different implications for the parties, and it is important that the parties at a technical and commercial level, come to an understanding of what this phase will entail.
True ‘acceptance testing’ is intended to be detailed testing of material features of the system to ensure that the system has achieved a level of stability, functionality and performance, that provides sufficient comfort for a customer to be able to choose whether to move to the next stage of development or ‘go live’, depending upon the complexity of the particular IT project in question.
‘Acceptance testing’ can also be a less involved ‘gut check’, to see if everything is going to plan, and whether the parties should progress to the next mini-stage of the project. The former is usually what is intended by contractual documentation rather than the latter (as the latter is undertaken outside the contract as part of the usual internal development of a system by the supplier).
The difficulty arises when the parties are at cross-purposes as to what the contractual acceptance testing is intended to achieve. If it is the more detailed acceptance testing, then it is the customer that is making the decision whether the system is suitable to move to the next stage or ‘go live’ based on agreed criteria. The supplier is not however agreeing at that stage that the system will definitely meet all of the specification, functionality and performance criteria. The supplier is also not warranting that as a result of successful acceptance testing, that the system will perform without any errors or other issues. In order for the supplier to be providing any level of comfort close to that, the acceptance testing phase would need to be much more comprehensive, as well as much more time and cost intensive – usually something which is unrealistic within the agreed timescales and cost parameters of the respective IT project.
With that backdrop in mind, it is inappropriate for a customer’s lawyers to be seeking heavy damages, termination rights and free fixes if the system has any issues post go-live on the back of there being successful acceptance testing – how can it be appropriate, when we have just identified above that acceptance testing is unlikely to ever be fully comprehensive?
As a dual qualified IT lawyer and software engineer, it always bring a smile to my face (and some sympathy for a customer when I am negotiating for a supplier) when a customer’s lawyer starts to draw an analogy between software being akin to suffering from the same degradation issues as car engines – it seems to be news to them when I explain that mathematical formulae do not degrade over time in the same way as a car engine! – but it is this genuine lack of understanding of the nature of software, which results in some lawyers spurring their clients on to insist on absurd and onerous remedy rights for a customer from go-live, based on successful acceptance testing.
So what can happen with software post go-live? Well there can be issues with aspects of the system which were not tested, or there might be performance issues arising due to future capacity issues not being properly agreed in the specification. However, this is not ‘car engine wear and tear or degradation’.
Sometimes there is recognition of the fact that the acceptance testing will never be comprehensive - but the customer’s lawyers may then attempt to render the acceptance testing phase as having no contrac-tual significance whatsoever, so that strong remedies apply from go-live irrespective of whatever happens during acceptance testing. This again is inappropriate.
the difficulty with imposing onerours remedies or fixes post go-live’, is that once a system is in a 'go-live' state, undertaking live fixes where live data is involved, is clearly higher risk than such changes being applied in a non-live environment. This therefore needs to be acknowledged by both parties.
The usual manner in which to mitigate this risk, is therefore, for parties to put the strong remedies in at the acceptance testing phase (which puts the onus on the customer to undertake proper acceptance testing, and on the supplier to ensure that it has properly pre-tested the system), and then rely upon a separate support contract to deal with issues post go-live, rather than linking the remedies back to the original system development and implementation contract. The support contract usually has lower liability limits than the original contract, because there are lower annual service charges associated with it (and any material issues should have been identified pre-go-live during acceptance testing). For a customer, it is important (as we mentioned at the outset above) that proper time and resource is expended on the acceptance process to mitigate the risks from a commercial and technical perspective (namely checking that the acceptance process is as comprehensive as necessary to check the customer’s key requirements), rather than cutting corners and trying to rely upon the contract to ‘fill in the gaps’.
Post go-live, certain types of fixes will be applied as ‘live fixes’, whilst other types will require the system being brought off-line. Again, the pricing arrangements and risk profile arrangements agreed between the parties will determine what will happen in such situations – for example, will the customer have business continuity measures in place to deal with such arrangements, or is that a service that the customer has openly agreed and paid for, such that the supplier should be providing such arrangements?
Of course, there is nothing to say that you cannot agree onerous provisions post go-live and link it back to the original contract, as parties can agree whatever they wish (within certain parameters of course!). In fact, there can be certain specific circumstances, where such remedies can be justified. However, where the decision to do so is simply an arbitrary decision, then it just means that if you are imposing a ‘hard contract’ from a customer perspective, then you are unlikely to generate much in the way of goodwill from the supplier if you require their assistance. Likewise, if the ‘toughness’ is originating from the supplier, then it is hardly conducive to building a longer term arrangement for future work and revenue.
The above should illustrate, that once again, the ‘planning’ aspect of the project and the contract can not be over-stated. Sufficient time needs to be built into the project to ensure that important aspects are not ‘glossed over’. It will cost more, and take more time, to deal with it through litigation subsequently, compared to having a co-operative approach upfront. It is therefore important, to take time to ensure that there is a genuine understanding by the parties as to the gateway processes created by acceptance testing, the associated implications and risks, and the equitable sharing of such risks and costs.
We will continue this theme in the next edition.