Inside the worlds biggest agile software project disaster
With millions down the drain, is it really fair to blame the IT squad for the UK governments recent agile misadventures?
In theory, it was a good idea - using a smart new methodology to unravel a legacy of bureaucratic tangles. In reality, execution of the world’s largest agile software project has been less than impressive.
By developing its flagship Universal Credit (UC) digital project – an initiative designed to merge six separate benefits strands into one – using agile principles, the UK Department for Work and Pensions (DWP) hoped to decisively lay the ghosts of past DWP-backed digital projects to bed.
Unfortunately, a report by the National Audit Office (NAO) has demonstrated that the UK government’s IT gremlins remain in rude health, with £34 million of new IT assets to date written off by the DWP on this project alone. Moreover, the report states that the project has failed to deliver its rollout targets, and that the DWP is now unsure how much of its current IT will be viable for a national rollout – all pretty damning indictments for an initiative that was supposed to be demonstrating the merits of the Agile Framework for central UK government systems.
Perhaps one of the most biggest errors for implementing an agile approach highlighted by the NAO is the failure of the DWP to define how it would monitor progress or document decisions and the need to integrate the new systems with existing IT, procured and managed assuming the traditional ‘waterfall’ approach.
Agile,which relies on small teams working at speed to develop software on an iterative basis, is ultimately dependent on successful management to bring the honeycomb clusters of projects to fruition in a timely way and make sure that things are ticking over as they should in individual teams. Whilst in theory the agile framework should allow for rapid response to issues as they develop, without someone to oversee these changes or provide effective direction, progress cannot be made. The DWP had a large number of teams involved in the UC effort, each with different priorities and objectives. All it would take was one problematic team for synchronisation of the whole project to be compromised.
On top of that, this was the first time that agile had ever been applied to a mass immense infrastructure project, one that was, from the outset, held by large contracts to fixed terms with large suppliers – a contradiction to the entire agile concept.
Ultimately, in spite of the bluster centering on IT failings, the real thorn in the DWP’s UC digital initiative is ”weak management, ineffective control and poor governance”. The overseers of the project were never really open to adopting the cognitive shift from the traditional waterfall mentality that would have enabled the agile methodology to properly take effect.
This morning Secretary for Work and Pensions Iain Duncan Smith was doing his best to pin the blame on the IT teams involved in the project, painting himself as the one lumped with the job of cobbling the botched affair back into something salvageable. However, there were warning flags appearing way before any development had even taken place – even as the DWP did their best to paint a positive spin on the situation.
The DWP’s ‘fortress’ mentality and ‘good news reporting’ culture is also something that the NAO has heavily criticised, and a quick glance at the UC digital implementation strategy date provides plenty of examples. For instance; back in September 2012, a spokesman for the DWP told Computer Weekly that things were all on track, saying that because the IT system was being built part-by-part on an agile basis, as well as bringing in existing systems, “It is being built and tested on time and on budget.” This optimism-laden missive was announced just one month after the project was ‘re-set’, something that has only come to light this week.
The UC programme is now being piloted in Greater Manchester at a single job centre, where 1,000 claimants draw benefits. In theory, this should steer the scheme closer to the agile principles, allowing developers to work on a pared-down trial and error approach, and taking on the challenge of creating lots of complex systems a step at a time.
Whilst UK government isn’t the first hierarchical entity to underestimate how difficult a complete overhaul of developmental systems can be to implement, it certainly will be interesting to see if this very public chastening will give it the incentive it needs to do everything it can to effect the sea change necessary for a truly agile system to ever take hold.
Image by Guilhem Vellut