The £12.7bn NHS computer programme is five years behind schedule and beset by criticism, viruses and fears over patient privacy. So should the world's biggest IT project be scrapped? Andy Beckett investigates

At some point last November, an infection began to spread unnoticed through the three hospitals that make up Barts and The London NHS Trust in east London. This was not MRSA but the Mytob worm, a common but potent computer virus. It steadily slowed and choked the 4,700 PCs of the trust's network. By noon on 17 November, a Monday, the network was effectively crippled.

The following day, the trust declared an "internal major incident". Ambulances carry-ing accident and emergency patients were diverted to other hospitals. Operations were postponed. The appointments system was suspended. Access to clinical information - usually quick and electronic - was maintained only by the slowest and most old-fashioned of methods: "runners" drafted in from the trust's administrative departments pounded the hospitals' endless twisting corridors with paper notes and printouts.

Scores of computer technicians from the private sector and from other London NHS trusts were brought in to eradicate the virus, but the PCs had to be decontaminated one by one. It was a week before the crisis was officially declared over, and a fortnight before the hospitals, some of the busiest in the capital, returned to normal. Afterwards, an official report found the virus had been able to infiltrate them because their anti-virus software "did not reach all [their] PCs and ... was configured incorrectly on some". The whole episode, the report concluded, had been "entirely avoidable".

The Barts incident, and a series of similar virus attacks on British hospitals since 2007, is the subject of an investigation by More4 News to be broadcast tonight. The More4 report reflects a longstanding concern about the vulnerability of the NHS to computer viruses and hackers. "The NHS have never really worked out the security architecture for their computer systems," says Martyn Thomas, a visiting professor at Oxford University Computing Laboratory. That could be said of a lot of large organisations; but for the NHS the robustness or otherwise of its IT infrastructure has a particular importance. A hint as to why comes at the end of the official report on the Barts episode: "As more and more patient data is only available on IT systems, the need for resilience within the network becomes more critical."

The NHS is in the middle of what is probably the biggest and most controversial civilian computer project in the world. The National Programme for IT has been running for seven years. It encompasses the conversion of billions of pages of patient records from paper to digital form; an electronic "choose and book" system to give patients power over their hospital appointments; an electronic prescription service linking GPs and chemists; a new NHS broadband network and internal email system; and near-instant digital access for clinicians to x-rays and scans. In short, the programme aims to transform the NHS in England (Scotland and Wales are developing their own health IT systems) into a modern computerised organisation - more efficient, more responsive, less prone to errors.

Without this hugely expensive transformation, in the view of the government, all its other increases in health spending and NHS reforms - one of the centrepieces of the last 12 years of Labour rule - will not work properly: "Effective IT underpins the future success of the NHS," Tony Blair was told at a Downing Street seminar on NHS computerisation in 2002. "The volumes of data involved," says Christine Connelly, who heads all the department of health's IT projects, "are like something from Nasa."

No other country has managed to connect up its healthcare system into a single digital whole. And nor, so far, have we: the National Programme for IT is five years behind schedule. This week, it was reported that the Conservatives, who have commissioned a review of the programme, could, if elected, abandon its scheme for computerising patient records, and instead let the data be stored on the internet by private companies such as Microsoft and Google. A Conservative spokeswoman refuses to confirm or dismiss the story. But the programme's difficulties are undeniable. "Overall, it's early 2015 that it will be finished," says Connelly, who previously ran and rescued a troubled giant computer project for Cadbury Schweppes, and has an air of weary determination. "It's certainly later than anyone would have hoped."

As the delays have built up, so has the impression of a government IT scheme, like many before it, gradually sinking into a swamp of technical difficulties, ethical disputes, incompetent contractors and Whitehall over-ambition and careless spending. "The estimated cost of the programme is £12.7bn," reported the Commons public accounts committee in January, "although this figure remains uncertain ... The programme is not providing value for money at present ... Little clinical functionality has been deployed, with the result that the expectations of clinical staff have not been met ... Patients and doctors have understandable concerns about data security ... the programme started with four main [private sector] suppliers but two have left."

The programme's many other critics have included the pressure group Privacy International, which in 2004 dubbed it Britain's "most appalling project" for allegedly eroding medical confidentiality; 23 of Britain's most prominent computer scientists, who in 2006 signed an open letter calling for the programme to be independently assessed for "its basic technical viability"; and the Conservative leader David Cameron, who in 2008 condemned the programme as "Labour's hubristic NHS supercomputer" and has regularly hinted since that it would be completely cancelled or radically rethought by a Tory government.

Since 2006 the programme's total cost has been predicted to rise to £20bn, even £50bn - half the current annual NHS budget. Connelly dismisses such suggestions: "These are not numbers I recognise. It's not over-budget. Up until the end of the financial year 2008-9, we have spent £4.5bn out of the £12.7bn." She also contests Cameron's characterisation of the programme: "When people talk about 'the NHS supercomputer', it's a throwback to the 70s. We don't have one single computer system.

We have a series of computer systems that talk to each other."

Yet large state spending projects with troubled reputations are currently vulnerable, whoever is in government. Connelly recently gave the IT programme's main private sector contractors until the end of November to make demonstrably better progress, or face the possible termination of their contracts.

At the Royal Free Hospital in north London, as elsewhere in the NHS, the new IT systems are only part of a broader, sometimes chaotic-looking refurbishment. The main entrance is being rebuilt. Ambulances squeeze between construction hoardings. Nearby offices have been stripped and stacked with plastic crates.

The crates are full of carefully packed old cardboard files - a year on from the arrival of the new computer networks at the Royal Free, one of the first hospitals to introduce them, paper is far from being abolished. "You could see this IT project as a journey," says Tim Peachey, the Royal Free's medical director, with the smooth bedside manner of a senior clinician. "We're on that journey, and we're pointed in the right direction ... But you could say it is never finished."

We go down to accident and emergency to see the new system in action. Amid the usual roomful of fearful and pained faces, there is a single keyboard and a flat screen, displaying a colour-coded list of patients. The computer graphics are utilitarian, bordering on crude, but the doctor on duty is happy with them.

He can track and classify patients. He can exchange medical data with other departments electronically - previously A&E had its own isolated IT system, and constantly had to send staff to other parts of the hospital "on errands". The doctor stands proudly beside the new computer. "It's had a few teething troubles," he says, "but now it's getting on very nicely."

In fact, last year the Royal Free had so many problems with the new system - network crashes, appointments lost or incorrectly scheduled - that it took more than six months to resolve them. The hospital lost £10m.

Other parts of the Royal Free's IT programme have been more immediately successful. In the x-ray department, a radiologist shows me a small room filled with more flat screens. Nowadays, she says, scans and x-rays can be digitally stored, examined in unprecedented detail, and almost instantly transmitted. "A lot of x-ray films got lost before," she says. "The filing system didn't work very well. We used to use bits of paper to measure things on x-rays. Get a bit of paper and copy marks on to it from a ruler. Now you can measure electronically onscreen down to fractions of a millimetre." Has the technology made her job less stressful? She shakes her head. "We just do more scanning than before. We need to catch up with other countries."

Some politicians and NHS managers have long hoped that computers will transform British healthcare. "IT programmes, initiatives and fashions ... have swept through the NHS since the 1970s," writes Sean Brennan in his 2005 book The NHS IT Project: The Biggest Computer Programme In The World ... Ever! "Huge projects such as the [Conservative-commissioned] resource management initiative of the early 1990s have been and gone and barely a ripple remains. Most of the computer systems purchased ... have long ago been quietly switched off."

These failed pilot schemes and technical dead-ends have been chronicled with outrage and relish by the press. But critics of NHS computerisation often fail to acknowledge its necessity. "In the last few years [paper] patient notes have got unmanageable," says a London doctor who has been working with NHS IT since the 70s. "In the past, if the patient had a thick file, you knew they had something wrong with them. Now they've got an ingrowing toenail."

Before Labour came to office in 1997, the NHS had a patchwork of computer systems, some ancient, some state-of-the-art, but almost all of them small and local and incompatible. Brennan notes there could be 40 in a single hospital. In 1998, at a gathering to mark the service's 50th anniversary, Blair spoke via an experimental medical video conferencing system to a clinician at the Royal Free. "The possibilities are enormous if we can get this right," Blair said afterwards. A few weeks later, the government announced that it would unify and greatly expand the use of computers in the NHS.

To the IT programme's critics, what resulted was too hastily conceived, too ambitious, too centralised, and lacked enough input from medical professionals. "Blair didn't understand IT," says Thomas. "The Department of Health didn't take proper advice. They over-claimed what the programme would achieve, to sell it to the public. In IT, every time someone boasts that they're trying to do something at the forefront of technology, 'the biggest in the world', you know it's going to fail." In 2007, ComputerWeekly.com, which has followed the NHS scheme in minute detail, characterised it as "an amorphous national programme without a simple, clear objective".

Connelly says some of the programme's flaws are unavoidable. "In a theoretical world, you write the perfect specification for an IT system before installing it. But the NHS doesn't stay the same. So you develop a system that's three-quarters perfect, put it in, move it on." When the private sector contractors involved in the project have not met this standard, she points out, they have not been paid until their work improves or, in the case of the Japanese conglomerate Fujitsu, have left the project altogether. Thomas concedes the contractors are partly to blame for the programme's troubles: "IT is a very immature industry. With a small project, with good people, you can get a good solution. If it's big, people typically lose control."

In a tiny hot office high up in the Royal Free, Mark Harber grapples daily with some of the NHS scheme's imperfections. He is a doctor in the kidney unit and confidently computer literate. But his department uses Cerner Millennium, an American software product heavily criticised by clinicians. "The system hasn't been designed for what it's like to use," he says. "It's not very Anglicised." Is Cerner prepared to make improvements? "They're responsive ..." Harber's voice trails off. "But it does take a long time." His narrow desk is still buried under drifts of paper.

In April, Cerner, which is based in Kansas, announced that, despite the recession, its first-quarter profits were up by 11%. "Cerner's chief financial officer," reported ComputerWeekly.com, "said the company was especially well-positioned to take advantage of ... the stimulus bill that President Obama signed into law on 17 February. The bill provides for money to be spent on modernising medical records."

Back in England, for all the frustration and politics surrounding it, the NHS IT scheme may well survive. Too much money has been spent, too many parts of it are in place, and the alternatives are not obvious. (The reported Conservative scheme to involve Google and Microsoft has already been criticised for its potential profit element, and for failing to cater for Britons without internet access.) Yet significant adjustments to the current programme are already underway. It is being made less centralised, and more sensitive to the needs of individual hospitals - more like what its critics say should have been installed in the first place. But Thomas remains sceptical: "What they're trying to do is make sure that it cannot be seen to have failed. There will be an orderly retreat, and they will declare it a victory."

Up in Harber's hot office at the Royal Free, even 2015 seems like an optimistic completion date. Peachey says it will take "five years" to achieve the free flow of medical data between the capital's health facilities, let alone across England. Harber stares at the clunky Cerner Millennium page on his PC screen and says with quiet exasperation: "Not all the things we were sold came as expected."

"Will it work?", Brennan asks in his book on the programme. "The answer to this question is probably 'eventually'. . . one day it will all come together." But whether the biggest civilian computer project in the world will endure is another matter. In the modern health service, Brennan points out, IT systems only survive for as long as NHS trusts are prepared to keep on paying to use and maintain them.

When and if the new IT systems become unloved relics, the Royal Free and other hospitals have plenty of dusty corners. Although for now they're still full of paper patient records. No one's had the time to scan them in yet ·

• More4's investigation into NHS computer viruses can be seen on More4 News tonight at 8pm.