Recently Wales Audit Office’s IM&T Audit Manager, Paul Cunningham, spoke to the British Computer Society at Aberystwyth University, about a WAO review of NHS data quality in a lecture entitled ‘Health data, what could possibly go wrong?’
In a two part blog, Paul gave us a flavour of what the talk was about. You can also view the presentation for a short time via http://bit.ly/bcsmidwales-healthdata (English only)
Our story begins in 2012 when my team was asked to undertake a review of data quality in NHS Organisations in Wales. This included a large scale analysis of data from every Patient Admin and Radiology system in Wales. This was the first time any such analysis had ever been undertaken and the results were very interesting.
The aim of project was simple enough; we wanted to find out if there were effective arrangements to promote data quality in NHS Wales. As with any audit, we investigated the issues by visiting NHS organisations around Wales where data was being processed, interview key staff and delving deep into the data itself. We concluded that arrangements ensuring the quality of data used to support clinical decisions in Wales were variable, but improving. That’s a fair, accurate and even handed assessment, if a typical, auditing response, but it doesn’t portray the issues we found with the actual data and doesn’t really reflect the complexity of the problem that the NHS is facing.
Let’s try and explain it another way. We all know the game rock-paper-scissors. The winner of the game is chosen on the following rules: Paper beats rock, Rock beats paper and Scissors cut paper. Simples! – only 3 items and only 3 rules to worry about. Now fans of the show “The Big Bang Theory” will know of the Lizard and Spock additions to the rules (scissors decapitate lizard, lizard poisons Spock, paper disproves Spock etc – for those uninitiated) more to the point, there are now five items and ten rules. My son and his friends have extended this even further taking it up to nine items and thirty-six rules! The number of elements rises, but the number and complexity of the corresponding rules and relationships rises even faster.
This is not a game, but with that in mind, we come back to the NHS which has to keep control of over 100 clinical data systems (we looked at 2 in detail here remember) and 3.1 million potential patients, all presided over by 8 NHS organisations providing and outsourcing care.
As you will have realised, our review barely scratched the surface, but we found that this theoretical complexity takes a very striking form in the real world: There are 3.1 million people living in Wales today, yet NHS data systems hold records for over 8.5 million people. That has to be a mistake of some sorts? The answer is simple: it’s not health tourism, nor is it migration, the cause is duplication.
Being John or Jane Malcovich…
We discovered that in many instances there was more than one record for one person (up to 52 records in one case). In some parts of Wales, this accounted for up to 40% of the records held. This has serious implications; the possibility of mixing up patients records and administering the wrong type of medication for example.
So how has this happened? Hospital processes differ and some are better than others in uniquely identifying patients. Newly installed and recently cleansed data banks tend to fare better than older ones. Without breaching data confidentiality, I can share with you that certain surnames are very common in Wales! Although this latter issue explains the set of 52 I told you about earlier, neither it nor the preceding ones account for the bulk of the duplicated records. Actually, we found that historic paper based patient notes are the major factor. Think about the real life process: for every data record that exists there is a corresponding piece of paperwork on file somewhere, created manually, over the years by clinical staff,; completed in their idiosyncratic handwriting, while sitting with real patients who might soon be moving on to another hospital where the process will be repeated. And, as we all know, real patients don’t stay at one hospital.
About the Author
Paul Cunningham is the Wales Audit Office’s IM&T Audit Manager and has many years’ experience working across all sectors of the WAO’s business, including NHS, local and national government clients. Prior to joining WAO, he lectured extensively in IT, computer science and business studies as well as providing consultancy for private industry. He is married, with three children – the youngest, Ashley, being the mathematical mind that created the complex game described above. He also has two granddaughters who live nearby in the Rhondda Valley.