It was with sadness that I learned of the recent death of the original Batman, Adam West. West played Batman absolutely straight and, as a 7-year-old, I eagerly awaited each new exciting episode to see from which fiendish and protracted means of death he would escape using some wonderful piece of equipment from his amazing utility belt. It was almost as if Batman knew exactly what kind of weird attack he would face.
Of course, like all super heroes, he wore a mask was to hide his real identity, millionaire Bruce Wayne. In order to maintain his anonymity in a pre-mobile phone age there were only two ways to communicate with Batman: the batphone which connected Commissioner Gordon’s office to the Batcave, or – if Batman was out for the evening getting seduced by Catwoman – there was a massive search light atop Gotham City Hall which could project the image of a bat onto the clouds, alerting Batman to the latest emergency, the Bat Signal. Once alerted, Batman and Robin could thwart any attack however weird.
In my 20s the series was repeated and I enjoyed it all over again but on a completely different level as I got all the satire and double-entendre that West pulled off with a completely straight face. I also had a different appreciation of Catwoman.
Batman would doubtless have produced a software patch from his utility belt on Friday 12 May when WannaCry was unleashed by The Joker, sorry I mean The Shadow Broker, who had stolen it from the American Secret Service. Too far-fetched for an episode of Batman.
At 13.20 on May 12 my CIO colleague Darren McKenna and I were having our third cup of coffee in the office. We were relaxed and looking forward to the weekend. At 13.21 we received an email from a CIO in a neighbouring trust (thank you, you know who you are) to say they were under attack by ransomware.
I have never seen Darren ‘spring’ before but he definitely sprang, gathering the IT team and issuing instructions. I don’t think he or the team got much rest over the next few days but happily, thanks to the selfless actions of our neighbour and the hard work of Darren and his team (and some luck), we avoided infection with WannaCry and we kept our EPR, trust email and telephony up and running throughout the crisis.
A few phone calls to members of the Health CIO and CCIO Networks revealed that the attack was at least national and that some organisations were “pulling up the drawbridge”, disconnecting from local networks, shutting down email and other systems.
The power of networks
At 14.15 I posted on the CCIO/CIO online discussion forum, Discourse, and at 14.31 one of our members named WannaCry as the culprit. The “Ransomeware attack Right Now” thread grew over the next few hours and became a good source of knowledge with over 200 posts. The Network worked when email had gone down or been turned off in the crisis.
My NHS background is in patient safety and I’m in the habit of carrying out “After Action Reviews”, which is a process for rapidly sharing the learning from an incident among those who were involved. I have usually carried them out in the aftermath of a suicide or homicide. It is a process borrowed from the military and is used for the immediate review of critical incidents.
I spoke with Ade Byrne, CIO Network chair, and with Digital Health editor Jon Hoeksma, and we agreed to hold a Digital Health best practice webinar exactly a week after the attack. Volunteers from different parts of the country gave full and frank accounts of their experiences.
The After Action Review manual recommends that, however dark the subject, a little gallows humour can help conversation, so we entitled the session “Brown Friday”. Over 130 Network members attended the webinar and helped establish the timeline and facts of the attack.
An After Action Review is not a full inquiry and it is not about blame and produces only a brief report of lessons that need to learned quickly. Within two days a report was produced and shared on Discourse where members can read it.
The Bat Signal
There are several recommendations you can read at your leisure but there is one key area that we need to attend to urgently and that’s the Bat Signal. As a set of organisations we rely heavily on email and one the first things that happened when people got wind of the problem was to fear their email and so shut it down.
WhatsApp groups, Discourse, Twitter and text message conversations were where most information was gleaned at the height of the crisis. The webinar revealed that the disruption caused by “the Drawbridge Effect” was far greater than WannaCry. Only 4% of the webinar audience reported actually getting infected. A lack of information caused a massive drawbridge effect.
One of the underpinning principles of AAR is absolute honesty with ourselves so that things can go better next time. We weren’t ready, despite Davey Winder’s prescient warning a week earlier – we weren’t ready. We had no Bat Signal, we didn’t have the Batphone number, we didn’t know whether to call Commissioner Gordon or Chief O’Hara, we didn’t know what the criteria were for lighting the Bat Signal, we weren’t sure if Commissioner Gordon would call us or even whether he knew our number.
Worryingly, we still have no Bat Signal – an agreed go-to system or systems for getting a message to the people who need to know when the computers are broken by the Joker/Shadow Broker. We need to sort that very soon.
In the interim I suggest that if you haven’t already got your CIO or CCIO on Discourse, you get them to join right away. Those of you who, like me, love to sail will be familiar with UHF Channel 16. You never use Channel 16 on your radio but you monitor it the whole time – it’s the distress channel, so you only use it when in distress. You don’t want to hear anything on Channel 16 because that means someone’s in big trouble and you might have to get involved.
Marcus Baw, can you set us up a Channel 16 on Discourse which will alert all members next time, for there will surely be a next time. But next time we’ll be ready. Keep this frequency clear.
23 June 2017 @ 17:17
An entertaining article – very similar to a discussion I was at yesterday in Durham.
Having an alternative communication method is key and channel 16 is the way to go for next time.
What people could not agree on was how do you stop clinicians from switching over to these alternative methods of communication that are not secure enough for the NHS e.g. Whatsapp, imessage, dicourse, etc. pretty secure from a lot of aspects but not deemed secure enough?
20 June 2017 @ 06:08
Very entertaining article, however I would like to add in reality “Batman” and “Robin” could not stop arguing about who was the most important (that brings back memories of when I was last @ NHS Digital in the north of England), oh well I suppose the truth was they both “counted” and relied on each other, what do others … think ?
23 June 2017 @ 13:46
The analogy holds true. There are too many egos and empire builders!
Wheel reinvention and not “invented here syndrome” cost us dearly in time and money and have held back progress for decades. Whether across organisations, within a particular geography even in the same organisation.
19 June 2017 @ 16:40
Raised as I was in the same TV genre as yourself, I chuckled to read about the need for a Bat Signal.
However on following the Wannacry outbreak from the comfort of semi-retirement I was reminded of Spitting Images when the Mrs Thatcher said that Ministers would be informed of the crisis “through the usual channels … BBC1, BBC2 and, of course, ITV. ”
I like the Channel 16 analogy – there are too many virus warning channels globally at present, most of them owned by proprietary brands. Perhaps one of the larger monopolies like Google would be well placed to act as a focal point for this channel? The threat is global with a Trojan allegedly released from North Korea within hours adversely affecting organisations as far apart as GPS in Warrington and Police Statins in Utrah Pradesh.