It was 7.30pm on July 10, and IT security manager Han Hann Kwang was about to leave his office when he got a call: A massive cyber attack had taken place, and his help was needed to neutralise the threat.
Mr Han, who is from the Integrated Health Information Systems (IHiS) - the technology outsourcing arm of public hospitals here - jumped into the fray with investigators from the Cyber Security Agency (CSA) who were activated on the same day.
Yesterday, both agencies gave reporters a look at what went on behind the scenes after the authorities found out that 1.5 million SingHealth patients had had their personal information stolen in a security breach.
At the CSA's 370 sq m Cyber Forensics Laboratory in the Ministry of National Development building in Maxwell Road, up to 80 analysts could be on 40 dual-screen workstations to analyse the network log files captured from the infected systems during a major attack.
The IHiS team was doing the same in its control centre, set up in a satellite office in Redhill. In the past two weeks, said Mr Han, his team has combed through "tens of terabytes" of information.
In the first week after the cyber attack was confirmed and the CSA was alerted, people worked round the clock to contain and investigate the threat.
"When a major incident is reported, CSA will deploy its response team onsite to investigate and determine the nature of the intrusion," said Mr Dan Yock Hau, director of CSA's National Cyber Incident Response Centre.
"After investigation, CSA will determine the appropriate measures that need to be taken to enhance the protection of the affected systems."
Like any criminal investigation unit would do, it sent agents to the scene armed with toolkits, in boxes weighing 20kg. Without divulging details, CSA said each forensic outfield kit was used to gather evidence at SingHealth's premises on the day the attack was discovered.
The equipment allows investigators to clone images of the compromised hard disk and extract system log files. These were taken back to its laboratory for more in-depth analysis to nail down the modus operandi and source of the attacker.
For more complex cases, CSA investigators may even need to do an on-site "triage" or system analysis. Similar to the procedure in a hospital emergency room, cyber security triage assigns degrees of urgency to the infection to decide the order of treatment for infected systems.
The investigation process which started on July 10 could take up to the end of the year as more parties are now involved.
Singapore's data privacy watchdog, the Personal Data Protection Commission (PDPC), is looking into whether there were security lapses by the IHiS and SingHealth, and whether they are liable for a fine of up to $1 million under the Personal Data Protection Act.
A four-man Committee of Inquiry (COI) headed by former chief district judge and current Public Service Commission member Richard Magnus has also been convened to look at how the attack was mitigated to draw lessons on ways to better protect public-sector IT systems which contain large databases.
The committee will also submit a report of its proceedings, findings and recommendations to Minister for Communications and Information S. Iswaran, who is also Minister-in-charge of Cyber Security, by Dec 31.
Mr Han said IHiS will be putting in place new IT safeguards to protect the public healthcare system, details of which he declined to reveal for security reasons.
CSA senior consultant Mr Lin Weiqiang, who was in the thick of the action following the revelation that SingHealth had been attacked, said: "It was hard work. We had to piece all the clues together like crime scene investigators. We were under immense pressure; we had to succeed."
