A LARGE part of the $20 million that the Singapore Exchange (SGX) will spend on infrastructure upgrades will go towards setting up a command centre so it can react better when future glitches strike.
The centre, to be completed by the end of the year, will allow the bourse to better monitor its data centre facilities, servers, networks and applications.
The move comes in response to criticism of SGX's crisis management from a Board Committee of Inquiry that it set up after a breakdown on Nov 5 last year.
INCIDENT RECOVERY ON NOV 5, 2014
- 2.18pm: Incident at the Primary Data Centre (PDC); no access to SGX IT systems
- 2.32pm: The third-party data centre provider informs SGX that only one of the two power sources is down
- 2.35pm: Crisis Management Team (CMT) is convened
- 2.51pm: CMT decides to formally call for a market halt. MAS is informed
- 2.58pm: Recovery initiated at PDC
- 3.05pm: SGX posts a website message formally announcing the market halt
- 3.10pm: The third-party data centre provider informs SGX of the total power outage
- 3.22pm: An SMS is sent to all SGX members about the market halt
- 3.30pm: Complete power outage information is surfaced to the CMT
- 3.48pm: SGXNet makes an announcement about the halt
- 4.19pm: A Twitter message on the market halt is sent
- 4.30pm: Securities market system recovers
- 5pm: Trading resumes on the securities market
- 5.38pm: Derivatives markets system recovers
- 5.51pm: Securities markets are closed
- 7pm: Trading resumes on derivatives market
Its report out yesterday noted that the market closure and subsequent trading resumption could have been managed better, and the recovery time shorter.
SGX's monitoring systems at the primary data centre during the incident did not provide enough information, which hindered decision-making, it said.
A voltage sag in both power lines to the centre saw a component in the emergency backup power supply generator malfunction and downstream switches could not cope. There was a momentary loss of power which resulted in a shutdown of all systems at the centre.
Initially SGX was told that only a partial power outage had occurred. This prompted it to activate a recovery at the centre instead of switching to a backup centre immediately.
It was later discovered there had been a total power outage and an immediate switch to the backup data centre would have been safe. But a delay in internal communication was a factor that led to the decision to continue with recovery at the primary centre.
SGX has since boosted the capabilities and capacity of its technology operations team, as well as improved crisis management procedures to shorten recovery time.
Industry players cheered SGX's move to ramp up investment in its technology infrastructure.
Mr David Gerald, president of the Securities Investors Association (Singapore), said SGX has done a lot to "spruce up" its IT systems in recent years, but there is still room for improvement in terms of its recovery processes.
"I'm glad that MAS (Monetary Authority of Singapore) has asked them to step up efforts in this area," he said, adding that much could be done to improve internal communications within SGX so that recovery could be faster.
He did not think public communication was "too bad", but agreed with the Board Committee of Inquiry findings that investors were not informed of the market halt "fast enough".
UOB Kay Hian senior executive director Esmond Choo noted the measures "reflect the seriousness of the breakdown (in terms of the blow) to the reputation of Singapore as a financial centre". He said while "SGX's infrastructure is very sound, errors at the margin can result in severe ramifications".
Remisier Desmond Leong said the commitment to improve the IT system showed SGX is proactive in preventing similar incidents from happening. He said: "As an exchange, the system has to work. It cannot just shut down. Or you will lose confidence from the investors, especially the foreign investors. This is a good step towards regaining investor confidence. In fact, the (IT) infrastructure is everything. It is the market. Reliability is very important."
The Board Committee of Inquiry comprises four directors who are all independent of SGX management. Mr Quah Wee Ghee chaired the committee. The rest are Mr Chew Choon Seng, Mr Kevin Kwok and Mr Lee Hsien Yang.