Volunteers fill critical Covid-19 data gaps left by Indian government

KOCHI, INDIA - As the highly infectious Omicron variant of Covid-19 races through the world's second-most populous country, harried Indians have turned to volunteer-driven data portals for information to keep themselves safe in this third wave.

Two such popular websites - covid19bharat.org and incovid19.org - have been collating data since Nov 1 from multiple clunky government sources, churning out user-friendly daily and cumulative updates on several parameters such as confirmed cases and vaccine doses right down to the district level.

In a country where the government has not only suppressed certain inconvenient healthcare data, such as Covid-19 mortality figures that exceed official estimates, but also failed to present information it is willing to share in an accessible manner, these easily navigable websites have become an indispensable source of critical information, including for researchers whose models help shape containment strategies.

These websites have picked up the baton from an illustrious predecessor, covid19india.org, which tracked the pandemic's ebb and flow in India from January 2020. By the time it ceased operations on Oct 31 last year, the site run by 300-plus volunteers had raked up more than 4.4 billion visits. It stopped operations as the volunteers wanted to return to their regular lives, with the pandemic easing after the second wave.

Mourning its closure on the portal's blog, one frequent user said checking the site had "become a habit like sitting down with a glass of mellow single malt after sundown", adding that its information had guided his major decisions through the pandemic.

The federal Ministry of Health and Family Welfare puts together data from various states and union territories daily to give only a cursory national and state-level cumulative update on parameters such as active cases and deaths on its portal and in press releases.

India's 28 states and eight union territories also publish these details district-wise on various platforms, including on social media, but with no standardisation in presentation and often on links difficult to trace.

It is these disparate and scattered data heaps from more than 740 districts that the portals run by volunteers consolidate at one spot and present neatly, including in downloadable formats.

"These days, there are a lot of efforts towards building models for predicting how the future months will look like with the new variant, or maybe you want to know what the reproductivity rate is for a certain district… Those kinds of research efforts require data, especially of the kind we are putting out," said Dr Asha Subramanian, the founder and chief executive of Semantic Web India, one of the several stakeholders behind incovid19.org

Dr Bhramar Mukherjee, chair of biostatistics at the University of Michigan, who has been tracking the pandemic's evolution in India, told The Straits Times (ST) it would have been "nearly impossible" to do state-level modelling without support from covid19india.org and covid19bharat.org

Professor Gautam Menon from Ashoka University, who has worked on Covid-19 modelling, said he and his collaborators used covid19india.org extensively and now rely on both its successors for information used in their predictive models.

"That the most trusted source of Covid-19 data comes from volunteer, non-governmental efforts, even as the government is the custodian of this data and indeed much more, should be a source of concern," Prof Menon told ST. "It does not speak well of the Indian government's commitment to open data and transparency."

He added that he would like to see more efforts placed on integrating the various databases under the government's control, so that they can become a source of actionable information in the current stage of the pandemic. India, for instance, has yet to create a systemic mechanism to disseminate data on critical issues such as vaccine efficacy and breakthrough infection rates.

While the two sites have benefited from covid19india.org's open source code, churning out these daily updates is no mean feat for volunteers who lend a hand every night after their regular day jobs.

A statement sent to ST by incovid19.org said the biggest challenge it faces is when states either do not publish data on time or provide district-level granularity. "Lack of regular and timely publication of data of some states remains a challenge and hence requires more time investment from us," it said.

And while some sources provide data in friendly machine-readable formats, freeing volunteers from manually keying them in, formats can change randomly, upsetting the workflow.

"The key challenge is making the data, which exists in multiple formats, languages and schedules, machine-readable, and keeping on top of sudden changes in formats," said a response from covid19bharat.org

Ms Rukmini S, a data journalist, said it is important to acknowledge how the government can restrict the scope of journalism without these independent efforts. Not having district-level data readily available, for instance, can offer a cursory, even misleading, view of the pandemic, with reporting focused only on state-level data or restricted to major cities.

Using data from covid19bharat.org, she tweeted on Jan 5 to say that while the rise in Covid-19 cases in Mumbai the day before (over 10,000 new cases) was worrying, nine other cities or districts had reported more than 1,000 new cases that day, indicating how deep the third wave's tentacles had spread into the country.

"If this data was not available, I would love to say that I would go to every state government's website and look at it every day, but I probably wouldn't. I would stop just at what the Ministry of Health and Family Welfare is producing every day and tell limited stories and miss a lot," she told ST.

The concern centres not just around the government's unwillingness to share Covid-19 data more widely and in a user-friendly manner, but also, in some cases, holding back data that produces an unflattering narrative, such as figures that show fatalities far exceeding official numbers.

In her recent book Whole Numbers And Half Truths, Ms Rukmini S. details a cat-and-mouse game around healthcare data between her and the government. On multiple occasions, when she used figures from the Health Management Information System to report on the severe curtailment of regular non-Covid-19 health services during the pandemic as well as excess Covid-19 mortality, the data went offline with the flimsy excuse of "server problems". This data was last published for May 2021, which coincided with the second wave.

There were multiple calls for the government to take over covid19india.org's effort when it decided to shut down.

When the government did not do so, a group of supporters and frequent users of covid19india.org decided to form covid19bharat.org to continue to fill the "vital gap" in India's Covid-19 data situation.

The new portals hope to keep going for as long as they can.

"The impact of the pandemic on the country and the metrics associated with it will form the foundation of many research initiatives for many years," said the team behind incovid19.org that got together to continue maintaining organised data for public use and research.

"We are aware that the pandemic may end, but the need to learn from it will continue for long and data will remain the backbone for such efforts."

Ms Rukmini S. remains sceptical of any change of heart in the government's data policy though.

"Covid-19 has become one of those intractable tugs of war where it has become so essential to this government to prove that they did a good job (of managing the pandemic) that any data it feels can counter this is just not going to be accepted."