We examined every Indigenous death in custody since 2008. This is why

Statistics around Indigenous deaths in custody are surprisingly hard to find.

That ought not to be the case. A key recommendation of the royal commission into Aboriginal deaths in custody in 1991 was for a national program to provide accurate, up-to-date information so that the scale of the problem and trends showing systemic failure never faded from public view.

The Australian Institute of Criminology (AIC) has run the monitoring program since 1992 in response to that recommendation.

But when Guardian Australia went looking for these figures three years ago, we could not find them.

It began in November 2015 with the inquest into the death of Yamatji woman Ms Dhu. It was one of the few cases of a death in custody to command national media attention, due to the sustained campaign led by her family and supported by activist groups. Not all families are capable of mounting such a campaign, and they should not have to.

Before the inquest, we wanted to know how many Indigenous people had died in custody since the royal commission handed down its final report.

After scouring the latest available reports and making data requests of every state and territory, many of whom said they did not have the time or the resources to dig up the numbers, we arrived at the vague figure of “more than 340”.

We later learned that figure was short by at least 30 people, probably more.

To get a more up-to-date figure, we would have had to compile coronial findings from every Australian jurisdiction as well as tracking any media reports or press releases from custodial agencies. We would also have to monitor social media for reported deaths in custody, because most prison services in Australia will only confirm a death in custody when asked to verify a report; they do not issue a media statement unprompted.

So that is what we did.

From May 2018, the reporting team has been reading every coronial finding relating to an Indigenous death in custody from 2008 onwards in Australia, and following up reports of cases that are still awaiting inquest. For the years 2010-2015, we collected information on every single death in custody in order to have a large enough sample size to produce meaningful analysis.

We examined 463 cases, 146 of which concerned an Indigenous person. We have determined that since the royal commission, there have been more than 407 Indigenous deaths in custody.

Using issues that coroner Ros Fogliani found in the treatment and care of Dhu as our initial guide, we checked each case against a number of data points.

We added other issues, based on what we knew of the treatment of others, including David Dungay Jr, Mr Ward, and Kwementyaye Langdon.

We learned that Indigenous people are more likely to die in police pursuits than non-Indigenous people, and that many of those who die in such circumstances are teenagers who committed minor crimes. We learned that Aboriginal people are more likely to receive some medical care while in custody, but twice as likely not to receive all the medical care they require.

We read stories of people complaining of excruciating pain but receiving no painkillers, of gravely sick people complaining of the symptoms of septic shock and being treated for dehydration, of families having to watch their loved being shackled to a hospital bed despite, on many occasions, being in a coma or reliant on machines to breathe.

The result is a resource that we hope will be used by researchers, lawyers, community advocates, and most importantly, families, who have lost someone in the justice system and are trying to navigate what can be a very complex and dense coronial process.

Things have undoubtedly improved since the royal commission, both in terms of monitoring and in terms of the rates at which Indigenous people die in custody. But we found that 43 of the 147 deaths we investigated were of people who were born since the royal commission’s findings were released.

The AIC collates data from coroners courts in each state and territory as well as data collection forms filled out by police and prison agencies and produces regular, cross-checked reports.

Recently, those reports have become less frequent, but that is set to change. There was a three-year gap between the last two monitoring reports, and the most recent report, published in April 2018, covers years 2013-14 and 2014-15, leaving a significant gap in public knowledge.

In a statement to Guardian Australia, the AIC said it had “recently decided to release reports on an annual basis.”

“This will allow for more timely and frequent reporting,” it said. “The next report to be released will cover the 2015–16 financial year.”

The AIC said its reports took some time to produce because of difficulty in gathering all the data and conducting the requisite validation and cross-checking. They also rely on external population data, which is usually not available until several months after the end of the reporting period, which coincides with the financial year.

It said that real-time reporting was not on the agenda, because it “would not allow for the validation and cross-checking of data required to produce a robust and statistically accurate report.” That is entirely appropriate.

We are not criminologists. Our aim is not to provide more statistics, but to share the stories behind those statistics and draw attention to failures and oversights that continue to occur and contribute to deaths in custody, despite countless pages of reports and coronial recommendations.

Indigenous Australians are among the most incarcerated people on the planet. That is a national shame. And because Indigenous Australians are disproportionately incarcerated, they die in custody at rates far greater than should be expected for a people who make up just 3% of the population.

We encourage you to read this database and get to know these stories.

In Australia, the crisis support service Lifeline is on 13 11 14. In the US, the National Suicide Prevention Hotline is 1-800-273-8255. In the UK, the Samaritans can be contacted on 116 123. Hotlines in other countries can be found here