Research Finds Serious Problems With Forensic Software
For Immediate Release
New research from North Carolina State University and the University of South Florida finds significant flaws in recently released forensic software designed to assess the age of individuals based on their skeletal remains. The researchers report that, on average, the software’s age estimates are off by more than 14 years.
“Estimating someone’s age at death, based on skeletal remains, helps to build a biological profile of the deceased,” says Ann Ross, a professor of biological sciences at NC State and co-author of a paper on the work. “That’s important information for identifying unidentified remains, and can also be important in law enforcement contexts.”
At issue is a publicly available computer program called DXAGE, which was released in 2018. The program estimates age-at-death based on bone mineral density.
Ross and collaborator Jonathan Bethard, an assistant professor of anthropology at USF, noticed that DXAGE’s estimates for adult female remains were based on a fairly small sample size – the remains of only 100 women.
To test DXAGE’s accuracy, the researchers used bone mineral density data from 470 women who took part in the National Health and Nutrition Examination Survey conducted by the CDC between 2007 and 2008. Specifically, the researchers plugged the bone mineral density data for each of those women into DXAGE and compared the resulting estimated age with each woman’s actual age.
The researchers found that, on average, the DXAGE estimates were off by 14.25 years – though there was significant disparity between age groups. For example, DXAGE could estimate the age-at-death of women who died in their 30s to within an average of 0.65 years. But for women who died in their 70s, DXAGE was – on average – wrong by 24.4 years. In other words, a woman who died in her early 70s could be estimated by DXAGE as having died in her 40s.
“We think this inaccuracy is primarily due to the small sample size used in developing DXAGE,” Bethard says. “It may also be due, in part, to DXAGE relying on remains from cemeteries, where they may have been buried for decades. Burial means that some of the minerals in the bones may have leached into the environment.”
The researchers stress that basing age on bone mineral density is valuable, because it is a quantitative way of assessing age at death.
“But it’s important to have a robust sample size for all age groups, and to use samples that have not been affected by environmental factors after death,” Ross says.
Another problem with DXAGE, Ross notes, is that the software makes use of a “black box” neural networking program.
“In other words, it’s not clear how the software works,” Ross says. “That poses a problem if forensic experts are asked to testify on how they arrived at their age-of-death estimates in a court of law.
“We already have a technique for estimating age-of-death based on bone mineral density, which relies on linear regression; I helped develop it,” Ross says. “That technique proved to be more accurate than DXAGE in estimating age for the 470 women we evaluated in this study. And it allows forensic experts to explain their estimates when called on to testify.”
The researchers note in their paper that these results highlight the importance of performing validation studies on web?based applications such as DXAGE.
“A lot of new software tools are becoming available to the forensic community,” Ross says. “We need to ensure the validity of these tools before putting them into practice.”
The paper, “Bone Mineral Density Adult Age Estimation in Forensic Anthropology: A Test of the DXAGE Application,” is published in the Journal of Forensic Sciences. Bethard is corresponding author. The paper was co-authored by Jacqueline Berger and Justin Maiers of USF.
The research was done with support from the National Institute of Justice, under grant number 2018-DU-BX-0197.
-shipman-
Note to Editors: The study abstract follows.
“Bone Mineral Density Adult Age Estimation in Forensic Anthropology: A Test of the DXAGE Application”
Authors: Jonathan D. Bethard, Jacqueline M. Berger and Justin Maiers, University of South Florida; and Ann H. Ross, North Carolina State University
Published: Dec. 14, 2018, Journal of Forensic Sciences
DOI: 10.1111/1556-4029.13987
Abstract: Estimating age-at-death of individuals represented only by skeletonized human remains is a fundamental aspect of forensic anthropological casework. Recently, several researchers have proposed that bone mineral density (BMD) is a useful predictor of age-at-death in forensic contexts. Navega et al. (JFS 63(2):497–503) developed an online application called DXAGE for calculating age-at-death from BMD parameters. This study tests the utility of DXAGE by utilizing data from the National Health and Nutrition Examination Survey (NHANES). BMD data from a female subsample (n = 470) of the NHANES 2007–2008 dataset were analyzed, and the relationship between predicted age and real age was examined. Inaccuracy was 14.25 years, and bias was -7.20 years. Results show that there is a weak correlation between predicted and actual age (r = 0.47) using the DXAGE application. While BMD data are potentially useful for predicting age age-at-death, the DXAGE application should be used cautiously in forensic anthropological contexts.
- Categories:
