Tiny Fossils, Big Tech

At NC State University, a team of researchers is using cutting-edge 3D modeling and artificial intelligence to solve a tricky problem: sorting microscopic, marine fossils no bigger than a grain of sand.

These fossils — called foraminifera, or “forams” for short — are tiny shelled organisms that have lived in Earth’s oceans for over 100 million years. When they die, their shells settle into the seafloor’s sediment, preserving chemical clues about patterns in ocean environments over time. For scientists studying past climates (a field known as paleoceanography), these fossils are gold.

The challenge lies in the sheer volume and minuscule size of the specimens. Evaluating forams requires sorting through hundreds of similarly shaped objects, a process that is both tedious and time-consuming. So three years ago, a team led by Edgar Lobaton created Forabot — an open-source robotic system for sorting and imaging forams.

“Identifying these fossils is very challenging, which is what led us to this work in the first place,” says Lobaton, co-author of a paper on the work and a professor of electrical and computer engineering at NC State.

However, Forabot remained a work-in-progress. To reliably sort objects smaller than the point of a pencil, the system’s hardware required constant, time-consuming fine-tuning. “We wanted to find a more efficient way to improve Forabot,” Lobaton explains.

Virtual Fossils As a Blueprint

The team’s answer was Foram3D, a novel technique that innovates paleoceanographic research by generating photorealistic, three-dimensional images of forams. 

To do this, the researchers modified an existing algorithm based on shell growth to produce detailed 3D facsimiles — precise virtual copies — of the fossils. This process involved incorporating the foraminifera’s underlying 3D geometry, such as internal shell structures, to create mathematically precise digital renderings of the fossils.

Lobaton’s team also worked with a paleontologist to ensure the facsimiles corresponded accurately to the characteristics of seven representative species of foraminifera.

The researchers then simulated the Forabot’s operational process. Using the newly captured 3D facsimiles, the researchers explored simulations of Forabot’s system and found ways to improve its accuracy from 82% to 89%. 

The simulations helped guide adjustments to the robot’s imaging system for improved accuracy, without requiring changes to the hardware. Once they found the optimal settings, the team was able to fine-tune the hardware more easily — which had been the most tedious step in the process of sorting and analyzing forams.

“These simulations helped us understand the best imaging conditions and are now guiding the development of a new robotic system focused on 3D reconstruction — an essential step toward further automating the identification of these microfossils,” says Sanjana Banerjee, a co-author and Ph.D. student in electrical engineering at NC State.

“Our work provides a strong foundation for studying the growth and morphology of a wide range of foraminifera species,” Banerjee says. “It also tackles major challenges in micropaleontology, such as limited data availability and accurate shape recovery.”

The Imaging Power of AI

The approach Lobaton and team used to develop the Foram3D technique holds promise for training and improving any robotic system that identifies and sorts objects with complex shapes. 

Using the 3D geometric foraminifera models, the research team tested how state-of-the-art AI models can reconstruct 3D shapes from only a sparse set of 2D images, mimicking more real-life scenarios such as imperfect lighting. The technology relies on using Neural Radiance Fields (NeRF), a deep learning technique. Unlike traditional 3D scanning, which requires a large number of high-quality images, NeRF technology uses a neural network to effectively “fill in the gaps” in a photo, such as a side or angle of an object hidden from view.

“The technique was created to improve robotic systems that sort and identify microscopic marine fossils used in climate research, but could serve as a blueprint for applications in a range of other fields,” says Lobaton.

“Potential use cases include microbe and pathogen isolation at the microscopic scale and sorting of agricultural produce at a larger scale.” 

Lobaton’s research team has made the code base used in this work open source so that other researchers can use it: https://github.com/ARoS-NCSU/Forams-3DGeneration.

The paper, “Foram3D: A Pipeline for 3D Synthetic Data Generation and Rendering of Foraminifera for Image Analysis and Reconstruction,” is published open access in the journal Marine Micropaleontology. The paper was co-authored by Turner Richmond, a former Ph.D. student at NC State; Michael Daniele, an associate professor of electrical and computer engineering at NC State; and Thomas Marchitto, a professor of geological sciences at the University of Colorado, Boulder.

This article is based on a news release from NC State University.