Spectroscopy Filters Eliminate ’Noise’ – and Researchers Now Have a Way to Determine How Well Those Filters Work

Spectroscopy is a powerful tool that allows us to identify, analyze and study objects and materials by analyzing color patterns, or light wavelengths. It’s used in everything from astronomical observation tools to the MRI machine in the hospital.

In a recent publication in the Journal of Applied Physics, NC State physicist David Aspnes and colleagues looked at different filters that are used to reduce noise in spectroscopy applications, and developed a method for quantifying the performance of linear noise-reduction filters in specific situations.

Aspnes sat down with the Abstract to talk about why, when it comes to noise reduction in spectroscopy, there’s no “one size fits all” solution.

The Abstract (TA): First, let’s talk about what “noise” means in the field of spectroscopy. Does it differ from application to application? In simple terms, what is noise in spectroscopy?

Aspnes: Noise is an undesired signal that originates either from the system itself or the surrounding environment. Noise can also be described as any degradation of information; in other words, it can take on a variety of forms. It’s probably best to first define what we mean by a spectrum. In the present context we can view this as a report of how the intensity of light varies as a function of wavelength or energy. We measure these intensities at equal intervals over some range from one end to the other, let’s say red to blue. Leaves appear green because they reflect more light in the mid (green) range than in the red or blue.

Now bands such as the green of a leaf tend to be rather broad, smoothly varying from one measurement to the next over a relatively wide range of points. Noise, on the other hand, can be described as essentially random fluctuations. One way of viewing this behavior is to think about waves. Information (the green) is slowly varying, so is described by long waves. Noise, on the other hand, is rapidly varying, and so is described by short waves. The wave picture separates noise and information in a way that is not possible by looking at a spectrum directly.

TA: Why is it so difficult to eliminate noise in these applications?

Aspnes: Actually, it’s not all that difficult: eliminate the short waves and you’ve eliminated the main contribution of noise. But that’s not quite fair because the real goal is to eliminate the noise AND keep the information. Here lies the challenge. For instance, you can eliminate noise completely by eliminating everything, but that defeats the purpose of spectroscopy. You can also preserve the information by keeping everything, but then you’ve kept the noise as well. So the real problem is finding the best compromise. We developed a cost function that allows you to quantify and therefore to optimize this tradeoff.

TA: You looked at four different filters – how did you determine a way to quantify their performance? Was that an equation? And can that equation be applied to different filters in different circumstances to figure out which one might be best to use?

Aspnes: We looked at four different filters belonging to the single category termed linear filtering. Linear filters work by attenuation, that is, by reducing or eliminating unwanted waves. That’s all they do. But some are better at this than others. Those that are too brutal at the long-wavelength end degrade the information. Those that pass too much at the short-wavelength end leak too much noise. The four filters that we looked at are a representative selection of a large number that have been proposed and are in use today.

To make the optimization process deterministic we derived an equation that quantifies degradation and leakage as separate terms. The equation shows that the two goals compete, so the perfect filter is impossible. The equation also contains information about the spectrum being processed, so it shows that a universal filter is impossible. But as a cost function it can be optimized for a given situation, leading to the best possible linear noise-reducing filter for this application.

TA: What are the next steps with this work?

Aspnes: There exists a second class of filters called nonlinear filters, which operate on different principles and which, owing to their nature, can outperform linear filters. We are finishing a second publication that analyzes this category and provides the necessary tools to optimize these filters as well.