To find rare frogs and birds, Pitt researchers are using machine learning algorithms and hundreds of microphones

It used to be that if you wanted to track down a rare frog, you’d have to go to a likely place and wait until you heard its call. The rarer the frog, the less likely it was you’d hear one.  

Now, there are better tools for that.  

“The technologies that we work with are designed mostly to give you a better chance of detecting things that are hard to detect,” said Justin Kitzes, an assistant professor of biological sciences in the Kenneth P. Dietrich School of Arts and Sciences.  

Kitzes makes use of tools for bioacoustics — the study of sounds made by animals — which, along with satellite imaging and DNA methods, is part of a new generation of conservation technologies that allow researchers to search more broadly and efficiently than ever before.  

At the beginning of a project, researchers in his lab place up to hundreds of sensors that listen in on an area of interest. Then researchers bring those recordings into the lab, where they sort the signal from the noise. And there’s plenty of noise.  

Each recorder can track about 150 hours of sound, and when the team deploys 50 sensors, as they did recently when searching for frogs in Panama, those hours add up.  

“7,500 is pretty small for us, because 50 recorders is actually a small deployment,” Kitzes said. “In our bird work, it’s more like 75,000 hours.” 

There’s no use in collecting eight continuous years of audio if you don’t have time to listen to it, though. The lab’s research owes thanks to two technologies made available in 2017: an inexpensive audio recorder that allows the team to deploy hundreds of sensors and an open-source platform that gave scientists the ability to develop machine learning tools to sort through the data.  

“That was really what kicked everything off,” said Kitzes. “Because that gave us an explosion of field data along with the ability to train deep learning models to analyze it.” 

Tracking birds using this technology is one main focus for the team.  

Another is its amphibian research, a collaboration with the lab of Biological Sciences Professor Corinne Richards-Zawacki as part of the RIBBITR program. That work, including biological sciences graduate student Sam Lapp and staff researcher Alexandra Syunkova, has the team focusing on sites in Pennsylvania, California, Panama and Brazil. The team’s Pennsylvania field work is based out of Pitt’s Pymatuning Laboratory of Ecology in Crawford County.

In one recent instance, audio recordings helped the researchers track down an elusive variable harlequin toad (pictured above) in an unlikely site in Panama that was only just beginning to recover from an outbreak of the deadly chytrid fungus. And just this year, the team published a study led by Lapp where they listened in on the underwater behavior of the endangered Sierra Nevada yellow-legged frog.  

Studies like the latter rely on training what's called convolutional neural network models — related to the ones used by tech companies use to recognize features in photos — to categorize different types of sounds when presented with a visual representation of the audio recordings.  

“We’re using the same kinds of models as Google and Amazon, where in your vacation photo albums they might be able to recognize a palm tree by a beach,” Kitzes said.  

But as high-tech as the work is, there’s no replacement for the eye of a trained human. Members of the lab always check some of the algorithm’s work to ensure that it’s looking for the right calls. It’s similar, Kitzes explains, to how he sees other uses of machine learning and artificial intelligence: Not as a replacement for the work of humans, but as a way to augment it.  

"The reason our lab exists is that we’re trying to make conservation biologists and ecologists more effective at their job,” said Kitzes. “So they can get out there, find more species, learn better about what’s impacting those species and, ultimately, take the actions that are necessary to conserve those species and protect biodiversity.” 

Photography by Corinne Richards-Zawacki