Software identifies animal species by the sounds they produce

Computer program can be used by biologists, environmental consultants and animal lovers.

By Elton Alisson

Agência FAPESP – Biologists engaged in field research with the aim of identifying animal species will soon have a new computer-based tool at their disposal.

Researchers at the University of Campinas’s Jacques Vielliard Neotropical Sound Archive (FNJV) in São Paulo, Brazil, in collaboration with colleagues at the same university’s Brazilian Amphibian Natural History Laboratory (LaHNAB) and Information Systems Laboratory (LIS), are developing a computer program to identify animal species by their vocalizations, such as birdsong, insect chirping and frog croaking.

The software is called WASIS, short for Wildlife Animal Sound Identification System. The beta version can be downloaded free of charge. The program resulted from the research project “NavScales: navigating through scales in space, time and domains of knowledge”, conducted with support from FAPESP under a cooperative agreement with Microsoft Research and coordinated by Claudia Maria Bauzer Medeiros, a professor at the university’s Computer Institute.

“The idea is that the software should be used both by researchers, to identify species in the field, and by environmental consultants, who need to collect data on the animal species present in a given area, or even by the general public, as plenty of people appreciate birdsong and other animal sounds,” said Luís Felipe Toledo, a professor at the university’s Biology Institute and one of the participants in the project.

According to Toledo, who is the Sound Archive’s curator, each animal species has a specific vocalization and always emits sound with a fixed pattern of frequencies (from high to low) and power (volume), while duration varies.

“This is the result of natural and sexual selection,” he said. “There are animals with vocalizations in the range of 2 kilohertz (kHz), while others emit sounds always in the 4 kHz range.”

Based on these two parameters, frequency and power, the software is capable of distinguishing the vocalizations of different species, such as birds, frogs and insects, among many others.

Comparison of recordings

To recognize a species, the software loads an audio file containing the vocalization of an animal recorded in the field by a researcher, detects the frequency and power ranges involved, and compares them with recordings of animal vocalizations stored in a database being built using the Sound Archive.

The system performs correlation analysis in a search of similarities between the audio file in question and the recordings in the database.

“The software indicates the probability of similarity between a recording of an animal’s vocalization and a recording already cataloged in the database,” Toledo said.

Users can set filters to help identify the species recorded. For example, if a recording appears to feature the call of a bird, the user can tell the software to compare the file only with recordings of bird calls in the database.

If it is impossible to tell which species emitted the sound, the user can tell the software to compare the file with all recordings in the database.

“Another setting filters recordings by location because animals in certain locations are known to vocalize differently from animals located elsewhere. So this setting can help refine search results,” said Leandro Tacioli, a Sound Archive technician who is also participating in the software development project and training with the support of a grant from FAPESP.

The beta version of the program is available only for the Windows operating system, but by mid-year the researchers plan to launch a final version 1.0 that will also run under Linux and MacOS. The database for the beta version currently contains some 200 vocalizations from the FNJV archive.

The researchers plan to add new recordings from the archive from time to time. “We’re in the process of feeding the database with new recordings, a task that requires a great deal of time,” Tacioli said.

“We have to listen to each complete recording and check that the quality is good before including it in the database.”

Assisting researchers

The software can also help researchers and technicians at the Sound Archive to identify animal species by their vocalizations.

As the largest sound archive in South America and one of the seven largest in the world, the FNJV archive contains more than 30,000 files of animal vocalizations that have been digitized and suitably protected.

In addition to these 30,000-plus recordings, most of which were produced from the 1960s on by French ornithologist Jacques Vielliard, the FNJV archive also holds 10,000 other audio files that have yet to be digitized and incorporated into the database. Vielliard, who pioneered bioacoustics in Brazil and died in 2010, was a professor at the University of Campinas (UNICAMP).

“Many of our recordings are hard to identify because we lack data such as the species and the date, time and place of collection,” Tacioli said. “The software can help us with the task of identifying the species in the recordings we have, as well as assisting other researchers with the identification of species they record in their own field work.”

Many researchers leave autonomous recorders running in forests or nature reserves for several days as part of their field work, he explained.

Listening to all these recordings one by one to try to identify the animals’ vocalizations requires a significant investment of time. “The software can load audio files and scan them to identify vocalizations by frequency,” he said.

The program will be used in a project the researchers at the FNJV archive are starting with the aim of compiling amphibian calls from around the world to try to establish the phylogeny (evolutionary relations) of this class of vertebrates.

“This is a job that can be done only with rich, highly diversified collections,” Toledo said.

Call for deposit scheme

At the end of January, the journal Science published a letter written by Toledo with Cheryl Tipp, Curator of Wildlife & Environmental Sounds at the British Library in London, and Rafael Márquez, a researcher at Spain’s Natural Science Museum in Madrid, calling for audiovisual recordings of animals to be mandatorily deposited with scientific collections to preserve them and assure access to them by researchers.

When researchers describe new species, they are obliged to collect specimens in the wild and deposit them with zoology museums, the letter argues.

When papers are published on DNA sequences of animals or plants, the data must be made available online in open-access databases such as GenBank, a collection of all publicly available nucleotide sequences maintained by the US National Institutes of Health (NIH).

However, this procedure is not mandatory for audiovisual recordings of animals, including recordings of vocalizations, films and photographs, which are as important as other data.

“What we suggest is that the editors of scientific journals require researchers to deposit audiovisual recordings with sound archives, museums, or any online open-access database, so that everyone has access to them,” Toledo said.

In the past, it was harder to store and share audiovisual recordings because the digital files were large and occupied many megabytes in databases.

Today, though, they can easily be compressed and sent to a sound archive such as FNJV or a museum to be cataloged and incorporated into the collections concerned. “When an audiovisual recording is deposited with a collection, it’s not only preserved but also made available for use by the scientific community,” Toledo said. “These recordings generate important data and are vital to both conservation and the advance of knowledge about biodiversity.”

The beta version of the WASIS program can be downloaded from LaHNAB’s website at www.naturalhistory.com.br/wasis.html.

The letter “The value of audiovisual archives” (doi: 10.1126/science.347.6221.484-b), by Toledo et al., can be read by subscribers to Science at www.sciencemag.org/content/347/6221/484.2.