Smartphone app uses computer vision to identify animals

CrowdPet compares photos of pets registered by owners with snaps of animals found in the street by volunteers.

By Suzel Tunes  |  FAPESP Research for Innovation – In July 2017, the Center for Animal Disease Control in the city of Vinhedo, São Paulo State, Brazil, began a census of animals reached by vaccination campaigns. The purpose is to maintain an up-to-date registry, above all for control of lost and abandoned animals in the streets. Identification and registration are being performed with the aid of CrowdPet, a smartphone app developed by SciPet, a startup incubated at the University of Campinas (UNICAMP).

Development of the app, which uses computer vision and artificial intelligence to identify animals, is supported by FAPESP’s Innovative Research in Small Business Program (PIPE).

CrowdPet is designed to combine two data sources: photographs of lost animals registered by their owners and photographs of animals sighted in the street by volunteers. “The app establishes a match between two images using computational visual recognition, and pinpoints by geolocation the place where the picture of the lost animal was taken,” says Fabio Rogério Piva, who is the CEO of SciPet and has a PhD in computer science from UNICAMP.

At the end of Stage 1 of the PIPE project, during which its feasibility was analyzed, SciPet completed a prototype capable of distinguishing cats and dogs from all other images with a 99% success rate. “Even if a user photographs people or objects, the system will register only photos of animals,” Piva explains.

The research project, conducted by students with grants under the supervision of Fernanda Andaló, SciPet’s CTO, was selected among finalists for the 2017 Inova UNICAMP Prize. The next step is even more complex, requiring accurate identification of each animal.

Redirecting the business plan 

The first version of SciPet’s business plan for CrowdPet chose people who wanted to find missing pets, connected in a kind of “social network”, as its sole target clientele. However, before completing Stage 1 of PIPE, SciPet was selected to take part in the Fourth Edition of FAPESP’s Training Program for High-Tech Entrepreneurs between July 31 and September 19, 2017.

During the training program, the SciPet team interviewed over 100 prospective customers. This listening and market prospection exercise showed that city governments needed to conduct a census before they could control the population of abandoned animals. “This is an unsolved problem. It’s hard to avoid double counting. Some cities use microchips, but the process involved is costly,” Piva says.

At that point, the firm made the strategic decision of offering the app to local government so that cities could provide a street animal monitoring service. “Besides Vinhedo, we’re also in touch with several other cities that are setting up animal welfare departments,” Piva says.

Cities are now considered the main partners for the project. “CrowdPet can be offered free of charge to the public through local government,” he explains. “We refined the concept and realized we wouldn’t be able to make a ‘pet solution’ work.”

Piva plans to add more functionality in future, including registration of volunteers and animal protection associations, lists of animals available for adoption, and rewards for people who find lost animals.

The challenge of animal recognition 

SciPet was founded in 2016. According to Piva, the firm’s first studies were based on a project called “Where is my puppy? Retrieving lost dogs by facial features”, conducted by researchers at UNICAMP’s Reasoning for Complex Data Laboratory (ReCoD Lab), with Professor Eduardo Valle as principal investigator. Valle is currently working with SciPet.

“This was one of the first attempts to apply biometry to animals,” Piva recalls. “It showed that human facial biometry methods aren’t very effective to identify dogs. Using a method specially developed for animals, they achieved 89% accuracy, equivalent to the success rate obtained by a human observer who specializes in dogs.”

However, the results of the project cannot be transposed directly to the real conditions for use of the CrowdPet app. “The study was based on photos taken with the animal always in the same position and at the same distance, i.e., centered. We have to work with photos taken in the street, in different positions and with varying lighting conditions,” Piva says.

Moreover, the state of the lost animal may also influence visual recognition, as it may be dirty or injured. It will therefore be necessary to have several photos of the same animal – and not just of its head but of the whole body, given that a characteristic mark on any part of the body could be decisive for recognition.

“The next step will be to reproduce the experiments described in the article ‘Where is my puppy?’ using photos taken in the wild. This is our biggest challenge,” Piva says. To surmount it, SciPet will be assisted by Professor Anderson Rocha as an associate researcher. Rocha is a specialist in machine learning for image recognition.

Company: Scipet
Site: crowdpet.io
Address: Rua Antônio Paioli, 320, Parque das Universidades, Campinas, São Paulo
Tel: [phone]
E-mail: [e-mail]