Marine biologists have crowdsourced a facial-recognition algorithm to help them identify whales on the spot.  What an unusual use of visual search!

There are only around 500 North Atlantic right whales left in the world, making them one of the most endangered of all whale species. This month, nearly that many data scientists raced to complete a project that might help researchers keep this small population from disappearing altogether. Their goal: Develop an algorithm that could identify any living North Atlantic right whale from a photograph of its face.


The contest was the brainchild of Christin Khan, a biologist at the National Oceanic and Atmospheric Administration’s Northeast Fisheries Science Center, who was looking for a way to solve a problem she and other whale researchers come across every day in their work. Khan is part of a team that flies aerial surveys over the waters off the U.S. East Coast to look for North Atlantic right whales. (Two other species of right whale live in the Southern Hemisphere and the North Pacific.) To keep tabs on their target population, researchers track all of the whales individually, using each animal’s distinctive facial markings to identify the ones they see swimming below. (Some even have names: Whale 1611 is Clover, whale 1006 is Quasimodo, whale 1250 is Herb.) But the process can be difficult and tedious.

Hence, visual search technology.  What’s interesting here is that rather than choosing a technology, the NOAA simply introduced a contest and let everyone compete for the best result.

The competition’s winning entry, announced in early January, came from a team at the Warsaw office of the data-science company Their algorithm could identify whales with 87-percent accuracy.


The team used a neural network, a kind of computer program that learns by example. The scientists trained the neural network to search for patterns among the photos, first on the scale of a few pixels, then with increasingly larger swaths of an image.

It’s interesting that a general algorithm like deepsense was best for such a specific use case.  This evolution occurred with text search also; at first, purpose-built algorithms were best for specific use cases, but over time better general purpose algorithms took over.

Facial recognition for whales … Who knows what the next interesting application for visual search will be!