Where's Waldo made easy with AI and facial recognition
20 August 2018 10:42 GMT Posted by • Nicholas Clark Bryan

The beloved childhood puzzle book Where’s Waldo has been given a 21st century twist. Creative agency redpepper have build a robot called ‘There’s Waldo’ that does all the finding for you. Even complete with a silicone hand to point him out.


‘There’s Waldo’ uses a metal robotic arm, a Raspberry Pi-controlled uArm Swift Pro, which is equipped with a Vision camera kit to allow it to use facial recognition software. The camera takes a photo of the page and then, using OpenCV, finds the possible Waldo faces. Google’s AutoML Vision service, having been trained on photos of Waldo, then receives the faces and identifies where he is. If the robot gets a match with a certainty of 95% or higher it will then point at where the face is on the page.

Google’s Cloud AutoML allows users to train their own AI without any previous coding knowledge. It was announced at Google’s I/O Conference last year and was released in January of this year. Cloud AutoML uses a drag and drop tool that allows anyone to create an image recognition tool and is currently being used by brands such as Urban Outfitters and Disney to improve e-commerce shopping experiences. It has even been able to identify ramen noodles by the shop they came from with 95% accuracy.

Matt Reed who is the Creative Technologist at redpepper told The Verge; “I got all of the Waldo training images from Google Image Search; 62 distinct Waldo heads and 45 Waldo heads plus body. I thought that wouldn’t be enough data to build a strong model but it gives surprisingly good predictions on Waldos that weren’t in the original training set.” Reed further said that he was inspired by Amazon Rekognition and its ability to recognise celebrities and wanted to experiment on a system that supported cartoons. Reed had no prior experience with AutoML and only trook around a week to code the robot in Python.

While no one is suggesting we take the fun out of Where’s Waldo by letting the machines do all the work it does demonstrate the impressive ability of the AutoML system. Looking to the future Reed suggested some other uses such as detecting fake or forged comic books and seeing which cartoon character the AI thinks a person looks like. While the AutoML was certainly created with a focus on business uses, such as more effectively selling products to customers, fun experiments such as ‘There’s Waldo’ offer a more creative avenue for using case data training and facial recognition software.