How many times have computers in science fiction done things that computers cannot do, like enhancing the resolution of an image from a security camera or picking out a perfect recording of a background conversation in a crowded room? Well, it turns out that we might be underestimating our software engineers after all. Two researchers at Carnegie Mellon University have figured out an algorithm that can detect where a picture has been taken, with an accuracy that's 30 times better than that of chance or random guesses. And their geolocation method may not be what you'd expect.

You'd think the best way to tell where a photo was shot would be to check for important buildings, landmarks in nature, or any signposts. Not so. Alexei A. Efros, assistant professor of computer science, and James Hays, a CS graduate student, developed their program to analyze the composition of photographs by creating and scanning histograms of image properties. Their algorithm examines the full profile of color and texture in each image, and also looks at various line features and geometric patterns. Then, it groups images of unknown location with images that have known details, and the geographical matching begins.

"We're not asking the computer to tell us what is depicted in the photo but to find other photos that look like it," Efros said. "It was surprising to us how effective this approach proved to be. Who would have guessed that similarity in overall image appearance would correlate to geographic proximity so well?"

So far, Efros and Hays have run their algorithm on a test set of 237 images, chosen for image quality, variety, and lack of easy geographical recognizability. When they ran their program, they successfully geolocated 16 percent of those test images to within 200 kilometers — and they also note that even geolocating an image within a country or region might still be helpful. These results, they say, are an encouraging jumping-off point for the larger field of geographical computer vision.

While a person might never be able to deconstruct and analyze a photo in this way, it's a piece of cake for a modern computer. And it probably wouldn't have been as easy to teach a computer to recognize photos the way a person would: It took a bit of outside-the-box thinking on the part of Efros and Hays to develop this new system. Perhaps we have only just scratched the surface of everything computers can do; to fully understand their extraordinary capabilities, it might be necessary to work within their limitations.

And in case you haven't figured it out yet, the picture above — one from the test set of Efros and Hays — was taken in the Netherlands.

Where in the World [Carnegie Mellon University]
IM2GPS Project

You’re #1: Ian Ingram, Artist-In-Residence at the Carnegie Mellon Robotics Institute’s CREATE Lab, has mounted a 15-foot tall foam finger in Steelers colors atop the Andy Warhol Museum. The finger points to individuals who “high five” any of a number of smaller foam fingers placed throughout the city. The finger encourages viewers to try to figure out where the finger is pointing and explore new and familiar parts of the city.

Reach, Robot: Composer and choreographer Grisha Coleman created an interactive musical installation. As pedestrians walk through Pittsburgh’s PPG Plaza, a web of cables suspended above their heads reacts to their motions. Walking, pausing, stepping, and reaching all trigger various sounds in the installation, immersing passersby in the musical and spoken works of the city’s African-American writers and artists.

Shelter: Garth Zeglin is a researcher at the Carnegie Mellon Robotics Institute whose work includes developing bipedal walking robots. But “Shelter” is a kinetic fabric robot. Viewers are invited to sit on a chair in the middle of two concentric fabric tents, which function independently of each other and respond to human presence as well as changes in light.

Extreme Animals: The Video Game: Inspired by his former job as the mascot at a Chuck E. Cheese, Matt Barton’s projects frequently include animatronic taxidermied animals. The animals of this work, created with Paper Rad member Jacob Ciocci, play frenzied video games, murmuring at and reacting to the game’s action. They glance back at approaching visitors, but promptly ignore them, returning their attention to the television screen.

ABB Basketball Arm: Perhaps the most well-traveled of the BigBots, this former automobile welder now shoots baskets at science museums across the country. The robot shoots free throws, with the angle and velocity of the ball selected by visitors, demonstrating concepts of precision and repeatability as well as how a single robot may serve diverse purposes.

The Look-See Tree: CMU grad student Ally Reeves designed a “Roving Art Cart” transported and powered by bicycle. The attached tree trunk lures viewers towards one of its six mini theaters, which come alive when it senses a human presence. Each theater displays an animatronic scene of animals whose existence is impacted by modern life; birds chirp cell phone rings and animals gather objects from the city for their nests.

Green Roof Roller Coaster: Gregory Witt and Joey Hays decided potted plants need a little more excitement in their lives, placing several young trees in a handmade rollercoaster on the roof of the Children’s Museum. The coaster’s cars monitor the vitals of the thrill-seeking vegetation and make sure they’re having a good time.

Rise and Fall: Artist Jennifer Gooch explores the nature of patriotic symbols and the ebb and flow of a nation’s dominance. Flags run up and down flagpoles and anthems play in a set loop. Visitors can send one flag to the top, but eventually it returns to its preset cycle.

Double-Taker: A robotic arm simulates human gestures and eye contact as it observes people outside the Center for Arts, making the occasional eponymous double-take. CMU Professor of Electronic Arts Golan Levin creates ocular art that changes based on the way the viewer views it.

prototype for an infinite array of semi-autonomous percussive devises (or Crickets): Kinetic sculptor Keny Marshall has created a network of cricket-like robots that knock out a certain pattern when they are “alive” and go silent when “dead,” that state determined through the rules of John Conway's “Game of Life.” Just like real crickets, this installation produces constant changes in harmony, with the added benefit that you can turn it off.

Mower: An android’s dream might be John Deere’s worst nightmare. An allusive electric sheep wanders the lawns of the Phipps Conservatory and Botanical Gardens, providing all the benefits of grazing animal without the need for a pooper scooper. Mechanical engineer and visiting professor at the Carnegie Mellon School of Art Osman Khan deliberately created a robot that solves a problem with already existing solutions.

The Robot 250 Festival ends July 27, although some events and exhibits continue into August.

[Robot 250]