When Apple posted a job listing Tuesday for a “Computer Vision specialist,” the tech-obsessed echo chamber began speculating which Apple product line would be employing this person’s expertise. Most intriguingly, the job posting made frequent reference to 3-D: “3-D geometry,” “3-D reconstruction” and “cameras and surfaces in a 3-D environment.”
So what exactly could Apple have up its sleeve? What is “computer vision” and how does it relate to 3-D?
The short answer: Apple could be delving into applications as familiar as simple 3-D video capture, to as arcane as real-time environment capture for an augmented-reality system.
“Computer vision is about enabling the computer or mobile device to make sense of a 3-D image the way humans do,” Forrester analyst Frank Gillett told Wired. “For this job application, Apple appears to be looking for someone who could help them think about how stereo cameras could look at a scene, and figure out how to do something useful for its owner.”
Gillett’s explanation suggests a much grander 3-D application than what we see in today’s mobile device. Currently, 3-D in the mobile space is defined by crappy stereoscopic cameras, and glasses-free 3-D displays with incredibly subtle — and sometimes imperceptible — 3-D spatial effects. Android smartphone manufacturers have shown off 3-D image- and video-capture in the LG Optimus 3-D Max and HTC EVO 3-D, but consumers haven’t warmed up to these simple implementations.
“3-D displays on smartphones today are nothing more than a gimmick,” Canalys analyst Pete Cunningham told Wired via email. “The lack of good content is a major challenge. There may be opportunities with 3-D gaming on tablets and smartphones in the future, but it is still a very niche segment.”
But looking toward the future, Apple could have the chops to turn consumer interest around.
Apple has been dabbling in the 3-D arts for quite some time, if patent filings are any indicator. One particularly interesting implementation is a 3-D display calibrated by eye positioning. It would provide subtle 3-D effects, like drop shadows that dynamically change depending on your position. Apple also won a patent for its own glasses-free 3-D display.
So what could Apple do with 3-D on an iPhone or iPad?
“I would expect that Apple is focusing entirely on 3-D environment capture — the idea that your iPhone could create a 3-D map out of the world around you with a simple swipe of the camera around your environment,” Forrester analyst James McQuivey told Wired. “It would create a meta view of the world. Who made the clothes that person is wearing? What architectural style is that pillar? All of that kind of information, if aggregated at the level of the operating system, could then be tapped into by many apps which would each then add value to the meta-view.”
Gillett said this would be useful for helping us quickly and intuitively understand information in a number of scenarios. For example, imagine you need to replace the wiper fluid in your car, and you’ve never done it before. You could use your smartphone camera to identify your car, and then receive a quick 3-D visual explanation of what to do — all with better spatial representation than 2D would provide.
Of course, Apple’s new computer vision specialist could work in other areas too.
“What it means, probably, is they want to render videos faster on things like video conference calls, or do refreshes without refreshing the whole screen,” Gartner analyst Ken Dulaney told Wired. Gillett said Apple may also want to tweak the rendering of an image, making it more understandable to the human eye, rather than just displaying it accurately.
Dr. Keith Price of the University of Southern California’s Computer Vision Laboratory thinks Apple could embark on something like Photosynth, a large-scale augmented reality product. Apple could also avoid the need for dual cameras by using a single camera and combining multiple images for a 3-D version.
How exactly any of this would be done is up to that doctorate-holding machine vision expert, and Apple’s multi-view stereo research group.
