Augmented Reality and Apps: A Natural Progression

Apps are getting more sophisticated every day. One of the technology factors that are moving this progress forward is augmented reality:

“Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is modified (possibly even diminished rather than augmented), by a computer. As a result, the technology functions by enhancing one’s current perception of reality. By contrast, virtual reality replaces the real world with a simulated one. Augmentation is conventionally in real-time and in semantic context with environmental elements, such as sports scores on TV during a match. With the help of advanced AR technology (e.g. adding computer vision and object recognition) the information about the surrounding real world of the user becomes interactive and digitally manipulated. Artificial information about the environment and its objects can be overlaid on the real world.” – What is Augmented Reality?

There’s no doubt that augmented reality is a game-changer, as this technology gives app developers the ability to layer digital input on top of the real world. For example, perhaps you’re visiting a national park; you could see points of interest and historical trivia pop up on your mobile device as you make your way around the grounds, effortlessly, just by activating an app. Or maybe you’re walking around a store and need to find something that seems to elude you. Instead of spending time trying to track down a harried store employee or trudging up and down the aisle in vain, your mobile device could locate the item via GPS and other sensors.  This will become the norm rather than the exception; according to a report from Juniper Research, which predicts that in 2013 AR apps on mobiles will generate revenues of almost $300 million, and that by 2017 2.5 billion AR apps will be downloaded onto mobiles or tablets every year. This info graphic from Trigg-AR paints a broader picture:

Examples of apps using augmented reality

It’s truly amazing what talented developers are able to come up with when it comes to augmented reality and the total app experience. For example, how about using social elements added on top of augmented reality to take advantage of crowd-sourcing:

“…AR is being used by a start-up company in San Francisco called CrowdOptic that can recognize which direction a crowd of people have their phones pointing. They can then invite others using that app to see what all those phones are seeing. For example, at a NASCAR race, fans that can’t see the entire track could point their phones at a distant turn and get photos and videos gathered by others who are closer to the action.” – Augmented reality is just beginning,

The BBC reports on several apps that take advantage of augmented reality to give artistic creations a whole new look; for example, Colorapp is an app that gives anyone the chance to see their drawings come to life; simply color the picture, then use the app to animate your picture in ways you never imagined; another app, called Tocaboca, lets you put yourself in a salon to play a little bit with a new look.

Movies are also getting the augmented reality treatment; a new app tied into the Hunger Games trilogy allows the user to point at a film poster gives “Easter eggs” like new wallpapers, a photo booth with characters from the film, and new tidbits from the movie trailer.

Developers now have the ability to build native apps for Google Glass with the new Glass Development Kit; one of these is called Wordlens:

“Google showed off a few of the first native Glass apps, and one of the coolest among them was Wordlens, a real-time, augmented-reality translation app. Wordlens works much like it does on the iPhone—foreign-language text targeted by the camera is translated on top of the video feed in real time. This is neat on a smartphone, but on a device like Glass it becomes much more powerful. Just by looking at text and saying "OK, Glass, translate this," the text on the Glass video feed is translated and placed above the original text. Wordlens' app uses the accelerometers to keep the virtual text aligned, all while working completely offline.” – Google launches Glass dev kit,

Of course, one of the most natural fits for augmented reality technology in apps has to be shopping. An app called Adornably uses augmented reality layers to help customers shop for furniture, giving users the ability to see what the piece would look like in their own space (a really helpful feature), something that is immeasurably helpful when you’re standing in the furniture showroom. Here’s how the app works:

“To use the app, you start by placing a standard-sized magazine in the center of the room you’re looking to furnish. Fire up the app and it’ll zero in on the magazine and then use those proportions to determine the scale of the room. Once you find a perspective you’re happy with, you can snap a picture at tap of a button.”

You can see a demo of the app below:

Augmented reality as everyday technology

As apps become more and more sophisticated, we should start seeing augmented reality as an expected technology in our “everyday apps”; i.e., the ones we use on a regular basis. One of the first prototypes of this kind of experience was created with MIT Media Lab’s SixthSense Device, conceived and designed through a couple of collaborative experiments at the MIT Media Lab, hangs around your neck and broadcasts computer-generated content onto a wall or any other surface. Users can interact with it, even dial a telephone number projected on their hands, move images with your hands on a wall, and email what you see to someone else.

Basically, the SixthSense device is a wearable PC that transforms any surface into a display screen that is completely interactive with the user, giving them the ability to pull in apps and content whenever and from where ever they need to, and then kicking them away when they’re finished. This isn’t a super sophisticated device; it’s just a webcam, a battery-powered 3M projector with a mirror, and an Internet-enabled mobile phone. Looking at what is currently being done with this project and imagining further implications as developers layer other sensors and capabilities on top of this is mind-boggling.

Augmented reality is taking our apps one step further, making information available to us before we realize we need that. The leader of the SixthSense group at MIT recognizes this need:

"We're trying to make it possible to have access to relevant information in a more seamless way," says Dr. Pattie Maes, who heads the Fluid Interfaces Group at MIT. She says that while today's mobile computing devices can be useful, they are "deaf and blind," meaning that we have to stop what we're doing and tell those devices what information we need or want….We have a vision of a computing system that understands, at least to some extent, where the user is, what the user is doing, and who the user is interacting with," says Dr. Maes. "SixthSense can then proactively make information available to that user based on the situation…. All the work is in the software. The system is constantly trying to figure out what's around you, and what you're trying to do. It has to recognize the images you see, track your gestures, and then relate it all to relevant information at the same time." –

What is the future of augmented reality?

App users are becoming increasingly more sophisticated in their expectations of apps. Just a few years ago, touch-enabled apps were seen as the pinnacle of what apps could offer, now this capability is expected as the norm in pretty much any app you download. Can we expect the same from augmented reality? Share your thoughts with us in the comments.


For more complete information about compiler optimizations, see our Optimization Notice.