Although I follow all display technologies, projectors have a unique quality you just won’t find in a flat panel LCD. Artists, engineers and experience designers have been making use of the ability to illuminate arbitrary surfaces to create some truly wonderful visual environments more and more. By relaying an image directly onto a surface, projectors offer an interesting approach to augmented reality (AR).

This technique is used often in advertising, where branding graphics will be generated onto a building or wall ideally in a crowded location, such as Grand Central Station. It’s also used as a tourist attraction or a means for generating buzz for an event, which happened very recently on Caesars Palace in Las Vegas during CinemaCon. These compelling presentations tend to really excite audiences, whether seen in person, or even just watching a YouTube video of the event later.

I’ve spent some of my career living between the two worlds of scientific exploration and artistic expression. I’ve been a consultant in the use of projector-based augmented reality for art, theater and advertising. A particularly compelling project I worked on was with a group of innovative artists known as “31 Down” in New York. As part of the Ontological Incubator, projectors were used to illuminate backgrounds (and even the naked bodies of actors) in order to create dreamlike states and offer a mechanism for storytelling in parallel (real objects and projected objects evolving together). I was proud of the result they achieved. Take a look at this quick clip from the play.

Of course, a lot of projector-based AR has its roots in the science of computer graphics and computer vision, which was developed in the late 90s primarily by Henry Fuchs and his PhD student at the time, Ramesh Raskar. The technical community refers to geometric alignment of projectors onto surfaces as shader lamps, where objects are texture mapped (or shaded) using projectors with images, video or graphics.

In the world of art, augmented reality has been used on both small objects ( i.e. texturing a face onto the model of a face, which is a bit creepy) and large scale mediums (from cars to buildings) to amazing effects like this video:


This concept is becoming incredibly popular, so much that Absolut has created an app to allow consumers to create their own augmented reality on any building in the world using Google Street View.

On a side note, techniques used to map a projected image into a 3D space are very much related to the mathematics that Mersive employs. We align multiple projectors into one seamless image to create a large-scale, beyond-HD display for simulation, planetariums, domes, virtual shopping and other displays. In our case, we may not know the shape of the surface beforehand, while AR systems typically require a geometric model of the target to be shaded. Nevertheless, the approaches are close cousins to one another.

About Christopher Jaynes

Jaynes received his doctoral degree at the University of Massachusetts, Amherst where he worked on camera calibration and aerial image interpretation technologies now in use by the federal government. Jaynes received his BS degree with honors from the School of Computer Science at the University of Utah. In 2004, he founded Mersive and today serves as the company's Chief Technology Officer. Prior to Mersive, Jaynes founded the Metaverse Lab at the University of Kentucky, recognized as one of the leading laboratories for computer vision and interactive media and dedicated to research related to video surveillance, human-computer interaction, and display technologies.

Submit Comment