Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now


Beyond Signage

Touchscreens, software, and QR codes combine for an interactive experience

Although the exhibits at the Saint John Paul II National Shrine in Washington D.C. use technology that is often seen in digital signage applications, in the hands of integrator/fabricator Design & Production and content creator Bluecadet, the palette broadens. In executing the vision of museum designer Gallagher & Associates, the team created an interactive media system that’s more about exploration and connection than signage. In its use of touch, content, and QR code integration, it demonstrates how to engage audiences in any environment where there is a story to be told, explored, and lingered over.

How do you tell the story of a saint? You don’t. You let visitors discover the story, following threads of narrative and imagery, responding to personal connection and inspiration where they find it, and where it leads them next. The life of Saint John Paul II lends itself to that kind of serendipity. It’s a good way to tell his unique story, and it also represents the man himself and his core mission to foster connection in its many forms.

The former Pope’s long service of global travel and community-building produced a spectacular amount of still and motion picture imagery, which the project team transformed into an immersive, user-controlled experience. The visitor experience in the gallery of the shrine includes an interactive 14-foot-long wall of life-size imagery (the World Travels touchscreen) and a nine-foot-long World Youth Day interactive table that contains imagery and information from Saint John Paul’s 100-plus appearances at these international gatherings. Both experiences are simultaneously controlled by multiple individual users, each following their own cinematic and interactive storyline.

Six Planar displays under a single hinged sheet of glass comprise the interactive World Travels touchscreen.

The content and user interface for the interactive displays, overseen by Philadelphia-based digital design firm Bluecadet (for museum design/planning company Gallagher & Associates of Washington, D.C.) integrate thousands of photographs and videos curated by the Knights of Columbus and the Vatican.

Bluecadet conceived the UI (user interface) and UX (user experience) of both the wall and the table through a thorough discovery process. “How do we package this content in a way that could accommodate multiple users and be enjoyable for the visitors?” asked Bluecadet’s founder/ CEO Josh Goldblum rhetorically. “We had many preliminary conversations. Did we want a giant globe? No. That gives the wrong idea. This isn’t about someone looking at the world from space. He was a man of the people. Also, we were designing this to work for multiple users simultaneously. Who would control the globe? It would be weird to have multiple globes …”

Ultimately they moved to the idea of the Touch Wall, which presents pre-set content when nobody is interacting with the wall. Proximity sensors within the LCD displays track approaching visitors and allow them to influence what images play—a key element of making digital signage personal and inviting. Visitors can select trips by browsing the photography-driven scrollable panels, or using a search interface to find out if the Pope visited their home country. Visitors can drag the content to a comfortable viewing height, ensuring an accessible, adaptable navigation.

Design & Production’s fabrication placed six Planar displays behind a single wall of glass, creating an elegant, contiguous multi-touch surface, with an IR sensor frame, says D&P’s senior systems engineer Dale Panning. The wall of glass is hinged for maintenance and “outweighs the displays.”

For the World Youth Day interactive table, Design & Production drew on its extensive experience fabricating similar tables for clients including many presidential libraries. They used four MultiTaction touch displays for a total resolution of 1920×4320 and backed them with a powerful computer and graphics cards, says D&P’s project engineer Jeremy Keith. This gave Bluecadet the platform and low latency they needed for a unique and processing-intensive content experience.

The table shows moving and still pictures as well as graphics and text, and the idea was to let as many viewers as could fit around its surface to each chose their own “story.” So one person could virtually trace the Pope’s visit to Manila and another Paris and another Denver, etc.

This sounds simple enough. Even young children can now maneuver this table intuitively, but that simplicity required leveraging MultiTaction’s sensor and processing technology, and integrating it with the right hardware and software bridges. Conceptually, the idea was to provide a series of user-operated “pucks”—small, coaster-like pieces of plastic, each carrying optical markers (similar to QR codes) to trigger a specific set of display events and sequences. With the puck, the user could linger in one spot, or move along the surface to the next stage of the “story”—the next stop in the visual journey—all the way to the end.

To make this concept work, the display needed to identify which puck was which and be able to trigger events that Bluecadet could use in its programming.

Bluecadet tapped into MultiTaction’s Codice technology to read 32 IR sensors embedded with the LED backlight for each MultiTaction LCD display. Codice allows the display to recognize and identify the optical markets using computer vision and object-recognition algorithms. Using an SDK, a developer can print out a physical optical marker, apply it to an object (a puck, in this case) and the display’s hardware/software complement can identify the unique code and the attached object’s location based on interpreting infrared reflections.

“Traditionally infrared-based technology has struggled with ambient light,” says David Saltsman, MultiTaction’s New York-based director for Eastern US and Canada. “Light sources like daylight and halogen also emit a level of infrared. So what MultiTaction offers is a feature called Hybrid Tracking. Normal infrared tracking simply tracks the infrared reflection but with Hybrid Tracking we’re capturing the shadows from external infrared light and combining that information with the reflections. Therefore, we’re able to have a much more accurate interaction of hands and/or objects even in ambient environments where traditional IR technologies falter.”

With the ability to capture that data from the display, Bluecadet could encode any instructions it wanted into the table’s custom software framework.

Of course, Bluecadet had to first test the MultiTaction displays to make sure they could deliver. “We knew this technology existed,” Goldblum recalls, “but not how good it was. You don’t know until you test it and get it in front of people whether it will actually support the experience you’re looking for. There could have been latency issues when you drag the imagery across the display. It might have staggered. Having a whole lot of users doing many different things at once might put too much tax on the processor.”

Design & Production provided prototypes of the table, which allowed Bluecadet to test the platform in its Philadelphia studio while creative teams concurrently built and shaped the imagery for the many different “stories” the table would reveal. Working with curators at the Knights of Columbus and the Vatican, Bluecadet gathered vast amounts of still and motion imagery and assembled a full team, consisting of a creative director, art director, user experience designer, product manager, content developer, and an editor. Clumps of imagery were refined, resized, and color graded using tools including Adobe’s Photoshop, Premiere, and After Effects. Meanwhile, a team of programmers, coding in C++ and making use of the Cinder open-source library, were building custom code for the table.

When the Bluecadet team was ready and had loaded all its custom software and media into the D&P PC (it would both control the presentation and serve the media), it came time for the all-important testing phase.

Four MultiTaction interactive displays are driven by a single computer and custom software for the interactive World Youth Day table.

They brought test subjects of all ages into the studio to put the table through its paces, including an entire Catholic school class courtesy of the Knights of Columbus. “They were really banging on the thing!” Goldblum recalls. “Fortunately, the hardware and software were both sturdy and capable of taking a beating. And they were also capable of delivering content the users felt was enjoyable.”

Among the discoveries the developers made was the fact that users were more likely to retain interest in their selected “story” if they have a clear sense of their own progress along the way. “Each story is finite,” Goldblum explains. “It has a certain number of ‘stops’ and it takes a certain amount of time to get through all of them.

“We also created this feature at the end where you could choose a message that would show up when the next person picked up that puck. That just indicates that something will happen to complete the experience and people were much less likely to lose interest when the experience provided that feedback loop.”

“We have seen significant growth in just the past few years,” says Saltsman of the interactive display market, noting that in addition to museums, other educational institutions, corporations and a wide variety of organizations are using interactive displays in creative ways. “I think it’s more true this year than ever before.

As the technology of our phones and tablets evolves, we’re seeing that same growth in the interactive display market too.”

“It helps deliver an experience that can’t be delivered any other way,” adds Goldblum of the type of interactive installations Bluecadet designs. “But you have to make sure the interface is gratifying. You can see this with apps at the app store. There are certain ones that are clunky. They don’t feel responsive to what you want to do. You don’t get some kind of gratification every time you do something with it. And there are others that people just feel completely engaged by.

“I think what you’re finding is the whole vocabulary of museums is changing rapidly,” Goldblum concludes. “It’s unusual if someone doesn’t carry some kind of a smart device around everywhere. This is the way people consume content now. Museums and other institutions and businesses will have to adapt or people will go other places.”

Featured Articles