Birth Of A Salesman
Mitsubishi mixes projectors and facial-recognition technology to take digital merchandising to the next level.
DOES IN-store multimedia really help retailers move products? Or is it just eye candy that busy shoppers ignore while making a beeline for what they need? Judging by a 2004 Nielsen Media Research study of Wal-Mart’s in-store TV network, not only are shoppers noticing multimedia, they’re increasingly stopping to watch it. Nielsen’s study found that shoppers watched Wal-Mart’s in-store TV network for an average of 7 minutes per visit, or 44 percent longer than they did two years earlier.
Those results are why Wal-Mart has begun replacing ceiling-mounted CRTs scattered throughout each store with 42-inch LCDs positioned directly adjacent to the merchandise that they’re supposed to help sell. That proximity makes them more effective in nurturing the sales process. For example, some Frito-Lay ads make a case for buying multi-packs of chips rather than individual bags.
To Mitsubishi, those trends suggested opportunity: “In malls, you constantly see video monitors everywhere,” says Paul Dietz, a principal member of the technical staff at the company’s R&D division, Mitsubishi Electric Research Laboratories (MERL). “We thought, ‘Why not projectors in retail?’”
Besides the growing use of multimedia in stores, another trend sparked MERL’s research into innovative ways for using projectors in a retail setting: “One of the problems is that most of the projector market is getting fairly commoditized,” Dietz says. “It started as a project to find interesting, new markets for projectors.”
Regardless of whether a retailer uses projectors or monitors, the motivation is the same: keep the store fresh and nurture the sales process. Fresh displays are particularly important to retailers that focus on everyday items rather than big-ticket purchases.
“Big stores such as Wal-Mart have a lot of repeat customers, so in order to try to increase purchases, they spend an enormous amount of resources constantly moving things around in the store to make it look new and different every time that the customer comes in,” Dietz says.
Change is good
Instead of changing content once a month or every few weeks, the ideal would be to change it constantly — as in, make it interactive so that the retail display is an on-the-spot response to customer actions. MERL tackled that need partly by adding the ability to sense proximity and then using AV to respond.
“If you’re interested in a particular shoe, you tend to go near it and pick it up,” Dietz says. “The system senses that, and it can put up some imagery to try to help influence that purchase decision.”
Here’s one scenario: A shopper walks up to a shoe display that’s splashed with images that tie into the product, such as a video of a runner for shoes designed for marathoners. When the shopper picks up a shoe, the system detects the movement and changes the part of the image near the shoe, such as by adding some brief text that has more information about the shoe. (Videos of this scenario and others are available at ftp://ftp.merl.com/pub/DigitalMerchandising/Digital%20Merchandising%20co… and ftp://ftp.merl.com/pub/DigitalMerchandising/PICT0124.MOV.)
By providing product information, interactive displays address another classic challenge: Striking a balance between having enough sales-people on hand to answer questions and keeping personnel costs at a manageable level. One obvious, low-tech solution is to pack the display with brochures and other information to help educate and nurture the buying decisions, but that can result in a cluttered display that produces information overload.
That approach also ignores the fact that salespeople do more than just provide information. “Our first-generation display did exactly that: It gave you more information about the shoe you were looking at,” Dietz says. “But we realized that if you watch salespeople work and the way media is done in stores, it’s not so much about information as it is about trying to make an emotional connection with the consumer. What we’re trying to do is to get that gut, emotional reaction: ‘I want this thing.’”
Finding the “taste space”
To spark that desire, MERL tapped its research on computer vision and graphics, which includes technology for stitching together multiple projectors to create large, high-resolution displays. MERL also leveraged its home-grown technology for identifying facial characteristics in real time.
“We can tell the race, gender, and a bunch of other demographic information for the people who are looking at that display,” Dietz says. “We can’t do it 100 percent, but we can do it a great majority of the time.”
That ability lets the display start to act more like a human salesperson, with the multimedia presentation changing on the fly just like a salesperson would change his or her pitch depending on the person they’re pitching to. (How exactly a sales pitch might be tailored to a specific demographic is up to the retailer.)
As exotic as that process might sound, it’s increasingly common. Amazon and Netflix, for example, use similar processes to create what’s called a “taste space”: a customer’s preferences based on past habits and purchases.
The catch is that the algorithms used to create taste spaces are incredibly complex, requiring tremendous amounts of computing power. As a result, taste spaces usually are updated by collecting the data over time and then periodically doing a giant set of calculations, such as each night.
That approach obviously falls apart in a retail setting, where a taste space would have to be calculated in the span of just a few seconds — the time it takes for a person to wander up to a display. “One of our mathematicians came up with a way to update that in real time,” Dietz says. “So you can have these giant models of taste space, and as new data comes in, you can instantly enter it and come up with a new prediction of what that person would like.”
Using that prediction, MERL’s digital merchandising system can tap a library of multimedia to create a customized sales experience for that shopper. “The system can automatically select among lots of different media based on the demographics and behavior of the people in front of it in order to optimize the chance of a sale,” Dietz says. “That’s going to revolutionize the way that people think about displays in retail stores.”
Building the business case
Unlike Wal-Mart’s in-store network, MERL’s system uses projectors rather than displays. That creates a couple of challenges, starting with shadowing: A retail display loses its effectiveness if shoppers create shadows when they’re milling around in front of it or reaching for a product.
The shadowing problem isn’t unique to retail. Some researchers have tackled the problem in classroom applications by using a mix of rear projection and cameras. For example, the Georgia Institute of Technology worked with SMART Technologies to develop an interactive whiteboard nearly 18 feet long and almost 5 feet tall. That system uses a camera to identify where shadows fall and then adjusts the projectors to get the image around the person. (For more information on Georgia Tech’s research, see “Projecting A Good Image” in the August 2004 issue of Pro AV.) To keep costs low — a major factor in whether a retailer can make a business case for it — MERL’s system forgoes cameras in favor of more projectors.
Another challenge is installation. “There’s the issue of getting the projection lined up with the shelves,” Dietz says. “If you’re a major retailer with 4,000 stores, setting up each of these and getting it aligned is going to be a disaster.”
To minimize the time and cost of installation, MERL embedded each display surface with optical fibers that detect light from the projectors. During installation, the projector shows a series of patterns that are unique at each pixel, so the system can tell which pixel is landing on which fiber. Armed with that location information, the software then uses a process similar to keystone correction to warp the imagery so that it’s aligned with the display. Dietz says it’s cheaper to do the alignment in software than it is to manually adjust the projector or lens.
Cutting the time and cost of installation improves the chances that MERL’s system will win over retailers. As for the equipment itself, MERL says that it’s impossible to estimate the cost at this point but stresses that the system is only part of the equation. “The hardware equipment costs are small compared to the cost of creative design,” says Adam Bogue, vice president of marketing and business development at MERL.
Another plus is that the system leverages existing projector and visual-recognition technologies rather than requiring custom-made equipment. “We have all of the pieces to do these displays right now,” Dietz says. “We have a large patent portfolio that covers this stuff. It’s just a matter of putting it together for a particular retailer’s environment and proving that this is the right approach.”
The result is a chicken-and-egg situation: MERL’s system has to prove that it can improve sales, but it can’t do that without more real-world installations. “We’re looking for partners,” Dietz says. “Nobody is going to believe us until we put it into some stores and have some solid numbers to show that it increased sales this much.”
Tim Kridel is a freelance writer and analyst who covers telecom and technology. He’s based in Kansas City and can be reached at[email protected].
