Shutterstock
I recently came across an article about an augmented reality app created by the Cheetos brand for SXSW. It uses your smartphone camera to show you what the world would look like if it were all made out of Cheetos; thus the app is named Cheetos Vision.
This kind of AR application is what you would call a stunt — a publicity stunt that generates buzz without offering something of any utility to consumers. That’s not a bad thing. AR is by no means an established channel for engaging with consumers. They’re still exploring it and getting used to it. The use-case in terms of what’s in it for consumers is more about delight than utility.
The problem with stunts is that they do not inspire a lot of creative thinking about what you can learn from consumers using them. Your measurement strategy is pretty much one of tracking an adoption wave. A stunt has the lifespan of a fad – some early adopters who are “in” and find it first, then a tipping point that turns into a swell of adoption, and then a fall-off as all the people who are going to adopt it have done so and anyone interested in adopting it won’t because the moment has already passed. There might be some measurement of usage within the app – how long did people use it per session, where were they when they were using it, etc. – but because by its nature it’s a one-trick pony, that, too, is going to be limited. The app itself doesn’t necessarily offer a lot of depth to explore, so there just isn’t going to be a lot to measure about how people use it.
Turns out, that lack of depth has also meant there is precious little information out there about how to measure AR results. Outside of education or industrial applications (like how to measure the performance of AR-using repair technicians), I found one article that even mentioned specifics of what to look for in AR results, and while it gave examples of some outcomes from specific AR applications by some brands and retailers, the specifics are cursory at best.
It’s OK, at this point in AR’s maturity, especially for brands and retailers, that there isn’t a lot of material out there on what to measure when deploying AR to consumers, but only so long as companies are collecting just about everything they possibly can about AR usage and, when they can, who is using it. If you have a good trove of data to start from (emphasizing the protection of consumer security and privacy), then you can ask a lot of basic questions about what happened with your AR project. And for both AR and VR, there are a lot of similarities to both web and video, so there are some good basic measurements to start from that have a lot of maturity in the digital world – for example, if there is a video or “cut scene” kind of component to your experience, what percent of users started it, what percent viewed to the end, what percent skipped it, etc.
But AR is different than VR, because it’s not self-contained. VR is more like a 3D website from a measurement perspective. There appears to be developing thought around setting up VR as “zones”, tagging specific activities within the VR world to each zone, and then measuring where consumers go, how long they stay there, what activities did they unlock or engage in, etc. Again, very akin to clicks, page views, time on page – all that web-based measurement that digital marketers know well.
AR departs from VR in a measurement sense, though, because it is an overlay on the real world. That means you need to be looking at how AR impacts real-world behavior at the same time that you need to look at how consumers use the AR features that are provided. With VR, you are seeking to understand the user’s immersion into a world and their activities there. With AR, you should be seeking to understand the user’s interactivity with the real world, and not just their usage within the AR app itself.
In out-of-home use cases, this is location-based and more at a macro-level. Think Pokémon Go – where did they go, what activities did they do? You might care if the consumer went to the park, if they fought a battle at a gym, if they caught some Pokémon. But you don’t really care where exactly in the park they went, if they went to places in the park that weren’t PG-activated, etc.
But when you’re talking about interacting with the real world within a retailer’s property, for example, like in a store, then your measurement strategy needs to be a lot more specific. For product brands that are creating AR experiences that might be used in stores (like product activation, for example), they may not care about where specifically in a store, or even if it changes consumer behavior other than again in the macro sense – do people who engage with our AR application buy more of our product? Does it change their perception of our brand?
For retailers, though, AR implementations, especially in stores, should augment measurement just as much as it augments reality. You can’t just rely on interactions within the app itself. You need measurement of customer behavior, and you need to be able to compare customer behavior for those who used AR vs. those who did not, and even if those using AR influenced the behavior of other consumers who saw them using it.
That means some engagement analytics, a connection to any loyalty or CRM the retailer has in place, and also video analytics in stores. If you’re not looking at physical customer behavior, most reliably measured through video analytics, then you’re leaving a lot of potential data about AR effectiveness in the dark.
Upgrade Your Measurement Strategy For AR In Stores
Measurement strategies are pretty simple to define. You can basically do it in three steps. One, define your objective, two, identify a range of potential outcomes that you want in order to achieve that objective, and three, identify how best to measure those outcomes. Let’s explore each one in the context of measuring AR effectiveness.
Define your objective
Retailers and brands hopefully care most about understanding the shopper journey. I tend to start from McKinsey’s model called the “Consumer Decision Journey”, which identifies five key steps: Awareness, Explore, Select, Purchase, and Advocate. I tend to add another step in there which is “Service” – looking at post-purchase activities that focus on enabling enjoyment of the products purchased.
For any project, it’s important not to be too scattershot. You always have to have a primary objective – you can’t have it all. But you also need to be aware of halo effects and identify a few of those to also track. Especially because we’re dealing with the intersection of a new technology where there is little established understanding of the impact on consumer behavior, and talking about a specific application of that technology into an environment where consumer behavior understanding has always been limited (that would be the store) – you have to be open to the unexpected. This is definitely a case where unintended consequences can potentially be a good thing.
In the context of the store, some sample objectives for an AR project might include:
- Raise awareness of our mobile app (Awareness)
- Opt-in to our email communications (Awareness)
- Explore a new product line in our stores (Explore)
- Buy add-on products to a heavily purchased item (Select)
- Learn how to use a product at home (Service)
- Share an in-store experience on social media (Advocate)
The important thing is to pick one, make sure it is defined and specific enough to be measurable, and then be flexible enough to consider other avenues to measure as your own understanding expands over time.
The range of potential outcomes
The next step is to define a range of potential outcomes based on your objective. To make it easy to hold on to what that means, let’s take the example of “Raise awareness of our mobile app”. In this case, the AR project might be adding an AR feature to your existing mobile app, like a game or an easter-egg kind of concept that encourages consumers to scan signs or products in the store to access content.
Some sample target outcomes that you may seek in using AR to raise awareness of your mobile app – did users…?
- Download the app
- Use the AR features
- Register / log in to the app
- Explore other features of the app
- Buy something using the app
There are time-based dimensions here as well. Do people who use the AR part of the app access the mobile app more frequently? Is there a time decay to their usage? Is it shorter or longer than people who did not use the AR features?
And then there are the potential ancillary outcomes – the unintended consequences of AR usage in stores that might either be positive or negative:
- Positive/negative consumer interaction with store employees
- Spontaneous interaction with other consumers
- Positive/negative perception of the retail brand
- Chatter about the experience on social media
- Differences in exploring multiple store categories
- Differences in in-store conversion rate or basket
Note that these ancillary outcomes are not cast as positive or negative. Based on the history of digital signage measurement in stores, it might be fair to expect, for example, that consumers develop a more positive image of the brand after using an AR experience as part of shopping in a store, especially when the retailer or brand is looking to be associated with concepts like being cutting edge or tapped in to specific tech-driven customer segments. But if the AR experience leaves consumers lacking, those ancillary outcomes could easily turn negative. The same thing can happen with customer-employee interactions. If consumers have trouble with an AR experience and expect store employees to help them, and store employees aren’t prepared for that, then the ancillary outcome of those interactions could easily turn negative.
Measurement strategy
Because of the interplay between the consumer, the app, and the real world around them, retailers and brands need to look beyond what happens within the app to have a shot at developing any understanding of what consumer usage really means.
In the example of raising awareness of the mobile app, retailers should (hopefully) already be able to measure app usage, and should also (hopefully) be able to measure behavior within the AR features of a mobile app. What’s missing, though, is in-store behavior. Because a lot of these applications are being built by people who are more digitally-oriented than store, it doesn’t surprise me to find that a lot of them are thinking they can capture in-store behavior simply by looking at the digital recording of usage. If you built an AR game that requires consumers to go to multiple places in the store to scan something in order to play the game, you can see based on the usage which places they went to. Except that this doesn’t give you visibility into the places they went where there was not some digital action to take. Or give you much insight into failed attempts – like, if the consumer did not find the activation point they needed to scan in a specific location, for example.
To get the full picture, you have to couple digital-based measurements with physical behavior measurements. Pretty much, your only option there is in-store video analytics.
Video analytics has really been waiting for its moment in stores. The camera prices are falling, the cost to install is getting easier thanks to wireless technologies and improved energy consumption and battery life. Retailers have a better understanding of what they’re missing out on by not measuring behavior in stores, thanks to the maturity of online analytics. But when it comes down to it, retailers often fall back on “I need to invest in customer engagement first” and “I can get close enough with digital-only measures” – especially when they’re facing the capital cost of putting multiple cameras in every store.
With AR, you don’t really need video analytics in every store to get the depth of data you need for measurement – it certainly helps, but it’s not a requirement. But it does mean you need to identify a representative sample of stores, and a pool of stores that is large enough and diverse enough to yield meaningful measurements of behavior that could be extrapolated into general statements about consumer behavior.
The Bottom Line
Measuring the effectiveness of AR is still fairly unexplored territory. Retailers and brands should be collecting as much data about use as they can. That means exploring AR-based data, the digital data surrounding the AR experience, and the physical behavior this inspires – all three. And the technology and its in-store applications are new enough that measurement needs to be approached from a mindset of collecting data to see what you aren’t thinking about, as much as looking to find the answers to the things you are.