(Machine Learning + Social Listening) x Instagram = Visual Listening

Visual Listening, The Future of Social Analysis

Visual Listening. A phrase so horrendously juxtaposed we had to italicise it to appear sarcastic. But as corny as it is, it’s an accurate description of a new discipline social agencies will be offering to brands. Social listening has been around for years now, and in that time the offering has stayed pretty much the same, the major developments have been the introduction of yet more graphs that take entire slides to explain. Until now.

Quietly the Machine Learning world (yes, that’s a thing) has been going about its business teaching computers to beat humans at our own games. Then ancient Chinese game of Go - which is considered more complex than Chess - was the last to fall, Google’s AlphaGo defeated the world’s best Go player in May, references to Terminator’s Skynet were made, it was very dramatic. But aside from that, a great number of developers and companies have been taking this greater understanding and power of AI and applying it to Image Recognition. Google, naturally, have been at it and have released the TensorFlow Object Detection API, and have made it open source to allow the community to develop more machine learning models. Subsequently there has been a large increase in the number of services offering the ability to recognise an object, a logo, even a breed of dog in your images. The social listening behemoths have been snapping these up and some have grown rapidly on their own, so now for the first time we can listen to social audiences based on the images they post, rather than just the text. Visual Listening.

Machine Learning and image analysis has confirmed to a 99% degree of accuracy that this is indeed a ridiculously cute dog in a cup.

Social platforms are now moving into this space to create a link to ecommerce: Pinterest Lens allows users to take pictures and run a search. It’s a visual discovery tool. Imagine taking a #foodporn picture of your meal then using it to find similar recipe Pins. Pinterest really want consumers to behave this way to drive ad spend on the platform. It's not 100% accurate yet, which is actually helping to inject the subject into mainstream conversation; the Turing test of puppy or bagel? And the 'Not Hotdog' AI from ‘Silicon Valley’ has even been made for real as popular culture has begun to adopt AI as a real part of our lives.

But what does this mean? What’s the insight? What’s actionable?

What does this mean for brands?

Numerous tools including Brandwatch and GumGum have estimated that approximately 80% of images online that include a brand’s logo, don’t mention the brand in the accompanying text. For example, Brandwatch state that Coca-Cola generates 230,000 visual mentions a month, meaning approximately 184,000 of them don’t contain a brand reference in the text and would be subsequently missed.

That’s a lot of images, but where’s the insight?

This is where we can start to go beyond the confines of likes, shares or “estimated impressions” and understand contextually how a brand is being shared socially. Image recognition isn’t restricted to logos, AI models can recognise objects such as mugs, phones, sunglasses, or scenery such as mountains, beaches or cityscapes. AI can even recognise facial features such as a smile, frown or grimace.

All of a sudden what we can search for is somewhat limited only by our imaginations (the technology has limitations too, of course). We can listen for images of a beer brand alongside a hamburger, at the beach… in Barcelona. Or perhaps we want to find super fans; authors who regularly post images of our brands, we could then identify the scenarios or objects that are most regularly present within those images to understand where and how the brand is used by its biggest advocates. Want to know if lifestyle influencers wear your sportswear brand? Now we can tell you. We can now begin to understand true brand use and relationships through natural advocacy. For example, some people will wear a brand with it acting as a recommendation in its truest form, but they will not write a tweet about it. These posts were previously lost to the social void. Not anymore.

Ok so there are insights to be had, are any of them actionable?

The cynic in me asked the same thing. You could tell Starbucks that thousands of pictures of their logo contained coffee and muffins, but they know that already. What can we get that’s useful? The truth is that it depends on the brand, their products or services. As previously mentioned, visual listening opens us up to a world of contextual information that we previously didn’t have access to, and this contextual information is brand dependant. Still that’s not answering the question, so here’s a list of six actionable use cases:

  1. More text If nothing else, visual listening gives access to 400% more text that we weren’t able to analyse before as it didn’t contain the brand or keyword mentions we search for currently. There’s a lot of keywords sentiment in there to understand.
  2. Crisis Comms Consumers love to moan on social media, especially when they’ve bought a faulty product. For example, there are a lot of FMCG brands that receive complaints around mould and foreign bodies due to faulty or broken packaging. This can be the first line of defence to identify an image before the press do and manage the conversation.
  3. Advocacy Influencers and celebrities take a lot of pictures, and even more pictures are taken of them. Understanding which influencer endorsements earn more organic mentions outside of explicit brand mentions will take the industry to the next level.
  4. Smile Take sentiment out of the equation and simply aggregate the number of images where consumers are visually happy with a product.
  5. Brand Partnerships Why recognise one brand logo when you could recognise two? Understanding the other brands that consumer’s use regularly together with your own can lead to some excellent campaigns and reach a much wider audience.
  6. Sponsorship Seeing the full value of earned and shared social posts that contain a brand logo but no mention of the brand.

There we have it, Visual Listening is a legitimate phrase and AI will most definitely take over the world at some stage, but we’re pretty happy about it (the first part at least). 

To find out how to use visual listening for your brand, contact Head of Mediabrands Society, Pete Durant on pete.durant@mbww.com.

The latest from the world of society

Introducing Encore: Why we can’t get enough of our Paid Social process!

Read full story

Four Reasons Why Selling with Social Makes it Easier to Face the Books

Read full story

Driving sales with Social. Can it be done outside Facebook?

Read full story