09/28/2015 05:11 pm ET Updated Dec 06, 2017

Why Marketers Need to Pry Open Digital Advertising's Black Box. (Hint: It's About Data.)

By Jason Uechi, Director of Engineering and Data Science at YP Mobile Labs

It's no secret that brands and agencies are swimming in an ocean of data and that the best are benefiting enormously from fresh insights that make their campaigns smarter and better targeted. Institutional change, specifically the restructuring of marketing organizations to successfully navigate these new waters, is hard, but the promise of "big data" to amplify creative and make it more impactful at scale is starting to be fulfilled.

While this infusion of data is transforming the practice of digital marketing for the better, it's also making it more complex. Marketers now have myriad buckets of data at their fingertips - demographic, behavioral, shopping, location, and so on - that they often struggle to make sense of it and to establish effective measurement standards that gauge whether data-rich digital campaigns are really resonating with consumers in the real world.

However, for all the data that marketers now have access to, there's still one gaping hole: the data marketers get -- or largely don't get -- from their tech platform partners.

To a large extent, digital advertising is still a black box, driven by invisible algorithms. Brands and agencies know their campaign results in terms of impressions, clicks, installs and other metrics, and they can track that against their spend, but they don't have much visibility into how their buy is actually moving through the digital advertising ecosystem. The problem has been compounded by the rise of programmatic, a notoriously opaque channel.

As a result, transparency has become a major concern for marketers, who are no longer satisfied with the wink and nod that used to suffice from their partners; they want true visibility from media sellers and ad-tech vendors to ensure they're getting maximum ROI and, importantly, an explanation of why the campaign is performing well. And they want insights that they can use to better plan and measure future campaigns.

As a result, tech platforms should look beyond performance results to satisfy marketers; they need to open up their black boxes and deliver valuable insights that will help brands from the very inception of a campaign. Platforms typically have thousands of anonymized data points for individual consumers gleaned through modeling and machine learning, and this data can be immensely valuable to brands and agencies, provided they have the right people to help interpret it.

In other words, brands need smart humans to complement the machines; the latter isn't of much use without the former. Of course, recruiting technical talent that also intuitively grasps the role data should play in amplifying creative work is an ongoing challenge for brands and agencies, but they're getting there.

Platforms are starting to get the message, and an increasing number have begun furnishing data insights to their marketer partners. Take Facebook, which introduced a free data product called "Topic Data" earlier this year; which shows marketers what users are saying about specific brands, products, events, activities and other subjects, making it a valuable resource for market research and campaign planning. And at YP, we're providing insights to retailers on which of the customers who walked into their stores had seen an ad beforehand. (Not surprisingly, those who click on retailer ads enter stores at a much higher rate.) It's a metric that goes well beyond impressions and clicks to demonstrate an ad buy's effectiveness.

So what comes next?

As data science continues to mature as a marketing discipline, we'll see brands and agencies apply it to their businesses in new and innovative ways. In particular, we've only seen the tip of the iceberg of what mobile location data can do, both in terms of targeting and measurement. Also keep in mind that consumer adoption of wearable devices at scale, which seems inevitable, will make location data even richer and potentially more accurate.  And, the expected rise of connected devices like smart refrigerators and smart cars could provide a whole new bucket of data to round out what marketers know - anonymously, of course - about consumer behavior. As an industry, we'll rush to make all of that meaningful for advertisers.

As more and more data is made available, freed from black boxes or even piped in from your fridge or toaster, the need for marketers to actively interpret human insights from abstract data is only going to become more acute. It's an exciting opportunity for those who want to be at the very active intersection of data and creativity.