SDS 677: Digital Analytics with Avinash Kaushik

This is episode number 677 with Avinash Kaushik, Chief Strategy Officer at Croud. Today’s episode is brought to you by Pathway, the reactive data processing framework, by Posit the open-source data science company, and by Anaconda, the world’s most popular Python distribution. 

00:00:20

Welcome to the SuperDataScience podcast, the most listened-to podcast in the data science industry. Each week, we bring you inspiring people and ideas to help you build a successful career in data science. I’m your host, Jon Krohn. Thanks for joining me today. And now let’s make the complex simple. 

00:00:51

Welcome back to the SuperDataScience podcast. Today I’m joined by Avinash Kaushik, a personal icon of mine for the past decade. It’s such an honor for me to have him on the show today. Avinash is Chief Strategy Officer at Croud, a leading marketing agency. Until recently, he was Senior Director of Global Strategic Analytics at Google, where he spent 16 years and where he launched the ubiquitous Google Analytics tool. He’s a multi-time author, including the industry-standard book, “Web Analytics 2.0”. He’s an authority on marketing analytics through his widely-read “Occam’s Razor” blog and “The Marketing Analytics Intersect” newsletter, which has 55,000 subscribers. And his prodigious posting of useful analytics insights has landed him nearly 200,000 Twitter followers and nearly 300,000 followers on LinkedIn. Today’s episode has a few deeply technical moments, but for the most part is accessible to anyone who’d like to glean practical digital analytics insights from a world leader in this space. 

00:01:48

In this episode, Avinash details the distinction between brand analytics and performance analytics and why both are critical for commercial success. He talks about his “four clusters of intent” for understanding your audience, delivering joy to them, and accelerating business profit. He talks about why it’s a superpower for executives to be hands-on with data, tools and programming. He talks about then his favorite data tools and programming languages, and he fills us in on how AI is transforming analytics today and provides his concrete vision for how AI will transform analytics in the coming years. All right, you ready for this enriching episode? Let’s go.

00:02:27

Avinash. Welcome to the SuperDataScience podcast. It’s so exciting for me to have you here. It’s actually surreal because when I started off as a data scientist, my very first job was at Omnicom, which is a big marketing company. And so in the weeks before I was starting, I knew I had this job offer. So I went to Union Square in New York. I went to the big bookstore there, the Barnes & Noble, and I went to the marketing analytics section and your book it was the obvious choice, “Web Analytics 2.0”. It was the obvious book to be studying at that time as somebody who wanted to be a digital marketing analytics expert. And so, yeah, surreal to have you here today. Thank you so much for coming on the show. 

Avinash Kaushik: 00:03:13

Oh, I’m thrilled to be here. Thank you for inviting me. I’m, I’m really excited to record this today. 

Jon Krohn: 00:03:17

Nice. Yeah. And so we’re filming live together in New York. And that’s because you recently started working at a company while they’re based in London, primarily, but they also have a big office in New York Croud – C R O U D. And actually, you’re not the first guest that we’ve had from Croud on the show. Because back when I started hosting the podcast two years ago we had Chris Tate who- 

Avinash Kaushik: 00:03:49

Oh, yes. 

Jon Krohn: 00:03:49

Connected us. So he was in episode 481. So, yeah. So you are the Chief Strategy Officer at Croud, which is a global full-service digital marketing partner for some of the world’s leading brands. What is a Chief Strategy Officer Avinash, and how does it tie to data and analytics? 

Avinash Kaushik: 00:04:09

Yes. No, that’s a, that’s a great question. So I was more interested in Croud the company and contributing to it. And so we just tried to figure out during the interviewing process, okay, what am I going to do? What is going to be my title? So, at the moment, at Croud, I’m responsible for product and engineering. I’m responsible for mergers and acquisitions. I’m responsible for analytics. And in taking those three areas and converting them into a competitive advantage for the company. And obviously, I help contribute to setting the company strategy, working with the board of directors, working with the chairman, our CEO. So it’s exciting that you’ve, you’ve already spoken with Chris. He’s our sort of CEO of the US business. And it’s great that you both know each other. 

Jon Krohn: 00:04:56

Yeah, he’s, it’s one of those examples of somebody that that I’ve known for years. But when I was starting off hosting the show, it was like, wow, this is a perfect podcast guest. 

Avinash Kaushik: 00:05:06

Yeah. 

Jon Krohn: 00:05:06

He’s an amazing speaker. 

Avinash Kaushik: 00:05:07

He is.

Jon Krohn: 00:05:08

He does great conference presentations. And he’s one of those people when he comes to like a cocktail party or something he’s an A plus at working room. 

Avinash Kaushik: 00:05:17

Oh, yes. No, he’s, he, he’s so good at what he does, personal or professional. He’s, he’s, yeah, I appreciate him. So analytics plays sort of like a really big role. You know, the best agencies in the world at the moment have figured out or need to figure out how to use data as a competitive advantage, because we’ve got so much data in the world, we can measure and track all kinds of things. Marketing is influenced by data in so many ways. So at Croud, we’ve built a practice centered on analytics. We have teams here in the US we have team in the UK, of course, that are figuring out how to take analytics very much from the bleeding edge. All these interesting kinds of things that we’ll talk about a little more today and create them as a way to create unhealthy profits for our customers. 

Jon Krohn: 00:06:10

Unhealthy profits. 

Avinash Kaushik: 00:06:11

Yes. And that’s, that’s my sort of ambition for agency when I met, I say, you know, we should, we should have a mantra. I love mantras, and I think our, Croud is going to be centered on this idea that we obsess about creating unhealthy profits for our customer. 

Jon Krohn: 00:06:26

Right.

Avinash Kaushik: 00:06:26

That’s the true north. 

Jon Krohn: 00:06:27

You want, you want it to be like the abundance of food that now most of the world has, where people have gone from starvation to just obese with profits, you’re going to be causing profit diabetes.

Avinash Kaushik: 00:06:37

No, it’s like a really great way to know, like, why do you wake up? Why do you come to work every day? What is the purpose of data? Why should you have less bureaucracy? Why should you invent things? It’s, it’s because we’re centered on our clients and not just creating activity for the clients. Like, any agency can run impressions and page views and or GRPs on TV. Oh, I’m not, I don’t care about activity. I care about outcomes. And so profit’s sort of a good way to center everybody on not just putting all this marketing in the world, but obsessing about the result of the marketing. And I think makes, makes Croud a little different than other companies. 

Jon Krohn: 00:07:15

I think that’s nice that it also makes sure that you are tied to the profit center objectives of the organization. 

Avinash Kaushik: 00:07:20

Yes. 

Jon Krohn: 00:07:20

Awesome. So in the marketing space, there are several terms related to analytics that sound really similar. And so maybe you can distill the differences for us. So digital analytics, marketing analytics, web analytics, are these the same thing or are there differences? 

Avinash Kaushik: 00:07:39

Yeah well, they’re, they’re becoming more and more they’re the converging, I think that’s a good way to say. I use marketing analytics as a phrase as an umbrella phrase under which everything else sits. So sometimes we’ll segregate things by saying, oh, this is just for our digital presence and, or this is just about the website itself. That’s where the web analytics piece comes from. Or sometimes we’ll say, oh, this is the full view of the business, the end-to-end view of the business. So that’ll include a company that has a digital presence, plus has stores, plus has a call center. Let’s put all of that into a massive data warehouse and sort of now you’re beginning to get into data sets that allow you to do data science. Because some of the other data sets are so two finite, there is not some truly sexy things you can do. So for me marketing analytics just sort of captures, is the umbrella under which all of these terms sit. And they sometimes just, in a larger company, they’ll refer to teams and say, oh, this is the digital analytics team, this is the offline analytics team. This is web analytics team. But, but unless you’re like a very massive company, they’ll probably have some phrase related to marketing, analytics, business intelligence, and then everything sort of subsumed underneath that. 

Jon Krohn: 00:08:51

Right. 

Avinash Kaushik: 00:08:52

That’s how I think about it. And marketing is just, just to say it’s the function and the purpose we’re solving for. 

Jon Krohn: 00:08:58

Right. Right. Right.

Avinash Kaushik: 00:08:59

And sometimes I’ll say, oh, marketing analytics and the only segmentation I’ll make after that is brand and performance. And it’s mostly because I don’t, I want the analytics purpose or the strategies purpose to be tied to, are we solving for media long-term outcomes or short-term outcomes. That’s basically the difference to me. 

Jon Krohn: 00:09:18

So the key distinction for you is more so brand versus performance. 

Avinash Kaushik: 00:09:22

That’s right. 

Jon Krohn: 00:09:22

So then tell us about those. 

Avinash Kaushik: 00:09:24

Sure, sure, sure, yeah. So a lot of, it’s an AND situation not an OR. A lot of people think about OR, and they’ll be segregated departments in company. I tend to think of these as an AND situation. And so brand marketing is all of the things we do to attach a collection of values, attributes to our brand. When I tell, say, Apple, you think of something. And I say, Chipotle, you think of something, or Lululemon, you think of something and it’s, it’s the thing that’s coming to your mind that’s the result of brand marketing. 

Jon Krohn: 00:09:53

Right. 

Avinash Kaushik: 00:09:54

And it’s, it’s really important that, that I sort of stretch brand marketing. Some of brand marketing is due to advertising you run. So these pretty wonderful ads you run on TV or, or on top of taxis in New York where we’re sitting today. But brand marketing to me is the, is the product, it’s the customer service, it’s sort of everything that you do as a company that creates an experience for the, for the consumer. In fact, the advertising you run makes a minority contribution to what people think about your brand. Your product probably is like 50, 60% of it. 

Jon Krohn: 00:10:31

Right. 

Avinash Kaushik: 00:10:32

Your customer service, another 20, 30% of it. Are you there when people really need you? 

Jon Krohn: 00:10:36

Right. 

Avinash Kaushik: 00:10:36

And by the way, the pretty ads you run constantly to make sure that people think a certain way about a company that does add a little bit to it. But if it’s actually those things, it’s evoking a particular ethos in the customer. So that’s brand marketing. Performance to me, it’s like, okay, we have a product or a service and let’s figure out how to put a competitive, competitively differentiated value proposition in the market that allows us to sell loads of product to meet our revenue goals and profit goals for this quarter, right. Or next quarter. So it, it’s sort of the, it’s a little more science, a little less art. Brand is sort of the reverse of it. But the purpose of both is to drive revenue. Is just that brand will drive it in the medium and long term, I say 6 to 18 months. And performance is driving outcomes for you between 0 and 6 months. It’s a very short-term view of driving revenue for us.

Jon Krohn: 00:11:29

Nice. Is it possible that the effectiveness of brand marketing is more difficult to measure than the performance one? Cause I can imagine what the performance one, and correct me if I’m wrong here, but just imagining here, that that might have a lot more to do with, oh, we have this campaign on Instagram, it got this many clicks and that led downstream to this many purchases of our widget. Whereas the brand marketing, it’s, it’s less about getting those actions in real time that are measurable, but it’s more about relying on the impressions and somebody seeing Lululemon in a particular way over time. 

Avinash Kaushik: 00:12:03

Yes. 

Jon Krohn: 00:12:04

From those impressions?

Avinash Kaushik: 00:12:05

Yeah. The horizon and the type of measurement you do has to be different between both in order for you to win those two things. I’m not sure that it is particularly, maybe a smidgen harder to do brand measurement versus performance. But they both have the unique challenges. So for brand, what you want to do is you’re going to solve unaided brand awareness or purchase intent. And so the measurement is all tied around, do we have the right sample? Are we asking the right people? Is it proper test and control?

Jon Krohn: 00:12:38

Ah, ah. 

Avinash Kaushik: 00:12:39

Because you, you, you don’t want to do, there’s a lot of mistakes people make there. And then we want to figure out oh, how do we do versus competition? So the amount of, actually the raw size of data you’ll collect in brand marketing is in kilobytes. In performance marketing, you’re collecting terabytes of data. 

Jon Krohn: 00:12:57

Right.

Avinash Kaushik: 00:12:57

But the challenge over the last five years in particular is people used to think, I’m going to dump 10 million into Google. That’s all I need to do. And by the way 50 million comes out at the other end, and that’s all there is to it. But one of the things we’ve figured out is influencing consumers across their entire journey means I have to be good at Google, I have to be good at Facebook, I have to be good at television, I have to do, be good at magazines, I have to do good at, be everything. So performance marketing has morphed into being the science of optimizing the portfolio of your actions. That’s where you begin to hear words like attribution. You begin to hear words like using machine learning in creative ways or media mix models.

00:13:39

So the effective practice of performance marketing requires difficult and challenging analytics just as it does for brand. But if you’re like a single channel, small business, all you do is run Facebook ads, then Facebook will give you all the analytics you need to see if you put a dollar in at one end, do ten dollars come out at the other end or three dollars come out at the other end. That part is simple. But as you go from medium to larger size company practicing sophisticated performance marketing is actually quite complex. 

Jon Krohn: 00:14:11

Cool. Are you moving from batch to real-time? Pathway makes real-time machine learning and data processing simple. Run your pipeline in Python or SQL in the same manner as you would for batch processing. With Pathway, it will work as-is in streaming mode. Pathway will handle all of the data updates for you automatically. The free and source-available solution, based on a powerful Rust engine, ensures consistency at all times. Pathway makes it simple to enrich your data, create and process machine learning features, and draw conclusions quickly. All developers can access the enterprise-proven technology for free at pathway.com — check it out! 

00:14:50

Well, there’s plenty more for us to still dig into- 

Avinash Kaushik: 00:14:52

Yes. 

Jon Krohn: 00:14:52

In this episode. You yourself have built quite a big brand. You’ve got a really popular newsletter. So, The Marketing Analytics Intersect, TMAI, is the name of your newsletter. And so what inspired you to begin writing this newsletter? Clearly, we can tell already that you have an enormous wealth of knowledge on marketing and analytics to share with us. But what inspired you to create this newsletter? How do you decide what topics to cover and how do you keep this big audience of 55,000 subscribers so engaged? 

Avinash Kaushik: 00:15:25

Yeah. I’d say it’s a, it’s not an easy, easy thing to do, for sure. You have fewer friends and you sleep less. I think it’s a secret formula. No, I used to write the, I think the blog. I have a blog Occam’s Razor, and it has existed for perhaps 17, 18 years. That’s sort of where I started writing and folks like Andy Beal and Guy Kawasaki and these guys sort of encouraged me to start writing. And so I started writing the blog. But as I got busier and busier, rather writing twice a week, I started writing the blog once a week and then once every other week, and then once every three weeks. And one of the things I decided really early on is I hated reading blogs that are all over the place. And so my blog is exclusively concentrated on marketing analytics, web analytics originally. But as I grew in my career, it sort of grew in thinking. But I didn’t write about anything, if I didn’t write about elections or my opinion about Justin Trudeau, nothing, just like very focused.

00:16:30

So about four years ago with, or four and a half years ago, with I thought, oh, you know what I don’t, I’m having less and less time, so I can’t write 1500 word blog posts. So I’ll just start a newsletter, and it’ll be shorter, it’ll be sweeter, and I could write every week and I could write about more kinds of content if I wanted to. Because it is just a little newsletter. Well, I didn’t realize. And that’s, that’s sort of how I started The Marketing Analytics Intersect. And I remember I signed up for the free one tiny letter. And they only let you have like 500 subscribers. And then after a week and a half, I already exceeded the limit, I moved to someone else. So I’ve moved three times as the newsletter has grown, but it has been a more personal way to write it. 

00:17:22

There’s nothing, it’s not public in any way. It’s not published anywhere. So I worry a little less about it getting indexed. It’s more free-flow. The challenge is, because I like writing now on average newsletter is still 1500 words, so I am back to writing a blog post every week. But in terms of inspiration and content, it has sort of been really interesting for me. One of the things I found after I wrote Web Analytics: An Hour a Day, my first book, is that you write a book, you become like famous a little bit in a tiny fishbowl, and you get speaking gigs and you maybe leave your job and you start becoming a consultant. But one of the things I noticed is other authors around me, they rarely wrote second books or, or after a while, I would hear very famous author, like three or four years after they wrote the book, and they were like completely disconnected from reality because they didn’t have real jobs anymore. 

00:18:14

And so I don’t blame them, it’s just you don’t have a real job. You just don’t understand. So one of the things I created as a deliberate choice is that I will always have a real job in the field that I am in. And I stayed at Google for such a long time, or I kept changing my jobs at Google, but I stayed there for a long time. And so for me, the source of ideas to write about is the real job I have. I actually have a job in the things I write about and I get frustrated with clients’ problems or designs. I actually use the tools, I actually write code. And so every week, like this week, it’s simply the, the blog post is about, I call it the Baklava of Brand Measurement, the layers, the layers of brand management. 

00:19:03

And it’s because I’ve met a bunch of CMOs over the last three weeks, and I keep running into the same challenge. I keep standing up the whiteboard explaining the same thing. And so I thought, oh, you know what, I was going to write a newsletter and I’m going to draw a design. The thing I keep drawing on a whiteboard, I’m going to draw it and I’m going to publish it in the world. So there’s like a little thing. So every week the inspiration is real work, and I think as long as I keep doing real work, I probably won’t run out of content to write. But it, it’s, it’s a hard choice to make because sometimes I’m like, oh, I think it would be kind of fun to just be a very expensive paid speaker. 

Jon Krohn: 00:19:39

Right. 

Avinash Kaushik: 00:19:39

Not, not having nine to nine job, things like that. But it’s a choice I’ve made. 

Jon Krohn: 00:19:45

Yeah. What so going back to the success is having fewer friends and less sleep. You know, sometimes I’ll complain to people how I’m, you know, I’m like, I don’t have much of a life, but I don’t know what I could do otherwise because we’re creating two podcast episodes a week, and the audience has that expectation. But I can’t just be a podcaster because what am if, if I’m podcasting about data science, and that’s all I do, I’m doing speaking and writing books, but I don’t know what it’s like building a machine learning company and deploying them into production, not right away, but eventually I’m going to be out of touch and- 

Avinash Kaushik: 00:20:20

That’s right. 

Jon Krohn: 00:20:20

It’s going to be a disaster. So this, the plates continue to spin and I wouldn’t have it any other way. 

Avinash Kaushik: 00:20:26

Yeah. And then the thing is, for me, the other thing it turns out I’m trying to figure out as, as I’m sure you have is, is how do I make this hobby or this thing into which I put extraordinary amount of time, every newsletter takes a lot-

Jon Krohn: 00:20:39

I bet.

Avinash Kaushik: 00:20:39

Of time and effort. 

Jon Krohn: 00:20:40

Yeah.

Avinash Kaushik: 00:20:40

And so, about two and a half years ago, in the middle of the Black Lives Matter movement, I thought, you know what, I’m going to make a paid version of the newsletter, and I’m going to charge people a hundred dollars a year to read me every week. And then what I decide is I’m going to take gross revenues from the newsletter. I’ll pay for Stripe and everything else and I’ll donate it to charity, and I thought, oh, like a couple hundred people will sign up.

00:21:07

And that’s still more money than zero. And I’m sort of really amazed that last year the revenue from the newsletter was $200,000. Which, which I donated to charity. And so in like two and a half years, the newsletter itself has contributed about four $25,000 to charity. And so now I have like this extra motivation to say this thing I’m really passionate about, I can convert into money and actually make, collectively me and the premium subscribers, are now putting some good into the world from our mutual passion. So now I have like this extra pressure to write good content every week. Because it’s not just for me, but it’s these three charities that I support with all this money. 

Jon Krohn: 00:21:51

Yeah. That’s great. And it goes to show how these external motivators can be much more meaningful than just, you know, adding a little bit more to your bottom line, personally. You mentioned something in there that you’re still hands-on with tools and with writing code. What kinds of hands-on tools and code do you write? 

Avinash Kaushik: 00:22:11

Yeah. So, for me, it has been as I have, I have matured in my career, I get to less and less hardcore hacking than I did early in my career. But right now, primarily for me like a, like a big big thing about me is that big passion of mine is that there’s always a lot of data, but how do you make people understand it as simply as possible? So one of my favorite hobbies is to go to D3.js and get all these amazing ideas for visualization and build complex cords and sunburst and streamgraphs to take some really interesting patterns in the data that are not visible in a table or a pie chart or I hate pie charts, sorry, stack bar charts. But actually create something really sophisticated that shows something complex inside the data in a very simple way. So I do a lot of JavaScript stuff. Then the other thing is I, over the last four years or so, I’ve done lots of things to figure out how to use machine learning algorithms in a more smarter way. Especially to answer the unknown-unknowns. And so, you know, classification decision trees or random forest algorithms. 

Jon Krohn: 00:23:22

For the known-unknowns. 

Avinash Kaushik: 00:23:24

Unknown-unknowns. 

Jon Krohn: 00:23:25

For the unknown unknowns.

Avinash Kaushik: 00:23:25

Yes, yes, yes. 

Jon Krohn: 00:23:26

For like, exploring and discovering new things in the data. 

Avinash Kaushik: 00:23:29

Yes, yes. So I always classify all data problems into these three categories, that you know, the no-knowns, the known-unknowns, and the unknown-unknowns. And I think I started exploring machine learning as a way to get better at the known-unknowns. Things I knew, but I didn’t know how to get the answer to and then so they’ll go with. But over the last three years, my obsession has been the unknown-unknowns, because that is where machine learning shines. Like I don’t have the brain to compute that many variables and coefficients and the hidden patterns inside the data. And I have so many more examples of those. So, so for me, coding a lot now is just figuring out, oh, all these things exist. Somebody’s already worked on it. Let me go grab them. Let me figure out how to use it for my things and edit these things and comment those other things out. So a lot of it, and of course using R and Tableau and all these other tools that we have in our field for a very long time. So using those. But I tend to obsess a lot more about strategic analytics, and then I have my colleagues like Konrad, who, who leads our analytics practice- 

Jon Krohn: 00:24:30

Konrad Kopczynski, he’s been on the show as well. 

Avinash Kaushik: 00:24:32

Oh, yes, yes. 

Jon Krohn: 00:24:33

From before he was a Croud, actually. 

Avinash Kaushik: 00:24:34

That’s right. Yeah. He was one of the companies that we acquired Impact Advisors. 

Jon Krohn: 00:24:39

Impact Advisors, yes. So he was on an episode number 465. And I was also going to say quickly, while I’m interrupting you with episode numbers, that you’re in very good company here with your dislike for pie charts, which is something that I evangelize against constantly and so we even, we have an episode 490, it’s called Say No To Pie Charts. Dedicated exclusively to this topic. 

Avinash Kaushik: 00:25:01

Yes, yes. I am so glad, and I highly recommend everybody go check that, because, I have one of my most popular posts on LinkedIn is Eat Pies, Don’t Chart Them. There you go. So, like my colleagues like Konrad, or my colleagues in the UK they tend to do more hardcore data science and coding and things, but for me, it, it’s a lot about, I tend to work a lot with VPs and CEOs and CMOs, and so for me it’s like taking an extraordinary amount of complexity from four-five hundred million of media spent across all kinds of marketing initiatives and figuring out how can I help make this really super simple for this very senior person to make very expensive decisions. And so my focus is, tends to be more in those kinds of things. 

Jon Krohn: 00:25:52

Yeah. And I think this gives you a superpower as an executive with so much experience already in the field, there are few that maintain using tools hands-on like this. I candidly had not prepared questions for you on that kind of technical detail, because I didn’t expect that you would still be doing that. But it, but it arms you with the superpower that you just, you just gave us a window into how useful that can be, because when there’s something, when you have hundreds of millions of dollars of spend to be looking over and to be digging into, if you are bounded by the limits of relying on a data engineer or a data scientist or machine learning engineer to be pulling those insights for you, you can’t be digging into it and getting the same kinds of insights as if you can say, huh, that’s interesting, or I wonder if there’s an unknown-unknown here. And you just explore it yourself, and then you visualize it beautifully in D3 and you can sell it. 

Avinash Kaushik: 00:26:53

That’s right. That’s exactly right. And, and, and this actually, your point is very good. I mean if, if our listeners, they, they are thinking about like, I want to grow up in my career and be a leader, so on and so forth. One of the things that I’ve learned through my career is I appreciate subject matter expert leaders, and I try to be one as well. Because for one simple reason, if I’m a leader who does at least some amounts of real actual coding and work it means I can imagine possibilities for my large team to execute. If you, if you don’t do that, it limits the imagination. You can, so let’s say Konrad and I are sitting and discussing like we did yesterday, and I’m like, eh, this doesn’t look good. We should do that. And Konrad’s like, oh, this is another great idea. If I’ve done the real work, I can continue that discussion and imagine new possibilities. But if you’re a leader who over time just becomes a leader, like a manager, there’s nothing wrong with it, please, we need more managers in the world. 

Jon Krohn: 00:27:52

That’s usually what happens. 

Avinash Kaushik: 00:27:53

Yeah. You lack imagination, which means you can lead your data science team, your analytics team, your marketing team in a less effective way. But if you can do real work, you can imagine new possibilities, you can chart an innovative future. 

Jon Krohn: 00:28:12

Yeah. I agree a hundred percent. Yeah, you’re preaching to the choir Avinash, yeah. So for our junior listeners, keep it up as you grow in your careers. And yeah, it can be worth it, you know I won’t embarrass this particular guest because I think this might have been something that he said in confidence and, but we’ll see. We have, I have an episode coming up with a guest, I’ll try to remember to bring this up with him. But I recently in a private conversation, he relayed that after many years like we’re talking decades with all of the incredible things that were happening with ChatGPT in late 2022, is that as that was exploding out, he said, I started learning modern programming languages, I started learning Python. 

00:29:00

Because it had been 20 years or whatever since he’d sat there with the terminal, but he spent his Christmas holiday, the days digging back into this and doing tutorials and becoming competent in modern machine learning because the NLP things that he was seeing were so mind-blowing. And for this person’s role, understanding the implications of that is critical. And so I think whether you are junior now and thinking about growing into that executive role, you know, we have that the advice for you today to maintain that hands-on technical capabilities, but even if you are not, even if you are a senior every little bit, that you can learn about how this stuff really works to remove the magic from the experience of using GPT-4, to be, to help you realize that this is something that you could be developing too. You could be training models that are involving proprietary data that have similar kinds of conversational capabilities as these models. But if you, if you don’t dabble at least a little bit in the technical details, it’s all just, it’s a black box. 

Avinash Kaushik: 00:30:11

Or you can’t imagine new possibilities for your business.

Jon Krohn: 00:30:13

Exactly.

Avinash Kaushik: 00:30:13

Because I find that I’m sitting in meetings and we’re discussing a problem about effectiveness, efficiency, and I’m like, oh, oh no, I set up a Discord server, I’m running Midjourney, and I’m doing this thing, and by the way, this thing I’m toying with, I’m playing with, here’s how it can completely transform this part of a practice. It’s that imagination. But, in that meeting, somebody actually spoke up and said, what’s Discord? That’s what I mean, I don’t look down on that person in any way, but it’s just that if you don’t know Discord, there are some possibilities you can’t imagine. That’s it. 

Jon Krohn: 00:30:49

Yeah. Can you send me a link to my Outlook mailbox about Discord, please? Very good. So yeah, I didn’t expect to go down this road with you, but really interesting. So back to your newsletter, TMAI you, you have a Venn diagram. And so there’s analytics and marketing, and then where they overlap, what’s happening in there? 

Avinash Kaushik: 00:31:16

That’s a good question. And the way I frame it is at, at the center of those two things sits Customer Delight and Profit, in that order. So I would say if you’re really great at practicing the art and science and marketing, and you’re good at the art and science of analytics, that the first thing you should obsess about is customer delight. Like, you want to deliver relevant, appetizing, you want to deliver amazing creative, you want to do, you want to know how to frequency caps so you don’t annoy people and things like that. So you, you deliver customer delight, and if you solve that problem first, then in exchange for that, a company’s ready to make profit. So I put it in that order. Same thing like, you know, like for our agency, we want to focus on delivering outcomes that are very powerful for the clients. Then you’ve kind of worked back to what’s in it for me. So the intersection of those two things are customer delight first, and then exchanging that for profit second. 

Jon Krohn: 00:32:13

Nice. This episode is brought to you by Posit: the open-source data science company. Posit makes the best tools for data scientists who love open source. Period. No matter which language they prefer. Posit’s popular RStudio IDE and enterprise products, like Posit Workbench, Connect, and Package Manager, help individuals, teams, and organizations scale R & Python development easily and securely. Produce higher-quality analysis faster with great data science tools. Visit Posit.co—that’s P-O-S-I-T dot co—to learn more. 

00:32:13

And then another big theme in your writing in your newsletter is incrementality-centric marketing. So can you explain the concept of incrementality and its importance for businesses in today’s digital landscape? And I think you might even have something that you alluded to prior to us hitting a record button, is that maybe this idea of incrementality applies to all aspects of life. 

Avinash Kaushik: 00:33:14

Yeah, exactly. No, it, it’s very true. So over the last few years, you know, we, we’ve been through the pandemic, and now we have these economic pressures in the market. So, very senior executives that begin to ask this really simple question what would happen if I did nothing in this area? And so, me, the trigger was a conversation with Alphabet CFO Ruth, an amazing inspiring leader. And I remember the first meeting I went to her, I said, oh, I’m this, I run global analytics for Google, and here’s all the performance and things. And she just looked at me and she said, what would happen if we just shut down marketing, all of it? It’s like thousands of people in like 25 countries. And Ruth’s question is, what she was asking is what’s the incrementality of marketing?

00:34:04

If we did, none of it, would all our profits dry up? And the answer is, so if you look at tools like Google Analytics, Adobe Analytics today on the digital side, or you look at your ERP or business intelligence tools that you’re running inside your company, you are reporting. So you open Google Analytics, it’ll report that we drove 10,000 conversions yesterday and we had spent 300,000 on marketing to drive those 10,000 conversions. And you’re very happy. It’s like, oh, marketing’s working really well. That, that’s like a lot of conversions. Incrementality is asking, what if I did not spend those $300,000? Would I have lost all 10,000 conversions? And it turns out the answer is you would’ve gotten 8,000 anyway. Right. And it’s because marketing does bring some amount of outcomes, but a lot of these people know your brand, they, you have word amount, you’ve got a great product, people are just going by your store.

00:35:00

And they’re like, I’m going to buy something. And so there’s this curve after which sort of the effectiveness of marketing dips off. And they’ve never really explored that, or the effectiveness of your sales team or the effectiveness of your HR team. It kind of dips at a point and you would’ve gotten some outcomes anyway. So for example, you shut down your sales team, you’ll still keep getting orders for a while. Right. You don’t need them for a while. Now, of course, maybe three months from now you’re like, oh my God, everything ran out. So incrementality is the art and science of saying, I’m spending $3 million on Google. How many of the outcomes I would’ve gotten anyway, that’s the thing. So, so what we, the, a very common way to measure the last one, for example, is Google has a, has a tool called CLS, Conversion Lift Studies, and they create proper test and control.

00:35:51

And they say, ah, today we’re claiming that 10 conversions were driven by Google, or we did a proper test and control. We show a certain more people your ads, we ensure other people your ads. And it turns out that of the 10 conversions you received that we are saying came from Google six, you would’ve gotten anyway. It’s four, were truly incremental. And so you are, it lead naturally leads to say, ah, what are the four, what are the keywords? Where are the people? Where did they come from? Can I find more of those kind of execution strategies that are truly incremental? So incrementality is a really great way to understand which parts of marketing are truly driving value that cannot be delivered by anything else the company is doing. And any executive that actually figures out how to do incrementality-centric marketing can meet the CFO, look them in the eye and say, this is what I delivered. 

00:36:50

And that is, it could not have been delivered by anything else. So then if she says, what if I shut down marketing, it could say, you would lose 27% of the sales, now that boom. Right. It’s like a superpower to have. And at the moment, most marketing is not judged by incrementality. It’s, it’s sort of this I call it claimed outcomes. And what it means is you’re not innovating, you don’t understand your effectiveness in a truly good way, et cetera, et cetera, et cetera. So I, I’m, I’m sort of a big proponent of figuring out what value do I add to my company that would not exist if I did not exist. That’s the mindset we want. 

Jon Krohn: 00:37:33

Nice. Yeah. Beautifully stated. Yes. So that covers the main topics that I kind of wanted to get from your marketing newsletter. And we had some nice divergence off into hands-on tools and that kind of thing. Let’s talk about Google more that you’ve been alluding to. So before Croud, you worked for Google for 16 years. That’s a long time. And in fact, you were part of the team that launched Google Analytics. Which is still today the market leader in web analytics software. How did you unlock the power of machine learning to realize Google’s marketing strategies? 

Avinash Kaushik: 00:38:13

Yes, so it’s really great. At Google I had three roles. So now I started with working very closely with the product team to launch tools. Google Antics was one of them. And I spent a bunch of time in sales figuring out how to take data and activate value for Google’s top 150 clients, people who spend tons of money. And I started sort of an internal Bain-McKinsey type consulting company inside Google. And in the last five years, I spent working in marketing using all that lessons on Google’s marketing itself. So I’ll give you two answers. The first and the third one. Inside Google Analytics once we, once we launch Google Analytics 2.0 and 3.0, one of the things, one of the things I, a key lesson for me is that availability of data makes companies less smart than I had expected. 

00:38:57

So now you’ve got all the data you wanted, all nice reports, beautifully organized, great visual design, and let’s say companies become 20% more data-driven. Now begin to realize that this assumption that that is necessary in experience and intelligence at the other [inaudible 00:39:016] analytics is flawed. A lot of people who got all this data, they still had to poke and prod and dig and segment and custom report and find the things they needed. And our users necessary didn’t have all those skills because they have many other jobs to do. So one of my first forays into using machine learning is we launched a feature called Intelligence inside Google Analytics. And so it’s like a tab, you click on it and my first discussion with the engineers was I don’t want a dashboard that just shows all the performance. 

00:39:47

I want a dashboard that tells our users, look at these things only, don’t look at everything. Only these things matter. So we started to use simple statistic to say these three, these elements we’re surfacing our three standard deviations away from the mean or two. Or by the way, you could move the slider and make it seven standard deviations [inaudible 00:40:07] if you wanted to. And see just the most extreme occurrences inside the data. And so what happened is it increased usage a lot more because we were thinking for you and surfacing you from the channels, you should look at the pages you should look at the KPIs you should look at. And that one page became like a portal to digging in where we’re sort of holding your hand and taking you into the tool. Instead 80 reports – too many, right? 

00:40:31

So that was the first foray into it. And then next thing we did was, was even more interesting. We were for example, really good at helping people understand what is your current revenue conversions average or value abandonment rate. But then I thought, oh, wouldn’t it be amazing if we could predict the future? So, so we launched like a set of tools based on some very simple machine learning algorithms in hindsight that would understand your revenue growth and things like that today and say, ah, this is how we anticipate you’ll end the quarter on. So they say, oh, I’ve made, made $500,000 already. We’re an, we are predicting based on all the factors, we’re analyzing tons of factors that you’ll end up at 750. Well, your target was a million and now you can go do something about. It’s like, oh my God, I’m going to go 250 short of my target and you could take action. 

00:41:23

So very simple predictive analytics. The next one was even more fun. We started to look at hundreds of millions of rows of data in consumer journeys and buy every anonymous cookie, anonymous cookie. We would predict users more likely to convert in the next X amount of time, Y amount of time, or Z amount of time. So now, rather than looking at this all these hundreds of thousands of people, because the average conversion rate for a website is something like 2%. So 98% are not going to convert. Now it is extraordinary, valuable to know in that 98% bucket, is there a cohort that is more likely to convert over the next seven days, because then I can segment them out and I can figure out how to do marketing to them in a different way. Because they have exhibited behavior that they’re more likely to convert. So I don’t have to spam 96%, I [inaudible 00:42:16] 2%. These are 2%, let me just figure out how to deliver relevant advertising. So that was the next foray into machine learning. 

Jon Krohn: 00:42:23

That, interestingly, that sounds to me like the incrementality. 

Avinash Kaushik: 00:42:28

Yeah. 

Jon Krohn: 00:42:28

These are two different kinds of approaches for identifying this maybe very small segment that are very likely to convert. And so targeting them really effectively. 

Avinash Kaushik: 00:42:38

Yes, exactly. And it’s, it’s figuring out how to drive hyper-relevance and give if in a way reduce the amount of money you’re spending on marketing, but the money you’re spending will have higher ROI. Because otherwise what you would do is annoy 98 people to get to those two. Now we’re saying no, you can just pull them out of the audience. Spend less money, but do something relevant for those two. And I think the thing that, I think the last bit that was I said I’m super proud of is, you know, on an average interaction, let’s say you bought this, this gorgeous shirt and- 

Jon Krohn: 00:43:12

Lululemon actually. Yes. 

Avinash Kaushik: 00:43:13

Me too. No, I’m not kidding. It’s actually Lululemon. So we’re both Canadians today. But let’s say you bought a shirt and Lululemon it’s unlikely that you bought it after seeing one of their ads or one visit to the site. You probably used an ad, you probably saw something on Facebook, you probably saw something else in Google, whatever. And the outcome of your purchase on Lululemon was this complex collection of interactions. And yet most of the credit, if Lululemon was analyzing that outcome, would say, oh he clicked on a Facebook ad converted, let’s spend more on Facebook. No, no, no, no. It was Facebook plus, plus, plus,plus. And so the last bit of machine learning that we did that continues to be one of the most bleeding edge things inside Google antics is to do data-driven attribution. Because it is almost impossible to figure out on Lululemon’s website across literally millions of people coming to the site with billions of touchpoints that they will have had prior to coming to the website. What is actually working? 

00:44:20

And that’s exactly the kind of thing you want to hand over to a machine learning algorithms because no human can visualize, understand, analyze that much complexity. So we built something in Google Analytics. The team has built something into Google Analytics, it’s called data-driven attribution. And it uses machine learning to figure out how to plot these journeys. And in the end, help tell Lululemon, you should spend X amounts on Facebook. You spend Y amounts on Google, you spend Z amounts on TikTok, and you should spend Q amounts on something else. And it allows you to do more effective marketing aligned with real consumer behavior. Rather than a theory in your head about how marketing actually works. 

Jon Krohn: 00:44:56

Did you know that Anaconda is the world’s most popular platform for developing and deploying secure Python solutions faster? Anaconda’s solutions enable practitioners and institutions around the world to securely harness the power of open source. And their cloud platform is a place where you can learn and share within the Python community. Master your Python skills with on-demand courses, cloud-hosted notebooks, webinars and so much more! See why over 35 million users trust Anaconda by heading to www.superdatascience.com/anaconda — you’ll find the page pre-populated with our special code “SDS” so you’ll get your first 30 days free. Yep, that’s 30 days of free Python training at www.superdatascience.com/anaconda 

00:45:42 Yeah, very cool. And this conversation that we’re having reminds me of, there’s so many very smart people that I’ve met in machine learning over the years who flocked to marketing because you could get more data on behavior than in any other industry. And so all of these kinds of examples that you’re providing around how machine learning can be used to really tease apart in a data-driven way, what’s the mix of marketing that’s leading to this end conversion of buying the Lululemon shirt or whatever? Yeah. 

Avinash Kaushik: 00:46:14

No, no, it is really cool. We have tons and tons of data and complex questions to answer. It’s like the perfect combination. You’ve got lots of data, not perfect, but lots of data and you’ve got some very complex questions. 

Jon Krohn: 00:46:26

Nice. And so how does that experience from the 16 years at Google allow you to be more effective as a chief strategy officer at Croud today? 

Avinash Kaushik: 00:46:37

Yeah, so one of the things I, as I worked with the top 150 or so companies on the planet as a part of working at Google, everybody from Chanel, to Expedia, to Ford Motor Company, to Pfizer and REMAX. And so just the experience of working with this depth and breadth of companies and different kinds of businesses sort of helps help me understand a lot better what kind of factors drive business practice and sort of I can bring that experience to our customer, our clients. But the other one is, I think that throughout my, when my mom asks me what I do, it’s really hard to explain. But, I sort of boil it down to this phrase, that I help people make better decisions in a business context. And so she gets that part. 

00:47:21

It’s like, I think what I do now at Croud is figure out how to use data, art and science, machine learning and this experience of actually working with these top 150 companies on the planet to create innovative marketing strategies. How to eliminate some of their beliefs accumulated over time that are no longer relevant. Like one of the things I I I’ve come to firmly believe is this idea of the marketing funnel is this [inaudible] it just [inaudible] I hate people who like the marketing funnel because it doesn’t exist. It does not exist. Right? And so I have this, this other framework I built based on all this experience and petabytes of analysis of consumers and how they behave. I call it intent-centric marketing. But I bring those kind of things back to our customers so they can create a non-normal competitive advantage. So for some of our clients that are already living at the bleeding edge- 

Jon Krohn: 00:48:19

And unhealthy competitive advantage. 

Avinash Kaushik: 00:48:21

Exactly, exactly. So you’re already doing great. Here’s how you pick and become greater still, or oh my God, you, you’re facing these amazing competitive headwinds. You have no idea how to actually get out of and create, differentia yourself. All right, let’s work with Croud, let’s help you figure that out. And it’s a lot of fun to do this business transformation. 

Jon Krohn: 00:48:39

Cool. Yeah. Something you share there with probably a lot of our listeners is an inability to explain to your parents or grandparents what is why people are paying you to do what you do. So a lot of, we’ve already, I’ve mentioned GPT-4 already in this episode. And we talked about ChatGPT already. So AI is something that you and I, and probably a lot of our listeners have been working with for many, many years. But now there’s this popular fascination with these tools and justifiably so. Cause all of a sudden now the capabilities that we see with these generative AI models, mind-blowing. So how do you, from a strategic standpoint how do you balance using advanced technologies, especially when these powerful new AI models come out with the need to maintain a human touch in marketing? 

Avinash Kaushik: 00:49:32

Yes. No, it’s, it’s really important, and I for one, you know, I’m, I’m not worried about humans becoming irrelevant for quite some time. It’s probably at some point, but not, not for a while. 

Jon Krohn: 00:49:45

We have varying perspectives. We had in a very recent episode we had Jeremie Harris, who is an AI safety expert. And so in episode number 668 as well as in an episode from two years ago, I can’t remember the episode number, and I’ve thrown out enough episode numbers already, I don’t need to give people more. But Jeremie Harris is, and it’s good that some people out there are very concerned, but he, he believes and he has lots of evidence to back him up that with the way that things that scaling these systems up is causing so many more capabilities than we anticipated. And it’s chipping away much more rapidly, I think, than people expected at these at this set of things that only humans can do that machines can’t. And so yeah, it’s interesting that some people take that to mean maybe we only have years, or at least we should be prepared for the possibility. Or maybe decades, but yeah, I’m with you. And I’ve even done counter episodes to [crosstalk 00:50:50] on here where I’ve said I’ve had a couple of episodes called AGI is Not Near, where I tried to dispel fears of a generally intelligent machine that’s, that’s more intelligent than us. I think that- 

Avinash Kaushik: 00:51:06

What I think, what I think really is what’s going to happen is it’s going to unlock our ability to do things that we enjoy more and aren’t against our strengths. I’ll give you a really simple example. One of the things we’re the, one of the big challenges in the agency world and the creative is figuring out, let me take one slight step back. If you look at what makes marketing effective in the real world, one shocking answer for me, and this was after I had published two bestselling books and I was relatively famous in a small fishbowl, I learned that 70% of the success of any marketing is the creative. It’s actually the picture on the billboard, it’s the ad video on the TV and Facebook or YouTube, et cetera. 70%. And I was like, what? 

00:51:52

I’ve been working on the other 30% so far in my life. And so I became obsessed over the last decade about building amazing creative that can be very effective. The other thing is, building creative is very expensive. So we’ll sit down in a room, we’ll have like 12 ideas. We have to kill 10 of them, because we can only make two of them because it’s expensive to make them. Then we’ll send them to some designers and prototypers over in India or Slovenia, because it’s like a little cheaper to make digital assets graphics. They’ll make it, they’ll send it back 10 days later. And now we have two ideas, we have to figure which one to pick and you go to market. So time-consuming, expensive. And my creativity from 12 got shrunk down to just two actually coming live from the whiteboard, right?

00:52:40

Now we replace that. It’s like there’s so much generative AI there. Just log into Midjourney and say here are my 12 ideas, and in 12 seconds it will actually visualize them, it will make them real. I need a bunny hopping around, Happy Easter, bunny hopping around and I with chocolates strewn over and there’s like a little rainbow and two clouds. Boom. Done. And so something where I could only visualize two of my 12 ideas. I can visualize 15 if I wanted to and I can see what they look and feel and experience like, and by the way, I’m like, oh, I like, so my creativity suddenly exploded. So what I enjoy as a human is not the process of building the graphics. I don’t enjoy that. What I enjoy is coming up with really interesting ways to connect to another human at the other end. And that’s like a really great metaphor for the kinds of things the AI will enable. Like right now, I cannot answer the unknown-unknowns. I simply can’t, I don’t even know what to ask. I don’t even like, I’m complete. So we had this campaign where we had spent like 60 million over time and we kept failing at it and I couldn’t figure out why we kept failing at it. By the way. We did segmentation, we did statistics, and we did upper and lower control limits, and we did everything and we couldn’t figure out why we keep failing. And it’s like a very frustrating experience. Instead we just hand it over to a machine learning algorithm and say, “Hey, here’s three years worth of data. Figure out why we suck. I don’t even know what to tell you why we suck. We just suck.”

00:54:18

And the thing is, like the handful of days later, it comes back with like this amazing decision tree and the first two things it says are the most important to getting this right. I hadn’t even thought that they should be in the arena of play. I hadn’t even thought about it. So what happened? Instead of being frustrated, this algorithm’s helped me find things that actually matter. And now I’m having more fun figuring out how to solve those problems. These are sort of two really different examples of how I see machine learning as an accelerant to me being able to do things I enjoy a lot more freeing me up for doing that, versus sort of the mundane stuff of figuring out how do I create few more reports in R. Power to the people, but I enjoy other things, in addition to that. 

00:55:07

So I think machine learning is going to going to be, going to be something across multiple spheres. I spent actually a bunch of time with on a project four years ago figuring out how to use data to figure out if somebody’s having a stroke. At the moment, you take a whole bunch of brain scans and then a human has to analyze it and on average it takes them 22 minutes or 24 minutes to analyze all that data and figure out if you’re having a stroke or not. Not, but the problem is in if you are actually having a stroke at minute eight your brain starts to starve of oxygen. So by the time I can come back with the answer, if you are having a stroke, some damage is already occurred. Now you feed those brains can, as they’re happening into a machine that is a machinery algorithm and with far greater accuracy as a human, it can predict you’re having a stroke. Now we’ve solved a problem that would’ve caused damage if the human was involved. Now we can minimize that stuff. For me, those are indicators of the direction we’re heading in. And I’m, I’m sort of really excited about it. 

Jon Krohn: 00:56:13

You’re talking to a techno-optimist host on a techno-optimist show. So that kind of example really brings a big smile to my face. You talking about Midjourney being used to be able to create 12 creatives for an ad instantly, as opposed to creating two creatives that take several days to get back to you. That gave me an interesting idea, and I wonder if this is already happening where we could today in marketing be having custom never before-seen creatives generated on the fly for an individual audience member. 

Avinash Kaushik: 00:56:51

There is nothing stopping us from doing that. I think it’s just figuring out how to [inaudible 00:56:56] rig the system at scale. So we, to be able to do that. 

Jon Krohn: 00:56:58

Exactly. It has to- 

Avinash Kaushik: 00:56:59

But, but at the moment- 

Jon Krohn: 00:57:00

Boundaries. 

Avinash Kaushik: 00:57:00

You’re really right. The first decade or so of the web digital, we were also excited that we could do personalized web pages, personalized creative in-advertising. And that dream died an ugly death because let’s say a million people come to SuperDataScience podcast website every month. And we wanted to create, let’s say for a million people, hyper-relevance would be a hundred thousand different types of the same page. That’s fine. A hundred thousand for a million is great. The problem is you cannot create a hundred, you cannot create a hundred thousand pages. You simply can’t. And so but going, going forward. 

Jon Krohn: 00:57:49

For, for our listeners where I’m not even going to cut an editor because it’s so funny, but Avinash thought that he’d scared my dog. He’s sitting on a chair next to him. But in fact it’s just that the dog walker, our dog walker Janine has arrived to pick him up.

Avinash Kaushik: 00:58:03

We should leave it in. This is good. Good stuff. 

Jon Krohn: 00:58:05

We’ll leave it in. It’s all organic.

Avinash Kaushik: 00:58:08

So I think that, that the challenge with making personalization was just content. We couldn’t produce that much content. But now with AI, both on the text as well as creatives it is actually entirely possible for us to create. And I think the thing we have to explore now is what kind of things do we want to personalize? What will people be most comfortable with and what’s going to be delightful versus not like, wait, wait, what? So I think we have to think those kinds of things through as humans and then build an experience. So for example, one thing that is very successful is, let’s say I’ve visited the website of Expedia three or four times at the end of it, Expedia’s homepage actually completely shifts to vacations on island resorts because it’s the only thing I’ve shown any proclivity towards. 

00:58:55

And so all the mountains are out, all the historic destinations are out, everything’s out. And that is good personalization. Because you understand and now you can start, create, using AI to create all kinds of really interesting text or creative to make that even more relevant to me. So that’s like a really great example where it would totally, totally doable, totally doable now and, and actually put in market. But on the other hand, if it’s like, oh we’re going to use something Avinash did six months ago and now we’re going to make him a creative and it’s no longer any relevant to me that would annoy me. So I think we have to think those things through. 

Jon Krohn: 00:59:32

Yeah. There’s pieces to figure out. Yeah. 

Avinash Kaushik: 00:59:34

We have the data and the tools now to actually do personalization in a way that we’ve never had. 

Jon Krohn: 00:59:38

Exactly. Yeah. It’s the kind of thing I wasn’t at Omnicom for very long, but it was the kind of thing that there was a, it was this recurring thing. Where people would come to you and say, yeah, you’re the data scientist here, how can you make this happen? And I was like, there’s no way. What are you talking about? And now you can do it. 

Avinash Kaushik: 00:59:55

You can do it. 

Jon Krohn: 00:59:56

Absolutely. And now, so now it’s, it’s getting all those guardrails in making sure that you’re not annoying people, making sure that you’re not offending people. That, that you need to be confident about. Awesome. So really exciting. I’m looking forward to seeing how these kinds of tools continue to change marketing analytics in the future. So we’ve, I mean we’ve now, with this kind of, this idea of Midjourney custom creatives, we’ve gone a little bit into the future into what’s possible in marketing. Do you have any other ideas of what could be unfolding over the coming years? What’s exciting in machine learning and AI and marketing? 

Avinash Kaushik: 01:00:30

So one of the, one of the easier answers in, in marketing is, how is this channel performing? Or how is email marketing doing? Or how is my website or store doing? I call this siloed answers. So in individual silos, it’s, it’s not that hard to get an answer around effectiveness and efficiency. So that’s easy. What is, what is harder is to say, how do does my digital presence work with my retail stores? You’re asking a harder question. Or you can say, I spend a million dollars on Facebook, million dollars on TikTok, million dollars on Google. Do I need to spend $3 million on digital marketing? Actually the answer turns out no, you can spend 1.7. Now you’re beginning to ask portfolio questions. So what I’m really excited about machine learning is in the past we were not able to answer portfolio questions with a degree of cleverness required because the type of analysis you need to do, the amount of complexity that is hidden inside the data you have to unpack is extraordinary.

01:01:34

No matter how big a brain you have and how experienced a data scientist is there, it’s like you’ve got in a very simple problem, you’ve got seven or eight dimensions being influenced by 60 variables. I mean, it is just, and that’s like a simple one. The complexity explodes as you go to portfolio-level decisions. And what I’m most excited to do invent over the next two years, let’s say very short term horizon, is figure out how to use Bayesian Belief networks and figure out how to use some of these other algorithms to make portfolio-level decisions in a very clever way. So a very simple example is some of the most sophisticated answers you need to do in marketing analytics for CMOs, CEOs, and CFOs is using done using something called media mix models. You understand across a whole bunch of things what’s actually attributing how to attribute value to the activity that you’re doing across owned, earned, and paid activities you’re doing.

01:02:29

And media mix models used to be done using traditional statistics and priors and all those kinds of things. And what we did in our work and what we do at Croud is actually hand all that over to machinery algorithms. And say, “Hey, you figured these things out.” And the complexity they can analyze and absorb is extraordinary. And not only are they able to answer the portfolio question, which is, “ah, your marketing added 27% of net incremental sales at a cost per incremental sale of $50”, let’s say, but by the way, here’s a contribution of television, here’s the contribution of email, here’s the contribution of billboards, here’s the contribution of Google. 

Jon Krohn: 01:03:09

The incremental contributions of that. 

Avinash Kaushik: 01:03:10

Exactly. That is just amazing. And in the cherry on the top, what we’ve done is we’ve built like a simulator on top of all this data so we can look forward and I can go in and say, I would like to sell 10 million phones and I want my brand consideration to go up six points. You can put in simulation. The simulation runs hundreds of scenarios. You tell it how much risk you’re willing to take. I say, I would like medium amounts of risk and it’ll spit out and say, in order to accomplish these business goals in the future, you need this much money. Here’s how you should allocate it. These are the kind of things you can do. That is transformational because you’re not relying on human gut anymore. It’s science telling you what you should do. It doesn’t mean that as a opinionated CMO, you can’t say, no, no, no, no, 10 more million on TV. fine. But at least your starting point is not all gut based. 

Jon Krohn: 01:04:03

Yeah. 

Avinash Kaushik: 01:04:03

It’s based on the sophisticated algorithm.

Jon Krohn: 01:04:04

The opinionated CMO likes their mom seeing the ads on TV. 

Avinash Kaushik: 01:04:09

By the way. Not, not, not, you’re joking, but in some instances, it happens. The CMO has insisted to me, radio works, we’d analyze all kinds of data, it didn’t work. So what we would do is we would we bought advertising on the radio stations along the commute of the CMO so the CMO would hear radio ads. But there was no proof it worked anywhere else really. [inaudible 01:04:31] So it’s not a joke. We do that sometimes. 

Jon Krohn: 01:04:34

Oh yeah. No, I believe it. It’s amazing how, you know, people unlike machines, or as far as we’re aware unlike machines, humans have these, you know, these very touchy feely things about, you know, their personal experience and they want to hear their brand on the radio on their commute. And they want their mom to say, oh, I saw your new ad and you look great in it on TV. And yeah, so people with big budgets, they, they do make those kinds of decisions, even if it’s not purely rational for the business. 

Avinash Kaushik: 01:05:04

I said, look, today, if you look at an average American Fortune 500 company, maybe 30% of the marketing allocation execution is data influenced, the rest is not. My goal is very simple. Flip the thing, it’s about 60 to 70% of the marketing should be data influenced. Then you’ve still got 30% of the spend, which you can do based on opinions, hypotheses, you can run experiments, you can try crazy stuff, and you can make sure that your best friends and your moms see your ad. That’s okay. That’s okay. All my, my hope is sort of very humble because you’re right. You, you got to, you have to figure out how to solve that human problem. Sometimes I’m way too analytical as somebody on the spectrum, you know, it’s like I want to follow the data and logic and I’ve sort of trained myself to say, no, you have to be perceptive to these kinds of human needs. So I set a target, 70% data influenced, or if you’re really good, 80% data influence, that’s still 20% of the budget to try all kinds of crazy imaginative things and innovate. 

Jon Krohn: 01:06:06

Awesome. Really exciting future ahead in marketing as well as in every industry. Yes. Because AI will touch all of it. Very exciting. So let’s touch on your book Web Analytics 2.0, which I opened up with saying, you know, that’s what introduced me to you in the first place many years ago. And so in that book, you provided insights into effectively measuring and analyzing web data. How have these concepts evolved since its publication? So the book came out in 2009. And so now we’re recording 14 years later. Oh. And I should mention for our listeners, you made an Easter reference there. And so we’re recording over Easter weekend, here in New York. And so, yeah. So anyways, so time is continuing to pass, so you guys are in the future listening to this. But yeah, we’re, we’re way in the future beyond the Avinash Kaushik who clicked submit on that a book manuscript back in 2009. So what’s changed in web analytics since then? 

Avinash Kaushik: 01:07:08

Yeah. Oh, oh, it is completely transformed. I think one of the, one of the, I I’ve told the publisher like, stop selling the book, but they insist cause I’m like, I have so much more I’ve done, but, but between Web Analytics – An Hour a Day, my first book, and Web Analytics 2.0 of the second book one of the things I deliberately did with Web Analytics 2.0 is made it a book about how you should think. So there’s a lot of concepts and how do you think about analytics, the approaches you should bring, the frameworks you should use. I think that the reason the book keeps selling and the publisher won’t stop publishing it is because it teaches you how to think about analytics. Some of the tools I mentioned in the book are dead. Some of the things screenshots I’ve used from, from things, they, you can’t do them anymore because those things don’t exist. 

01:07:56

But the way you think continues to be relevant, which is what I’m sort of really proud of. And so I said like, if I write books, I should just write books about how to think because they can survive a lot the time rather than a user manual for a tool. The tool’s going to go away. The thing, a handful of things that I think have accelerated as I reflect on that is I couldn’t think of doing machine learning anything in 2009. Like I couldn’t, I only started working with machine learning like eight, nine years ago. So that has changed. Our ability to collect and understand way more data exists today than it did at the time. So the possibilities have expanded exponentially. But the thing that, you know, I’m really sort of super excited about is if you did web analytics in 2009, you perhaps sat in a silo reported to the director of digital and had mediocre influence inside your company. 

01:08:51

People who do web analytics now are in charge of pretty much the most amazing existence for any company because it covers all digital, the entire business is web now, basically, even if you have retail stores. and so what has changed, I think that I am most excited about is if you are analyzing digital data now, web data, you are sitting in meetings with the CMO, you’re influencing this chief strategy officer to figure out what the next set of strategy should be for the company. So the amount of influence that has grown over time amount of things you can drive to actually move fundamental profitability business is just transformed there. I’m sort of so really, really excited about that. 

Jon Krohn: 01:09:35

Awesome. Yeah. That is really exciting. And that’s a, that’s a really great way of framing just how people not just in marketing, but people who are doing analytics, doing machine learning, we used to be kind of off in a corner and now it’s front and center. Where, you know, you mentioned there, the CMO, the CSO, often the CEO is saying today, what’s going on with all these things that are changing? And the CEO wants to know the AI strategy of the company directly. 

Avinash Kaushik: 01:10:07

No, that’s exactly right. That’s accurate. and it touches every single aspect of the business. I did a project with a very large South American company. They run a whole bunch of supermarkets. And the problem I solved with them, this is two years ago, is we were trying to figure out how to make sure that the right product is in the right supermarket and make logistics across South America more efficient. And we handed over what should go in which truck every single night across hundreds of supermarkets in these four countries onto machine learning algorithms. It reduced the logistics costs and profitability in the company by like 6%. By the way, that is insane. It is insane by just handing something to an algorithm. And customer delight improved because the right products went in the right shelf, in the right store every single day. So this just tells you it’s not just web analytics or marketing analytics, like every part of the business where you have tons of data and you have extraordinary number of variables to balance … hand it over to algorithm, it can do it better than you and enjoy the ride. 

Jon Krohn: 01:11:13

Nice. Yeah. Great summary point. So a few more things from your book that I’d like to touch on with the time that we still have. So for example, you discuss a big part of your book is your intent-centric framework. So there’s four clusters of intent. Tell us about those. I think that’s a really good example of one of the kinds of insights from your book that is equally relevant today as it was in 2009. 

Avinash Kaushik: 01:11:41

Yes, yes, so, I love, I love this idea. So if you’re doing advertising in particular, it’s not uncommon for the VP of X or Y to say, “Hey, we would like to reach women 18 to 34 with this kind of a message”, so on and so forth. That’s, so, it’s called demographic-psychographic-centric marketing. It’s say, people who earn this much money stay in this area, yada, yada, yada. Now the problem is, as you create those, those requests, maybe no woman 18 to 34 wants to hear from you. Maybe what you’re saying is not relevant to any of them. And yet throughout its history, marketing has been demographic-psychographic-centric. It doesn’t understand what you want. It has no signals around it just says, oh, you’re a man. You like Lululemons. So you know what, I think you should see ads for Ford Mustang. 

01:12:30

I don’t know what says that, but that’s kind of how it works. It has no sense of you intend that you actually are very passionate environmentalist, I I’m assuming and you like hybrid cards, let’s say. Right. It has no sense of it. So one of the things that the web has enabled is for us to understand intent based on expressed behavior. So we understand the, some of the content that you might be consuming. We understand what you had searched for. All these things are throwing off behavioral signals that can read in as intent. So the starting point isn’t saying, I want to reach all 18 to 34-year old women. Right. The idea is to say, “oh, I’m in the business of selling these products too and can I, can you help me find the intent where the message or a story about those products will be relevant to that audience?” 

01:13:22

And it might turn out that it’s only 10% of women, 90% of it is men. You would never know that unless you analyze intent. So the simplest example of intent is you search for something in Google, right? There is your intent, answer that, AdWords does that really well. Right? But let’s say you read certain pages in the New York Times, consistently, that’s an expression of your intent and what you’re interested in. Or you go read reviews about thing X or thing Y and there’s an expression of intent. So I’ve broken things into four clusters of intent we should solve for. And the first one is your largest addressable audience. And that’s your See intent cluster. And it’s the largest addressable, that word addressable says, you have to analyze intent to understand that this person at the other end, for them, what I have to say is relevant, and that’s going to interrupt people and annoy them.

01:14:18

The next cluster of intent is Think, which is the largest addressable qualified audience with some commercial intent. So the reviews and staff. And then you’ve got people, another cluster with Do. People who at the moment are in market and they’re showing off all this intent around wanting a product right now. And so that’s a third cluster. And the one that marketing most ignores is the Care cluster of intent, which is your extra loyal customers. The most profitable customers are always extra loyal that almost maybe one or 2% of marketing is directed towards them. Right. And that is insane to me. It doesn’t matter. So See, Think, Do, Care. Those are the clusters of intent. And when you think about the marketing you’re doing, you have to figure out what cluster of intent am I solving for? Am I using the platforms I have access to do marketing where I can identify these clusters of intent and ensure that your marketing is relevant to those people? 

01:15:17

So if you’re solving, a lot of people say, I want to do social media. Okay, what cluster of intent are you solving for? Because if you run Buy now, Buy now, Buy now ads on Facebook or TikTok, you, all you’re doing is harming your brand. Right? So people are going to think moron. Right? It is not the intent that’s being expressed there. TikTok, Facebook really good at the See cluster of intent. And you’re like, ah, so it can’t be Buy now, Buy now, it has to be something else. Oh, what is it? Oh, what’s the intent? Now you do what is known as hyper-relevant native advertising. So in this platform, it’s very unique. You have to express things a certain way. It can’t be yelling at people Buy now, Buy now, Buy now, here’s a coupon. They’re not there to look for a coupon. Now there are other places you should do coupon advertising. So, I hype, encourage people to think about what clusters of intent they want to solve for, and then do kind of marketing and advertising in that cluster of intent that is most relevant at that point. There is a point at which you tell people, here’s a coupon, but that’s a different cluster of intent. Don’t mix them up. You, when people execute, Do advertising on Think platforms, there’s very little ROI.

Jon Krohn: 01:16:29

Right, right. 

Avinash Kaushik: 01:16:29

Likewise, See ad, See marketing on Do platforms is insane as well. So it’s a really good way to think about relevance, intent, and make sure that marketing delivers highest possible ROI. 

Jon Krohn: 01:16:43

Awesome. Crystal clear, the four key clusters of intent: See, Think, Do, and Care. And we need to be approaching each of those differently in how we market to those different kinds of audiences. So that’s a perfect example of the myriad concepts that you’ve invented over your career. So let’s talk a bit about how you have grown that career. And so we’ve talked a lot about growing businesses with analytics, but let’s switch to personal growth. So what are the most significant challenges and opportunities you’ve encountered in your career that have helped you grow your skillset and expand your horizons?

Avinash Kaushik: 01:17:25

Yeah. Oh, you’re very kind. And that’s, that’s a, that’s a really good question. One of the things and I’m sure this is actually exactly true for you, is if you want to keep doing something for a very long time, you have to figure out how to keep growing and learning. You cannot write a blog or a newsletter or podcast or hundreds of episodes without constantly investing in your growth so that you are relevant, you know what’s going on. So for me, one of my rules is that every single week I set aside four hours to learn something new. And it could be coding, it could be actually following a chocolate blogger. I’m not a great cook. I just love the way that they talk. I love the way that they express things. And I might send it to my wife and say, oh, I like these brownies or whatever. 

01:18:09

Right? But invest in learning something new. Make time for it. That’s been, that’s been good. Then constantly wanting to learn and experiment with tools and algorithms and things in our space. That’s also very clear choice. Actually, the choice that is the more non-obvious one in terms of self-learning is just learning how to be a more effective human being in different situations. So one of my, one of my practices I learned from Steve Bennett, I used to work at Intuit, he was the CEO. He was like a really big feedback junkie. And he’s like, you should always be asking people for feedback. You don’t have to listen to everything, only ask feedback. So I always ask people feedback. So I’ll get done with a meeting and I’ll send a quick note and say, could you tell something that I did that was very effective and something that you wished I did that I did not? 

01:18:59

So I’m always asking people for feedback and it’s a great source for me to figure out how can I be a more effective colleague? How can be a more effective leader? How can I be more effective this? Because at the end of the day, like I’m in the business of persuading people to do things differently. And in order for, to me to do that data is necessary, but not sufficient. That was a big growth thing for me because it’s like, we got data, it’s [inaudible 01:19:28] let’s go. Actually, it’s not, it takes a lot more than data to persuade people. And so investing in myself, so I’m more perceptive about these things. Am I more effective team participant, am I a more effective leader, actually has ensured that the kind of things I want to get done happen at a faster scale and bring everybody along. 

01:19:45

And it’s not, not, it hasn’t been easy for me. I made tons of mistakes. One of my biggest mistakes as I worked for Silicon Graphics and I was this global project manager and launched massive petabyte data warehouse, updated in real time from 14 countries. I was so proud of it. But I went to DirecTV and after like a few months they were like, I was not as effective as a leader. And in the moment somebody said to me, oh, you manage people like resources. I was like, oh f***. Sorry! I [inaudible 01:20:15], yes, it’s true. Bacause I grew up as a project manager the previous few years. I had line items and net charts and actually people are not resources. They’re people. And there was a mistake I made. Is, I had to learn how to unwire that from my system.

01:20:31

And it has made me a more effective leader. So what, what I urge our audiences, yes, look, learn tools and algorithms and play with Midjourney and ChatGPT4 and all that stuff, but don’t, don’t ignore this really big piece of the puzzle about what makes you more, what could make you a more effective team participant. What could you make your more effective leader? And for me, I’m an agent of change. I walk into the room wanting to shake things up. That is my job. I’m blunt. So it’s even more important to figure out how to do those things well because I walk into a room to challenge status-quo. And please consider investing in those things as well. It will make you a more effective person. 

Jon Krohn: 01:21:11

Yeah. Great. I love both of those things. Setting aside four hours a week to learn something new. Sounds brilliant. I think I need to be learning more about chocolate. And yeah. Being effective, dealing with humans, asking for feedback. I think this is so important. I was mentioning to you before we started recording that Ben Taylor well-known data scientist, he recently started his own podcast called Atomic Soul. And he was here last, a week ago at the time of filming, recording one of the first episodes of that show with me as the guest. And I hadn’t been a guest on a show in years. But one of the takeaway points that I had for the audience was very similar to this, which is that we, as we become effective in business, we often learn that we need to be asking for feedback in business. 

01:22:04

And we do like formal 360’s and so much useful information comes out of that. And so my takeaway point in this Atomic Soul podcast was that I encourage you to glean that same kind of feedback in your personal relationships as well. There’s no point in there being, it, it’s amazing when you, when you ask somebody this question, what was effective, what wasn’t? You know, what could I be doing better in our personal relationship with your family members, with friends with romantic partners? It’s amazing how there can be something that they just, they, they, they, they couldn’t find the right time to bring something up where, and once they do, you’re like “Oh, I can do that. Absolutely. I had no idea that that was annoying you or that that was a problem. I don’t care about that thing. I don’t hold onto that. It’s not a part of my ego.” Yeah. 

Avinash Kaushik: 01:22:59

No, it’s good, good. My wife does my performance reviews. Keeps me humble and growing. So I validate this advice. 

Jon Krohn: 01:23:11

Yeah, yeah. Yeah. Very funny. Yeah. Perfect. So we’ve obviously talked a lot about your, your book Web Analytics 2.0 in this episode, and you, it’s not your only book, but I was wondering if you have a book recommendation beyond your own works for our audience? 

Avinash Kaushik: 01:23:28

Yeah, I mean, there’s plenty of sort of books on in the space that I could recommend, but I remember I talked about this idea that one of the big things you want to invest is stretching your imagination, understanding possibilities. So I’ll recommend The Three Body Problem by Cixin Liu. 

Jon Krohn: 01:23:44

Oh yeah. 

Avinash Kaushik: 01:23:45

It’s, it’s a sci-fi book. I’m, I’m a big sci-fi fan and it is a modern sci-fi book in the sense that, you know, the Dune series or the Old Man series and they’re all good. But what Cixin Liu has done really good job in The Three Body Problem is used modern science, modern math to imagine a future and it’s a three part series. One in three are exceptional, but the way that he imagines what could happen with our contact with alien civilization, it just, it’s just so exceptionally well done with modern science and it’ll keep you sort of guessing about the threads that are going to come out. And it, it really made me think very deeply. So I recommend the three [crosstalk 01:24:29]

Jon Krohn: 01:24:29

Nice. Yeah. That’s, that’s the kind of book that you hear people be very passionate about recommending. So it sounds like a great one. And then, so final question for you, Avinash is how people should follow you after this episode. We’ve talked about some of those already. You’ve got the Occam’s Razor blog, you’ve got the TMAI newsletter. Where else should people be following you, social media anywhere? 

Avinash Kaushik: 01:24:50

Oh, yeah, I mean, I sort of stopped being on Twitter the day Elon made a bid for Twitter. So I don’t know what to do for my 200,000 followers there. But LinkedIn is a great place. I’m on LinkedIn every other day at least writing short articles. And then they can just sign up for the newsletter because it’s free and people can sign up for it and I write it every week and it’s a lot of fun. 

Jon Krohn: 01:25:14

Nice. All right, Avinash, thank you so much for taking the time to be on the show today. It’s been amazing for me to get to spend time with somebody that I’ve idolized for so long. And yeah, maybe we’ll have the opportunity to have you on the show again in the future to hear how things are developing. 

Avinash Kaushik: 01:25:29

Great. Thank you so much for having me. 

Jon Krohn: 01:25:36

What an amazing experience to meet Avinash and record this exceptional episode with an icon like him. In today’s episode, Avinash fills in on how brand analytics is more of an art. It evokes a particular ethos and data on it are typically measured in kilobytes. In contrast, performance analytics is more of a science. It can be directly tied to revenue and profit goals and data on it can be measured in terabytes. He talked about how he loves using D3.js for building beautiful charts and conveying concepts to key stakeholders, how he uses R and Tableau to explore data hands-on and discover unknown-unknowns. How machine learning and marketing enables the discovery of small audience segments that are most likely to convert. How AI like Midjourney unlocks creative capabilities, meaning a dozen creatives can be explored instantly instead of a couple of creatives days later. 

01:26:24

And he talked about how techniques like Bayesian belief networks will increasingly enable portfolio-level marketing questions to be tackled in an automated way. As always, you can get all those show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Avinash’s social media profiles, as well as my own social media profiles at www.superdatascience.com/677. That’s www.superdatascience.com/677. 

01:26:49

I encourage you to let me know your thoughts on this episode directly by tagging me in public posts or comments on LinkedIn, Twitter, or YouTube. Your feedback is invaluable for helping us shape future episodes of the show. All right. Thanks to my colleagues at Nebula for supporting me while I create content like the SuperDataScience episode for you. And thanks of course to Ivana, Mario, Natalie, Serg, Sylvia, Zara, and Kirill on the SuperDataScience team for producing another enriching episode for us today. 

01:27:13

For enabling that super team to create this free podcast for you we are deeply grateful to our sponsors whom I’ve hand selected as partners because I expect the products to be genuinely of interest to you. Please consider supporting the show by checking out our sponsors’ links, which you can find in the show notes. And if you yourself are interested in sponsoring an episode, you can get the details on how by making your way to jonkrohn.com/podcast. Finally, thanks of course to you for listening. It’s because you listen that I’m here. Until next time, my friend, keep on rocking it out there and I’m looking forward to enjoying another round of the SuperDataScience podcast with you very soon.