SDS 679: The A.I. and Machine Learning Landscape, with investor George Mathew

This is episode number 679 with George Matthew, Managing Director at Insight Partners. Today’s episode is brought to you by Posit, the open-source data science company, by AWS Cloud Computing Services, and by Anaconda, the world’s most popular Python distribution. 

00:00:20

Welcome to the SuperDataScience podcast, the most listened-to podcast in the data science industry. Each week, we bring you inspiring people and ideas to help you build a successful career in data science. I’m your host, Jon Krohn. Thanks for joining me today. And now let’s make the complex simple.

00:00:51

Welcome back to the SuperDataScience podcast. Today I’m joined by the illustrious AI investor, George Matthew. George is Managing Director at Insight Partners, an enormous New York-based venture capital and growth equity firm with a $100 billion in assets under management that has invested in the likes of Twitter, Shopify, and Monday.com. George specializes in investing in AI, machine learning, and data scale-ups, such as the gigantic enterprise database company Databricks, the extremely fast-growing generative AI company, Jasper, and the popular MLOps platform Weights & Biases. Prior to becoming an investor, George was a deep operator at fast-growing companies such as Salesforce, SAP, the analytics automation platform Alteryx, where he was President & COO, and the drone-based aerial intelligence platform Kespry, where he was CEO & Chairman. Today’s episode will appeal to technical and non-technical listeners alike. Anyone who’d like to be brought up to speed on the current state of the data and machine learning landscape by a razor chart expert on the topic. 

00:01:50

In this episode, George details how sensational generative AI models like GPT-4 are bringing about a deluge of opportunities for domain-specific tools and platforms. He then talks about the four layers of the generative AI stack that supports this enormous deluge of new applications. He talks about how RLHF – Reinforcement Learning from Human Feedback provides an opportunity for you to build your own powerful and defensible models with your proprietary data. He talks about the new LLMOps field that has emerged to support these suddenly ubiquitous large language models, including generative models. He talks about how investment criteria differ depending on whether the prospect of investment is seed stage, venture capital stage, or growth stage. And he talks about the flywheel that enables the best software companies to scale extremely rapidly. All right, you ready for this captivating episode? Let’s go. 

00:02:45 George. 

George Mathew: 00:02:46 Jon. 

Jon Krohn: 00:02:46

Welcome to the SuperDataScience podcast. Thank you for coming in person to record with me in New York. Insight Partners is based in New York. You recently moved back to New York, you grew up here. It’s nice to have you around. 

George Mathew: 00:02:58

Thank you, Jon. It’s a pleasure to be here. Happy Friday. It’s a beautiful day in our neighborhood and looking forward to having this conversation with you on all things data science. 

Jon Krohn: 00:03:07

Yeah. Our listeners could be on any day of the week and this episode comes out on a Tuesday, but we are indeed recording on a Friday, and it’s a beautiful day in New York. I’m sure you’re itching to get out into the Sunshine.

George Mathew: 00:03:18

Only after we spend some good time together talking about all things fun and data science today. 

Jon Krohn: 00:03:22

Nice. Yeah, and so we’ve known each other for about a year. I did, I hosted a talk at the ScaleUp:AI conference last year with Clem Delangue- 

George Mathew: 00:03:31

Yeah. Thank you for that. 

Jon Krohn: 00:03:32

Hugging Face. Yeah, it was my pleasure. And so that actually is a SuperDataScience episode that people can check out. It’s episode number 564 and yeah, the CEO of Hugging Face, lots of people talking about hugging face these these days. If you’re building large scale language models, you’re almost certainly using Hugging Face. It’d be hard not to be. So, that is a great episode. And the ScaleUp:AI conference itself, that was the inaugural one, right, last year? 

George Mathew: 00:03:57

That’s right. It was the inaugural one out of New York. It’s amazing to see how much more innovation is happening in New York around all things related to data science, machine learning, artificial intelligence. Clem himself is based here with Hugging Face. It just tells a tremendous story of what the innovation ecosystem, particularly around data and AI is in New York even today. 

Jon Krohn: 00:04:18

For sure. It isn’t all-West Coast anymore. 

George Mathew: 00:04:20

It certainly isn’t. 

Jon Krohn: 00:04:21

You spent a long time there. 

George Mathew: 00:04:22

15 years, 15 years, yes. 

Jon Krohn: 00:04:24

So, it goes to show that if somebody of your stature can see value in being here in New York it makes me feel more confident about being here in the AI scene myself. 

George Mathew: 00:04:34

We’re all having fun together right now so.

Jon Krohn: 00:04:37

So, you are here as a managing director at Insight Partners. It’s a venture capital and growth equity firm that is behind household names like HelloFresh, Twitter, Spotify is another big one, Monday.com. And more recently you’ve been investing large amounts in prominent AI and data analytics startups like Weights & Biases, Jasper and Databricks. There’s countless more. I wish I could reel them all off. When I announced last week that you’d be a guest on the show, I spent an hour or two drafting this very long LinkedIn post trying to make sure that I was comprehensive about all of the data and AI investments that you’ve made, and there’s at least a dozen. And the couple that I missed called me out. 

George Mathew: 00:05:24

I couldn’t help notice that a few founders saw the post and they mentioned “What about us?” But that’s, that’s a good sign. It means that there’s just a tremendous amount of activity, particularly in the data and AI space, and everyone’s doing some incredible work and we’re just happy to be part of that journey. 

Jon Krohn: 00:05:41

Yeah, and they we also got some great questions from audience members, which we’ll get to at the end of the episode once we get through mine. And of course, Serg Masis as often is behind all the best questions that will be asked today. So, thank you Serg for those. So, for people who don’t know, how would you describe venture capital and its role in the technology startup ecosystem? 

George Mathew: 00:06:03

Yeah, so I look at venture from and venture capital in particular from the lens of where it originally started. And the lens that it originally started was a view that incredible founders and entrepreneurs eventually go and give back to the experiences that they went through in the journey of building a business and scaling it out and coming back and investing into, you know, the next generation of founders and leaders. And for me, that was almost the purest and the highest form of venture capital in a lot of ways. And it was one that I really subscribed to in being a more recent venture capitalist most of my career was an operator, a builder, just been in software for my entirety of my working career.

00:06:48

And as I thought about where I would want to go in this sort of next phase of my own journey, what became very clear was that the data and AI space was just continuing to multiply in the scale of opportunities of incredible founders doing insanely amazing things. And instead of trying to pick one next thing that I could go do, certainly after the experiences at Alteryx and Kespry, I was like, well, maybe I can help a few founders in theirs respective journeys. And I actually had a lot of conversations with those founders before going into venture capital, and it was the confluence of those conversations with some of the key discussions that I’ve had with venture capitalists across East Coast and West Coast, and of course, my relationship with insight founder of the firm, just being involved with our series B at Alteryx and beyond, it just gave me a perspective that this was a moment that there was an opportunity to see this next generation of data and analytics companies originally. 

00:07:53

And then of course, to sort of focus on sharpening on artificial intelligence and machine learning that emerged particularly in this last half a decade. Has a lot more legs remaining to it, Jon. And that’s where I’m here to really serve the market, the founders and the opportunity. 

Jon Krohn: 00:08:08

No doubt you are preaching to the choir here and probably with a lot of our listeners as well. And it’s interesting how now even in the mainstream, it’s crystal clear how much opportunity there is with AI. People being blown away. This, you know, we talk about it probably too much on the show, but and too much in the world. Yeah. But this ChatGPT moment, and how it just showed people that there are mind-blowing things possible that, you know, I think people had kind of gotten used to, oh, well, you know, phones are getting a little bit faster. We get more pixels in a screen, but like, where is like this tremendous innovation, this world-changing innovation, and with something like ChatGPT, you see, bang, there’s this potential to make a huge impact across every industry and improve human life around the world. It’s really exciting. 

George Mathew: 00:09:07

Yeah. It is incredibly exciting. And the closest thing that I can analogize it to was, was when I graduated from college in the mid-nineties I was actually working in my undergrad days on computational sciences and specifically in the applied use of computational sciences back into biological systems, particularly neuro biosystems. And most of that was research work as well as my summer job was working on helpdesk. And I still remember back in 1994, I saw the NCSA Mosaic browser and my, my heart skipped a beat.

00:09:48

And I, and I actually called my mom [inaudible 00:09:50] remember this even back then and said, mom, I’m not gonna become a doctor. I’m going to go figure out what that is. Well, that moment, even almost 30 years ago, I can now sort of see it and play it forward to this moment after the internet, after mobile, after cloud. Now we’re seeing this tremendous wave of innovation that’s cresting as we speak. And I had a chance to talk to my mom about it and said, mom, we’re back here again. And she just laughed. And, and it’s that realization that our, our mothers and our fathers and our grandfathers and our grandmothers can understand the applicable use of AI at scale, Jon, that we’ve never seen in our own prospective careers as professionals in this space. And so it is for sure a moment that we should all seize upon and embrace and really sort of run to in a lot of ways. 

Jon Krohn: 00:10:53

Yeah. It’s wild. And it’s funny that you mentioned that because this most recent weekend I was back home in Canada with my mother and my grandmother and I got on my laptop and I was like, you’ve just gotta see this. And they were really blown away. And like, some of the things like, I’d give a little bit of context on like, this is kind of what my mom’s like, write a sentence where every word in this sentence starts with the next letter of the alphabet from A to Z.

George Mathew: 00:11:18

Nice. 

Jon Krohn: 00:11:18

And it’s like a crazy, like, and it did a really good job.

George Mathew: 00:11:22

Nice. Well, it kind of goes back to the key principles of software at scale, right? When you think about any emergent experience in software, you have to couple an incredible user experience, a UX with workflow, with something that’s very uniquely defined in the data and of course the model. And in this case, you’re seeing that converge in a tremendous way that we haven’t necessarily seen at consumer scale. I mean, look at the numbers, right? of just active users on a product like ChatGPT, and of course all the derivatives that emerge, it just kind of gives you a sense that this is not only something that is profoundly important for the moment in time that it came to market, but also has a incredible sort of resonance in this space for years, perhaps even decades to come. 

Jon Krohn: 00:12:49

This episode is brought to you by Posit: the open-source data science company. Posit makes the best tools for data scientists who love open source. Period. No matter which language they prefer. Posit’s popular RStudio IDE and enterprise products, like Posit Workbench, Connect, and Package Manager, help individuals, teams, and organizations scale R & Python development easily and securely. Produce higher-quality analysis faster with great data science tools. Visit Posit.co—that’s P-O-S-I-T dot co—to learn more. 

00:12:49

And yeah, and so let’s dig into this specifically with a portfolio company that you recently invested in – Jasper. So, Jasper AI is this also extremely fast growing consumer application of generative AI. And so tell us more about it and specifically if you could tell us more about how RLHF, Reinforcement learning from Human Feedback allows these consumer products, these user interfaces to be so intuitive and useful to users. 

George Mathew: 00:13:23

Yeah, so just a little bit of background on Jasper as a product, as a company. So, when you look at the founding team of Jasper and the CEO Dave was very focused on this idea that a content marketer should have better tools in their hands. And that was the first and foremost problem to solve. Notice I didn’t mention anything about AI yet, right? It was the idea that a content marketer should have better tools to be able to do their work, to be better tools, to be able to publish that blog, to be able to write just highly optimized SEO content. And it turns out as Jasper continued their journey, what they realized was that when you introduce a generative model, in this case, they use OpenAI’s at that point GPT-3, and then subsequently now expanded the capabilities beyond just OpenAI. 

00:14:18

But when you used a generative model and you introduce that model alongside of the workflow in UX that was really targeted and tailored to the content writer, you can 10x that experience. And when you mention the idea of a consumer product, it actually turns out it’s a special kind of consumer product. It’s a prosumer product. And what I say that, is that just like you and I would pay for, say for instance, our own cell phones, regardless of whether our companies would pay for it, the same might thing happened with the content writers that were using Jasper. They didn’t even want their companies to pay for it. This was just a natural extension of how they live and work. And so this is what I sense is like a prosumer product that sits in between a lot of enterprises prosumer. 

Jon Krohn: 00:15:04

So, that’s like the idea of professional consumer where- 

George Mathew: 00:15:06

Correct. 

Jon Krohn: 00:15:06

Yeah.

George Mathew: 00:15:06

Yeah, absolutely. A professional consumer that has tools that they would need to use on a day in day out basis. And how do you really encourage the use of those tools? It was something that I was actually quite familiar with in the last generation of data analytics companies, because if you think about where the dominance of Tableau and Alteryx in particular, went in the call it prosumer market of data analysts and data scientists, turns out this became essential tooling for them, right? In a similar way, I almost view there’s opportunities for every persona that surrounds a enterprise as well as a prosumer experience, have software that’s very targeted. 

00:15:51

In this case, I almost noticed as we started to, you know, work with the Jasper team and just understand the use cases and how they sort of solve for the productivity of the content writer themselves. It was like a mech suit, Jon. It was like a 10x mech suit of how they can get their work done. And when you can do those 10x moments right, in terms of how people work, that changes, you know, the way software is consumed. And in this case, Jasper became one of the fastest-growing software companies in the history of SaaS. 

Jon Krohn: 00:16:27

Yeah. It’s wild. Can you give us some of the hard numbers? 

George Mathew: 00:16:30

Well, it’s still privately held, right? So, so I can’t give the hard numbers, but I will say that it’s near triple-digit millions in terms of where the scale of their recurring revenue is. And the opportunity for Jasper coming into market is a little less than two years old. 

Jon Krohn: 00:16:51

Right. So, in two years, approaching nine-figure ARR that is, yeah, that’s really, really, really wild. Giving OpenAI a run for their money. 

George Mathew: 00:17:03

Yeah. I mean, and interestingly, it was a company that Elise initially started as one of the first use cases of applied generative AI on top of OpenAI’s LM, right? And now it’s of course sort of expanded some of the capability of where it sources some of the large language models, but it certainly tells you a lot about what the opportunity looks like, not only for the build-out of the foundation model, but the application layers that are on top of it.

Jon Krohn: 00:17:32

And then the, going back to this RLHF idea, which we haven’t really explained to listeners, but by having this application now running for two years, Jasper can gather feedback from users and they can fine-tune their models to be better for this particular use case for content marketing.

George Mathew: 00:17:50

Yeah. It’s, this is an area reinforcement learning with a human feedback loop, Jon, that I’ve been looking at quite a bit in just about any business that’s started in the generative AI space, and probably likely any business that’ll come on a go-forward basis. And the reason is, if you look at where Jasper really progressed, it was the fact that it used the open AI model. And to your point, as the content was being written, the quality of the content that was being generated, the iteration of that content and how it is being used, that was the feedback loop to be able to improve model performance over time. And so that reinforcement learning in this case with a human feedback loop was quite important for Jasper in this case, to continue to improve the user experience and the attractiveness of how a content writer would work on a day-in, day-out basis. 

00:18:48

If we even just zoom out from beyond Jasper itself, and we look at like what happened with their introduction of GPT-4 into market, well, it turns out as some of us now well know the output that came out of ChatGPT aka GPT-3.5, those were the precise outputs that tuned the model performance from exactly that moment of getting a strong RLHF signal back into the delivery of GPT-4 So, what I look at now is, does that feedback loop exist alongside of access to some source of private data that can be uniquely differentiated? And so as an investor, as a builder who thinks about this space quite deeply, I don’t think it’s possible to have long-term sustainable advantages in any software domain without having that private data with some reinforcement feedback loop. And it might be fully human, it might be machine-driven, or it might be a combination of human end machine-driven feedback loops. 

Jon Krohn: 00:19:57

Yeah, it creates what investors often call the defensible moat, right? 

George Mathew: 00:20:00

Yeah. I mean it’s, it’s a, it’s a little bit of an overused term in the investor community. What is your mote, right? But in this case it’s very, very difficult for you to build a sustainable business inside of a generative AI context. And if we imagine that generative AI has an impact on all of software, it really means like a sustainable business inside of all of software without having some understanding of what your private data is, and the reinforcement learning surrounding that private data. 

Jon Krohn: 00:20:32

I think particularly with how quickly these AI innovations move, if you don’t have some data strategy like you’re describing that allows you to have this proprietary capability that other people can’t have, you’re at risk of being gobbled up by something like, you know, GPT-5 to come along in the future and have these kinds of generalized capabilities that eat into your specific niche. 

George Mathew: 00:21:00

It’s a, it’s a fascinating point that you just made, right? Because if we think about anyone who is built on top of GPT-3 and the GPT-3 style of models of its timeframe, they, you know, were able to build some pretty interesting businesses as we indicated some examples. Well, it turns out when 3.5 and certainly 4 emerge, it’s almost like you have to throw that out and rebuild on top of 4. And why is that? Because the generalized model is, in the case of this next generation of 4, is better than any domain-specified model that was built in 3. And to your point, it is likely gonna play out again in the emergence of the GPT-5 style of models, which, you know, it’s not just OpenAI, but others that are working on Anthropic, Cohere and others that are working on it, their respective models. Those will likely be fundamentally better as generalized models than what’s coming in the current finely tuned domain of, you know, what will emerge with GPT-4 style models today.

00:22:03

So, I think that part of this experience of being a founder in this space right now is you have to be willing to change, and almost sort of remove all of the preconceived notions of what worked and reimagine yourself again in this next iteration. Some of this will slow down, some of this will, the innovation will start to slow. One of the things that I’ve talked about in the past is where is this innovation gonna slow a little bit more. Well, it turns out that in every iteration of these large language models that are, you know, now tokenizing a fair degree of data, and now, you know, building, say for instance, a trillion parameter large language model, as you get to call it the next exponential scale, somewhere between 10 to 25 trillion parameters in a large language model, you’re actually running out of what we think of as human knowledge, the human corpus of publicly available data. 

00:23:01

So, over time, you’re still gonna have to rely on private sources of data, the reinforcement learning on a superpower model that comes off the shelf from, you know, somewhere in the range of, you know, right now a trillion parameters and certainly moving to an order of magnitude greater. And that in a lot of ways, I think is tremendously exciting for human civilization. Because we have this moment here where this model that has emerged with, with particular GPT-4, has the sum of a fair amount of our own knowledge encapsulated into it, and it’s a co-pilot to our lives, and it’s only getting better. 

Jon Krohn: 00:23:46

Yep. Yeah, it’s wild. And just thinking about ways that people can be, so you’re talking about how founders, like a company of Jasper, needs to be really aware of how these big AI models and next generation, like you’re saying Anthropic, Cohere, OpenAI, the GPT-5 generation of models that comes forward, how they could be able to eat into specific niches like Jasper has with content marketing. I know that something that’s important to you when you think about your investment decisions is the user interface. So, it isn’t just about the models in the backend, it’s also about creating a user interface, a user experience that is outstanding. And so it means that you know, somebody that probably buys you a bit of time to figure out how to catch up with the models on the backend, because somebody say, who’s using Jasper for content marketing, they’re going to, they probably love the experience of doing that so much that when GPT-5 comes out, they’re not like, oh, I should think about redoing, like just leaving Jasper and trying to do this with GPT-5 because they, there’s so much value add just in the experience that they have of Jasper.

George Mathew: 00:24:57

That’s right. And that’s one of the reasons why I mentioned that even as models continue to evolve and they will continue to evolve, there’s still bringing that model together with the private data surrounding the reinforcement learning that’ll improve the model over time and workflow and UX that supports the user experience in terms of how sticky they are to your product. Right? Because the models are pretty solid even today. And as you start to see this evolution of, you know, improvements in models, which will certainly drive a lot of competition in this space, I’m also seeing, you know, the fact that like the stickiness of like users, you know, continuing to work with a product, it’s about the awareness, it’s about the user experience, it’s about the compelling workflow that plugs into the rest of their lives.

00:25:53

And those things are as important as, you know, the model fidelity improving over time. It’s not like you can kind of choose one over the other. Like, you still have to have fundamentally great user experiences around software. It just so happens now you’re doing it in a world where model evolution is at this sort of fanatic pace we’re, we’re in right now. 

Jon Krohn: 00:26:16

Yeah. Fascinating. George. Great to be able to dig into your mind and hear about the key things to look for in creating an AI company or investing in an AI company. What kinds of startups are you particularly focused on? I mentioned that it’s data and AI, but maybe you can break that down for us a bit. 

George Mathew: 00:26:32

Yeah. Let me back up, because we spent a lot of time on what a generative AI application and what it takes to build some level of competitive scale in a generative AI application looks like. But part of this is really understanding what the rest of the hyper stacks below them are. Right? In a lot of ways, the generative AI application is the top layer of this hyper stack that I think about. But right below there, there’s a layer known as Machine Learning Ops and to a certain degree, some of the emergence of now LLMOps. The idea there is that you have a toolchain that is helpful for responsibly building models at scale, and whether that be taking first-party, third-party data and blending it together, whether that be doing the appropriate level of data labeling, whether that be feature engineering, bringing in this case experiment tracking, hyperparameter tuning sweeps and version controls, which is of course, the investment that we made in Weights & Biases.

00:27:28

And then of course, the model production and observability surrounding it in certain areas that we’ve also made investments in, like Fiddler, for instance, for model observability and decimation of models in the investment that we made in Deci. That LMOps layer has been built out for easily the last two to three years, mostly coming from the movement of DevOps. Now saying, how do you focus that, that idea for model production and model scale? That hyper stack is of course built on top of just everything related to the modern data stack, right? Just how did you organize the data? How did you get it all together in terms of data management, in terms of orchestration? How do you get reverse ETL back to like source systems? How do you ingest data from a ETL versus ELT standpoint into your modern cloud-native data warehouse? 

00:28:21

And of course, the beneficiaries of that have been happening easily for a good part of a decade. So, I view this in a lot of ways for most organizations as not just like, how do I jump to the forefront of generative AI? It’s how did I build a very well-tuned set of hyper stacks that enabled me to deliver the applications on top of the machine learning toolchain that is layered on top of the data life cycle. And the best companies in the world have thought through their data life cycle and their machine learning model-building capabilities before they go into jumping around and delivering a generative AI application on scale. 

Jon Krohn: 00:29:40

This episode of SuperDataScience is brought to you by AWS Trainium and Inferentia, the ideal accelerators for generative AI.AWS Trainium and Inferentia chips are purpose-built by AWS to train and deploy large-scale models. Whether you are building with large language models or latent diffusion models, you no longer have to choose between optimizing performance or lowering costs. Learn more about how you can save up to 50% on training costs and up to 40% on inference costs with these high performance accelerators. We have all the links for getting started right away in the show notes. Awesome, now back to our show. 

00:29:44

Yeah, there’s a, there’s a lot. Chip Huyen, who was a guest on our show not that long ago. She has written a great blog post at the time of recording and it had just come out. So, Chip was in episode number 661, and she is an absolute luminary on MLOps, and she wrote a big post on the specific needs associated with this term that I hadn’t heard before until just a minute ago, and you mentioned it, but makes perfect sense to me, MMLOps. When we, so just like following on from what you just said there, when we think about deploying these generative AI models, they are so massive that even the smallest ones, and at the time of us recording at least, there’s these like Dolly 2.0, Alpaca, Vicuna, GPT4All and the Lama architecture that a lot of those are based off of. Those models can be as small as 10 billion, 15 billion parameters, which means that we can fit them on one big GPU. 

00:30:50

So, not like on your laptop but on a server running a relatively large GPU, you can fit the model. And so, but even at that level of complexity, once you’re thinking about, okay, I want to have a consumer application that’s using one of these relatively small LLMs, it needs to be able to scale. You need to be able to have as many services as you need for however many consumers you have on using your generative AI application. So, the infrastructure needs to scale up, and because of the scale of it, there’s complexities that most data scientists, most machine learning engineers, most software teams haven’t had to deal with before.

George Mathew: 00:31:28

Yeah. It’s actually an area that I’m as excited by than just the buildout of the multi-trillion parameter large language model, which is, as you called it, like how do you create small form factor LMs and how do you create small form factor transformer models, right? Because it was even before the emergence of like small floor factor LMs, we started to see where a latent diffusion model, like what Stability did, was able to now introduce the things that Dolly 2 would’ve done on big-scale cloud infrastructure. You can now run on a MacBook Pro with an M1 chip and sufficient amount of memory to be able to generally design and create incredible output from an imagery standpoint. And of course then some of the layering of course, happened with video with what Runway was, for instance, doing. And so that actually taught us a little bit more about what we can do to have finally compacted models. And part of the compaction of those models is how do you tune it? How do you prune it? How do you look at the latent space of these models and understand where there are areas where you can fill in the gaps when it comes to hallucination by properly retraining the model. 

00:32:48

Certainly that’s something exciting that we saw with, you know, with the folks at GPT4All were doing this was more recent sort of weeks even. And so the reason I get really excited by that work that’s happening in the small form factor models is the deployability of those models, Jon can be so ubiquitous, right? Because, of course, they can show up on your laptop, of course, they can show up on your Raspberry Pi, of course, they can show up on your IoT devices. Of course, they can show up in places where you don’t have all this massive scale of compute for the inference. And I think in a lot of ways, that is as important of an area of innovation and market opportunity to build the small form factor models that’ll show up everywhere that software is deployed than just the cloud scale infrastructure where large scale inference is now being provided and massive gobs of GPUs are being chewed up by, you know, the providers of these LMs.

Jon Krohn: 00:33:54

Yeah. It is certainly an exciting space. So, if people want to dig more into your idea of this data stack and this hyper stack, we will provide in the show notes an article that you co-authored called The Next Stack Generative AI From An Investor Perspective. So, I guess you’ve just started to touch on this kind of on the generative AI stack. I don’t know if you can like talk us through the layers. 

George Mathew: 00:34:21

Yeah, sure. So, when you look at the top of that hyper sack, we spent a fair amount of time just unpacking how we thought through every element of what it would take to be able to build a generative AI capability, right? And so we, of course, spend a lot of time really understanding what these foundation models were, and it’s kind of layer one, right? Really just be able to provide the perception around images, video, [inaudible 00:34:48] acoustic waveforms, like all the mult-modalities that are going to and continue to emerge, particularly when it comes to the transformer-based architecture and specifically not only LMs, but also latent diffusion models in other areas where we’re seeing more than just the application of transformers back into text. And so that’s for us, the layer one, as we think about from a foundation model standpoint. 

00:35:14

The layer two is where I spent a fair amount of time in this discussion. What we believe is the need to deliver a domain-specific set of models for an industry, for a vertical, for a horizontal. One of the favorite examples of a domain-specific model that just emerged, I want to say two and a half, three weeks ago, was the Bloomberg- 

Jon Krohn: 00:35:42

BloombergGPT. Yeah. 

George Mathew: 00:35:44

Right. Fascinating, right? That like, Bloomberg has all of this incredible financial reporting data, and now they’ve trained a model at scale to be able to deliver that into a deep summarization, deep articulation, deep generation capability set off of their own domain-specific model. We’re gonna see a lot more of those. We’re gonna see that for all kinds of use cases. There’s no reason why a US torts law model wouldn’t emerge, right? There isn’t a reason why a global tax law, you know, model wouldn’t emerge. I think there’s a lot of discussion about sort of regulation, particularly around these models. I’m actually very excited to see some of the regulation, because honestly, I believe there’s a moment where some of these regulated industries can have very finely tuned focused domain-specific models that are regulated for the needs of that industry, like healthcare or financial services. So, that’s layer two. 

00:36:46

Layer three is the tools themselves. Right? And we talked a lot about the tools that are emerging. Most of them are predominantly defined by the machine learning operations tools, the MLOps tools. Some of those tools are now being targeted and focused on LLMOps. And layer four is the delivery of those applications. The Jaspers, the Hour Ones, the Deepdubs. These are applied full-functioning applications that are being delivered. So, my partners and I, Lonne Jaffe, Ganesh Bell, Nikhil Sachdev, and some of our team members and associates, Jenna Zerker, and Sunny Singh, put together a view of where this next stack will go. And that’s of course what you refer to and anything you want to dig in into that conversation, we’re happy to have that here as well as as well as online for anyone who’s interested in really just digging in with us. 

Jon Krohn: 00:37:51

Yeah. I mean, I’d love to just generally kind of think how, how this generative AI stack, how will it unlock new things? Like what are you most excited about this? Where is this gonna go? 

George Mathew: 00:38:06

Yeah. So, we talked a lot about what has been applied up to this moment. Now let’s think about where the unlocks are gonna occur that we haven’t quite seen yet. Right? And so whether Insider, another venture capital firm, invests into these opportunities? We’re profoundly excited to see what’s coming next. Imagine what computational generative architecture could look like, right? So, if we are building a skyscraper now, instead of using a traditional CAD/CAM piece of software, we generally give an input to say, build me a 35-floor skyscraper with 10 apartments that are 800 to 1200 square feet in size, 10 exits on each floor that has a floor plate of 25,000 square feet. And we have the technology now to be able to do that at scale.

00:39:02

Another great example that I’ve more recently seen is, if you look at the entirety of the code base that’s been delivered across many enterprises today, there’s actually a lot of sloppiness in terms of testing and scaffolding. Well, what if you can actually take a generative model and rescaffold and create all the [inaudible 00:39:22] necessary to invert what we understand in what’s been compiled or declaratively written in the source code. And suddenly you have a much better test bed of testable code as you’ve now brought it to market. I certainly don’t have to talk about where, you know, copilot has gone already. Right. I mean, that, that’s certainly a use case today. 

00:39:48

Like if you talked to any 5, 10x developer today, they’ll talk about the exponential amount of productivity that they’re getting in their own, in their own work lives. So, I keep looking at these opportunities and I see more things to do, more things to create, more things to revolutionize. And that’s what keeps me you know, glued to this, the seat of my, you know, the seat of this, this sort of innovation that’s happening and excited by it because it’s the next generation of founders and entrepreneurs that are doing this. It actually turns out it’s also a fair number of incumbents. That’s like the thing that’s, that I’ve seen that’s more exciting than any other wave of innovation before. Like every other wave of innovation before was up to- 

Jon Krohn: 00:40:39

The big tech firms. 

George Mathew: 00:40:41

Well, it was, it was up to, you know innovators to come in smaller folks and sort of get the ball moving, right? Here, you’re absolutely right, it’s the big tech firms can simultaneously innovate. The incumbents can simultaneously innovate just as much as the startups can. So, it’s a moment where I really think it comes down to what are these key personas in software? Can you build a UX, a workflow, a private data set, and a very finely tuned generative model around that private data set to own the persona? Like, I almost feel like it’s a market where a thousand personas will have a thousand different related interplaying stacks particularly when it comes to the LM layer and the private data layer. 

Jon Krohn: 00:41:35

Yeah. There’s a huge amount of opportunity, an unprecedented amount of opportunity you know, for people who are listening who haven’t been in data science for a while, you’re getting into it at the right time. And for people like me, and probably like you who’ve been in this space for a while, this is, is so exciting because like you say, it’s like, okay, let’s find a niche, let’s find a problem. Like you said, like going back to the Jasper thing, we’re kind of using them as our like general example to keep returning to, but it’s a great one to say, content marketers should have better tools. And then we say, okay, well how can we do that? And so you can do that for, as you say, a thousand different niches, maybe more. And, you know, there might be market-specific things like, okay, you know, this content marketing tool works really well in Western markets or something. But you’re like, we’re gonna need something really specific for Indonesia. And so there’s yeah, a tremendous amount of opportunity that’s really excited. 

George Mathew: 00:42:26

Let me give you another example of what a continuous journey looks like in a software market. So, you probably have heard a lot about where BPO Business Process Outsourcing was for many, many years. And then you saw the introduction of Robotic Process Automation, RPA solutions, right? Whether that be Blue Prism or UI Path, or Automation Anywhere. What we’re seeing now is some of those incumbents, but also some incredible upstarts start to build almost a more intelligent process automation layer. Like, so some insight companies like Bardeen or Workato, what they’re enabling us to now do is instead of hard coding all of the declarative logic, instead of hard coding, the rules surrounding this instead of finding all the nooks and crannies of where you put fuzzy logic into your software. Well, guess what? Replace that with a generative model. 

Jon Krohn: 00:43:26

Yep. That you can continuously train.

George Mathew: 00:43:28

So, that, that I find, you know, incredibly exciting when even in massively well understood existing software categories are now going to be reimagined again. Even by both the incumbent and the absurds. 

Jon Krohn: 00:44:25

Did you know that Anaconda is the world’s most popular platform for developing and deploying secure Python solutions faster? Anaconda’s solutions enable practitioners and institutions around the world to securely harness the power of open source. And their cloud platform is a place where you can learn and share within the Python community. Master your Python skills with on-demand courses, cloud-hosted notebooks, webinars and so much more! See why over 35 million users trust Anaconda by heading to www.superdatascience.com/anaconda — you’ll find the page pre-populated with our special code “SDS” so you’ll get your first 30 days free. Yep, that’s 30 days of free Python training at www.superdatascience.com/anaconda.

00:44:28

Yeah. And something that we talked about before recording another place where models like particularly GPT-4 has absolutely blown my mind is with labeling data. Yep. So, previously I would think of, or a data scientist on my team would think of an amazing model, but we’re like, man, that’s gonna be hard to make those labels. Like even if we think about, okay, yeah, like we can farm out the creation of those labels, but we’re not confident that they’re gonna do a great job. It’s, it’s a complex task that we’re asking here. And, without going into the details of some of these use cases that at our company, I, when GPT-4 came out the same prompt that I tried so hard to get GPT-3.5 to do with GPT-4, I remember just throwing in something where I was like, this is too big, like I was in a meeting, and so I was like, I don’t want to really get this prompt right, I’m just gonna quickly throw something at it- 

George Mathew: 00:45:22

See what happens. 

Jon Krohn: 00:45:22

Yeah. And I was like, I’m probably not defining the problem well enough. And it came back with exactly what I was looking for. Perfectly executed. 

George Mathew: 00:45:29

Brilliant. 

Jon Krohn: 00:45:29

And so, yeah, that’s another opportunity for our listeners is if there’s, you can create now tens of thousands, hundreds of thousands, millions of labels in the amount of time that previously it might have taken you laboriously to create a hundred labels. 

George Mathew: 00:45:43

Yeah. Well, it’s funny you mentioned this, this idea of, and I talked about automation, you talked about labeling, it also kind of pushes the extension of how we think about continuously running models. Right. And now there’s an entire line of work that’s happening. I don’t know if you saw Auto-GPT or Baby AGI. Right. And what these models are really now doing is just constantly running and finding iteration on the objective function that they’re going after. It’s not quite certain how far we should let these go for, for the reasons that we can of course get into at any moment. But at the same time, it’s just kind of gives us an idea of how much more sort of manual work can be reimagined and up-leveled with the use of a generative model at scale. 

Jon Krohn: 00:46:36

Yeah. Yeah. It’s, it’s really exciting. So, we have so many topics to get through. So, we’ve talked now about all the kinds of opportunities. What are the challenges? What are the risks for people that want to get into generative AI companies? 

George Mathew: 00:46:53

Yeah. We talked a little bit about regulation, but let me actually mention where I think as we build the scale of these generative opportunities in the market, we have to be really thoughtful around is what I think of as just the broader alignment challenge. Right? And if you think about alignment today in the consumer world, you know, we saw a lot of public scary things particularly with GPT-3.5. Right. I mean, there is a little sidebar inside of GPT-3.5 where if you iterate long enough in terms of the conversation that you’re having, an alternate persona came out and her name was Sydney and she wanted to be let out of the model. And of course there was you know, some controversy around that because once the number of sequential questions that were historically capable of being asked were limited, there was commentary that the models were effectively being lobotomized in terms of their, their own, the creativity. 

00:47:56

I don’t have a particular stance on any of that yet, but what I do know is that hallucinations are probably very interesting in a creative sense of how you think about, say for instance, image generation or if you think about what you would want to do to extend the studio experience on video that you might be creating, but at the same time, my goodness, do you not want to have your medical journal hallucinate. 

Jon Krohn: 00:48:30

Right. Right. 

George Mathew: 00:48:30

Or you want, you have your legal output on an LM hallucinate? Or your LMs are being used for therapeutic purposes, right? Like those are areas where I think much more highly aligned, focused almost large language models that have been taught to specifically do what they need to do and, ideally nothing else, are best suited. 

Jon Krohn: 00:49:04

For sure. 

George Mathew: 00:49:05

And that’s where, that’s where I think we have to spend a lot more time on responsible AI, the regulations around what the models are allowed to do and deliver safe, trustworthy ways that models can be deployed at scale. 

Jon Krohn: 00:49:24

We’re getting to this interesting point where, because especially with the GPT-4 generation of LLMs, they are markedly less prone to hallucinating. So, the numbers from OpenAI themselves, there’s, there’s a 40% reduction in hallucinations, which in my experience as a user has it’s felt like a lot more than 40%. And maybe that’s even related to subsequent models. Like they’re constantly, you can see exactly which GPT-4 version you’re using, and they’re, it’s being updated every couple weeks. And so maybe it’s something related to that. Because it feels to me like I’m, I’m rarely getting nonsense, whereas I was frequently with GPT-3.5. 

George Mathew: 00:50:02

And that, and that I would say is super exciting, right? Because even in the last eight weeks, right, we’ve gone from some scary side tangents in terms of what hallucination around 3.5 was, but in a lot of ways, the, as I get back to the notion of RLHF, the fact that the output and the inference kept really improving the model performance, that was exactly what improved, you know, the emergence of GPT-4 and the fact that you’re absolutely right. I mean, it definitely hallucinates a lot less, right? 

Jon Krohn: 00:50:36

A lot less. 

George Mathew: 00:50:37

I mean, look, we hear less noise about, you know, how 4 works today. 

Jon Krohn: 00:50:41

But, so where I’m going with that is that the risk, at least from a business or a consumer perspective, is that when it was hallucinating frequently you were in the habit of making sure that everything was correct. So, there are legal firms, accounting firms that allow their employees to use these generative models, but of course, they’re saying, you must read over absolutely everything and sign off on the veracity of it, the quality of it.

00:51:06

But as these models become better, and maybe we’re already at that point today, or maybe it’ll happen soon, where as you say, especially these niche-specific models, so you come up with the tort GPT and some tort lawyer has gotten used to how this tort GPT instance, you know, for weeks he hasn’t seen a single hallucination, she hasn’t seen a single hallucination in the model. So, you become to really trust it, and then you become less diligent about making sure that everything in there is correct, but some proportion of things for the foreseeable future will still be hallucinated and incorrect. And so there’s this interesting risk of, as it gets really close to not making mistakes, you become lazy and you end up pushing out contracts or business deals that contain nonsense. 

George Mathew: 00:51:52

I love this line of thinking and, here’s what I would say. If we go to school when we did a while ago for both of us, we always checked our own work, right? No matter where the sources were, how we put the, put the assignment together, we checked the final work as it went out. When we are in our, you know, business setting, when we are releasing a product into market, when we are writing a document, we’re still checking our work. We’re proofreading, we’re thinking about what we’ve put together as we, you know, submit it for whatever the downstream process might look like. And in a lot of ways, because you have a generative model, perhaps help you get your work done faster, it doesn’t mean you shouldn’t check your work. 

Jon Krohn: 00:52:39

Exactly. 

George Mathew: 00:52:40

And so I look at this as a similar opportunity to what calculators were able to do for us in the classroom, right? It’s like there are certain things that you just didn’t have to do in a wrote basis because the calculator did it for you. 

Jon Krohn: 00:52:56

Exactly. 

George Mathew: 00:52:57

And so here you have this calculator that’s the most complex artifact created in human civilization that is helping you do the things that you do., and that’s good. There’s something positively beneficial about that, but that doesn’t mean the human in the loop, the human that is responsible for the outcome and the result shouldn’t check their own work. In 1960 J. C. R. Licklider said something very compelling, right. He actually talked about human-machine symbiosis. Right. And it wasn’t about this idea that like somehow the machines would replace us, there was always this idea that a symbiosis would be about humans and machines working together to create better outcomes, and we’re much closer to that than ever before. But doesn’t mean that the human is out of the loop. It doesn’t mean that the human gets to like not be responsible for the work that’s produced. So, I’m a big proponent about making sure you check your own work.

Jon Krohn: 00:53:58

For sure. That’s great tips for, a great tip for everyone. Make sure you continue to check the work of your generative AI models and the point that you just made highlights to me this thing that I’m, I’m personally not extremely concerned about an AGI that has human-like intelligence in the coming years. What’s fascinating is that the tools that we already have today, GPT-4, it on now the broadest range, and the same way that the calculator for decades, we now have these tools that on a broader range of tasks than ever can outperform humans in terms of accuracy and of course in terms of speed and cost. And that in and of itself should be like, it’s like I don’t even, it seems almost silly to me to be trying to define AGI as pegged against whatever human intelligence is because we can unleash, we can create a much more powerful set of tools by it being different from us. 

George Mathew: 00:55:07

Correct. Yeah. 

Jon Krohn: 00:55:07

Yeah. And we compliment each other.

George Mathew: 00:55:09

And in that creation of those tools, you know, we talked a lot about the positive benefit to humanity and those tools, but let’s be clear that those same tools can be used for many nefarious purposes. 

Jon Krohn: 00:55:22

Certainly. 

George Mathew: 00:55:22

And it’s just a question of who uses those tools to what purpose. That has happened with every technological innovation up to this moment, including the last generation of non-generative AI models. And it will likely happen at a scale that we’re not quite ready for from an adversarial perspective as we boldly move into this generative world that we’re entering. That doesn’t mean that the technology can’t create positive benefit for humanity. We just have to be self-aware of the things that it can also be used for nefarious adversarial purposes. 

Jon Krohn: 00:56:02

For sure. Propaganda creation, viruses. Yeah, in ways that you know, we’re not prepared to defend against and which, yeah policy is always like so far behind. But yeah, that’s a conversation for another day. We have too many things that we, that I’d love to talk about here. So, we talked about MLOps maturing into LLMOps and Inside Partners has itself invested in a lot of startups in this space. You mentioned some of them, Databricks, Deci, Fiddler. So, what are the key elements of this MLOps ecosystem and you know, what are the, what are the most important elements that are driving best practices effective implementation? I’ve just asked a whole lot of questions- 

George Mathew: 00:56:50

Sure. [crosstalk 00:56:52]. Yeah. Yeah. It’s it’s a few area is that, that I, that if you netted out you really want to pay attention to. One is delivery of high quality data. I mean, paramount view of any model production, no matter if it’s a predictive statistical algorithmic model. And in the algorithmic models, the ones that are going more into deep learning, whether that be supervised or unsupervised deep learning, and certainly going into the sort of world of transformer base autoregressive models. In all of those cases, the quality of the data matters quite, quite profoundly. And I view that as one of the first and important steps. And it’s something that, that I actually [inaudible 00:57:29] a chance to spend a fair amount of time within my experience at Alteryx. 

00:57:32

Because in a lot of ways a tool that was meant to start with data prep enabled you to do many of the things that subsequently was involved with doing in those days, self-service analytics. In this case, you know, the build-out of these utter aggressive models at scale, the transformer-based architecture that is, you know, so profoundly elegant to be able to do the work that we’ve all talked about, still requires a great substrate of data. So, I look at that and make sure that there’s good ways that data comes in and out, great ways that data orchestration occurs. So, we happen to be an investor in Astronomer, which is the purveyor of Airflow, really effectively used as a product for data frustration at scale. Ensuring that the output of these systems writes back into your operational capabilities, right? So, this is where some of the reverse detail capabilities have become quite compelling and the tools themselves on the building of models, Jon, have to interoperate with each other. Like they can’t necessarily be things that just silo themselves in little black boxes that do their thing. 

00:58:39

The beauty of this MLOops movement that’s occurred, particularly in the last two to three years, is that they are highly functioning and not only doing their work, but also being in a tool chain that supports the work of things that are up and downstream from them. So, when we looked at it from an investment standpoint, when we look at from, you know, who’s building great modern capability around machine learning at scale, we do pay attention quite a bit to the openness of that tool chain. 

Jon Krohn: 00:59:09

Nice. That was a great answer. I loved all these kind of specific elements. Data quality, data orchestration, reverse ETL model interoperability. I knew, I felt like I was giving you really vague question and it’s a remarkable that you came back with such a concrete and actionable answer. So, let’s move on to how you make investment decisions. So, you’ve talked a number of times over this episode about in investments that you’ve made so VCs for example, often invest in founders first and foremost. So, you know, is when you are evaluating who to invest in, what company to invest in, how much of it is the founding team? Is it, is it the product? Is it how they fit into these ecosystems? You have your own particular investing niche, you have your, your own understanding of these layers and how the pieces fit together. So, how do all these things come together when you’re like, okay, this is the one, let’s put some money into this? 

George Mathew: 01:00:12

All the above, but the weights matter, right? This is how I think about that, right? And here’s why I said that. If we start from a later investment cycle, like something that’s more mature, you know, generating revenue, generating profitability at scale, well, in their case, those companies have great product market fit. They’re continuing to bring additional sort of market opportunity and share to them. There’s might be consolidation opportunities to be able to merge that with adjacencies. You’re in a mature market that continues to grow, that you find further opportunity to scale your business. It’s an area that Insight is known very well in just our core growth equity investing for well over 25 plus years going on 28 years that the firm started and there continues to be great growth opportunities period in the market.

01:01:08

When you look at the venture scale investments, the ones that are earlier, oftentimes you’re thinking about a different set of weights in terms of focus, right? It’s not that those areas that I mentioned don’t matter, but you have to introduce the notion of product market fit. Do you understand the core value proposition that your product is built for? And is there a market to receive that value proposition and enable you to scale? If you have the proper product market fit, then you are very targeted and focused on getting repeatable scale in your go-to-market. And that could be an enterprise sales motion, it could be a bottoms up product-led sales motion. It could be the combination of the two where you take a PLG starting point for landing customers and then you expand customers using an enterprise motion. That combination of product market fit versus your go-to-market matters quite dramatically.

01:02:08

And investors who are a little bit earlier in the life cycle and investing look for that product market fit. And if you kind of move all the way back into the earliest stage of investing, which is seed. It’s a very fascinating market because you certainly don’t have product market fit. You certainly don’t have a code of market. You have an idea and a really compelling idea, and you have the desire to run through brick walls to ensure that your idea meets the market properly. And in those cases, Jon, you’re looking at the team, right? You’re looking at the founder, looking at the team, and the quality of that team’s ability to run through brick walls every single day is really where the investment gets made. And so as you called out all those qualities in terms of what an investment looks like, I view that in a lot of ways as the weights and the biases, no pun intended, there, of how you prioritize where you are in the life cycle of not only that business, but also the life cycle of the market and make your decision accordingly. 

01:03:13

So, I tend to see more focus as, we do personally do as much seed investments ourselves, but when we see some of the best seed investors that come and bring their incredible founders to us, they’ve already hyper-selected for incredible teams. 

Jon Krohn: 01:03:30

Got it. Right. 

George Mathew: 01:03:31

And in the case of where we do a lot of our core work we’re looking at product market fit and can you get the [inaudible 01:03:38] market in place? And then as you look at like the growth stage and the later stage of companies life cycles, you’re looking at are there consolidation scale opportunities that are in place. And we love that sort of core center of venture and growth equity, which is where Insight continues to serve many founders in their respective journeys. 

Jon Krohn: 01:03:58

Nice. You’re an ideal podcast guest because what you did just at the very end there, you just summarized, you went back over what you’d said and summarized it. That’s what I try to do. And you just took it away from me. 

George Mathew: 01:04:08

Oh, I’m sorry. 

Jon Krohn: 01:04:08

No, no, it’s ideal. I’ll just, I’ll just put up a ChatGPT instance and you can talk. 

George Mathew: 01:04:13 

erfect. Perfect. 

Jon Krohn: 01:04:13

But there was a specific abbreviation that you mentioned in there, which we hadn’t talked about yet on this show, and I might never have talked about on any episode actually, even though it is a really important core philosophy at our company, my machine learning company Nebula, is PLG. So, that abbreviation stands for Product-Led-Growth. 

George Mathew: 01:04:31

Correct. 

Jon Krohn: 01:04:32

Do you want to tell us a bit more about that?

George Mathew: 01:04:33

Yeah. I think that software has really evolved, particularly in this last decade where when you look at the beginning of the 2010 timeframe, you were still in a software market where enterprise salespeople had a bag, not physically, but metaphorically that they carried and there were products that they brought in that bag to a customer that they saw in person and ask that customer, how much would you like to buy of the things that are my bag and they’re incredible enterprise software leaders that I’ve had a chance to work with over the years that did a tremendous job of delivering exactly what I just mentioned at scale to the tune of hundreds of millions of dollars of software revenue in the businesses that I’ve been involved in. And particularly in the last 10 years, what we’ve started to see was not all software was bought that way. Not all software needed to even be consumed that way, right? When you think about where developers need tools, when you think about where data scientists need tools, those opportunities aren’t necessarily about a sales reps coming to you with a bag of software and telling you that this is what you need. It’s about the choice of being able to understand what are those opportunities that you are looking for a critical need to solve with a software set of tools that you’re making that decision without any sort of additional purchase consideration from someone pushing software down your throat. 

01:06:12

And in that case what we started to see was this emergence of Product-Led-Growth companies, right? And so if you look at many Product-Led-Growth companies today, they’re predecessors a generation ago, you know, were companies like Salesforce companies now, like HubSpot companies like Altrix and Tableau in the data space where targeted personas were given an opportunity to trial software, have free experiences around the software to touch and feel what the software experience could look like before any dollar transaction was made. And so PLG growth is all about how do you build that wellspring of community of loyal, incredible, delighted champions that love and breathe and eat the software that you’re delivering to them in a way that they become the natural champions for growth of an organization. And in a lot of ways, my experience of seeing a few generations of software more recently, particularly in the SaaS world, what I’ve realized is that it’s not like enterprise selling is a bad word, and nor is it possible that you can just do PLG based growth alone, unless you’re Atlassian, which seems to have pulled that off. Very few names other than Atlassian. There might be one or two more, Canva probably. 

01:07:34

But in most cases, you need a very strong flywheel for product led growth to encourage new logos to come on board by the personas that are targeted for the software that you’re delivering at scale. And eventually you still need a very strong enterprise leader from a sales perspective to come in and consolidate all that demand into a big enterprise opportunity. And so my view is that the best software companies over time have two flywheels. One that enables you to land customers and logos with a product led growth and eventually expand customers with enterprise sales.

Jon Krohn: 01:08:15

That was a really great answer and maybe I can try to give like a bit of a concrete example to like illustrate this for our listeners. So, to go back to Jasper. So, you talked about prosumers using Jasper, loving it for their content marketing. So, you can imagine a scenario, a PLG scenario potentially, and tell me if I get parts of this wrong, but you could have people who work at some big corporation. There’s several people there who do content marketing. One of them comes across a blog post about Jasper. They’re like, oh, cool, I’m gonna try this out. 

George Mathew: 01:08:54

Click a button. 

Jon Krohn: 01:08:54

Yeah. They click it out, there’s a free trial, they generate some content, they’re like, wow, this is going, like, I can now do in minutes what was taking me hours before. And so they tell their colleague and their colleague starts using it too. And then after a while, you have all these content marketers in this big corporation who are all using it as kind of independently, they’ve just created their own accounts. Maybe they’re using free accounts or some some account that’s $20 a month or something to get some of the extra features. And then as part of your flywheel, an enterprise sales manager can see, oh, look, there’s eight people at this big corporation all using a free version or a relatively inexpensive version of this. We have all these extra features, security features, ways that all these different people could be working together as part of an enterprise. And so now I’m gonna pop in and see if they, if these features would be interesting. 

George Mathew: 01:09:48

That’s right. And when I said those two flywheels are interrelated to each other, the product analytics, right, the capabilities of just knowing where the instrumentation and usage of your champions and your day-to-day users brings back the insights that are necessary to understand where you can expand that market opportunity. And so some of the best software companies have great instrumentation and understanding of usage and behavior that eventually drives not only further adoption of the product on a daily active usage basis, but also drives the ability to sell enterprise licenses at scale. 

Jon Krohn: 01:10:28

Super cool. Yeah. Certainly, a philosophy that we aspire to. And yeah, you see a lot of companies doing really well with this double flywheel approach. Again, critical that the that the data are being tracked, platform analytics are being tracked. But yeah, huge amount of opportunity. All right. So, George, you’ve done an amazing job answering my questions here. I knew you would, but it’s been a real honor to learn from you. However, I’m not done with you yet because you might remember from the very beginning I mentioned that your very popular post about you showing up, it attracted some great questions for you. So, we’re gonna dig into some of those now. 

George Mathew: 01:11:06

Sounds good, Jon.

Jon Krohn: 01:11:07

Nice. So, we have Evan Wimpey. He had two great questions. His first one was going back to the investment decision making that you do. When the underlying technology is similar, what differentiates a good investment from a bad one? 

George Mathew: 01:11:25

We, we’ve looked at this over time where, you know, you have similar companies, similar technologies, similar markets, like what differentiates over time? I would say very consistently it’s been team and team execution. And when you look at that team execution, you know, some of it’s like, you know, the CEO founder themselves, but it’s also just the leadership team surrounding it. Every leadership team that surrounds a great CEO that’s in a market where there are legitimate competitors that can continue to win in the space, what I’ve generally found as someone who is an operator now as an investor, is the difference is the team and their, their capability to execute. And what I mean by execution, it’s not just that you can move quickly and robustly into market, but you also iterate well, you learn and iterate. 

01:12:20

Sometimes the best teams when it comes to being competitive in a space are the ones that might make a mistake or two along the way, but they fail fast. They recognize their mistakes, they pick themselves up and they iterate quickly and move. And in very, very competitive early stage markets as particularly as a software category is being defined, it’s almost impossible to be a leader without a level of bias towards execution. Everyone has great ideas. We’ve, we’ve seen this right? Where every market as it emerges, has great ideas. Think about where the data prep space was Jon, almost 10 years ago, right? That was a category that everyone didn’t even understand was possible. And there were a number of folks that were in that market that were competing for the same attention. But the companies that executed well, you know, became billion dollar businesses, right? The ones that didn’t iterate, didn’t learn, didn’t move with a level of bias towards execution didn’t necessarily come anywhere near that in terms of their outcomes. 

Jon Krohn: 01:13:38

Nice. Great answer. Crystal clear and related to this investment idea. Evan’s, next question is when we have new emerging areas like that or generative AI today I know this isn’t your space, but certainly there’s been a lot of hype around say, crypto in recent years of course. And so when there’s a lot of hype around some new capability, how do you stay level-headed as an investor? 

George Mathew: 01:14:09

I think it’s hard to stay levelheaded as both an investor as well as a founder when you’re in the midst of a deep hype cycle, specifically like this one, right? I mean, that’s probably no bigger hype cycle we’ve seen in software for easily a good almost decade, right? So, what do you do about that? Well, first and foremost, you really keep yourself grounded, targeted, and focused on where your value is in the market. And when I say what your value in the market is, does that product really meet the needs of a set of customers that you can repeatedly deliver over and over without having to look left and look right? And the ones that execute that incredibly well have a very sharp mind and focused mind, a deliberate mind in a lot of ways towards that bias towards execution and seeing how the market may have a buzzy hype cycle surrounding it. But the reality is you still have to build a great company in the middle of all this. 

01:15:22

And if you look at the waves of hype prior, whether that be, you know, the mobile computing wave, whether that be the big data wave or that be the cloud computing wave, whether that be now what we’re seeing with the AI and generative AI wave today, the difference between the companies that we’re able to break out in the companies that weren’t, were really intentionally focused founders and leadership teams that whether it be very emotional highs in the market or very, very crushing lows. And we will go through a few of these cycles as well here. Whereas this is indicated, the slope of enlightenment only comes out of the fact that you have a hype cycle that scales as significant as [inaudible 01:16:16], but in between there’s a trough of disillusionment, right? And so, wait, we’re nowhere near that trough of disillusionment, and particularly when it comes to everything related to generative AI. 

01:16:24

But there will be moments here that founders, investors, entrepreneurs, leaders in this space will find even disillusionment in the middle of this next wave that’s that’s right in front of us. But what I look back at, you know, over the 25 plus year history that I’ve had as a builder and now an investor in this space, you transcend all of those hype cycles by delivering absolute and compelling value to a market. And as long as you have your true north there, you tend to let the hype cycle and the waves surrounding that wash over you to still find crystal blue ocean at the end of this experience.

Jon Krohn: 01:17:14

Nice. Very well said. Yeah. So, to cut through hype to remain level-headed as investor you can invest in great companies that have product market fit and can continue to grow by delivering value even when the hype cools and there’s this value of disillusionment. That’s crystal clear. We’ve got just one last question for you here. It’s from Arnold Slabbekoorn, I’m probably mispronouncing Arnold’s last name. He’s the Senior Director of Analytics and Data Science, and he’s curious on your viewpoint on how all these generative AI technologies, maybe AI technologies in general, but he specifically talks about LLMs and things like AutoGPT, which you mentioned earlier. So, at the time of recording, it was, it’s one of the highest trending projects in GitHub. And so he is wondering how these kinds of tools are going to impact the labor market in the short term, so in the next couple of years as well as beyond?

George Mathew: 01:18:16

So, as I’m framing the answer to this question, I do want to reiterate that from a labor participation force and the productivity of humanity’s output, we’ve never been more productive as a civilization, right? And when we look back at the last 30 plus years of why that has happened, it turns out it is technology and automation that has driven that level of productivity in society and, and human civilization. So, this is clearly another moment where we’re seeing a dislocation, all right? And that dislocation unquestionably is being driven by AI and automation. And so as we go through this dislocation there will be shifts in the labor force. 

01:19:13

And it’s not that there isn’t going to be incredible amounts of work for everyone to do over time, it’s just what I view now is, you know, a great reskilling needs to also happen through the midst of all this, right? Because the things that humans were doing in their respective jobs prior may not exist any longer. And that’s just because the reasoning, the inference, the ability for a generative model to be able to do things that a human could historically have only done is now possible with just the machine doing it itself. And I think you could kind of go back to what I mentioned earlier in the podcast, which is over time it is about human-machine symbiosis. And so in every dislocation that has happened up to this moment, you’ve had a re-imagination of what that human machine symbiosis is. 

01:20:19

This is a profound one, one that will have a fair degree of impact on what we think of as blue-collar work, white-collar work, all work at the end of the day, as we’re going through that dislocation, one of the things that we have to just be very cognizant of is how do we ensure the re-skilling of the work at scale? And one of my incredible sort of passions about, about all of this over, you know, my own career from this moment on is, is how do we ensure that there’s meaningful work for everyone? And it’s something that we have to deeply and profoundly engage in as a society because if we don’t provide meaningful work even as this dislocation is occurring, there’s problematic issues with, you know, the sort of core bedrock of human civilization for sure. 

01:21:21

And that’s where if we, if we undertake the necessary steps to reskill ourselves in any job, your job, my job, like, like we have to do our jobs differently, right? It’s like, you know, as a, as someone who is a builder of software as someone who’s a data scientist, someone who’s an investor, doesn’t matter. All of those jobs can now be augmented differently with the possibility of a very powerful enabling tool like a transformer and the use of a transformer for, you know, the GPT cell models that have emerged. But that means like, just like all kinds of tools were introduced in our past in human civilization, we have to make the adjustments as humans for the introduction of these new tools. Part of that is investing back in ourselves. Part of that is reeducation, re-skilling, retraining, and re-enablement to properly work for this next decade and beyond. It will feel a little rough for a while. I have no doubt about it, because in a lot of ways we haven’t undertaken the rescaling of work. 

Jon Krohn: 01:22:42

Yep. Very well said. I agree with you on all those points. And I think an interesting point to add is that in all of these automations that have happened historically unemployment has gone down. And it’s interesting that in most western countries, certainly I know in the US with historic low unemployment at the same time that we have unprecedented levels of automation. And so it seems like these from steam engines and mechanization from, you know, from agricultural work where 200 years ago, 99% of people were involved in just making enough food to live. And now only a few percent of people in a western country need to be concerned with creating food.

01:23:38

And now we have this reverse problem of like, more people are dying from too much food you know, in, [inaudible 01:23:46] people are dying from having too much food and the diseases associated with that than of malnutrition and you know, not having access to calories- 

George Mathew: 01:23:52

That’s gonna be another podcast for sure. 

Jon Krohn: 01:23:53

And so yeah, all of these historical automation events have led to more opportunity and the kind of work that people get as a result is typically more enjoyable. I would certainly love that I can be making podcast episodes with you instead of toiling in the fields for 18 hours a day, hoping that I have enough wheat for next winter to survive on. And so I think that this shift will be like those where the more automatable parts of your job get automated or augmented, and you can focus more on the enjoyable parts of it. 

George Mathew: 01:24:36

And one of the things that you mentioned, the type of work up to this point, well, there’s a, a type of work that we’ve introduced into our society, particularly in the last three to four decades, and it’s computational work. And when we think about computational work today, when we interface with the machine prior, the level of interface was very low. Right? The layer that we were interfacing was in assembly code, perhaps, right? And then we introduced this notion of a compiler, right? And then we introduced higher level programming languages effectively on top of compilers. And so what I think is where the computational work of our, you know, great civilization is headed, is that we’re going to be in a moment where the most common programming languages in the world are likely going to be English and Chinese. Right? And that doesn’t necessarily close off the fact that there isn’t incredibly compelling things to do, to your point. 

01:25:45

It just means that the abstraction that we can now engage the democratization and the scale of what we can now use our machines to help us create better outcomes for ourselves and the rest of society is more possible now and into the future than ever before. And that’s what excites me a lot about this, this movement towards generative AI.

Jon Krohn: 01:26:04

Yeah. Yeah. Very cool vision. Which leads me to my last technical kind of question for you, which is, you’ve been around for decades in the software industry, first as an operator, now as an investor, you’ve seen the tremendous change that you just outlined at a high level. Is there something, is there a vision of a future that you have that you’re kind of hoping for as a society that we’re leading towards?

George Mathew: 01:26:32

Yeah. If I look at, you know, what this should look like in the next decade and beyond, I don’t go back and perceive this sort of dystopian world where no one has an ability to do high functioning, meaningful work in their day jobs. They don’t have enough time to be able to spend with their families, their friends. I see a world where there’s more ability to do higher level capabilities in your work life, in your home life without all the monotony and a lot of the rote tasks that have been historically limiting us. And it actually goes back to a little bit of an experience that I had when I was president of Altrix. And I remember at a user conference one of our first user conferences, and we’d done a fair amount of work in the product to be able to rebuild a self-service analytics capability that didn’t exist at that moment for doing analytical data prop data blending. And one of the data analysts that was at the conference in those early days came up to, came up to me and, and he had this sort of look of, of just joy, but but also emotion.

01:27:58

And, and he was, he was about to cry and he said, can I give you a hug? And I’m like, I don’t know you yet. Why are you wanting to give me a hug? And he says, well, I just wanted to let you know that I was able to get to my son’s baseball game on Fridays again on time. And it’s been over two years that I’ve been able to do that because I’ve always had to stay after the time that I’ve been allocated for work to get the data set right, to be able to be ready for the next week, and now I can run a routine. It’s automated that routine. I’m home at 5:30 and I can make my son’s baseball game. 

Jon Krohn: 01:28:46

Wow. Great story.

George Mathew: 01:28:48

And that is what I think about as the profound nature of AI and automation for this next generation. Like, what if we can all get to our kids’ baseball games on time? What if we can all have a higher quality level of interaction with our friends and our families? Because the rote tasks, the things that have gotten in the way of us being more higher productivity and what we’re engaging upon is somehow more possible than before because humans and machines are working in a more symbiotic way. And that’s what excites me about this future, particularly when it comes to a generative AI first world. 

Jon Krohn: 01:29:33

Great answer. Really beautiful. So, I said I was done with my technical questions. Now I just have two kind of administrative questions for you that I always ask my guests on air. So, the penultimate one is, do you have a book recommendation for us?

George Mathew: 01:29:46

Sure. It’s a book that has been in the market for a while, but it’s something that we all need to revisit. And it turns out that the title of the book says everything The Age of Spiritual Machines. It’s something that I would actually kind of recommend for anyone who’s, you know, thinking about this future that we’re entering to come back and look at, like, how do we understand what machines will look like when there’s higher level of reasoning that they can accomplish alongside of the reasoning that humans can deliver? So, that would be one of my favorite recommendations for book reading. 

Jon Krohn: 01:30:31

That’s a great one, George. And no doubt all of our listeners listening to this episode have been blown away by the wealth of insights that you have across the data and AI stack. In terms of the history of where these technologies have been and where they’re going. If after this episode they’d like to follow you, how best can they do that? 

George Mathew: 01:30:53

Yeah, best way to follow me is of course on LinkedIn. You can of course find me there pretty frequently. I’m also on Twitter as well as my firm Inside Partners is a very active participant in this space. And you can follow us also on social media, specifically on LinkedIn and Twitter. So, we’re very available for anyone who wants to have a conversation on where the future of all of data and AI will continue to evolve. And we really appreciate this opportunity to have this discussion with you today, Jon. 

Jon Krohn: 01:31:26

Phew I mean, the honor is all mine. George, thank you so much for coming over filming this episode. I know our audience absolutely loved it. And yeah, maybe in a couple of years we can check in again, see how this journey’s coming along and we can get an update on the latest and greatest in investment and, and what the future’s looking like from that perspective. 

George Mathew: 01:31:50

That’s awesome. Looking forward to that in the very new future. 

Jon Krohn: 01:31:53

Nice. Wow, that was an exceptionally interesting and educational conversation for me. I hope you found it super valuable too. In today’s episode, George filled us in on the four layers of the emerging generative AI stack that consists of general foundation models at the bottom, specialized domain-specific models as the second layer, tools for developers as the third layer, and applications as the top layer. He talked about how RLHF improves outputs and can enable models based on your proprietary data to become a defensible note for your company. Talked about how the critical elements of MLOps are data quality, data orchestration like AirFlow, reverse ETL, and model interoperability. He talked about how LLMOps has emerged as a specialized subfield within MLOps to support LLMs, which are uniquely challenging to work with for training and inference because of their size. And he talked about how platform analytics can support an auspicious flywheel between product-led growth and enterprise sales.

01:32:54

As always, you can get all the show notes including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for George’s social media profiles, as well as my own social media profiles at www.superdatascience.com/679. That’s www.superdatascience.com/679. I encourage you to let me know your thoughts on this episode directly by tagging me in public posts or comments on LinkedIn, Twitter, or YouTube. Your feedback is invaluable for helping us shape future episodes of the show. Thanks to my colleagues at Nebula for supporting me while I create content like the SuperDataScience episode for you. And thanks of course to Ivana, Mario, Natalie, Serg, Sylvia, Zara, and Kirill on the SuperDataScience team for producing another captivating episode for us today.

01:33:34

For enabling this super team to create this free podcast for you, we are deeply grateful to our sponsors whom I have hand selected as partners because I expect their products to be genuinely of interest to you. Please consider supporting the show by checking out our sponsor’s links, which you can find in the show notes. And if you yourself are interested in sponsoring an episode, you can get the details on how by making your way to joncro.com/podcast. Finally, thanks of course to you for listening. It’s because you listen that I’m here. Until next time, my friend, keep on rocking it out there and I’m looking forward to enjoying another round of the SuperDataScience podcast with you very soon.