78 minutes
SDS 849: 2025 AI and Data Science Predictions, with Sadie St. Lawrence
Subscribe on Website, Apple Podcasts, Spotify, Stitcher Radio or TuneIn
Sadie St Lawrence returns for her 4th annual prediction episode on the Super Data Science Podcast. Together with host Jon Krohn, they reflect on 2024’s most transformative trends—like agentic AI and enterprise AI monetization—and predict what's coming in 2025, from AI-driven science to the skills data scientists need to stay ahead.
Interested in sponsoring a Super Data Science Podcast episode? Email natalie@superdatascience.com for sponsorship information.
About Sadie
Sadie St. Lawrence is head of AI at SSL Innovations and the Founder of Women in Data. Women in Data is an international non-profit organization represented in 55 countries with a community of over 60,000 data leaders, practitioners, and enthusiasts. Women in Data has earned recognition as a Top 50 Leading Company of the Year and as the #1 community for Women in AI and Tech. Before founding SSL innovations and Women in Data, Sadie gained extensive experience in data science and AI strategy, providing consulting services to healthcare, high-tech, and government organizations. She has educated more than 500,000 individuals in data science and has developed numerous machine learning programs. Sadie’s accomplishments include being named one of the Top 30 Women in AI (2022), one of the Top 10 Most Admired Businesswomen to Watch in 2021, a Top 21 Influencer in Data (2021), and one of the Top 30 Most Inspiring Women in AI (2021). She is also a recipient of the Outstanding Service Award from UC Davis (2019) and just recently, included in Dataleum’s 30 Outstanding Women in Data, 2023.
Overview
What does the rise of agentic AI mean for your career? How will AI reshape the devices we use every day? In this annual roundup of data science trends past and future, Sadie St Lawrence returns with her fourth annual forecast to tackle these questions and more.
Here are Sadie’s five bold predictions for 2025:
In this episode you will learn:
Items mentioned in this podcast:
Follow Sadie:
Sadie St. Lawrence is head of AI at SSL Innovations and the Founder of Women in Data. Women in Data is an international non-profit organization represented in 55 countries with a community of over 60,000 data leaders, practitioners, and enthusiasts. Women in Data has earned recognition as a Top 50 Leading Company of the Year and as the #1 community for Women in AI and Tech. Before founding SSL innovations and Women in Data, Sadie gained extensive experience in data science and AI strategy, providing consulting services to healthcare, high-tech, and government organizations. She has educated more than 500,000 individuals in data science and has developed numerous machine learning programs. Sadie’s accomplishments include being named one of the Top 30 Women in AI (2022), one of the Top 10 Most Admired Businesswomen to Watch in 2021, a Top 21 Influencer in Data (2021), and one of the Top 30 Most Inspiring Women in AI (2021). She is also a recipient of the Outstanding Service Award from UC Davis (2019) and just recently, included in Dataleum’s 30 Outstanding Women in Data, 2023.
Overview
What does the rise of agentic AI mean for your career? How will AI reshape the devices we use every day? In this annual roundup of data science trends past and future, Sadie St Lawrence returns with her fourth annual forecast to tackle these questions and more.
Here are Sadie’s five bold predictions for 2025:
- Agentic AI will be the dominant trend, moving beyond single applications to create specialized networks that can autonomously handle complex tasks, though security and permissions between platforms remain a challenge.
- AI integration into everyday devices will accelerate, from augmented reality glasses with real-time translation to more sophisticated personal computing experiences, though not all integrations will prove valuable.
- AI-driven scientific research will expand significantly, building on current successes where AI-assisted researchers achieved 44% more new material discoveries and 39% more patents.
- Enterprise AI monetization will be crucial as companies seek returns on massive hardware investments.
- The demand for AI engineering skills will surpass traditional data science skills, though this represents an evolution of the role rather than a replacement — requiring practitioners to build on existing technical foundations with new capabilities.
In this episode you will learn:
- (03:30) 2024 AI trend recap
- (19:23) Comeback of the year: Google
- (27:29) Wow moment of the year
- (40:20) Looking ahead to 2025
Items mentioned in this podcast:
- Human-Machine Collaboration Institute
- Women in Data
- Data Bytes Podcast
- Observer Planner
- NVIDIA
- Anthropic
- Daniel Kahneman
- Microsoft Copilot
- GitHub Copilot
- ChatGPT
- Klarna
- Waymo
- Tesla
- NeurIPS Conference
- Gemini 2.0
- Notebook LM
- Perplexity
- Sakana AI Scientist
- SDS Episode 745 – 2024 Data Science Trend Predictions
- SDS Episode 812 – Focus on AI-driven scientific discovery
- SDS Episode 843 – Insights on enterprise AI monetization challenges
- SDS Episode 847 – How to become an AI engineer
- SDS Episode 781– Sol Rashidi – Expert on profitable AI projects
- The Super Data Science Podcast Team
Follow Sadie:
Podcast Transcript
Jon Krohn: 00:00:00
This is episode number 849 on Data Science Trends for 2025 with Sadie St. Lawrence.
00:00:12
Welcome to the Super Data Science podcast, the most listened to podcast in the data science industry. Each week, we bring you fun and inspiring people and ideas, exploring the cutting edge of machine learning, AI, and related technologies that are transforming our world for the better. I'm your host, Jon Krohn. Thanks for joining me today. And now let's make the complex simple.
00:00:46
Happy New Year and welcome back to the Super Data Science podcast. I hope you had a wonderful 2024. To start you off on the right foot in 2025, for today's episode, we've got our annual data science trend prediction special for you again this year. In today's episode, which will appeal to technical and non-technical listeners alike, we cover how Sadie's predictions for 2024, which she made a year ago on this show, how those predictions panned out. We award our wow moment of 2024, our comeback of the year, our disappointment of the year, and our overall winner of 2024. And then, of course, we speculate on what 2025 will bring us, including Agentic AI coverage, AI in everyday edge devices, the remarkable way AI is transforming scientific discovery with numbers that will surprise you, what massive GPU investments by tech giants tell us about AI monetization in 2025 and the unexpected shift coming to data science careers.
00:01:42
As with our 2022, 2023 and 2024 predictions episodes, our special guest again this year is the clairvoyant Sadie St. Lawrence. She's a data science and machine learning instructor whose content has been enjoyed by over 600,000 students. She's the founder and CEO of the Human Machine Collaboration Institute, as well as being founder and chair of Women in Data, a community of over 60,000 women across 55 countries. On top of all that, Sadie serves on multiple startup boards and is host of the Data Bytes podcast. All right, you ready to join Sadie and me on this visionary episode? Let's go.
00:02:23
Sadie, welcome back to the Super Data Science Podcast for another year of forecasting. This is your fourth consecutive year where you are leading us into the future, into the exciting future AI changes that are going to be even faster than ever in 2025. Welcome.
Sadie Lawrence: 00:02:41
Thank you, Jon. I cannot believe it's been four years that we've been doing this. As everyone says, it's so cliche at the end of the year, where does the time go? But now I really feel like where did the time go because we've been doing this for four years, and so eventually maybe five years. We have to do a full five-year recap or something on how many were we off, how many were we on?
Jon Krohn: 00:03:02
Yeah, that's a really good idea because even thinking about, even just as I was looking back to see how many times you've done this just now, before we started recording, I was thinking, wow, 2021 predictions. What would those have been like?
Sadie Lawrence: 00:03:15
Do we even want to look? Not in this this year, not this year, save it for next year.
Jon Krohn: 00:03:17
No, next year.
Sadie Lawrence: 00:03:20
You have to listen to the podcast for a whole another year to see what we said in 2021.
Jon Krohn: 00:03:26
Yeah, yeah, yeah. But yeah, in 2021, we made predictions for 2022. You've always been pretty on the money. I'll quickly recap your predictions for 2024 before we find out what you've been up to over the last year. So for 2024, your number one prediction was that there would be much, much more demand than ever before for GPUs and other kinds of AI hardware accelerators, which would open the door for new players to compete with established giants like NVIDIA. And absolutely, I mean, NVIDIA has become, I mean it's crazy to think from last year it already seemed like it had a crazy stock price and we've seen it increase so much more over 2024. So you're absolutely right there. But simultaneously, we have also seen lots of players come in. Tensor processing units have evolved a lot. AWS is training them and Inferentia chips are doing exciting things. For example, this giant new cluster that's been announced that Amazon is involved with Anthropic with hundreds of thousands of Trainium chips. So I think you were spot on with number one there.
Sadie Lawrence: 00:04:36
Yeah, I am really excited that there is more diversification in the market, although we may start need predicting these trends a little further out because I read today that NVIDIA employees, I think like 75%, are millionaires just because of how much the stock has rise.
Jon Krohn: 00:04:50
Really?
Sadie Lawrence: 00:04:51
So I'm like, well, if only we predicted this out a few years ago, maybe I would be in a little bit better financial position today. But regardless, let's start looking at some of those new players coming into the market, and I think it's just going to be great for everyone involved, particularly startups and getting access to the compute that they need.
Jon Krohn: 00:05:11
For sure. Your second point for 2024 was that we'd have LLMs as a new operating system, what you called an LLM OS, that large language models would transform the way that people interact with machines, meaning that you don't need to be using your keyboard or a phone screen quite as much, that you can have face and voice recognition, and you've been absolutely spot on with that.
Sadie Lawrence: 00:05:38
In my opinion, I think this one is a little bit of a miss personally, and for the reason is that I think our habits take much longer to change as humans. I don't know how much, I haven't seen the stats of how much people are using voice mode, but we're so used to interacting with individual applications. I know there's computer use from Anthropic. I don't see it fully integrating into a full operating system yet. I think we're starting to see the tip of the iceberg, but I think there's still a lot more to come in this space. And so I'm going to be a little bit more mean on myself in this one and say this was only half accurate, but more to come.
Jon Krohn: 00:06:21
Okay, well, you made up for that with number three, which number three could not have been more on the money. So number three, you discussed how we will make advances in LLM capabilities beyond just scaling the size of the network or scaling data. You said that we would have a slow thinking model. Innovative approaches will aim to replicate human slow thinking processes as described by the famous economist Daniel Kahneman, who passed away actually in 2024. And yeah, you said this could lead to significantly enhanced capabilities in logic, reasoning and mathematical tasks, potentially requiring fewer training data. And o1 from OpenAI exemplifies this trend, which I'm sure we're going to see other of the huge hyperscaler companies are going to be scaling along this inference scaling lever that they now have available to them.
00:07:20
And it's interesting, this is for now, it's relatively constrained to problems that you can easily break down into subparts. So like math problems, computer science problems, these can be broken down into intermediate steps where you can use reinforcement learning to say, "Okay, at this first intermediate step, we've done a good job, we verify that we did it correctly. Let's move on to the next intermediate step." And so it'll be interesting to see in maybe in 2025 how the hyperscalers can take innovations like o1 and extend them beyond these relatively narrow, easy to break down into intermediate step tasks, which could be a big challenge.
Sadie Lawrence: 00:08:03
I think where we're going to see it move to is more of scaling in terms of specific domains. And a lot of where I'm pulling this from is from psychology and biology. If we look at brain development, if we pulled scaling laws into brain development, we would just think the bigger the animal or the mammal, the larger the brain, the more intelligence that they have. That is not always the case. What we find within biology is animals in particular develop a unique type of intelligence based on their environment and the environment that they decide to thrive in. And I think we're actually going to see that happen a lot this next year where, okay, we've realized that slowing down thinking works, but that's maybe for really complex problems, not for all types of problems. So I think we're going to move beyond scaling in some really interesting ways this next year, and we're just, again, seeing the tip of the iceberg of what that looks like with models like o1.
Jon Krohn: 00:09:03
Yep, yep, yep. Exciting times as we'll talk about more later. O1 is a really exciting innovation for me. I'm already making spoilers on things that are to come. So that was number three. Number four for 2024 was tool consolidation via LLM APIs. So you said that function calling APIs of LLMs will unify and integrate diverse systems and applications leading to streamlined workflows and a consolidated enterprise toolset. So this has ended up coming to life with this Agentic AI term where you have Agentic AI frameworks that have blossomed over 2024 and have allowed LLMs to make use of lots of tools, online capabilities, interacting directly with computers, like all of the functionality of a computer as opposed to just text in, text out. And so, I mean, this one you probably won't be able to disagree when I tell you that you were on the money for number four.
Sadie Lawrence: 00:10:05
Yeah, I'll try and disagree with you a little bit. I was hoping to see some more tool consolidation in terms of, let's say people using ChatGPT for example, and being at their go-to place for analysis, for coding, for presentations. I still am seeing a little bit of specialized tools. I think Copilot's actually done a little bit better job of it just because it integrates so heavily within the Microsoft Suite.
Jon Krohn: 00:10:32
The GitHub Copilot?
Sadie Lawrence: 00:10:33
No, Microsoft Copilot.
Jon Krohn: 00:10:34
Microsoft Copilot, yeah.
Sadie Lawrence: 00:10:35
Yeah. Just from an enterprise level, we rely still so heavily on Microsoft Office products. And so-
Jon Krohn: 00:10:43
Do you use Microsoft Office products?
Sadie Lawrence: 00:10:45
Okay, I have a heavy opinion on this. I use it in one company now. In my previous company, we were all the startup tools, which were G Suite, Slack, et cetera. Which do I prefer? Definitely the G Suite, Slack, but we work with more enterprise clients and so that's why we switched over to the Microsoft Suite.
Jon Krohn: 00:11:16
Oh, boy, that only for an interesting 2025 for you.
Sadie Lawrence: 00:11:18
It's going to be rough. I don't know what's going on with Search and Outlook, but I'm sure a lot of people feel my pain on this. But I think in terms of tool consolidation, Microsoft is just set up as a leader in that space just because of how much dominance they have in the enterprise space today.
Jon Krohn: 00:11:34
Nice, yeah. And then we've gone off-piste a bit here, but to go for your fifth and final prediction for 2024 before we move on to our next topic category entirely was you talked about there being workplace upheaval, so that sophisticated code interpreters like ChatGPT's, advanced data analysis, that that empowers business users to perform data analytics on their own. And so this, it redefines traditional analyst roles. And I understand that you might have some interesting stats for us on how the 2024 job market changed that bear out what you described last year with your expectation of workplace upheaval.
Sadie Lawrence: 00:12:15
Yeah, unfortunately, we're seeing this heavily impact tech. And people who aren't in tech I think are taking the wrong conclusion from this, which is tech is not a good place to be in and the job market is difficult, but particularly in high-tech companies, we're seeing a lot of layoffs because they know and understand how to implement these tools. So there's a posting from Fred where it showed software development job postings on Indeed in the United States. We kind of reached the peak of that in 2022, and that has been just on a sharp decline since then and at one of its lowest rates since 2020. I think what's interesting about this is that peak coincides right when ChatGPT came out, we saw this new AI boom. And so you can see directly a correlation between technical jobs being affected by AI, particularly because that's one of the easiest place to implement it, but also, technical people are the ones who are aware of what this technology can do and how to implement it.
Jon Krohn: 00:13:25
Yeah, I read just the other day that Klarna, they allow you to break your payments, you buy a stereo, you can do four quarterly payments at the same price as if you were just buying the thing outright immediately. And so Klarna, K-L-A-R-N-A, there was recently a report that they haven't done any hiring basically since that peak that ChatGPT was released at and they've just been using normal attrition to reduce their workforce size. And they were replacing call center people, I presume software developers, don't know that for sure, with AI.
Sadie Lawrence: 00:14:09
Yeah, I think where this is going to get really hard for is recent college graduates or less experienced individuals. That is where AI really shines. I talk about using AI tools a lot, like having a really excited intern. They have deeper knowledge than me and probably fresher knowledge than me in particular areas, but having that context is what they're missing. And so if you are in that position of being a recent college graduate or early on in your career, this is going to be a really key time to expand your skills, but particularly your business skills because a lot of the basic tasks are what leaders are going to look for AI to replace.
Jon Krohn: 00:14:53
Yeah, good shot there. And we will have more later in this episode on what we think for 2025 and how careers and skills will be impacted by the AI tools that are coming and becoming richer as we go.
00:15:08
That gives a recap of our predictions, really your predictions for 2024, which certainly you couldn't even argue with me that you got three of the five spot on. So the demand for compute, the slow thinking model, workplace upheaval, tool consolidation, maybe could've been... I guess you weren't expecting it to kind of be the Agentic way that it ended up proliferating, but we are going to be talking about that a lot more in today's episode. And then the LLM OS, you felt like that wasn't quite right, but still lots of ripe capability there. I think as these kinds of tools, things like Apple Intelligence becoming more useful, more widespread, we'll see LLMs be more of the voice, the face, direct interface with the code behind the scenes, with the back end as opposed to needing to rely on typing like we still have mostly been this year.
00:16:08
All right, quickly, Sadie, before we get to, we're going to do an interesting section next that we've never done before, which is, and this is your idea, I love it, is that we'll pick an overall winner for the year. We'll pick a comeback of the year, a wow moment and a disappointment of the year. But before we get to that, tell us a bit about what your year has been like, something in particular that interests me and is quite timely because this episode is coming out just before New Years, and so this is definitely the time that people are thinking about how they can be restructuring their life to be more successful, more happy in the coming year. And you have your first physical product. So after a decade of creating popular digital products, you now have a physical product that people can order or pick up at their bookstore and could transform their 2025.
Sadie Lawrence: 00:17:08
Yes. This has been such a passion project of mine because I, for the past 10 years being a true data person, have been tracking what I've been doing in half an hour increments for the last 10 years. And I have gained so much from this experience that I decided I wanted my own planner. And a lot of people are like, "Wait, it's tracking, but it's a planner. How does this combine?" I'll explain how it all works together. To be able to do it in a more effective manner.
00:17:37
And so I created a planner called The Observer, and the whole name behind The Observer is to actually observe what you're doing with your time because I am a true believer that actions speak louder than words and we often set goals for ourselves, but then rarely reflect on how we spend our time to align with those goals. And so the whole idea is that you just fill in daily what you did in your day and just that whole process of taking a second to be like where did I spend my time and what did I go to really triggers something in your brain to question, am I aligning my time with my goals? And so there's some pre-setup in terms of imagining the life that you want, putting those into increments for quarters and months and weeks. And then the most important part is that reflection part on a daily basis where you track your time.
00:18:29
It's been a super fun project. Again, it was just a passion project of mine that then people would ask me how I do it. So I created a Shopify account on a weekend and then next thing we know now we're in stores. So it's been really a pleasure to create.
Jon Krohn: 00:18:43
Nice. And we'll be sure to include a link to that, which is theobserver.store and we'll have observer.store in the show notes so that you can pick up your own Observer, see how you are spending your time, imagine how you could be spending your time differently and maybe have a more fulfilling, a more productive life. Cool.
00:19:04
All right. Let's move on to now the section that we promised just before we talked about your planner, which is picking our... Should we do overall winner last?
Sadie Lawrence: 00:19:16
Yeah, let's do winner last.
Jon Krohn: 00:19:19
Okay. So then should we do comeback of the year first?
Sadie Lawrence: 00:19:22
Yes.
Jon Krohn: 00:19:23
Okay, cool. So for our comeback of the year, I'll go first if you want, if want to go first.
Sadie Lawrence: 00:19:30
Okay, yes. And I will say Jon and I have not discussed these, so I'm also really excited to hear what his are and we may have the same ones. I'll let you go first. I have mine written down. I promise I won't change it once I hear yours, but yeah, what was your comeback of the year?
Jon Krohn: 00:19:41
Google.
Sadie Lawrence: 00:19:44
Same, yes.
Jon Krohn: 00:19:45
Oh, same? Yeah.
Sadie Lawrence: 00:19:47
They had too much good stuff that they saved for last. I mean, even in these last few days, it's been incredible. I think it was today Veo 2 came out, Willow AI studio, NotebookLM.
Jon Krohn: 00:20:01
Yeah, Gemini 2.0, it's across benchmarks across the board. Gemini 2.0, they're now competing at the forefront again, like you would've expected Google to be all along with any kind of AI. Two years ago, if we'd done back in 2021 when we were making our predictions for 2022, you would've anticipated that Google would be the front-runner still in 2025. And while they are not clearly the front-runner like they were back then, there are absolutely other competitors out there, some of which we will probably be talking about when we make our predictions about overall winner and so on. I don't want to try to spoil it too much, but it's not a unipolar AI world anymore. It's a multipolar AI world, which is great. It's great to have lots of smart people working in lots of different labs with their own takes on how things should be done safely and how the envelope can be pushed effectively.
Sadie Lawrence: 00:21:14
Yeah. I'm curious, since Google is both of our comeback for the year, do you think that they're going to gain traction in the market or because OpenAI has such a brand presence, when you think of AI and Gen AI, I'm guessing most people think of OpenAI right away? Do you think Google's going to have a hard time coming back from that even though technically you and I both see them as the comeback of the year?
Jon Krohn: 00:21:42
Yeah, it's an interesting question. It probably depends on where you are. I know that I recently came back from a trip in San Francisco and lots of people there seem to be... It's interesting. I went to an event, this Gen AI event where on your name tag, you wrote your name as well as your favorite AI tool. And a lot of people had ChatGPT written on their name tag, but there were also a lot of people verbally expressing disappointment with OpenAI's year in 2024. So it's interesting.
00:22:21
I think OpenAI absolutely had pole position in late 2022 with the release of ChatGPT and they maintained that in 2023 with the release of GPT-4, but I think people were expecting, and who knows, maybe by the time this episode goes out, there will be like a GPT-4.5 release or something like that. But yeah, I think people with the Delta between GPT-3 and 4, I think there was a lot of expectation that scaling alone, scaling data, scaling your number of weights in your model, that would continue to yield the crazy advances that we saw between GPT-3 and 4. And that hasn't borne out.
00:23:08
We are seeing OpenAI still absolutely be competing at the threshold with things like their o1 model, which I absolutely love. We already talked about that earlier in the episode. Scaling at inference time has a lot of potential, especially with tasks that can be broken into intermediate steps and validated at each of those intermediate steps, but yeah, it'll be interesting to see where OpenAI goes and how they are able to monetize or not. That's something that Google still enjoys. They still enjoy this monopoly oversearch and they're so dominant in advertising, they can afford to make some missteps or be a bit slower and still catch up over time. They have huge amounts of compute. They have, I think, probably still the strongest and largest concentration of AI talent. And so I don't know. It's interesting to see where they'll go. OpenAI is potentially more vulnerable, not so much to another giant, existing giant like Google, but other upstarts like an xAI or an Anthropic.
Sadie Lawrence: 00:24:23
Yeah, I think it's shaping up to be a really exciting 2025 because now that we're about two years into this Gen AI movement, it seems like everybody is at the starting line again. And I think that that's just going to make a really interesting year for us next year. But I would agree with you. Should we do the disappointment in terms of who the... Can we jump to that now?
Jon Krohn: 00:24:49
Yeah, let's do it. Let's do disappointment of the year. What's yours?
Sadie Lawrence: 00:24:52
Yeah. I will say it is OpenAI and I think it was really hard just because they set expectations so high.
Jon Krohn: 00:24:59
Yeah, I wrote them down and I scratched it out for something that I thought I of that's even more disappointing to me.
Sadie Lawrence: 00:25:04
What's more disappointing?
Jon Krohn: 00:25:06
Apple Intelligence.
Sadie Lawrence: 00:25:07
Oh, that didn't even cross my radar because I haven't even used it. Yeah, that would be the highest disappointment because that was just what a flash in the pan and just does anyone use it? That's my question. Who uses it?
Jon Krohn: 00:25:24
I don't know. No one's talked to me about it, but there's certainly been a lot of hype around its potential. And yeah, I mean, Apple still hasn't, at the time of us recording this at least, they haven't figured out how to embed... I think for them, a really tricky thing for Apple is that security is so important to them and a reliable product experience is so important to them. And LLMs are fundamentally neither of those things, especially if you're having to send off your user requests to OpenAI's GPT models for processing because Apple doesn't have its own in-house LLMs that are capable of the range of tasks that consumers expect today. So yeah, they're in an interesting pickle, but same thing to Google. Apple has huge amounts of revenue, amazing margins, and they have time to figure this out.I would not be surprised if in another five years from now, Google and Apple are still dominant players in technology.
Sadie Lawrence: 00:26:30
I was going to say that I hope that next year our comeback of the year will be Apple and Apple Intelligence. I mean, yes, they want things to be right, but I mean, who's used the photos app. That was not right. I think we all can agree that the new release of the Photos app was a total miss as well. So maybe Apple Intelligence, and they're tied with OpenAI with as the backend behind a lot of that as well, so they can tie together in both of our disappointments, but I really hope to see both of these back end the game as our comeback for next year because there's so much potential from the phone integration with AI that I would really hope to see this next year.
Jon Krohn: 00:27:11
For sure. All right, so yeah, our comeback of the year, we both agreed on Google. Our disappointment of the year, your topic was my second topic with OpenAI, and it sounds like Apple Intelligence did disappoint both of us. Let's move on now to our wow moment of the year. Maybe I'll go first on this one since you got the last one. And so, for me, I've already alluded to this, it's o1 from OpenAI. It's interesting that simultaneously that expectations were so high for them that they could both be the disappointment of the year and the provider of our biggest wow moment.
Sadie Lawrence: 00:27:48
Yeah, I think that just shows how high expectations were, are and they continue to be within AI. I think that all of us in AI now are almost TikTokified. I don't even know if that's a word, but in terms of wanting that quick dopamine hit of if something isn't happening this week or something that's not wowing us or blowing us away, we just write it off. So it's interesting that you have them as your wow moment when it's also my disappointment because I think it really just ties into expectations are high and we are looking for that next dopamine hit in AI every single week, if not every day.
Jon Krohn: 00:28:32
What's your wow moment?
Sadie Lawrence: 00:28:33
So my wow moment is not necessarily from an overall use, but just from a human level of when I just listened to it, and this should give you a key to what it is, but I was just truly impressed. And that was with the NotebookLM and the podcasting.
Jon Krohn: 00:28:53
That's number two. That's my number two.
Sadie Lawrence: 00:28:53
And the reason why it just was so human to me, and that's why it wowed me is their expressions, the way that they talked. It felt like you and I talking on a podcast. So just from a human level, is it going to change the world? I don't know, but I just thought it was cool. And so that was my wow moment.
Jon Krohn: 00:29:15
Absolutely. I almost had that as my number one as well. And we did an episode of this podcast, number 822, which came out in late September. In that episode I expressed how blown away I was by NotebookLM, and I also air in its entirety a 12-minute podcast episode about my PhD dissertation, which is so boring, but these fake podcast hosts did manage to make it seem exciting. And so I included it in full, in the episode and people were blown away. That must be one of my most commented posts of the year of a large number of people reaching out and saying, "Wow, I hadn't heard of this, or I hadn't used this, and now I have used it and it blew me away. Here's what I tried." So yeah, that was really cool. I think it was a wow moment for a lot of people.
Sadie Lawrence: 00:30:10
Yeah. I'll add one sub-wow moment in there which may not get talked about.
Jon Krohn: 00:30:13
Sub-wow.
Sadie Lawrence: 00:30:13
I hope we have a sound effect for that too, sub-wow, or maybe it's its own sound effect, but I recently got a Tesla and the full self-driving on that is incredible. And I was just blown away because as a kid, my mom was like, "Hey, you really need to learn to drive and do all these things." And I told her one day, "I will have somebody who drives me around." I did not think would be a robot in full self-driving, but here we are today. So just to have childhood memories of saying something and then to be living it today is truly incredible.
Jon Krohn: 00:30:50
That also, I've got to add my sub-wow moment, which is-
Sadie Lawrence: 00:30:53
Sub-wow.
Jon Krohn: 00:30:57
... which is Waymo. I had my first Waymo experience this Northern Hemisphere summer, and that was really cool, having a car, because I think that's another level of autonomy beyond Tesla's full self-driving, right? Where with Tesla's full self-driving, you need to have somebody sitting behind the wheel. But to have the Waymos now in San Francisco and at the time of recording also in Scottsdale, Arizona, I think, you can just use the Waymo app and a driverless car comes up, picks you up, you get in it and it drops you off. I almost want to make that my biggest wow moment of the year. I don't know how I didn't think of it right off the bat, but I mean, because that physical presence because that's...
00:31:44
I come back to the Waymo example a lot with when people ask me, when people find out I work in AI, as the quote, a lot of people completely outside of AI will say things like, "Oh, contentious," and I'm like, "Really? Oh, I wasn't aware it was so contentious." And they're like, "Well, yeah, I'm a creative or I have lots of friends who are creatives," and I can see that okay, yeah, I can see why it's so contentious. But for me, I guess I'm so often seeing big changes and benefits. But there are, you know, the Waymo example is one that I come to frequently to say, this is a... as opposed to something that's happening on your computer screen, this is a physical, very obvious manifestation of AI that when you experience that, when you call a Waymo car, get into it and drops you off somewhere, you see the steering wheel spinning all on its own and it's making great driving decisions, that makes it clear that in the future, in the not too distant future, we don't need drivers. We don't need human drivers.
00:33:03
In the United States, in most states, the number one job is truck driver. And there's tons of related jobs that support the truck driver, people working in cafes along the roadside and that kind of thing. You don't need that. Self-driving cars don't need cafes or motels. And so it's going to make a really big impact. And given the upheaval that will be caused by this, there's things that we need to be doing as a society in terms of retraining people because this AI shift should end up being, just all other automations in the past, it should provide people with more interesting work than ever before. And I mean, this time, there's talk about this time being different, but all past increases in automation have led to more employment and lower unemployment. So I don't know. I've touched on a lot of topics there, but I haven't let you speak for a long time. So, Sadie.
Sadie Lawrence: 00:34:07
No, I was really lucky this year to hear one of the co-CEOs of Waymo talk, and one of the things that she said was, "We are building the single best driver." And I found that really interesting because she talked about how they have over a hundred thousand fleet of cars out driving, but she talked about it as one driver. They talk about it as a single brain, a single driver brain, and they're only building one. And that just resonated with me so much because it really gives us perspective of what intelligence and machine intelligence can do at scale, right? You only need to build one of the best single drivers and you can change a whole industry. And so I think that's something just to think about. Get really specific in the models that you're building and the domain that you're building because when you do that at scale, it's incredible.
Jon Krohn: 00:35:06
Agreed. That's a really nice way of thinking about it. I hadn't heard it phrased that way before, but now that's a great meme to be thinking about industry by industry. How can I create a brain for some specific task that can be outstanding at that specific task? And then, because it's software, you can just replicate it as much as you like, software update to the fleet of a hundred thousand cars transform the industry. Yeah. Cool. All right, so then that leaves us just with overall winner still to select. Do you want to go first or do you want me to go first? I feel like this was all your idea. And so maybe you should go first on this one.
Sadie Lawrence: 00:35:48
Okay. My overall winner for 2024 is open source, and particularly I think just us-
Jon Krohn: 00:35:58
That's not a company.
Sadie Lawrence: 00:36:02
... as individuals... Okay. I could choose Meta then and Meta's Llama-
Jon Krohn: 00:36:04
Oh, wow. Okay.
Sadie Lawrence: 00:36:05
... particularly if you want me to get specific, but I really think it's us as consumers. I think we are going to be the winners of AI because it is open source. I'm seeing more and more startups entering the market. And just the capabilities of the models that are available, the costs are getting cheaper and cheaper. So it's an open source community. We saw that also from a legal perspective with some of the bills that were being passed in California that got turned down and was really great for the open source community. So that's why I say open source in general from a company perspective, what Meta's done with Llama. And from an individual perspective, I think it's a consumer who's really going to win from all of this.
Jon Krohn: 00:36:50
Wow. Nicely done. Nicely done. You went very lowercase M, Meta, with your answer. I just picked a company which is Anthropic.
Sadie Lawrence: 00:37:05
I had a feeling this was going to be yours. Please tell me the love for Anthropic of why it's your overall winner.
Jon Krohn: 00:37:12
Yeah, anyone who knows me knows that I'm a big Claude fan. I use it more... I subscribe to Gemini, I subscribe to ChatGPT Plus, I subscribe to you.com, but I end up at least 80% of the time using Claude as my weapon of choice. I don't end up needing the kind of o1 capabilities on a lot of the tasks that I'm doing. Probably that could be related to tasks that I'm doing are relatively common coding tasks related to neural networks. And on that kind of thing, Claude 3.5 Sonnet does an outstanding job debugging things. I don't need all that power of o1. I just get an answer quickly from Claude. And same kind of thing, things related to the podcast, summarizing episodes for me, transcribing things, Claude just does an amazing job. It does the best job of any of the LLMs, private LLMs that I'm regularly using.
00:38:18
There are still places where I use Gemini. If I have very large files, I think Gemini is great. I haven't used Gemini 2.0 that much. Maybe if I'd been using it a lot more in recent days, I'd say, wow, this can replace Claude for me. But I also, I love the UI of Claude. Think they've done a really good job of it. It has this friendliness and this warmth to me that I don't get from any of the other tools that I subscribe to. All the other ones seem like they're designed to look really futuristic, but it creates this coldness, whereas somehow, yeah, Claude manages to give me the warm fuzzies.
Sadie Lawrence: 00:39:00
That's the first time I've heard a comparison of what tool makes you feel better. I like that. We can maybe add that to one of the winning awards next year. I'm curious though, because you didn't mention Perplexity, you use Perplexity. Is that in your repertoire? That's something that I have been using-
Jon Krohn: 00:39:16
I don't use Perplexity much.
Sadie Lawrence: 00:39:18
... a lot. I love Perplexity.
Jon Krohn: 00:39:21
Oh, yeah?
Sadie Lawrence: 00:39:21
Mm-hmm. So maybe add that to... Tell me how, if it gives you any more fuzzies, I'm curious about. It will be hard to replace Claude, but yeah, Perplexity is something that I would say in the past two months, I've really started to dive into a lot more and love just the references and have pretty much replaced search with Perplexity now.
Jon Krohn: 00:39:43
Wow, there you go.
Sadie Lawrence: 00:39:46
Christmas shopping, wasn't able to find a product that I needed to buy for someone. They were all sold out. Thanks to Perplexity, was able to find it, snag it, and have a very happy Christmas.
Jon Krohn: 00:39:59
Cool. Yeah, I think I've ended up using you.com for some of those same kinds of Agentic tasks, but yeah, I'll definitely try. I've only done a few searches in Perplexity. I don't know why that ends up happening. It's definitely a blind spot that I've felt on other podcast episodes in the past, so I'll have to spend some more time with that.
00:40:19
All right, so we have been... This episode is all about forecasting the future, but like your planner, so far we've been looking back at 2024, but I also think that sets the stage for the predictions that we are making in 2025. And it also gives you a sense of how reliable we are as crystal ball readers for the forthcoming year and largely good, largely good. Largely you can trust us based on our performance in 2024 and the previous years, which you can go back and listen to if you want to.
00:40:50
So for 2025, the big number one topic that you highlighted, Sadie before we started recording and I agree with 100%, is Agentic AI. There's no question. I could speak for a long time about this, but you are the guest, so I'm going to just let you go first, please.
Sadie Lawrence: 00:41:09
Yes. I mean, this has started in what I call the fall conference season to boil up as a key topic, I think it's going to take over all conferences will be about Agentic AI next year, but more importantly, the number of startups in this space. I think there's over 600 as of today, and who knows if that's even an accurate count of it, but it's really the next step of what I think all of us as consumers are also looking for is we're looking for more of those autonomous agents where we don't want to be copy and pasting from different applications. We wanted to just go on our behalf.
00:41:47
And I think the, I don't know if it was a meme or an actual article from this year where it was a woman who said, "We got AI and it creates things, but I was hoping it would fold my laundry or do basic tasks." And that to me is really where the need for Agentic AI comes in. That's more of a robotic task is folding your laundry, but particularly, we wanted to now, what I would say is leave the confines of an application and go do actual tasks for us autonomously on our behalf. I think that the trust in AI has built that we as humans are ready to maybe unleash it to that next step. We see a little bit of that with computer use, but more importantly, I think we're going to see a lot of companies who just like Waymo are building a specialized brain for a particular application, one that will be the best ever social media marketer, one that will be the best financial analyst. And so we'll have these specialized models who will be able to truly be agents on our behalf and take autonomous steps.
Jon Krohn: 00:42:55
Yeah, it's going to be absolutely transformative. I think to go into a bit more detail on that viral post that you were mentioning there, I think it tied into the woman who wrote that post saying, "I wanted automation to be taking away the mundane tasks from me," like you said, folding laundry and leaving the creative tasks like video production, artistic endeavors to me, but instead AI's taken that away and I'm left folding the laundry. But Agentic AI systems are a step on the way to having even in the real world more Waymo-like physical embodiments, making changes and being able to fold your laundry for you. That is coming and it seems like now in our lifetimes for sure, that we will have machines that can do this kind of stuff and hopefully most people, if not everyone, will be able to afford machines like that. These should be widely available, not something that's just for some small percentage of the planet.
00:44:03
I recently came back, at the time of recording, I've just come back from NeurIPS, Neural Information Processing Systems. This is the 30th year it's been running and it's I think safe to say the most prestigious AI conference for academics that is out there. And at this conference, the Agentic AI trainings, workshops, some of them were so, there was so much interest in them that there was a crowd of people outside the room unable to squeeze into the standing room. And so that gives you a sense of how popular this topic is. Fei-Fei Li gave one of the keynotes at NeurIPS this year, and she talked a lot about how her company, WorldLabs, is creating sophisticated data sets that will allow agents to explore 3D visual worlds as opposed to just being right now visual agents are becoming pretty good at recognizing two-dimensional images, but that doesn't necessarily make them great at exploring the real world and being able to fold your laundry.
00:45:14
So yeah, really exciting things happening in Agentic AI. I am completely smitten with Agentic AI and we've been doing podcast episodes on it a fair bit recently. We have many more planned for 2025, early 2025, with Agentic AI experts. So look out for those. Yeah, I'm going to be creating a half-day or full-day Agentic AI hands-on training for ODSC East, Open Data Science Conference East in Boston this spring. Yeah, I think it's a fast-moving space, but it's unquestionably where these things are moving and it's a testament to how far along we've come with LLMs and their ability to be accurate because if you're going to have AI agents going off and doing tasks autonomously that you've assigned them, or having an AI agent master that's spinning up a bunch of slaves, that's a really rough word to be using. There's got to be a better one. But in computer science, that is often the word that's used, but sub-processes, sub-agents, sub-wow agents.
Sadie Lawrence: 00:46:28
Sub-wow, there we go.
Jon Krohn: 00:46:31
In order for any of that to work effectively, you need to have accuracy. If your LLM is hallucinating 10% of the time, that's an extremely large amount if you're going to have a large number of processes running. But if you get to a 1% or a fraction of 1% error rate, and with things like o1 from OpenAI, being able to check their work step-by-step, this evolution in LLMs, the way that they've advanced, it allows us now to be in this Agentic AI moment that we're in.
Sadie Lawrence: 00:47:06
I'll just add one bit of caution on this one where I can see it not manifesting in the way that we may hope for and fully imagine, which is I think we have a lot to figure out in permissions and access. And so just because your agent can go and do something and is accurate, we switch quite frequently between applications on our computer, and giving it that automatic permission may not so much be an issue from a human perspective, but companies playing nicely with each other. And so I think that will be really interesting to figure out is, okay, will Apple give Microsoft access? They've done it with having Outlook as one of your mail applications, but how does that look like once different agents are released into that platform? So I'm curious to see how that will all work out this next year.
Jon Krohn: 00:48:02
Nicely said. Yeah. I mean, this is the wall that we run into with LLMs or Agentic AI frameworks being able to be effective is around data security, privacy. It would be ideal to be able to take a whole giant enterprise and say, "Okay, Agentic AI system, here is all the information," but of course, if you do that, then you're opening it up to abuse. I mean, because then all of a sudden someone in the software engineering department can write an agent that's like, "Provide me with the pay packages of everyone in the company."
Sadie Lawrence: 00:48:38
Exactly, right? Or who's not to say some other agent can't come in and fool your current agent to think that they're a helper. I mean, there's so many ways that this could go wrong, but I think the difficulty of this task that we're facing right now is really on the security and privacy side of things.
Jon Krohn: 00:48:55
Nice. All right. So that's number one. Agentic AI is our big prediction for 2025. I feel very safe in this prediction. Seems like a layup. Number two from you is AI integration into everyday devices. So this is a higher risk one from you, I guess similar to your LLM OS prediction for 2024. But yeah, tell us about your idea here with these everyday devices. Give us some examples.
Sadie Lawrence: 00:49:24
Yes, I think this is a playoff of the LLM operating system and maybe more of a stepping stone to get there. But where I'm seeing this go is recently I got the Meta glasses. I particularly just like buying different tools and applications just to test things out and was really impressed with them overall. I'm going to Mexico next week and I really want to test out the real time translation. So I think that we're going to start to see AI just streamlined into more and more of our devices.
00:49:58
We talked about our disappointment with Apple Intelligence, but again, I think that they have so much cash flow, they have access to so much talent, we could see them make a comeback next year. And then Microsoft this year released what they called their AI computer. I know they took it off the market for a little bit because people were concerned about the privacy of it, but it was essentially taking snapshots of your screen throughout your workday and giving you recommendations and starting to train how it can be its own assistant with you. Again, back to the privacy and security concerns, they took it off, but I think next year we'll have something figured out where we'll see AI woven more seamlessly into everyday devices more than we have today.
00:50:44
On the last one there, I will say I do think some will not always be smart weaving into products. When I went to Costco this year and saw AI on a toothbrush, an electric toothbrush, that's how you knew it has gone too far. So I didn't say it will all be positive, but we will see it continue to pop up in our everyday devices.
Jon Krohn: 00:51:04
Yeah. Where I was staying in Vancouver, I would walk from the Airbnb that I was in Kitsilano, beautiful area of Vancouver. I'd walk from my Airbnb to F45, which I'd never done before. Have you ever worked out with F45? You ever done that?
Sadie Lawrence: 00:51:18
No, I've walked by one, but I've never walked in one.
Jon Krohn: 00:51:22
That was pretty cool. It was like Sadie and I both do CrossFit and they don't have barbells. You don't spend a lot of time on technique, but in 45 minutes you do get a good mix of cardio and strength work. It was an interesting experience for a week. But anyway, when I was walking in between my Airbnb and F45, every day I would walk past this golf pro shop and all of the posters in this golf pro shop were for drivers that had AI in the name. So it's things like AI smoke. And there isn't an-
Sadie Lawrence: 00:52:01
Wait, golf drivers with AI in them?
Jon Krohn: 00:52:04
I don't think... Honestly, I didn't look into this. My assumption was that there isn't AI in the club. I mean, I don't understand how that could be, but my assumption is that somehow AI is involved in the design of the club in some way. That's got to be it.
Sadie Lawrence: 00:52:22
Maybe there's some tracking on it that there's an app that you upload, the data of how you swung and what-
Jon Krohn: 00:52:30
Maybe that is what it is.
Sadie Lawrence: 00:52:30
I don't know. I am not too into... As listeners can hear, neither of us sound like we're that into golf to help us out here.
Jon Krohn: 00:52:37
No, I'm so glad.
Sadie Lawrence: 00:52:38
Some of the audience will have to give us some help on this one.
Jon Krohn: 00:52:41
Yeah, please tell us, comment on social media and let us know whether there's computer chips in golf clubs now. It would not be shocking if they were. Like you say, they're in toothbrushes. They will be increasingly around.
Sadie Lawrence: 00:52:57
Will in your brain soon.
Jon Krohn: 00:52:59
Well, yeah, that is happening more too. So Agentic AI, number one. AI integration into everyday devices, number two. Number three, I love this one. You see AI-driven scientific research and innovation becoming a much bigger thing. I absolutely agree. I'm going to quickly preempt what you're going to say with a couple of recent episodes that I released on this topic. So number 812 was on this Japanese company that's full of Google DeepMind alums called Sakana, S-A-K-A-N-A. They released this AI scientist paper that was able to draft papers specifically on machine learning at that time, and they were planning on spreading to other industries as well, but being able to write papers, come up with ideas for papers and run the experiments, get the results, write up the results independently, and did a pretty good job. And this kind of thing will only get better, especially in that kind of environment where the AI system can actually be doing experiments.
00:54:04
And I think we will see more... I don't know if we'll see big examples of this prominently in 2025, but it is definitely the future that big pharmaceutical companies or big energy companies, they will allow AI systems to run physical labs. And so for episode 812, for this AI scientist, the reason why they stuck with machine learning problems is because these experiments could be run in silico, it could be run on computer hardware, but in the not too distant future, as I'm sure you're about to say, Sadie, these AI systems will control physical labs that can also do physical experiments. And so science will accelerate because of that. The machines don't need to sleep. And Ed talked about in episode number 835 with you.com's co-founder, CTO, Brian McCann, Brian talked a lot in the episode about how scientific discovery AI systems will be able to do kinds of scientific discoveries that a human never could because AI systems are trained on all knowledge and no human scientists are expert across all domains. Anyway, I kind of, hopefully I didn't take too much air of your AI scientist balloon, Sadie.
Sadie Lawrence: 00:55:22
No, I think it just shows how much excitement we both have for this area. And we saw just the beginning of this where I think it was Time Magazine shared an article about how ChatGPT was better than doctors in diagnosis. And just to your exact point with scientific discovery, we have AI models that consume so much more information than we ever could. And being a doctor, yes, you go to lengthy years of medical school, but keeping up with all of the new drug discoveries, all of the new scientific discoveries, there's just not enough time. And so I'm really excited for this just from a medical side in terms of not only new drug developments and new diagnosis, but that support for doctors in the healthcare system, and then just in terms of the new inventions of what haven't even been created.
00:56:16
And so there was a paper that came out this fall from MIT. It's about artificial intelligence, scientific discovery, and product innovation. And what they found was that AI-assisted researchers achieve 44% new material discoveries, 39% more patents, and had a 70% raise in new product prototypes. So just really truly incredible results. And I think this adds to this utopia future that I know you often talk about and I hope to actively build with AI, particularly in the realm of science, medicine and innovation.
Jon Krohn: 00:56:54
Yeah, but speaking of the dystopian aspects of these kinds of things, I think that same study showed that use of tools like ChatGPT to accelerate research led to job dissatisfaction increases as well for the scientists.
Sadie Lawrence: 00:57:09
Yeah. Okay, this is so fascinating to me because the job satisfaction definitely depends on the job type. When looking at knowledge workers, so there were two studies, one from Slack and then one from Microsoft, knowledge workers love using AI for their day-to-day jobs, whereas scientists, and what I would say is more specialized individuals feel that it's taking away the joy of what they were doing. And I find this so fascinating because my hypothesis is that as a scientist or as a doctor, you spend many years going to school to have a particular knowledge and then you feel like you're being replaced. Whereas my hypothesis for knowledge workers is you're just try to get a job. Maybe it's not what you majored in. You don't really like your boss. You're just trying to get by. If AI will come in and do part of that job for you, then the happier you are. And so to me, that's my hypothesis behind it, but if someone does a study, please let me know if you test this hypothesis.
Jon Krohn: 00:58:15
That sounds reasonable to me there. All right, so we've done three of our five predictions for 2025. Agentic AI, AI integration to everyday devices. Number three just now is AI-driven scientific research. Number four is enterprise AI monetization. Sadie, tell us about that one.
Sadie Lawrence: 00:58:34
Yeah, this year we talked about new competitors entering the market with compute. We also, I think, saw some investment in nuclear and in energy. I think we're going to continue to see more of that next year, but I think we'll start this year to see what is that investment turning into. And so for me, what I'm really watching keenly next year is XAI, so Elon Musk, subsidiary of X, but now the AI company and the supercomputer that they built where they broke the coherence bottleneck. And I think this is going to be really our first test fully on the scaling laws and if we're going to actually break the scaling laws or we are going to be constrained by the scaling laws or if we're going to actually have another major breakthrough in this place.
00:59:29
And so this year I saw a ton of investment. I think Mark Zuckerberg talk about how he had bought 600,000 GPUs from NVIDIA. You'll have to correct me if I'm wrong on the exact number, but I mean, people were buying so much, I'm ready to see the return on this investment and I think we're going to continue to see more investment in infrastructure this next year, but particularly in energy and in nuclear. Jon Krohn:
00:59:57
Fact-checking you, Meta plans to have nearly 600,000 NVIDIA H100 GPUs by the end of 2024, says the AI overview provided automatically to me by Google Gemini. As of March 2024, Meta had 350,000 H100s. Again, yeah, so that is... Yeah, it's interesting. These quotes are all coming from the beginning of the year. So it was like a January 2024 article where Mark was saying that by the end of 2024, he expects them to be running 600,000 H100 GPU equivalents, which is interesting because then it doesn't necessarily mean NVIDIA H100s. It could be like an AWS Trainium 2 chip or that kind of thing, but something of roughly that capability.
Sadie Lawrence: 01:00:57
Yeah, because my understanding that xAI's 100,000 GPU AI cluster is the largest that has been built to date.
Jon Krohn: 01:01:06
Yeah, I think it could be that Meta's are distributed for different programs, different geographies. It's not, whereas I think that xAI cluster is one single cluster training one LLM.
Sadie Lawrence: 01:01:23
Yes.
Jon Krohn: 01:01:25
Nice. It is interesting, this enterprise AI monetization topic, Sadie, because obviously as we spend these huge amounts of money on hardware, on talent, it means that the impacts on the bottom line need to be substantial for the project to ultimately be profitable. And this is something that we've talked a lot about on this podcast recently with guests. We had a great one, episode number 843 with the CEO of a company called Protopia, Eiman Ebrahimi. He talked a lot about these trade-offs between profitability and security and how so many projects, AI projects end up getting stuck in proof of concept purgatory because you can quickly spin up using Claude 3.5 Sonnet or OpenAI o1, some cutting edge LLM capability. You're like, you can create a Jupiter notebook or some simple Gradio application within a few hours or over a weekend that does amazing things.
01:02:37
But then you bring that to your manager, to a VP at your company and you say, "Look at this amazing tool I built," and when you start to get into the nitty-gritty of, okay, but how will we make this performance and cost-effective in production while also ensuring data security, enterprise AI monetization is a tricky point. And it also means that it provides a lot of opportunity for people out there who are listening to specialize in thinking about these kinds of problems because as it becomes easier and easier to use an LLM to spin up the code that you're going to use or just use an existing API to provide all of the intricate AI functionality of your platform, it's these kinds of questions like, how can I do this profitably for me or my business, that will allow you to really get ahead in data science.
Sadie Lawrence: 01:03:28
Yes, as much as data scientists are numbers people, I unfortunately fill the financial numbers are what are often forgot to be run when they build these models. And not just from the cost to actually put this model into production and continue to maintain it, but what is that return on investment? And so, yes, if anybody wants to get ahead in their career, just include a small financial model with your proposal and I think you'll get through the red tape a lot faster.
Jon Krohn: 01:03:58
For sure. And if you're looking for an episode we did this year that digs into that idea, a lot of profitable AI projects, is number 781 with Sol Rashidi. Sadie, do you know Sol?
Sadie Lawrence: 01:04:10
Yes, I do. She's been on the Data Bites podcast.
Jon Krohn: 01:04:13
Sure.
Sadie Lawrence: 01:04:15
Met her at a couple conferences before. She's great.
Jon Krohn: 01:04:18
She is really great. I hope to have her on the show again soon. That episode ended way too quickly and I had so much more to talk to her about, really well reviewed. So yeah, Sol, maybe we'll have her back on the show in 2025.
01:04:32
All right, but speaking of episode length, we are reaching the final, the fifth and final prediction for 2025. This one is maybe controversial. We'll end up seeing what happens here. I wanted to touch on with one of these predictions how data scientists or people working in AI are going to be impacted by all these innovations in 2025. And we touched on something there. We already talked just at the end of number four on enterprise AI monetization, how helpful it can be to be, like you said, that idea of including a financial model as part of your proposal. That is probably going to cut through red tape a lot more quickly. So that's great advice there.
01:05:16
So financial savviness can be something, but in terms of the "hard skills" of data science or AI, I expect that in 2025, the demand from the market, from employers for AI engineering skills will surpass the demand for data science skills. And when I say data science skills, I mean I guess everything else that data scientists do other than doing AI engineering, which includes things like using existing third-party API's, building prototypes, linking a bunch of APIs together, maybe downloading open source model weights and fine-tuning them, or even fine-tuning a closed source model, which is something that you can do, all those kinds of AI engineering skills. I think that the demand for those will surpass all other data science skills put together.
01:06:12
Now, that doesn't mean that all those other data science skills aren't valuable, even from things like data visualization to data communication, which are relatively soft, but even harder skills like statistics and other kinds of machine learning approaches, regression models, none of that stuff is going away. But in terms of what the market is asking for, I think we're going to see there's going to be surplus in demand for AI engineering skills specifically. And so that seems clearly to me to be an area to be leaning into as a listener to this show and add that as a complement to whatever existing data science skills you have. You don't need to be... Or you could actually even listen to our most recent Tuesday episode, which was number 847 with Ed Donner in which he talks about how you can become an AI engineer and you could jump right to that without necessarily developing other data science skills. But I think the most powerful way of approaching this is to have the AI engineering skills that Ed talked about in detail in episode 847 and using those to complement whatever existing skill set you have.
Sadie Lawrence: 01:07:24
Yes. I mean, to add on to what you said, I would say I think it's really a consolidation of roles and additional skills added on. If you signed up for any career in technology, you've signed up for a lifelong learning approach. And so it's just continued to expand. Just as you mentioned, it's not saying your data visualization skills are going to go away, not at all. Your Python, your SQL, none of that's going to go away. It's just going to be a continual evolution in building onto it.
01:07:53
Something that came to my mind as you were sharing this was when I entered the corporate world was in 2014, and at that time, moving to the cloud was a big deal. I remember meeting people in the IT teams who were like, "Yes, I used to manage these servers and these products," and they were nervous about moving to the cloud because they thought there was going to be less to manage, but what it came to find out is just the shape of their job changed. Instead of actually physically managing servers, you were managing permissions within the cloud. And so I see it somewhat similar in the role of data science as well. Maybe you're not training a model from scratch anymore, but fine-tuning a model and building onto it, really looking at how does that integrate into your existing products, into your pipeline. And so while it may seem like your job is changing or evolving, you're probably actually going to end up doing more, doing more roles and taking a higher approach to the orchestration of how it's done.
Jon Krohn: 01:08:59
Nicely articulated, Sadie. Yeah, spot on. All right, so I think that rounds up where we are. Oh, I have one last thing to add to this number five. I just consulted my notes here quickly, which is that something that's really annoying for me that has happened as a result of people integrating LLMs into everything that they're doing and allowing people to do more with less thanks to AI, the most annoying thing is how on LinkedIn now every post I make, I get several completely banal comments written by obviously generative AI. And if I believed that what was written there... Okay, let's say you have some thought that you want to express about some post on LinkedIn or some social media platform and you want to clean up the language around it, great. But so often, these comments are obviously there's no meaningful opinion in there-
Sadie Lawrence: 01:10:08
There's no soul to it,
Jon Krohn: 01:10:10
There's no soul to it.
Sadie Lawrence: 01:10:12
I completely agree. I don't even know how people do it. Maybe there's some extra premium subscription that I'm not on because I'm like, if you're taking time to copy, paste... But yes, the soul of it is missing. I mean, my whole thing is that misspellings are the new captures, right? Actually, throw in some misspellings, throw in some weird punctuation, just make it real and raw. And particularly if you're commenting on Jon's post, make sure that it has some soul in it and no more AI comments on that side.
Jon Krohn: 01:10:46
Yeah. So that is my conclusion for 2024. If you're out there doing that, please reconsider. You're just filling us with noise, humans with noise.
01:11:00
Well, Sadie, thanks so much again for the fourth year in a row running, making these predictions with us. I love how we spent more time on the retrospective stuff this year. I think it colored our predictions a bit more and was maybe a bit of fun for listeners in terms of what they think about as the overall winners this year, the comeback of the year, the wow moment, the disappointment of the year. I would love to hear from all of you in the comments with your human typed comments or even maybe Gen AI augmented, but just your real opinion at the soul of that comment on this episode to hear what you think about us doing the format this way and what your predictions are for 2025. Sadie and I would love to hear those and I'm sure we'll both comment in reply using our fingers on a keyboard.
Sadie Lawrence: 01:11:49
Or some voice to text because that always gets real messy and fun. I can't 100% guarantee it'll be fingers on a keyboard, but it will at least be from my soul to your soul.
Jon Krohn: 01:12:00
Nice. All right, so yeah, Sadie, anything else you'd like to add before we wrap today's episode? I don't think it makes sense for us to do the... Well, I'm not going to ask you for a book recommendation because we do this every year unless you have something that you think this year, a book recommendation that people need to know about something that touched you in 2024.
Sadie Lawrence: 01:12:18
Ooh, books that touched... I wouldn't say there was a book that touched me this year. I think my only comment and thought to wrap this up is one, just like how lucky we all are to live in this time and to work in this space. And maybe you're trying to get a job in this space, but the fact that you're even listening to this podcast shows that you have a deeper interest, a deeper knowledge, and that's a super lucky position to be in. And I hope that all of the awesome things that we talked about in this show, people go take advantage of, use these tools, are inspired and really create that utopian future that you and I hope for we dream of, we can see because to me, it's just such an exciting time to be alive. I'm so excited for this year ahead of us and I think it's going to be a really transformative year and I can't wait to connect with everybody and see what we got right, see what we got wrong, but most importantly, make adjustments along the way.
Jon Krohn: 01:13:19
For sure. Nicely said, Sadie. So for people who want to listen to you between this predictions episode and the one coming in about 365 days, Sadie, how should people follow you or connect with you?
Sadie Lawrence: 01:13:34
Yes, social media is great. I do love LinkedIn, X, Instagram, all of the things, I'm a big fan of. I do have a Substack as well, so I do share out what happened on a monthly basis. Happy to connect in all the common ways.
Jon Krohn: 01:13:50
Nice. I think when we did our very episode, our very first predictions episode or one of my first episodes with you on this podcast, your Instagram had been hacked or something like that, but now you're back on track?
Sadie Lawrence: 01:14:03
I'm back and better than ever, yes.
Jon Krohn: 01:14:05
Nice.
Sadie Lawrence: 01:14:05
So I had to start from scratch again. I had a viral post last year that gave me lots followers because I had a typo on it that said, "I married my dog." So there you go. This is why you still want to write your posts yourself because those typos can turn into your biggest win.
Jon Krohn: 01:14:26
What had you really done? Carried your dog?
Sadie Lawrence: 01:14:29
No, I think I had added something... I don't even remember now. It was like I got married and something about I have a cute dog. Anyways, my marriage and my dog got intertwined and through some editing and unfortunately they came together.
Jon Krohn: 01:14:44
That's funny. Nice.
Sadie Lawrence: 01:14:46
People are still in the comments arguing, "Did she marry her dog or not?" So you can join in on the debate.
Jon Krohn: 01:14:52
All right. Sadie, thank you so much for taking the time again this year. I know how valuable your time is and I'm always appreciative of it. Thank you so much and we'll catch you again next year.
Sadie Lawrence: 01:15:03
Sounds great. Thanks, everybody.
Jon Krohn: 01:15:10
What an exciting year we have ahead in 2025 indeed. In today's episode, Sadie predicted first that Agentic AI will be the dominant trend moving beyond single applications to create specialized networks that can autonomously handle complex tasks. Those security and permissions between platforms remain a challenge. Her second prediction was that AI integration into everyday devices will accelerate from augmented reality glasses with real-time translation to more sophisticated personal computing experiences, though not all integrations will prove valuable. Her third was that AI-driven scientific research will expand significantly, building on current successes where AI-assisted researchers achieved 44% more new material discoveries and 39% more patents than researchers who weren't AI-assisted. And Sadie's fourth is that enterprise AI monetization will be crucial as companies seek returns on massive hardware investments. And finally, our fifth prediction for 2025 is that demand for AI engineering skills will surpass traditional data science skills, though this represents an evolution of the role rather than a replacement, requiring practitioners to build on existing technical foundations with new AI engineering capabilities.
01:16:23
As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Sadie's social media profiles, as well as my own at superdatascience.com/849. Thanks of course to everyone on the Super Data Science podcast team for their work all year long. That includes our podcast manager, Sonja Brajovic, our media editor, Mario Pombo, partnerships manager, Natalie Ziajski, researcher Serg Masis, our writers Dr. Zara Karschay and Sylvia Ogweng, and our founder Kirill Eremenko. Thanks to all of them for producing another visionary episode for us today to wind down the year.
01:16:58
All right, if you enjoyed this episode, please share it with people who you think might like it. Review the episode, of course, on your favorite podcasting app or on YouTube, subscribe if you're not a subscriber, but most importantly, I just hope you'll keep on tuning in. Thanks for listening in 2024 and I hope to see you a bunch more in 2025. I'm so grateful to have you listening and I hope I can continue to make episodes you love for years and years more to come. Until next time, keep on rocking it out there and I'm looking forward to enjoying another round of the Super Data Science podcast with you very soon.
This is episode number 849 on Data Science Trends for 2025 with Sadie St. Lawrence.
00:00:12
Welcome to the Super Data Science podcast, the most listened to podcast in the data science industry. Each week, we bring you fun and inspiring people and ideas, exploring the cutting edge of machine learning, AI, and related technologies that are transforming our world for the better. I'm your host, Jon Krohn. Thanks for joining me today. And now let's make the complex simple.
00:00:46
Happy New Year and welcome back to the Super Data Science podcast. I hope you had a wonderful 2024. To start you off on the right foot in 2025, for today's episode, we've got our annual data science trend prediction special for you again this year. In today's episode, which will appeal to technical and non-technical listeners alike, we cover how Sadie's predictions for 2024, which she made a year ago on this show, how those predictions panned out. We award our wow moment of 2024, our comeback of the year, our disappointment of the year, and our overall winner of 2024. And then, of course, we speculate on what 2025 will bring us, including Agentic AI coverage, AI in everyday edge devices, the remarkable way AI is transforming scientific discovery with numbers that will surprise you, what massive GPU investments by tech giants tell us about AI monetization in 2025 and the unexpected shift coming to data science careers.
00:01:42
As with our 2022, 2023 and 2024 predictions episodes, our special guest again this year is the clairvoyant Sadie St. Lawrence. She's a data science and machine learning instructor whose content has been enjoyed by over 600,000 students. She's the founder and CEO of the Human Machine Collaboration Institute, as well as being founder and chair of Women in Data, a community of over 60,000 women across 55 countries. On top of all that, Sadie serves on multiple startup boards and is host of the Data Bytes podcast. All right, you ready to join Sadie and me on this visionary episode? Let's go.
00:02:23
Sadie, welcome back to the Super Data Science Podcast for another year of forecasting. This is your fourth consecutive year where you are leading us into the future, into the exciting future AI changes that are going to be even faster than ever in 2025. Welcome.
Sadie Lawrence: 00:02:41
Thank you, Jon. I cannot believe it's been four years that we've been doing this. As everyone says, it's so cliche at the end of the year, where does the time go? But now I really feel like where did the time go because we've been doing this for four years, and so eventually maybe five years. We have to do a full five-year recap or something on how many were we off, how many were we on?
Jon Krohn: 00:03:02
Yeah, that's a really good idea because even thinking about, even just as I was looking back to see how many times you've done this just now, before we started recording, I was thinking, wow, 2021 predictions. What would those have been like?
Sadie Lawrence: 00:03:15
Do we even want to look? Not in this this year, not this year, save it for next year.
Jon Krohn: 00:03:17
No, next year.
Sadie Lawrence: 00:03:20
You have to listen to the podcast for a whole another year to see what we said in 2021.
Jon Krohn: 00:03:26
Yeah, yeah, yeah. But yeah, in 2021, we made predictions for 2022. You've always been pretty on the money. I'll quickly recap your predictions for 2024 before we find out what you've been up to over the last year. So for 2024, your number one prediction was that there would be much, much more demand than ever before for GPUs and other kinds of AI hardware accelerators, which would open the door for new players to compete with established giants like NVIDIA. And absolutely, I mean, NVIDIA has become, I mean it's crazy to think from last year it already seemed like it had a crazy stock price and we've seen it increase so much more over 2024. So you're absolutely right there. But simultaneously, we have also seen lots of players come in. Tensor processing units have evolved a lot. AWS is training them and Inferentia chips are doing exciting things. For example, this giant new cluster that's been announced that Amazon is involved with Anthropic with hundreds of thousands of Trainium chips. So I think you were spot on with number one there.
Sadie Lawrence: 00:04:36
Yeah, I am really excited that there is more diversification in the market, although we may start need predicting these trends a little further out because I read today that NVIDIA employees, I think like 75%, are millionaires just because of how much the stock has rise.
Jon Krohn: 00:04:50
Really?
Sadie Lawrence: 00:04:51
So I'm like, well, if only we predicted this out a few years ago, maybe I would be in a little bit better financial position today. But regardless, let's start looking at some of those new players coming into the market, and I think it's just going to be great for everyone involved, particularly startups and getting access to the compute that they need.
Jon Krohn: 00:05:11
For sure. Your second point for 2024 was that we'd have LLMs as a new operating system, what you called an LLM OS, that large language models would transform the way that people interact with machines, meaning that you don't need to be using your keyboard or a phone screen quite as much, that you can have face and voice recognition, and you've been absolutely spot on with that.
Sadie Lawrence: 00:05:38
In my opinion, I think this one is a little bit of a miss personally, and for the reason is that I think our habits take much longer to change as humans. I don't know how much, I haven't seen the stats of how much people are using voice mode, but we're so used to interacting with individual applications. I know there's computer use from Anthropic. I don't see it fully integrating into a full operating system yet. I think we're starting to see the tip of the iceberg, but I think there's still a lot more to come in this space. And so I'm going to be a little bit more mean on myself in this one and say this was only half accurate, but more to come.
Jon Krohn: 00:06:21
Okay, well, you made up for that with number three, which number three could not have been more on the money. So number three, you discussed how we will make advances in LLM capabilities beyond just scaling the size of the network or scaling data. You said that we would have a slow thinking model. Innovative approaches will aim to replicate human slow thinking processes as described by the famous economist Daniel Kahneman, who passed away actually in 2024. And yeah, you said this could lead to significantly enhanced capabilities in logic, reasoning and mathematical tasks, potentially requiring fewer training data. And o1 from OpenAI exemplifies this trend, which I'm sure we're going to see other of the huge hyperscaler companies are going to be scaling along this inference scaling lever that they now have available to them.
00:07:20
And it's interesting, this is for now, it's relatively constrained to problems that you can easily break down into subparts. So like math problems, computer science problems, these can be broken down into intermediate steps where you can use reinforcement learning to say, "Okay, at this first intermediate step, we've done a good job, we verify that we did it correctly. Let's move on to the next intermediate step." And so it'll be interesting to see in maybe in 2025 how the hyperscalers can take innovations like o1 and extend them beyond these relatively narrow, easy to break down into intermediate step tasks, which could be a big challenge.
Sadie Lawrence: 00:08:03
I think where we're going to see it move to is more of scaling in terms of specific domains. And a lot of where I'm pulling this from is from psychology and biology. If we look at brain development, if we pulled scaling laws into brain development, we would just think the bigger the animal or the mammal, the larger the brain, the more intelligence that they have. That is not always the case. What we find within biology is animals in particular develop a unique type of intelligence based on their environment and the environment that they decide to thrive in. And I think we're actually going to see that happen a lot this next year where, okay, we've realized that slowing down thinking works, but that's maybe for really complex problems, not for all types of problems. So I think we're going to move beyond scaling in some really interesting ways this next year, and we're just, again, seeing the tip of the iceberg of what that looks like with models like o1.
Jon Krohn: 00:09:03
Yep, yep, yep. Exciting times as we'll talk about more later. O1 is a really exciting innovation for me. I'm already making spoilers on things that are to come. So that was number three. Number four for 2024 was tool consolidation via LLM APIs. So you said that function calling APIs of LLMs will unify and integrate diverse systems and applications leading to streamlined workflows and a consolidated enterprise toolset. So this has ended up coming to life with this Agentic AI term where you have Agentic AI frameworks that have blossomed over 2024 and have allowed LLMs to make use of lots of tools, online capabilities, interacting directly with computers, like all of the functionality of a computer as opposed to just text in, text out. And so, I mean, this one you probably won't be able to disagree when I tell you that you were on the money for number four.
Sadie Lawrence: 00:10:05
Yeah, I'll try and disagree with you a little bit. I was hoping to see some more tool consolidation in terms of, let's say people using ChatGPT for example, and being at their go-to place for analysis, for coding, for presentations. I still am seeing a little bit of specialized tools. I think Copilot's actually done a little bit better job of it just because it integrates so heavily within the Microsoft Suite.
Jon Krohn: 00:10:32
The GitHub Copilot?
Sadie Lawrence: 00:10:33
No, Microsoft Copilot.
Jon Krohn: 00:10:34
Microsoft Copilot, yeah.
Sadie Lawrence: 00:10:35
Yeah. Just from an enterprise level, we rely still so heavily on Microsoft Office products. And so-
Jon Krohn: 00:10:43
Do you use Microsoft Office products?
Sadie Lawrence: 00:10:45
Okay, I have a heavy opinion on this. I use it in one company now. In my previous company, we were all the startup tools, which were G Suite, Slack, et cetera. Which do I prefer? Definitely the G Suite, Slack, but we work with more enterprise clients and so that's why we switched over to the Microsoft Suite.
Jon Krohn: 00:11:16
Oh, boy, that only for an interesting 2025 for you.
Sadie Lawrence: 00:11:18
It's going to be rough. I don't know what's going on with Search and Outlook, but I'm sure a lot of people feel my pain on this. But I think in terms of tool consolidation, Microsoft is just set up as a leader in that space just because of how much dominance they have in the enterprise space today.
Jon Krohn: 00:11:34
Nice, yeah. And then we've gone off-piste a bit here, but to go for your fifth and final prediction for 2024 before we move on to our next topic category entirely was you talked about there being workplace upheaval, so that sophisticated code interpreters like ChatGPT's, advanced data analysis, that that empowers business users to perform data analytics on their own. And so this, it redefines traditional analyst roles. And I understand that you might have some interesting stats for us on how the 2024 job market changed that bear out what you described last year with your expectation of workplace upheaval.
Sadie Lawrence: 00:12:15
Yeah, unfortunately, we're seeing this heavily impact tech. And people who aren't in tech I think are taking the wrong conclusion from this, which is tech is not a good place to be in and the job market is difficult, but particularly in high-tech companies, we're seeing a lot of layoffs because they know and understand how to implement these tools. So there's a posting from Fred where it showed software development job postings on Indeed in the United States. We kind of reached the peak of that in 2022, and that has been just on a sharp decline since then and at one of its lowest rates since 2020. I think what's interesting about this is that peak coincides right when ChatGPT came out, we saw this new AI boom. And so you can see directly a correlation between technical jobs being affected by AI, particularly because that's one of the easiest place to implement it, but also, technical people are the ones who are aware of what this technology can do and how to implement it.
Jon Krohn: 00:13:25
Yeah, I read just the other day that Klarna, they allow you to break your payments, you buy a stereo, you can do four quarterly payments at the same price as if you were just buying the thing outright immediately. And so Klarna, K-L-A-R-N-A, there was recently a report that they haven't done any hiring basically since that peak that ChatGPT was released at and they've just been using normal attrition to reduce their workforce size. And they were replacing call center people, I presume software developers, don't know that for sure, with AI.
Sadie Lawrence: 00:14:09
Yeah, I think where this is going to get really hard for is recent college graduates or less experienced individuals. That is where AI really shines. I talk about using AI tools a lot, like having a really excited intern. They have deeper knowledge than me and probably fresher knowledge than me in particular areas, but having that context is what they're missing. And so if you are in that position of being a recent college graduate or early on in your career, this is going to be a really key time to expand your skills, but particularly your business skills because a lot of the basic tasks are what leaders are going to look for AI to replace.
Jon Krohn: 00:14:53
Yeah, good shot there. And we will have more later in this episode on what we think for 2025 and how careers and skills will be impacted by the AI tools that are coming and becoming richer as we go.
00:15:08
That gives a recap of our predictions, really your predictions for 2024, which certainly you couldn't even argue with me that you got three of the five spot on. So the demand for compute, the slow thinking model, workplace upheaval, tool consolidation, maybe could've been... I guess you weren't expecting it to kind of be the Agentic way that it ended up proliferating, but we are going to be talking about that a lot more in today's episode. And then the LLM OS, you felt like that wasn't quite right, but still lots of ripe capability there. I think as these kinds of tools, things like Apple Intelligence becoming more useful, more widespread, we'll see LLMs be more of the voice, the face, direct interface with the code behind the scenes, with the back end as opposed to needing to rely on typing like we still have mostly been this year.
00:16:08
All right, quickly, Sadie, before we get to, we're going to do an interesting section next that we've never done before, which is, and this is your idea, I love it, is that we'll pick an overall winner for the year. We'll pick a comeback of the year, a wow moment and a disappointment of the year. But before we get to that, tell us a bit about what your year has been like, something in particular that interests me and is quite timely because this episode is coming out just before New Years, and so this is definitely the time that people are thinking about how they can be restructuring their life to be more successful, more happy in the coming year. And you have your first physical product. So after a decade of creating popular digital products, you now have a physical product that people can order or pick up at their bookstore and could transform their 2025.
Sadie Lawrence: 00:17:08
Yes. This has been such a passion project of mine because I, for the past 10 years being a true data person, have been tracking what I've been doing in half an hour increments for the last 10 years. And I have gained so much from this experience that I decided I wanted my own planner. And a lot of people are like, "Wait, it's tracking, but it's a planner. How does this combine?" I'll explain how it all works together. To be able to do it in a more effective manner.
00:17:37
And so I created a planner called The Observer, and the whole name behind The Observer is to actually observe what you're doing with your time because I am a true believer that actions speak louder than words and we often set goals for ourselves, but then rarely reflect on how we spend our time to align with those goals. And so the whole idea is that you just fill in daily what you did in your day and just that whole process of taking a second to be like where did I spend my time and what did I go to really triggers something in your brain to question, am I aligning my time with my goals? And so there's some pre-setup in terms of imagining the life that you want, putting those into increments for quarters and months and weeks. And then the most important part is that reflection part on a daily basis where you track your time.
00:18:29
It's been a super fun project. Again, it was just a passion project of mine that then people would ask me how I do it. So I created a Shopify account on a weekend and then next thing we know now we're in stores. So it's been really a pleasure to create.
Jon Krohn: 00:18:43
Nice. And we'll be sure to include a link to that, which is theobserver.store and we'll have observer.store in the show notes so that you can pick up your own Observer, see how you are spending your time, imagine how you could be spending your time differently and maybe have a more fulfilling, a more productive life. Cool.
00:19:04
All right. Let's move on to now the section that we promised just before we talked about your planner, which is picking our... Should we do overall winner last?
Sadie Lawrence: 00:19:16
Yeah, let's do winner last.
Jon Krohn: 00:19:19
Okay. So then should we do comeback of the year first?
Sadie Lawrence: 00:19:22
Yes.
Jon Krohn: 00:19:23
Okay, cool. So for our comeback of the year, I'll go first if you want, if want to go first.
Sadie Lawrence: 00:19:30
Okay, yes. And I will say Jon and I have not discussed these, so I'm also really excited to hear what his are and we may have the same ones. I'll let you go first. I have mine written down. I promise I won't change it once I hear yours, but yeah, what was your comeback of the year?
Jon Krohn: 00:19:41
Google.
Sadie Lawrence: 00:19:44
Same, yes.
Jon Krohn: 00:19:45
Oh, same? Yeah.
Sadie Lawrence: 00:19:47
They had too much good stuff that they saved for last. I mean, even in these last few days, it's been incredible. I think it was today Veo 2 came out, Willow AI studio, NotebookLM.
Jon Krohn: 00:20:01
Yeah, Gemini 2.0, it's across benchmarks across the board. Gemini 2.0, they're now competing at the forefront again, like you would've expected Google to be all along with any kind of AI. Two years ago, if we'd done back in 2021 when we were making our predictions for 2022, you would've anticipated that Google would be the front-runner still in 2025. And while they are not clearly the front-runner like they were back then, there are absolutely other competitors out there, some of which we will probably be talking about when we make our predictions about overall winner and so on. I don't want to try to spoil it too much, but it's not a unipolar AI world anymore. It's a multipolar AI world, which is great. It's great to have lots of smart people working in lots of different labs with their own takes on how things should be done safely and how the envelope can be pushed effectively.
Sadie Lawrence: 00:21:14
Yeah. I'm curious, since Google is both of our comeback for the year, do you think that they're going to gain traction in the market or because OpenAI has such a brand presence, when you think of AI and Gen AI, I'm guessing most people think of OpenAI right away? Do you think Google's going to have a hard time coming back from that even though technically you and I both see them as the comeback of the year?
Jon Krohn: 00:21:42
Yeah, it's an interesting question. It probably depends on where you are. I know that I recently came back from a trip in San Francisco and lots of people there seem to be... It's interesting. I went to an event, this Gen AI event where on your name tag, you wrote your name as well as your favorite AI tool. And a lot of people had ChatGPT written on their name tag, but there were also a lot of people verbally expressing disappointment with OpenAI's year in 2024. So it's interesting.
00:22:21
I think OpenAI absolutely had pole position in late 2022 with the release of ChatGPT and they maintained that in 2023 with the release of GPT-4, but I think people were expecting, and who knows, maybe by the time this episode goes out, there will be like a GPT-4.5 release or something like that. But yeah, I think people with the Delta between GPT-3 and 4, I think there was a lot of expectation that scaling alone, scaling data, scaling your number of weights in your model, that would continue to yield the crazy advances that we saw between GPT-3 and 4. And that hasn't borne out.
00:23:08
We are seeing OpenAI still absolutely be competing at the threshold with things like their o1 model, which I absolutely love. We already talked about that earlier in the episode. Scaling at inference time has a lot of potential, especially with tasks that can be broken into intermediate steps and validated at each of those intermediate steps, but yeah, it'll be interesting to see where OpenAI goes and how they are able to monetize or not. That's something that Google still enjoys. They still enjoy this monopoly oversearch and they're so dominant in advertising, they can afford to make some missteps or be a bit slower and still catch up over time. They have huge amounts of compute. They have, I think, probably still the strongest and largest concentration of AI talent. And so I don't know. It's interesting to see where they'll go. OpenAI is potentially more vulnerable, not so much to another giant, existing giant like Google, but other upstarts like an xAI or an Anthropic.
Sadie Lawrence: 00:24:23
Yeah, I think it's shaping up to be a really exciting 2025 because now that we're about two years into this Gen AI movement, it seems like everybody is at the starting line again. And I think that that's just going to make a really interesting year for us next year. But I would agree with you. Should we do the disappointment in terms of who the... Can we jump to that now?
Jon Krohn: 00:24:49
Yeah, let's do it. Let's do disappointment of the year. What's yours?
Sadie Lawrence: 00:24:52
Yeah. I will say it is OpenAI and I think it was really hard just because they set expectations so high.
Jon Krohn: 00:24:59
Yeah, I wrote them down and I scratched it out for something that I thought I of that's even more disappointing to me.
Sadie Lawrence: 00:25:04
What's more disappointing?
Jon Krohn: 00:25:06
Apple Intelligence.
Sadie Lawrence: 00:25:07
Oh, that didn't even cross my radar because I haven't even used it. Yeah, that would be the highest disappointment because that was just what a flash in the pan and just does anyone use it? That's my question. Who uses it?
Jon Krohn: 00:25:24
I don't know. No one's talked to me about it, but there's certainly been a lot of hype around its potential. And yeah, I mean, Apple still hasn't, at the time of us recording this at least, they haven't figured out how to embed... I think for them, a really tricky thing for Apple is that security is so important to them and a reliable product experience is so important to them. And LLMs are fundamentally neither of those things, especially if you're having to send off your user requests to OpenAI's GPT models for processing because Apple doesn't have its own in-house LLMs that are capable of the range of tasks that consumers expect today. So yeah, they're in an interesting pickle, but same thing to Google. Apple has huge amounts of revenue, amazing margins, and they have time to figure this out.I would not be surprised if in another five years from now, Google and Apple are still dominant players in technology.
Sadie Lawrence: 00:26:30
I was going to say that I hope that next year our comeback of the year will be Apple and Apple Intelligence. I mean, yes, they want things to be right, but I mean, who's used the photos app. That was not right. I think we all can agree that the new release of the Photos app was a total miss as well. So maybe Apple Intelligence, and they're tied with OpenAI with as the backend behind a lot of that as well, so they can tie together in both of our disappointments, but I really hope to see both of these back end the game as our comeback for next year because there's so much potential from the phone integration with AI that I would really hope to see this next year.
Jon Krohn: 00:27:11
For sure. All right, so yeah, our comeback of the year, we both agreed on Google. Our disappointment of the year, your topic was my second topic with OpenAI, and it sounds like Apple Intelligence did disappoint both of us. Let's move on now to our wow moment of the year. Maybe I'll go first on this one since you got the last one. And so, for me, I've already alluded to this, it's o1 from OpenAI. It's interesting that simultaneously that expectations were so high for them that they could both be the disappointment of the year and the provider of our biggest wow moment.
Sadie Lawrence: 00:27:48
Yeah, I think that just shows how high expectations were, are and they continue to be within AI. I think that all of us in AI now are almost TikTokified. I don't even know if that's a word, but in terms of wanting that quick dopamine hit of if something isn't happening this week or something that's not wowing us or blowing us away, we just write it off. So it's interesting that you have them as your wow moment when it's also my disappointment because I think it really just ties into expectations are high and we are looking for that next dopamine hit in AI every single week, if not every day.
Jon Krohn: 00:28:32
What's your wow moment?
Sadie Lawrence: 00:28:33
So my wow moment is not necessarily from an overall use, but just from a human level of when I just listened to it, and this should give you a key to what it is, but I was just truly impressed. And that was with the NotebookLM and the podcasting.
Jon Krohn: 00:28:53
That's number two. That's my number two.
Sadie Lawrence: 00:28:53
And the reason why it just was so human to me, and that's why it wowed me is their expressions, the way that they talked. It felt like you and I talking on a podcast. So just from a human level, is it going to change the world? I don't know, but I just thought it was cool. And so that was my wow moment.
Jon Krohn: 00:29:15
Absolutely. I almost had that as my number one as well. And we did an episode of this podcast, number 822, which came out in late September. In that episode I expressed how blown away I was by NotebookLM, and I also air in its entirety a 12-minute podcast episode about my PhD dissertation, which is so boring, but these fake podcast hosts did manage to make it seem exciting. And so I included it in full, in the episode and people were blown away. That must be one of my most commented posts of the year of a large number of people reaching out and saying, "Wow, I hadn't heard of this, or I hadn't used this, and now I have used it and it blew me away. Here's what I tried." So yeah, that was really cool. I think it was a wow moment for a lot of people.
Sadie Lawrence: 00:30:10
Yeah. I'll add one sub-wow moment in there which may not get talked about.
Jon Krohn: 00:30:13
Sub-wow.
Sadie Lawrence: 00:30:13
I hope we have a sound effect for that too, sub-wow, or maybe it's its own sound effect, but I recently got a Tesla and the full self-driving on that is incredible. And I was just blown away because as a kid, my mom was like, "Hey, you really need to learn to drive and do all these things." And I told her one day, "I will have somebody who drives me around." I did not think would be a robot in full self-driving, but here we are today. So just to have childhood memories of saying something and then to be living it today is truly incredible.
Jon Krohn: 00:30:50
That also, I've got to add my sub-wow moment, which is-
Sadie Lawrence: 00:30:53
Sub-wow.
Jon Krohn: 00:30:57
... which is Waymo. I had my first Waymo experience this Northern Hemisphere summer, and that was really cool, having a car, because I think that's another level of autonomy beyond Tesla's full self-driving, right? Where with Tesla's full self-driving, you need to have somebody sitting behind the wheel. But to have the Waymos now in San Francisco and at the time of recording also in Scottsdale, Arizona, I think, you can just use the Waymo app and a driverless car comes up, picks you up, you get in it and it drops you off. I almost want to make that my biggest wow moment of the year. I don't know how I didn't think of it right off the bat, but I mean, because that physical presence because that's...
00:31:44
I come back to the Waymo example a lot with when people ask me, when people find out I work in AI, as the quote, a lot of people completely outside of AI will say things like, "Oh, contentious," and I'm like, "Really? Oh, I wasn't aware it was so contentious." And they're like, "Well, yeah, I'm a creative or I have lots of friends who are creatives," and I can see that okay, yeah, I can see why it's so contentious. But for me, I guess I'm so often seeing big changes and benefits. But there are, you know, the Waymo example is one that I come to frequently to say, this is a... as opposed to something that's happening on your computer screen, this is a physical, very obvious manifestation of AI that when you experience that, when you call a Waymo car, get into it and drops you off somewhere, you see the steering wheel spinning all on its own and it's making great driving decisions, that makes it clear that in the future, in the not too distant future, we don't need drivers. We don't need human drivers.
00:33:03
In the United States, in most states, the number one job is truck driver. And there's tons of related jobs that support the truck driver, people working in cafes along the roadside and that kind of thing. You don't need that. Self-driving cars don't need cafes or motels. And so it's going to make a really big impact. And given the upheaval that will be caused by this, there's things that we need to be doing as a society in terms of retraining people because this AI shift should end up being, just all other automations in the past, it should provide people with more interesting work than ever before. And I mean, this time, there's talk about this time being different, but all past increases in automation have led to more employment and lower unemployment. So I don't know. I've touched on a lot of topics there, but I haven't let you speak for a long time. So, Sadie.
Sadie Lawrence: 00:34:07
No, I was really lucky this year to hear one of the co-CEOs of Waymo talk, and one of the things that she said was, "We are building the single best driver." And I found that really interesting because she talked about how they have over a hundred thousand fleet of cars out driving, but she talked about it as one driver. They talk about it as a single brain, a single driver brain, and they're only building one. And that just resonated with me so much because it really gives us perspective of what intelligence and machine intelligence can do at scale, right? You only need to build one of the best single drivers and you can change a whole industry. And so I think that's something just to think about. Get really specific in the models that you're building and the domain that you're building because when you do that at scale, it's incredible.
Jon Krohn: 00:35:06
Agreed. That's a really nice way of thinking about it. I hadn't heard it phrased that way before, but now that's a great meme to be thinking about industry by industry. How can I create a brain for some specific task that can be outstanding at that specific task? And then, because it's software, you can just replicate it as much as you like, software update to the fleet of a hundred thousand cars transform the industry. Yeah. Cool. All right, so then that leaves us just with overall winner still to select. Do you want to go first or do you want me to go first? I feel like this was all your idea. And so maybe you should go first on this one.
Sadie Lawrence: 00:35:48
Okay. My overall winner for 2024 is open source, and particularly I think just us-
Jon Krohn: 00:35:58
That's not a company.
Sadie Lawrence: 00:36:02
... as individuals... Okay. I could choose Meta then and Meta's Llama-
Jon Krohn: 00:36:04
Oh, wow. Okay.
Sadie Lawrence: 00:36:05
... particularly if you want me to get specific, but I really think it's us as consumers. I think we are going to be the winners of AI because it is open source. I'm seeing more and more startups entering the market. And just the capabilities of the models that are available, the costs are getting cheaper and cheaper. So it's an open source community. We saw that also from a legal perspective with some of the bills that were being passed in California that got turned down and was really great for the open source community. So that's why I say open source in general from a company perspective, what Meta's done with Llama. And from an individual perspective, I think it's a consumer who's really going to win from all of this.
Jon Krohn: 00:36:50
Wow. Nicely done. Nicely done. You went very lowercase M, Meta, with your answer. I just picked a company which is Anthropic.
Sadie Lawrence: 00:37:05
I had a feeling this was going to be yours. Please tell me the love for Anthropic of why it's your overall winner.
Jon Krohn: 00:37:12
Yeah, anyone who knows me knows that I'm a big Claude fan. I use it more... I subscribe to Gemini, I subscribe to ChatGPT Plus, I subscribe to you.com, but I end up at least 80% of the time using Claude as my weapon of choice. I don't end up needing the kind of o1 capabilities on a lot of the tasks that I'm doing. Probably that could be related to tasks that I'm doing are relatively common coding tasks related to neural networks. And on that kind of thing, Claude 3.5 Sonnet does an outstanding job debugging things. I don't need all that power of o1. I just get an answer quickly from Claude. And same kind of thing, things related to the podcast, summarizing episodes for me, transcribing things, Claude just does an amazing job. It does the best job of any of the LLMs, private LLMs that I'm regularly using.
00:38:18
There are still places where I use Gemini. If I have very large files, I think Gemini is great. I haven't used Gemini 2.0 that much. Maybe if I'd been using it a lot more in recent days, I'd say, wow, this can replace Claude for me. But I also, I love the UI of Claude. Think they've done a really good job of it. It has this friendliness and this warmth to me that I don't get from any of the other tools that I subscribe to. All the other ones seem like they're designed to look really futuristic, but it creates this coldness, whereas somehow, yeah, Claude manages to give me the warm fuzzies.
Sadie Lawrence: 00:39:00
That's the first time I've heard a comparison of what tool makes you feel better. I like that. We can maybe add that to one of the winning awards next year. I'm curious though, because you didn't mention Perplexity, you use Perplexity. Is that in your repertoire? That's something that I have been using-
Jon Krohn: 00:39:16
I don't use Perplexity much.
Sadie Lawrence: 00:39:18
... a lot. I love Perplexity.
Jon Krohn: 00:39:21
Oh, yeah?
Sadie Lawrence: 00:39:21
Mm-hmm. So maybe add that to... Tell me how, if it gives you any more fuzzies, I'm curious about. It will be hard to replace Claude, but yeah, Perplexity is something that I would say in the past two months, I've really started to dive into a lot more and love just the references and have pretty much replaced search with Perplexity now.
Jon Krohn: 00:39:43
Wow, there you go.
Sadie Lawrence: 00:39:46
Christmas shopping, wasn't able to find a product that I needed to buy for someone. They were all sold out. Thanks to Perplexity, was able to find it, snag it, and have a very happy Christmas.
Jon Krohn: 00:39:59
Cool. Yeah, I think I've ended up using you.com for some of those same kinds of Agentic tasks, but yeah, I'll definitely try. I've only done a few searches in Perplexity. I don't know why that ends up happening. It's definitely a blind spot that I've felt on other podcast episodes in the past, so I'll have to spend some more time with that.
00:40:19
All right, so we have been... This episode is all about forecasting the future, but like your planner, so far we've been looking back at 2024, but I also think that sets the stage for the predictions that we are making in 2025. And it also gives you a sense of how reliable we are as crystal ball readers for the forthcoming year and largely good, largely good. Largely you can trust us based on our performance in 2024 and the previous years, which you can go back and listen to if you want to.
00:40:50
So for 2025, the big number one topic that you highlighted, Sadie before we started recording and I agree with 100%, is Agentic AI. There's no question. I could speak for a long time about this, but you are the guest, so I'm going to just let you go first, please.
Sadie Lawrence: 00:41:09
Yes. I mean, this has started in what I call the fall conference season to boil up as a key topic, I think it's going to take over all conferences will be about Agentic AI next year, but more importantly, the number of startups in this space. I think there's over 600 as of today, and who knows if that's even an accurate count of it, but it's really the next step of what I think all of us as consumers are also looking for is we're looking for more of those autonomous agents where we don't want to be copy and pasting from different applications. We wanted to just go on our behalf.
00:41:47
And I think the, I don't know if it was a meme or an actual article from this year where it was a woman who said, "We got AI and it creates things, but I was hoping it would fold my laundry or do basic tasks." And that to me is really where the need for Agentic AI comes in. That's more of a robotic task is folding your laundry, but particularly, we wanted to now, what I would say is leave the confines of an application and go do actual tasks for us autonomously on our behalf. I think that the trust in AI has built that we as humans are ready to maybe unleash it to that next step. We see a little bit of that with computer use, but more importantly, I think we're going to see a lot of companies who just like Waymo are building a specialized brain for a particular application, one that will be the best ever social media marketer, one that will be the best financial analyst. And so we'll have these specialized models who will be able to truly be agents on our behalf and take autonomous steps.
Jon Krohn: 00:42:55
Yeah, it's going to be absolutely transformative. I think to go into a bit more detail on that viral post that you were mentioning there, I think it tied into the woman who wrote that post saying, "I wanted automation to be taking away the mundane tasks from me," like you said, folding laundry and leaving the creative tasks like video production, artistic endeavors to me, but instead AI's taken that away and I'm left folding the laundry. But Agentic AI systems are a step on the way to having even in the real world more Waymo-like physical embodiments, making changes and being able to fold your laundry for you. That is coming and it seems like now in our lifetimes for sure, that we will have machines that can do this kind of stuff and hopefully most people, if not everyone, will be able to afford machines like that. These should be widely available, not something that's just for some small percentage of the planet.
00:44:03
I recently came back, at the time of recording, I've just come back from NeurIPS, Neural Information Processing Systems. This is the 30th year it's been running and it's I think safe to say the most prestigious AI conference for academics that is out there. And at this conference, the Agentic AI trainings, workshops, some of them were so, there was so much interest in them that there was a crowd of people outside the room unable to squeeze into the standing room. And so that gives you a sense of how popular this topic is. Fei-Fei Li gave one of the keynotes at NeurIPS this year, and she talked a lot about how her company, WorldLabs, is creating sophisticated data sets that will allow agents to explore 3D visual worlds as opposed to just being right now visual agents are becoming pretty good at recognizing two-dimensional images, but that doesn't necessarily make them great at exploring the real world and being able to fold your laundry.
00:45:14
So yeah, really exciting things happening in Agentic AI. I am completely smitten with Agentic AI and we've been doing podcast episodes on it a fair bit recently. We have many more planned for 2025, early 2025, with Agentic AI experts. So look out for those. Yeah, I'm going to be creating a half-day or full-day Agentic AI hands-on training for ODSC East, Open Data Science Conference East in Boston this spring. Yeah, I think it's a fast-moving space, but it's unquestionably where these things are moving and it's a testament to how far along we've come with LLMs and their ability to be accurate because if you're going to have AI agents going off and doing tasks autonomously that you've assigned them, or having an AI agent master that's spinning up a bunch of slaves, that's a really rough word to be using. There's got to be a better one. But in computer science, that is often the word that's used, but sub-processes, sub-agents, sub-wow agents.
Sadie Lawrence: 00:46:28
Sub-wow, there we go.
Jon Krohn: 00:46:31
In order for any of that to work effectively, you need to have accuracy. If your LLM is hallucinating 10% of the time, that's an extremely large amount if you're going to have a large number of processes running. But if you get to a 1% or a fraction of 1% error rate, and with things like o1 from OpenAI, being able to check their work step-by-step, this evolution in LLMs, the way that they've advanced, it allows us now to be in this Agentic AI moment that we're in.
Sadie Lawrence: 00:47:06
I'll just add one bit of caution on this one where I can see it not manifesting in the way that we may hope for and fully imagine, which is I think we have a lot to figure out in permissions and access. And so just because your agent can go and do something and is accurate, we switch quite frequently between applications on our computer, and giving it that automatic permission may not so much be an issue from a human perspective, but companies playing nicely with each other. And so I think that will be really interesting to figure out is, okay, will Apple give Microsoft access? They've done it with having Outlook as one of your mail applications, but how does that look like once different agents are released into that platform? So I'm curious to see how that will all work out this next year.
Jon Krohn: 00:48:02
Nicely said. Yeah. I mean, this is the wall that we run into with LLMs or Agentic AI frameworks being able to be effective is around data security, privacy. It would be ideal to be able to take a whole giant enterprise and say, "Okay, Agentic AI system, here is all the information," but of course, if you do that, then you're opening it up to abuse. I mean, because then all of a sudden someone in the software engineering department can write an agent that's like, "Provide me with the pay packages of everyone in the company."
Sadie Lawrence: 00:48:38
Exactly, right? Or who's not to say some other agent can't come in and fool your current agent to think that they're a helper. I mean, there's so many ways that this could go wrong, but I think the difficulty of this task that we're facing right now is really on the security and privacy side of things.
Jon Krohn: 00:48:55
Nice. All right. So that's number one. Agentic AI is our big prediction for 2025. I feel very safe in this prediction. Seems like a layup. Number two from you is AI integration into everyday devices. So this is a higher risk one from you, I guess similar to your LLM OS prediction for 2024. But yeah, tell us about your idea here with these everyday devices. Give us some examples.
Sadie Lawrence: 00:49:24
Yes, I think this is a playoff of the LLM operating system and maybe more of a stepping stone to get there. But where I'm seeing this go is recently I got the Meta glasses. I particularly just like buying different tools and applications just to test things out and was really impressed with them overall. I'm going to Mexico next week and I really want to test out the real time translation. So I think that we're going to start to see AI just streamlined into more and more of our devices.
00:49:58
We talked about our disappointment with Apple Intelligence, but again, I think that they have so much cash flow, they have access to so much talent, we could see them make a comeback next year. And then Microsoft this year released what they called their AI computer. I know they took it off the market for a little bit because people were concerned about the privacy of it, but it was essentially taking snapshots of your screen throughout your workday and giving you recommendations and starting to train how it can be its own assistant with you. Again, back to the privacy and security concerns, they took it off, but I think next year we'll have something figured out where we'll see AI woven more seamlessly into everyday devices more than we have today.
00:50:44
On the last one there, I will say I do think some will not always be smart weaving into products. When I went to Costco this year and saw AI on a toothbrush, an electric toothbrush, that's how you knew it has gone too far. So I didn't say it will all be positive, but we will see it continue to pop up in our everyday devices.
Jon Krohn: 00:51:04
Yeah. Where I was staying in Vancouver, I would walk from the Airbnb that I was in Kitsilano, beautiful area of Vancouver. I'd walk from my Airbnb to F45, which I'd never done before. Have you ever worked out with F45? You ever done that?
Sadie Lawrence: 00:51:18
No, I've walked by one, but I've never walked in one.
Jon Krohn: 00:51:22
That was pretty cool. It was like Sadie and I both do CrossFit and they don't have barbells. You don't spend a lot of time on technique, but in 45 minutes you do get a good mix of cardio and strength work. It was an interesting experience for a week. But anyway, when I was walking in between my Airbnb and F45, every day I would walk past this golf pro shop and all of the posters in this golf pro shop were for drivers that had AI in the name. So it's things like AI smoke. And there isn't an-
Sadie Lawrence: 00:52:01
Wait, golf drivers with AI in them?
Jon Krohn: 00:52:04
I don't think... Honestly, I didn't look into this. My assumption was that there isn't AI in the club. I mean, I don't understand how that could be, but my assumption is that somehow AI is involved in the design of the club in some way. That's got to be it.
Sadie Lawrence: 00:52:22
Maybe there's some tracking on it that there's an app that you upload, the data of how you swung and what-
Jon Krohn: 00:52:30
Maybe that is what it is.
Sadie Lawrence: 00:52:30
I don't know. I am not too into... As listeners can hear, neither of us sound like we're that into golf to help us out here.
Jon Krohn: 00:52:37
No, I'm so glad.
Sadie Lawrence: 00:52:38
Some of the audience will have to give us some help on this one.
Jon Krohn: 00:52:41
Yeah, please tell us, comment on social media and let us know whether there's computer chips in golf clubs now. It would not be shocking if they were. Like you say, they're in toothbrushes. They will be increasingly around.
Sadie Lawrence: 00:52:57
Will in your brain soon.
Jon Krohn: 00:52:59
Well, yeah, that is happening more too. So Agentic AI, number one. AI integration into everyday devices, number two. Number three, I love this one. You see AI-driven scientific research and innovation becoming a much bigger thing. I absolutely agree. I'm going to quickly preempt what you're going to say with a couple of recent episodes that I released on this topic. So number 812 was on this Japanese company that's full of Google DeepMind alums called Sakana, S-A-K-A-N-A. They released this AI scientist paper that was able to draft papers specifically on machine learning at that time, and they were planning on spreading to other industries as well, but being able to write papers, come up with ideas for papers and run the experiments, get the results, write up the results independently, and did a pretty good job. And this kind of thing will only get better, especially in that kind of environment where the AI system can actually be doing experiments.
00:54:04
And I think we will see more... I don't know if we'll see big examples of this prominently in 2025, but it is definitely the future that big pharmaceutical companies or big energy companies, they will allow AI systems to run physical labs. And so for episode 812, for this AI scientist, the reason why they stuck with machine learning problems is because these experiments could be run in silico, it could be run on computer hardware, but in the not too distant future, as I'm sure you're about to say, Sadie, these AI systems will control physical labs that can also do physical experiments. And so science will accelerate because of that. The machines don't need to sleep. And Ed talked about in episode number 835 with you.com's co-founder, CTO, Brian McCann, Brian talked a lot in the episode about how scientific discovery AI systems will be able to do kinds of scientific discoveries that a human never could because AI systems are trained on all knowledge and no human scientists are expert across all domains. Anyway, I kind of, hopefully I didn't take too much air of your AI scientist balloon, Sadie.
Sadie Lawrence: 00:55:22
No, I think it just shows how much excitement we both have for this area. And we saw just the beginning of this where I think it was Time Magazine shared an article about how ChatGPT was better than doctors in diagnosis. And just to your exact point with scientific discovery, we have AI models that consume so much more information than we ever could. And being a doctor, yes, you go to lengthy years of medical school, but keeping up with all of the new drug discoveries, all of the new scientific discoveries, there's just not enough time. And so I'm really excited for this just from a medical side in terms of not only new drug developments and new diagnosis, but that support for doctors in the healthcare system, and then just in terms of the new inventions of what haven't even been created.
00:56:16
And so there was a paper that came out this fall from MIT. It's about artificial intelligence, scientific discovery, and product innovation. And what they found was that AI-assisted researchers achieve 44% new material discoveries, 39% more patents, and had a 70% raise in new product prototypes. So just really truly incredible results. And I think this adds to this utopia future that I know you often talk about and I hope to actively build with AI, particularly in the realm of science, medicine and innovation.
Jon Krohn: 00:56:54
Yeah, but speaking of the dystopian aspects of these kinds of things, I think that same study showed that use of tools like ChatGPT to accelerate research led to job dissatisfaction increases as well for the scientists.
Sadie Lawrence: 00:57:09
Yeah. Okay, this is so fascinating to me because the job satisfaction definitely depends on the job type. When looking at knowledge workers, so there were two studies, one from Slack and then one from Microsoft, knowledge workers love using AI for their day-to-day jobs, whereas scientists, and what I would say is more specialized individuals feel that it's taking away the joy of what they were doing. And I find this so fascinating because my hypothesis is that as a scientist or as a doctor, you spend many years going to school to have a particular knowledge and then you feel like you're being replaced. Whereas my hypothesis for knowledge workers is you're just try to get a job. Maybe it's not what you majored in. You don't really like your boss. You're just trying to get by. If AI will come in and do part of that job for you, then the happier you are. And so to me, that's my hypothesis behind it, but if someone does a study, please let me know if you test this hypothesis.
Jon Krohn: 00:58:15
That sounds reasonable to me there. All right, so we've done three of our five predictions for 2025. Agentic AI, AI integration to everyday devices. Number three just now is AI-driven scientific research. Number four is enterprise AI monetization. Sadie, tell us about that one.
Sadie Lawrence: 00:58:34
Yeah, this year we talked about new competitors entering the market with compute. We also, I think, saw some investment in nuclear and in energy. I think we're going to continue to see more of that next year, but I think we'll start this year to see what is that investment turning into. And so for me, what I'm really watching keenly next year is XAI, so Elon Musk, subsidiary of X, but now the AI company and the supercomputer that they built where they broke the coherence bottleneck. And I think this is going to be really our first test fully on the scaling laws and if we're going to actually break the scaling laws or we are going to be constrained by the scaling laws or if we're going to actually have another major breakthrough in this place.
00:59:29
And so this year I saw a ton of investment. I think Mark Zuckerberg talk about how he had bought 600,000 GPUs from NVIDIA. You'll have to correct me if I'm wrong on the exact number, but I mean, people were buying so much, I'm ready to see the return on this investment and I think we're going to continue to see more investment in infrastructure this next year, but particularly in energy and in nuclear. Jon Krohn:
00:59:57
Fact-checking you, Meta plans to have nearly 600,000 NVIDIA H100 GPUs by the end of 2024, says the AI overview provided automatically to me by Google Gemini. As of March 2024, Meta had 350,000 H100s. Again, yeah, so that is... Yeah, it's interesting. These quotes are all coming from the beginning of the year. So it was like a January 2024 article where Mark was saying that by the end of 2024, he expects them to be running 600,000 H100 GPU equivalents, which is interesting because then it doesn't necessarily mean NVIDIA H100s. It could be like an AWS Trainium 2 chip or that kind of thing, but something of roughly that capability.
Sadie Lawrence: 01:00:57
Yeah, because my understanding that xAI's 100,000 GPU AI cluster is the largest that has been built to date.
Jon Krohn: 01:01:06
Yeah, I think it could be that Meta's are distributed for different programs, different geographies. It's not, whereas I think that xAI cluster is one single cluster training one LLM.
Sadie Lawrence: 01:01:23
Yes.
Jon Krohn: 01:01:25
Nice. It is interesting, this enterprise AI monetization topic, Sadie, because obviously as we spend these huge amounts of money on hardware, on talent, it means that the impacts on the bottom line need to be substantial for the project to ultimately be profitable. And this is something that we've talked a lot about on this podcast recently with guests. We had a great one, episode number 843 with the CEO of a company called Protopia, Eiman Ebrahimi. He talked a lot about these trade-offs between profitability and security and how so many projects, AI projects end up getting stuck in proof of concept purgatory because you can quickly spin up using Claude 3.5 Sonnet or OpenAI o1, some cutting edge LLM capability. You're like, you can create a Jupiter notebook or some simple Gradio application within a few hours or over a weekend that does amazing things.
01:02:37
But then you bring that to your manager, to a VP at your company and you say, "Look at this amazing tool I built," and when you start to get into the nitty-gritty of, okay, but how will we make this performance and cost-effective in production while also ensuring data security, enterprise AI monetization is a tricky point. And it also means that it provides a lot of opportunity for people out there who are listening to specialize in thinking about these kinds of problems because as it becomes easier and easier to use an LLM to spin up the code that you're going to use or just use an existing API to provide all of the intricate AI functionality of your platform, it's these kinds of questions like, how can I do this profitably for me or my business, that will allow you to really get ahead in data science.
Sadie Lawrence: 01:03:28
Yes, as much as data scientists are numbers people, I unfortunately fill the financial numbers are what are often forgot to be run when they build these models. And not just from the cost to actually put this model into production and continue to maintain it, but what is that return on investment? And so, yes, if anybody wants to get ahead in their career, just include a small financial model with your proposal and I think you'll get through the red tape a lot faster.
Jon Krohn: 01:03:58
For sure. And if you're looking for an episode we did this year that digs into that idea, a lot of profitable AI projects, is number 781 with Sol Rashidi. Sadie, do you know Sol?
Sadie Lawrence: 01:04:10
Yes, I do. She's been on the Data Bites podcast.
Jon Krohn: 01:04:13
Sure.
Sadie Lawrence: 01:04:15
Met her at a couple conferences before. She's great.
Jon Krohn: 01:04:18
She is really great. I hope to have her on the show again soon. That episode ended way too quickly and I had so much more to talk to her about, really well reviewed. So yeah, Sol, maybe we'll have her back on the show in 2025.
01:04:32
All right, but speaking of episode length, we are reaching the final, the fifth and final prediction for 2025. This one is maybe controversial. We'll end up seeing what happens here. I wanted to touch on with one of these predictions how data scientists or people working in AI are going to be impacted by all these innovations in 2025. And we touched on something there. We already talked just at the end of number four on enterprise AI monetization, how helpful it can be to be, like you said, that idea of including a financial model as part of your proposal. That is probably going to cut through red tape a lot more quickly. So that's great advice there.
01:05:16
So financial savviness can be something, but in terms of the "hard skills" of data science or AI, I expect that in 2025, the demand from the market, from employers for AI engineering skills will surpass the demand for data science skills. And when I say data science skills, I mean I guess everything else that data scientists do other than doing AI engineering, which includes things like using existing third-party API's, building prototypes, linking a bunch of APIs together, maybe downloading open source model weights and fine-tuning them, or even fine-tuning a closed source model, which is something that you can do, all those kinds of AI engineering skills. I think that the demand for those will surpass all other data science skills put together.
01:06:12
Now, that doesn't mean that all those other data science skills aren't valuable, even from things like data visualization to data communication, which are relatively soft, but even harder skills like statistics and other kinds of machine learning approaches, regression models, none of that stuff is going away. But in terms of what the market is asking for, I think we're going to see there's going to be surplus in demand for AI engineering skills specifically. And so that seems clearly to me to be an area to be leaning into as a listener to this show and add that as a complement to whatever existing data science skills you have. You don't need to be... Or you could actually even listen to our most recent Tuesday episode, which was number 847 with Ed Donner in which he talks about how you can become an AI engineer and you could jump right to that without necessarily developing other data science skills. But I think the most powerful way of approaching this is to have the AI engineering skills that Ed talked about in detail in episode 847 and using those to complement whatever existing skill set you have.
Sadie Lawrence: 01:07:24
Yes. I mean, to add on to what you said, I would say I think it's really a consolidation of roles and additional skills added on. If you signed up for any career in technology, you've signed up for a lifelong learning approach. And so it's just continued to expand. Just as you mentioned, it's not saying your data visualization skills are going to go away, not at all. Your Python, your SQL, none of that's going to go away. It's just going to be a continual evolution in building onto it.
01:07:53
Something that came to my mind as you were sharing this was when I entered the corporate world was in 2014, and at that time, moving to the cloud was a big deal. I remember meeting people in the IT teams who were like, "Yes, I used to manage these servers and these products," and they were nervous about moving to the cloud because they thought there was going to be less to manage, but what it came to find out is just the shape of their job changed. Instead of actually physically managing servers, you were managing permissions within the cloud. And so I see it somewhat similar in the role of data science as well. Maybe you're not training a model from scratch anymore, but fine-tuning a model and building onto it, really looking at how does that integrate into your existing products, into your pipeline. And so while it may seem like your job is changing or evolving, you're probably actually going to end up doing more, doing more roles and taking a higher approach to the orchestration of how it's done.
Jon Krohn: 01:08:59
Nicely articulated, Sadie. Yeah, spot on. All right, so I think that rounds up where we are. Oh, I have one last thing to add to this number five. I just consulted my notes here quickly, which is that something that's really annoying for me that has happened as a result of people integrating LLMs into everything that they're doing and allowing people to do more with less thanks to AI, the most annoying thing is how on LinkedIn now every post I make, I get several completely banal comments written by obviously generative AI. And if I believed that what was written there... Okay, let's say you have some thought that you want to express about some post on LinkedIn or some social media platform and you want to clean up the language around it, great. But so often, these comments are obviously there's no meaningful opinion in there-
Sadie Lawrence: 01:10:08
There's no soul to it,
Jon Krohn: 01:10:10
There's no soul to it.
Sadie Lawrence: 01:10:12
I completely agree. I don't even know how people do it. Maybe there's some extra premium subscription that I'm not on because I'm like, if you're taking time to copy, paste... But yes, the soul of it is missing. I mean, my whole thing is that misspellings are the new captures, right? Actually, throw in some misspellings, throw in some weird punctuation, just make it real and raw. And particularly if you're commenting on Jon's post, make sure that it has some soul in it and no more AI comments on that side.
Jon Krohn: 01:10:46
Yeah. So that is my conclusion for 2024. If you're out there doing that, please reconsider. You're just filling us with noise, humans with noise.
01:11:00
Well, Sadie, thanks so much again for the fourth year in a row running, making these predictions with us. I love how we spent more time on the retrospective stuff this year. I think it colored our predictions a bit more and was maybe a bit of fun for listeners in terms of what they think about as the overall winners this year, the comeback of the year, the wow moment, the disappointment of the year. I would love to hear from all of you in the comments with your human typed comments or even maybe Gen AI augmented, but just your real opinion at the soul of that comment on this episode to hear what you think about us doing the format this way and what your predictions are for 2025. Sadie and I would love to hear those and I'm sure we'll both comment in reply using our fingers on a keyboard.
Sadie Lawrence: 01:11:49
Or some voice to text because that always gets real messy and fun. I can't 100% guarantee it'll be fingers on a keyboard, but it will at least be from my soul to your soul.
Jon Krohn: 01:12:00
Nice. All right, so yeah, Sadie, anything else you'd like to add before we wrap today's episode? I don't think it makes sense for us to do the... Well, I'm not going to ask you for a book recommendation because we do this every year unless you have something that you think this year, a book recommendation that people need to know about something that touched you in 2024.
Sadie Lawrence: 01:12:18
Ooh, books that touched... I wouldn't say there was a book that touched me this year. I think my only comment and thought to wrap this up is one, just like how lucky we all are to live in this time and to work in this space. And maybe you're trying to get a job in this space, but the fact that you're even listening to this podcast shows that you have a deeper interest, a deeper knowledge, and that's a super lucky position to be in. And I hope that all of the awesome things that we talked about in this show, people go take advantage of, use these tools, are inspired and really create that utopian future that you and I hope for we dream of, we can see because to me, it's just such an exciting time to be alive. I'm so excited for this year ahead of us and I think it's going to be a really transformative year and I can't wait to connect with everybody and see what we got right, see what we got wrong, but most importantly, make adjustments along the way.
Jon Krohn: 01:13:19
For sure. Nicely said, Sadie. So for people who want to listen to you between this predictions episode and the one coming in about 365 days, Sadie, how should people follow you or connect with you?
Sadie Lawrence: 01:13:34
Yes, social media is great. I do love LinkedIn, X, Instagram, all of the things, I'm a big fan of. I do have a Substack as well, so I do share out what happened on a monthly basis. Happy to connect in all the common ways.
Jon Krohn: 01:13:50
Nice. I think when we did our very episode, our very first predictions episode or one of my first episodes with you on this podcast, your Instagram had been hacked or something like that, but now you're back on track?
Sadie Lawrence: 01:14:03
I'm back and better than ever, yes.
Jon Krohn: 01:14:05
Nice.
Sadie Lawrence: 01:14:05
So I had to start from scratch again. I had a viral post last year that gave me lots followers because I had a typo on it that said, "I married my dog." So there you go. This is why you still want to write your posts yourself because those typos can turn into your biggest win.
Jon Krohn: 01:14:26
What had you really done? Carried your dog?
Sadie Lawrence: 01:14:29
No, I think I had added something... I don't even remember now. It was like I got married and something about I have a cute dog. Anyways, my marriage and my dog got intertwined and through some editing and unfortunately they came together.
Jon Krohn: 01:14:44
That's funny. Nice.
Sadie Lawrence: 01:14:46
People are still in the comments arguing, "Did she marry her dog or not?" So you can join in on the debate.
Jon Krohn: 01:14:52
All right. Sadie, thank you so much for taking the time again this year. I know how valuable your time is and I'm always appreciative of it. Thank you so much and we'll catch you again next year.
Sadie Lawrence: 01:15:03
Sounds great. Thanks, everybody.
Jon Krohn: 01:15:10
What an exciting year we have ahead in 2025 indeed. In today's episode, Sadie predicted first that Agentic AI will be the dominant trend moving beyond single applications to create specialized networks that can autonomously handle complex tasks. Those security and permissions between platforms remain a challenge. Her second prediction was that AI integration into everyday devices will accelerate from augmented reality glasses with real-time translation to more sophisticated personal computing experiences, though not all integrations will prove valuable. Her third was that AI-driven scientific research will expand significantly, building on current successes where AI-assisted researchers achieved 44% more new material discoveries and 39% more patents than researchers who weren't AI-assisted. And Sadie's fourth is that enterprise AI monetization will be crucial as companies seek returns on massive hardware investments. And finally, our fifth prediction for 2025 is that demand for AI engineering skills will surpass traditional data science skills, though this represents an evolution of the role rather than a replacement, requiring practitioners to build on existing technical foundations with new AI engineering capabilities.
01:16:23
As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Sadie's social media profiles, as well as my own at superdatascience.com/849. Thanks of course to everyone on the Super Data Science podcast team for their work all year long. That includes our podcast manager, Sonja Brajovic, our media editor, Mario Pombo, partnerships manager, Natalie Ziajski, researcher Serg Masis, our writers Dr. Zara Karschay and Sylvia Ogweng, and our founder Kirill Eremenko. Thanks to all of them for producing another visionary episode for us today to wind down the year.
01:16:58
All right, if you enjoyed this episode, please share it with people who you think might like it. Review the episode, of course, on your favorite podcasting app or on YouTube, subscribe if you're not a subscriber, but most importantly, I just hope you'll keep on tuning in. Thanks for listening in 2024 and I hope to see you a bunch more in 2025. I'm so grateful to have you listening and I hope I can continue to make episodes you love for years and years more to come. Until next time, keep on rocking it out there and I'm looking forward to enjoying another round of the Super Data Science podcast with you very soon.
Show all
arrow_downward