Podcastskeyboard_arrow_rightSDS 842: Flexible AI Deployments Are Critical, with Chris Bennett and Joseph Balsamo

17 minutes

Data ScienceArtificial Intelligence

SDS 842: Flexible AI Deployments Are Critical, with Chris Bennett and Joseph Balsamo

Podcast Guest: Chris Bennett and Joseph Balsamo

Friday Dec 06, 2024

Subscribe on Website, Apple Podcasts, Spotify, Stitcher Radio or TuneIn


In this Five-Minute Friday, Jon interviews Chris Bennett and Joseph Balsamo on the importance of flexibility in the way we deploy AI models, Dell’s brand positioning in the AI space, and whether GenAI’s business applications stand up to the hype.
 

About Joseph Balsamo
Joseph Balsamo is the Senior VP of Product Development at Iternal Technologies, leveraging 30+ years of expertise in AI solutions, global IT leadership, and business strategy. He leads the development of patented Blockify technology, enhancing LLM accuracy by 40x and enabling rapid, scalable AI adoption. Joseph has pioneered cloud migrations, data security, and compliance solutions, aligning technology with business goals to deliver impactful results. Known for building agile global teams, he empowers enterprises to reimagine workflows and achieve operational excellence.

About Chris Bennett
Chris Bennett is the Global CTO for Data and AI Solutions at Dell, specializing in cloud-scale infrastructure and enterprise workload strategies. A visionary technology strategist and product innovator, Chris excels in ideating, prototyping, and launching customer-driven solutions. As a public speaker and technology evangelist, he inspires organizations to embrace cutting-edge technologies. With a focus on building high-performance teams, Chris drives transformative innovation in modern IT infrastructure and AI adoption.

Overview
Chris Bennett, Global CTO for Data and AI Solutions at Dell, and Joe Balsamo, Senior VP of Product Development at Iternal Technologies, sit down with Jon Krohn at the ScaleUp:AI conference in New York.

Jon first wanted to know the story behind how Dell and Iternal came to work together. Dell and Iternal’s partnership has a long history, with Iternal having been part of a proposal creation platform with Dell, helping sales teams create personalized proposals many years back. Ongoing partnerships are not unusual for Dell: Chris says that it’s important for the group to have longstanding connections with trusted providers, especially considering how the business value of AI is still in its early stages. So early in the game, Chris believes agile companies like Iternal can really help Dell to access use-cases at a variety of scales and niches.

Flexibility is essential to Dell’s business strategy and internal operations. Chris added that Dell also offers considerable flexibility to users. Dell powers a user’s tool chain, supporting them in building AI-powered applications, however large the infrastructure suite may be. Dell is, essentially, all about enabling that flexibility and then keeping it going throughout the project. What this means is increasing interoperability via their partnerships and Dell’s own software-defined and cloud-based solutions.

When deciding what AI tool to use, Chris appealed to listeners to understand a combination of factors: What works best, the performance, implementation and resiliency costs, and what other software and applications it supports.

Finally, Jon asked Chris and Joe about the ubiquity of GenAI across industries, and whether its uses are overplayed in business. Joe responds that, though it is good to know that more people are paying attention to the benefits of GenAI, business leaders should be focused on generating business outcomes, not AI. Decision-makers should rather look at what they need and find a platform that will solve that problem, and that they should preferably seek small wins with the help of AI before they look to turnaround their business.

Listen to the episode to hear how Chris troubleshooted an issue with an inflexible client, and why it’s so important to understand a client’s business problem inside out!

Items mentioned in this podcast: 

Follow Joseph

Follow Chris

Did you enjoy the podcast?
Jon Krohn: 00:00:00
This is episode number 842 with Chris Bennett and Joseph Balsamo.

00:00:19
Welcome to the Super Data Science podcast, I'm your host, Jon Krohn. Today’s episode features the highlights of a session I recently hosted on “Flexibility in AI” and this has not one but two guests. The first is Chris Bennett, who is Global CTO for Data and AI Solutions at Dell and the other is Joseph Balsamo, who is Senior VP of Product Development at Iternal Technologies. Today’s episode should be interesting to anyone. In it, Chris and Joe detail why it’s critical to be flexible in our deployment of AI models and why the term “generative AI” is overhyped. Ready? Let’s jump right into our conversation which was recorded at the Scale:Up AI conference in New York a few weeks ago. The interview you’re about to hear came hot on the heels of my interview with superstar Andrew Ng. You can check out his interview in episode 841, so that’s what I’m referring to at the beginning of this conversation. All right, let's cut right into it.

00:01:16
Welcome back. Second stage. I hope you enjoyed that session with Andrew Ng. I'd say a tough act to follow, but we've got great guests now as well, for this session. We've got Chris Bennett from Dell, and Joe Balsamo from Iternal, which if you're Googling that, it sounds like Eternal, like Eternal Paradise, but it's with an "I," kind of like "IT" at the beginning. Although, I tried to dig into that and there apparently made it be no basis to that etymology that I've hallucinated. So, that's an LLM hallucination from me.

00:01:48
So, let's first talk about how Dell, a market leader in GPU-enabled systems, like AI systems, and Iternal, a comparatively small firm, came to work together. And what do you guys do together today?

Chris Bennett: 00:02:03
So, thank you, first of all. And thank you Scale.

Jon Krohn: 00:02:07
Scale:Up AI

Chris Bennett: 00:02:08
Scale:Up AI, yeah, wow. There was my LLM moment. So we had been working together for just under a year, I think. Maybe more.

00:02:20
Actually, if now that I think about it, we've been working together for a very long time, because they are behind a proposal creation platform that Dell has been running for quite some time. So, bringing AI to our sales teams and helping them create individual, personalized proposals. But the interesting thing to me about Iternal, and we have an entire ecosystem of partnerships with the smallest of the small ISVs, all the way up to the largest of large companies that you can imagine, in IT and beyond, and you know, one of the things that we are trying to do, is we're trying to build a stable, if you will, of trusted providers, where we have worked with them in the past. We've delivered exceptional results to our customers um on top of our infrastructure, or in cloud, or in a hybrid mode, however that manifests itself.

Jon Krohn: 00:03:11
Nice. Anything you'd like to add, Joe?

Joseph Balsamo: 00:03:13
Yeah. I mean, I believe the business value of AI is still really in its early stages, and you kind of need that complementary scale and niche, or specific use cases, in order to align to where you are, whether you're crawling or running. And I think the combination of Dell and the combination of an Iternal could really help you with those types of cases, whether they're small or large.

Jon Krohn: 00:03:35
Very nice. Chris, for those of our listeners who are not already familiar, can you explain the AI services that Dell provides? I think when people think of Dell, it's a no-brainer that this is a hardware company. You can instantly see the Dell logo on the top of a laptop, or on a server, and maybe some people out there aren't immediately thinking of Dell and the AI services they provide. So, fill us in on those.

Chris Bennett: 00:03:59
Yeah, the interesting thing about our company and I've been here for a long time. I'm not going to say how long, but this might help.

Jon Krohn: 00:04:07
His beard.

Chris Bennett: 00:04:08
My beard yeah. So the interesting thing about Dell is we've been built from a college dorm room, all the way up to you know, the data center and consumer services provider that we are today, over the last 40 years. The last 40 years of execution and strategy and acquisition have led us to this sort of pivotal moment in this industry. You know, we've been through many epics in the IT space. And one of the things that we've done deliberately is we have built, again, back to the ecosystem. We've built a massive ecosystem of partnerships. And we've also built up capabilities in our own consulting business that complement our partners to bring services to our customers, no matter where they are in their continuum, right? So, if they're just trying to figure this AI thing out, they're under, they're really, maybe they're a little bit terrified of it, maybe they have some use cases identified, but they're not exactly sure how to identify the data that they're going to need to use, etc. One of the things that we can bring, if we don't have that discrete service in our own stable, we've got partners that do. So, I think, from an end-to-end solutions company, I would pose that we have unique capabilities that no one else in the marketplace has. Whether you think about the desktop with AI-enabled devices, to the edge computing devices that are AI-enabled and GPU-enabled, to the data center, to the world's largest, you know, large language model training farms, we are the infrastructure behind most of that.

Jon Krohn: 00:05:39
Very cool. Very cool. One of the advantages of working with Dell, I understand, is the flexibility, when you're using these kinds of AI services. Flexibility between doing your AI training, or inference, on-prem, in fully in a cloud, or in some kind of hybrid model, between your own premises and some third-party cloud. And Dell also offers, if you are working with a cloud partner, flexibility in your vendor choice. So, tell us a bit more about that advantage of flexibility.

Chris Bennett: 00:06:13
I think, I think when it comes to flexibility, there's couple of, there's couple of dimensions to that, right? The first dimension to that is, what tool chain are you going to use, right? We're not a tool chain company. We're the infrastructure that powers it, right? And when you think about the platforms that need to be built on top of the infrastructure, the platforms that need to be built need to be, let's call it portable, right? So, if I'm building something in cloud, and I'm piloting things, and I'm testing it in cloud, that's fine, right? We've got software-defined platforms that run cloud enviro hyperscaler cloud environments. We've got software that same exact software-defined platform runs on premises in data centers and colos. And I think the important thing, in my mind, is as you're considering building these AI-powered applications, right? Which encompasses a very large suite of infrastructure, both software and hardware-defined, you have to think about it in a way that, if I want to move this one day if I want to pick it up from my hyperscaler cloud platform and move it to a colocation provider, or go the other way, right? I've got to be able to do that. So, if I use a bunch of proprietary tooling, and it leaves me stuck in a particular consumption model, that's going to require refactoring, which is going to significantly increase the cost to move, and it decreases your flexibility. So, that's kind of what we're all about. We're all about flexibility. We're obviously have a strong bias toward on-prem, but we interoperate pretty much where our customers live. So, and that's our motto. You know, we preach, bring AI to your data. So, if all of your data is in a hyperscaler cloud, let us help you, with our partners and with some of our software-defined solutions. Let us help you realize that vision in cloud. And then, you know, run your applications and your workloads where it makes the most sense.

Jon Krohn: 00:08:00
You said that it's obvious that you'd have a bias towards on-prem, and I guess that's because you're hardware, a hardware company. If I'm thinking about an application that I'm building, why should I be thinking about potentially having an on-prem application, or a hybrid situation, as opposed to just using a third-party cloud?

Chris Bennett: 00:08:20
I think a lot of it really just dictates to, again, where that customer is, right? If they're all in on hyperscaler cloud, it might be career-limiting for me to say this, but maybe you should run your AI applications in hyperscaler clouds, because that's where your data is, right? So, we believe that fully the large majority of data going forward, and even historically, is being produced at the edge. And what that means to us is, not in a traditional data center location. We think the data warehouses, etc. live in consolidated spaces, whether it be in cloud or on-prem, but really, where kind of the action is happening now is at the edge, right? We're seeing, in almost every industry, where you know, new data is being created in volumes, very large volumes. So, I think, you know, again, where our customer is where we want to meet them. We don't want to be, we don't want to dictate that you must and you shall and you will, right? It's, where are you, and how can we help you advance yourselves? It's kind of a new dimension for us, right? We're kind of used to backing a truck up to a loading dock and saying, "Hey, I've got 3,000 servers in here. How many can I leave with you?" right? So, it might be a little bit of an uncomfortable conversation for us to have, and it's conversations that we're having in new areas, but it's an important conversation.

Jon Krohn: 00:09:44
Nice, very cool. Do you have examples, Chris probably, of examples where a client was inflexible, unlike Iternal? And so we've talked about how important it is to be flexible. If you have examples of where a client was inflexible and how that led to disastrous results.

Chris Bennett: 00:10:01
From a flexibility perspective, I think there are so many different ways to get this wrong, and there are so many different ways to get it right, right? So, if I tie myself to a model, what I might find is, as my user count goes up, or my interaction with my AI system goes up, that model might have a premium over something that might be hosted on-prem, right? So, if I'm using an API-based model to host my application, and it's very, very difficult if I've written my application specifically to that model. So, I think, I you know, I'm not going to I won't call out any companies, but I think, from a from a flexibility perspective, we would encourage you to look at all of the options, look at every option that's out there, understand what works the best, but also understand what is the performance cost, what is the resiliency cost, and what is the actual dollar, or euro, or yen, or wherever you happen to live, cost. What is the cost of implementing it specifically to one ecosystem, right? We are all about open. We have a silicon diversity strategy. We have a platform diversity strategy. We support open source. We support, you know, fully integrated stacks from our silicon providers, and we support integrated stacks from our software partners. So, I think, you know, when it comes to success stories, I prefer to think that we have far more success stories. I will give you one very quick example, and it'll be internal-facing. We publish..

Jon Krohn: 00:11:41
Internal or Iternal?

Chris Bennett: 00:11:42
Internal to Dell. So, very early days, we built an application, and the idea of the application was back when the usage and deployment patterns for augmented workloads workflows were not well-understood, right? So, RAG is what we call it today. Terrible term. I hate it, but it's a term. And it's a year old in August.

Jon Krohn: 00:12:05
Responsible Augmented Generation.

Chris Bennett: 00:12:06
Yes. Imagine. It's actually mature. So, we deployed this platform in a pilot stage to about 1,000 users inside of Dell, and it was a very easy-to-understand format. The UI was incredible. It worked really, really well, until you asked it a question. And what we learned was, the data set, the corpus of knowledge that we used to train the model, right, because we thought you had to train a model, and then the systems that we were pointing to, that contained the enterprise data, they were all behind what I'll call a paywall. So, if I, as the user, had access to that paywall, I could ask it anything I wanted to, and it would answer me authoritatively.

Jon Krohn: 00:12:46
Right.

Chris Bennett: 00:12:47
But if I didn't, it would it would literally hallucinate a response, because the model would respond in the best way it knew how. And at the time, again, I'm not gonna tell you what model this was, it's an open model, but it's a model that was in its infant stages. It didn't really understand how to say, I don't know the answer to your question. So it would fabricate the question. So, and again, and this all goes back to, when you're doing this, you need to understand the consumer of your application. You need to understand the rights that the consumer has to access the data that this application has access to, right? Don't use system as the login for HANA when you're using the HANA agent to connect your reg solution to your AI application.

Jon Krohn: 00:13:31
Nice, great story. I'm glad you were able to give us something. If I asked about a disaster and you didn't have any, it would have been a bit of a dull question.

Joseph Balsamo: 00:13:37
Iternal never had a disaster.

Jon Krohn: 00:13:42
One last one for you, Joe, before we get to it. We've got some great audience questions that have come up. But one last one for you is, do you think that we're using the term generative AI excessively? It is a term that we hear a lot, especially in data science and the AI world, but in enterprises in general. At a conference like Scale:Up AI today, you hear a lot about generative AI. Do you think it's overhyped?

Joseph Balsamo: 00:14:06
I think that business leaders want to generate business outcomes not AI, right? Is it overhyped? I mean, it's getting attention that needs to be there. So, I like that aspect of it. I like that people are paying more attention to it. Not everything is generative, and you might not need generative for what you need. You need to look at your business cases and find a platform that is right for you. Find a company that's going to allow you to get as close to turnkey as possible with the technology and try to realize small results before you try to realize big results, right? And again, make sure you know what your data is and make sure you have the confidence in your data before you go at it. That's where I've seen those disasters.

Chris Bennett: 00:14:49
Let me just add one quick thing to that. You know, the adage of rising tide lifts all boats. It's very, very true here, right? When generative AI, generative AI, first of all, it's been around for a long time. But the practical application of it sort of exploded two years ago -ish and when that happened, all of the other AI platforms that people had deployed, machine learning, traditional AI, whatever, and however that manifested itself, they all almost instantly became more capable. They didn't all instantly become better, but they all became more capable as a result of the innovation curve that was happening at such a massively fast speed. So I think that's, you know, it's not really overhyped, but I think when people think of generative AI, they think of AI in general, maybe I should trademark that. But you know, it really is a question of, you know, we now have capabilities with machine learning to use datasets in a way that we hadn't even imagined before, right? Use cases across businesses are exploding because now it's no longer, it's actually left to imagination now, right? If I can imagine it, I can probably do it.

Jon Krohn: 00:15:59
And it also provides you access to all kinds of unstructured data, which is most of the data that we have. So that was a big point at the end of our conversation with Andrew in the preceding session, was that with generative AI tools, now you have access to all this unstructured data. And he gave a real time demo of using, you know, typing in some keywords and that pulling out specific frames in video fully automatically without any labels being explicitly trained. So, lots of flexibility thanks to generative AI. So yeah, Chris Bennett from Dell, AI and Data Solution CTO over at Dell and Joe Balsamo, VP of Product Development at Iternal. Thank you both for your time today and your excellent answers on flexibility in AI systems.

Chris Bennett: 00:16:44
Thanks so much.

Jon Krohn: 00:16:46
All right, I hope you enjoyed today's conversation with Chris Bennett and Joseph Balsamo on why it's critical to be flexible in our AI model deployments. To be sure not to miss any of our exciting upcoming episodes, subscribe to this podcast if you're not already a subscriber. But most importantly, I just hope you'll keep on listening. Until next time, keep on rocking it out there and I'm looking forward to enjoying another round of the Super Data Science podcast with you very soon.

Show all

arrow_downward

Share on