Stop Dreaming, Start Doing: AI Where It Actually Matters

01We really need to stop talking about the latest and greatest model and like what we can possibly do with AI and you know, having all these conversations where we dream and we need to just start using it to do some things. People are getting there. It's just adoption is a lot slower than I it's a slower than I thought it would be. 100%. I feel like that that opening line for every one of them is this changes everything. Why aren't we talking about everybody in the middle? That's what we're that's what that's kind of what we're we're going to talk about today is we've got all this We've got all this conversation about you know Big, big companies spending bazillions of dollars on data centers or new models, the latest and greatest. And then on the other end of the spectrum you've got these really tiny AI startups that everybody wants to talk about.

02⁓ But all there's a ton of people, and this is kind of like you know, our whole podcast is there's a ton of people that work in the middle market with companies that maybe aren't as sexy, but we're the they're they're driving the economy. Yeah, what we talk about, right? Main Street is the economy. Exactly. Exactly. And so yeah, I just feel like we're really missing that whole piece of the conversation and you just don't see a lot of like, okay, how does a financial model work? How does IT get work done in a normal business now that we have AI? Well, 'cause we want to talk about AI, right? And that's kind of what we talking about today is the fact that like the conversation is not talking about AI. It's about what do I get better, faster, stronger with AI? What's the output? I mean, isn't that the same thing that we've been talking? I mean, like we have for a long time talked about what capability do you want to have at the end of this project that you don't have now or that needs to be enhanced?

03That's the whole question CFOs, CIOs have been asking for a long time, just to understand whether or not the investment is even worth it. I mean, like you know, what we were talking about was the fact that like, you know, when I turn on the news or I get come I'm sitting here at the computer, I get here in the morning and ⁓ I sit down and every newsletter I get is this new model dropped and it has this many parameters and it runs, you know, you you will soon have this on you know, this this one runs on a local machine and you can run this on your desk and you don't need to have, you know, thirteen Mac minis running in your closet, ⁓ any I mean like but but It's just talking about stuff. Yeah, it's not talking about the output or the value created by the stuff. I mean, it's kind of up to everybody else to say, All right, now I've got this stuff. What do I do? How do I make money with it? I mean, that's really the biggest thing is like how do we make money with this new stuff? I think I'd like to explore that a little bit with you today.

04⁓ that's that's ⁓ I think it's a big it's a missing piece and something that everybody needs to get their mind around. Well, and so there you know, I mean, I still lean heavily into the technical pieces of this and and and the technical pieces, right? I mean, I think people need to understand what is happening, but like what is happening isn't just like that's not the value, that's the that's the raw ingredients, right? That's the the the the tools that we have. and ⁓ you know we we tr A lot of the news right now is, you know, arms race on different models and how do we either make them as big as possible, or ⁓ we make them ⁓ you know, there is this argument about making them more efficient, and then there's this argument for like, well, ⁓ we're gonna make them as small as possible. And Fit ⁓ on your phone, your watch, whatever it is, that way your wearable doesn't have to send your health data someplace. It just lives on your phone and in your ring. Right. And so yeah, that's it it's good, it's useful.

05There's there's maybe a privacy, you know, related concern that people want to discuss and and put it on their phone. And that might be a good use case, right? But again, the whole point is so that you can track your fitness or your health and you want to do it privately, right? So that's your value, right? Not the fact that you have a small model. And ⁓ The fact that the the news keeps talking about, you know, the the newsletters, the the the talking heads that you know that I get to s you know that c end up feeding my really nerdy feed is, you know, about ⁓ things like models and quantization and you know and and ⁓ you know Stop right there with quantization. Why because that's gonna be a new word for a lot of people. So explain quantization. The easiest, you know, kind of like definition, and it's and it's close, is the fact that like large language models, you know, at the end of the day, it's just a giant wall of numbers. It's a lot of numbers with decimal point value. So it's very, very precise and it's millions of numbers.

06And if you want millions of digits of data to be less data, you compress it. You round it, or you compress it and you have less data. Is it wrong data? No, it's maybe less precise. Maybe. ⁓ these large language models are incredibly resilient to ⁓ compression and ⁓ this quantization, quantizing, doing this compression from 16 bits of accuracy, decimal points-ish, down to four or to eight. ⁓ it is great. It works. It is how ⁓ the ⁓ NVIDIA Spark ⁓ puts a lot of its ⁓ models on its hardware and tries to run them at a speed that is useful for end users. It's ⁓ plays into the stuff that I do on my computer here. ⁓ and it's not bad. I mean, ⁓ there's nothing inherently bad about it. But when we say you can run it on your machine, and there's a lot of papers that get put out that say, ⁓ I mean, like Th there's a whole thing around this one's like like hard for me just because it's like math because ⁓ this one point five eight bit quantization, which makes no sense, except like it's like a weird logarithm like number that like math nerds should know. But ⁓ it fast doesn't mean good.

07Good doesn't mean small, right? Small doesn't mean good, you know, ⁓ it just means that it's a tool and Yeah. I I think one analogy, just to maybe drive this home a little bit. And you can either agree or disagree with whether this is just too much of an oversimplification, but I think it helps the listeners. The quantization that we're talking about is basically saying right now you have Google Maps or Apple Maps, whatever. And You give me your address and I type it in and I go right there because it's using Lawn Lat exactly, you know, it's using GPS and all the satellites and everything, and it takes me right to your house. And then you quantize it and it's basically like saying, Hey, go three streets up and I'll be the fourth house on the right. So it's a simplified way of kind of getting to the same place. And when you simplify it like that, just to further the analogy. Is it your right or my right? I I don't I don't really know. So that's where the precision is, that's where you lack some of the precision.

08So when you make these models smaller, you're basically like giving somebody directions to the place you want to go, old school, like when you and I grew up. Right. Where the store used to be and the tree fell over and There's an old sign and No, it's not. But you're still gonna see, you know, where it was. I promise. But but do you get what I'm saying? Not there anymore. It is it is a good example. It it is less fidelity, but not wrong. And size and quantization of these models, I mean, I think the the ⁓ one, yes, it g it doesn't give you a wrong answer, it may give you a less precise answer. And it is purely a function of trying to make it where the speed of its processing of information can occur with less memory, right? So on your phone with 16 gigs of RAM. versus your Mac with 48 gigs of RAM or your NVIDIA Spark with 128 gigs of RAM. ⁓ the the speed is absolutely and it is like physically tied to how much memory and the speed of the of of the the memory on your computer. And and so making it smaller makes it faster.

09Sometimes you don't need rocket grade answers, right? Like giving somebody instructions like who who's smart can can go four blocks up, two blocks over. They'll get it. But that nuance is missed in a lot of the, you know, the the reporting is just about the arms race or the reporting is just about, you know, who's copying who or or the other end, right? ⁓ you know, how'd you describe this the other day? Like, you know, the the ⁓ I don't know, but whatever you're gonna say, it sounds like I said something smart that stuck with you. So so keep going. I like this. You were you called you call you called it a barbell, right? I you looks like a barbell graph. Yeah, yeah, yeah. 'Cause yeah, yeah w one side's frontier labs, the other side is startups that are building something. Right. ⁓ back to your back to your comment though about these small models and you know, the way I mean they're really they do have their place.

10I think I think one of the architecture concerns I have when people are using these is a lot of times they do want to use them on a machine, like locally. On like so when I say on a machine locally, I mean like I've got a desktop under my desk. That's where it lives. Who else has access to it? So I mean in FPNA, like that's always been or finance, I should just say, F PNA, financial planning and analysis for those who are listening that wouldn't know. Yeah. So but in finance, you know, we've always had We've always had this problem. We always try to push things into the cloud or into a server that everybody can access the data on because you can't, you know, if one person's out on vacation, the work still needs to get done. Some of these smaller models, I think people are trying to use them right now to accomplish some tasks. I I know that there are like analysts and folks out there that are downloading models so that it stays private and they don't Leak company information, but they're the only ones that can do it. Charles is the only one that has access to the model.

11So we're kind of like gone back to well, I need an Excel spreadsheet, you know, on this laptop, and all the data is in a CSV that I save in this file folder. So, you know, making these smaller models, even like if they're on a DGX Spark, which is a tiny machine that you've kind of mentioned, NVIDIA makes it and it's basically a computer that looks like I don't know, what is it? It's about the size of a Y y we would say cigar box because we're as I know as old as we are. Man, I was thinking like a C D player or something like that. I'm like, this doesn't work. I couldn't think of but anyway, it's it's like, let's put it this way, it's about like six by six and maybe two and It's ma it's Mac Mini sized. It's about the size of a Mac Mini. There you go. Well anyway, I mean, how do you you you have to figure out a way to make that not local. You've got to hook that up to your network to allow other people to use it. Well and yeah, if that's the goal, right, is to is to make it useful for more than one person.

12Well Up you're in a company. That's what we're talking about. We're not talking about a solopreneur here. We're talking about mid market companies. So it's like great, you've got one. That's cool. Only Mark can use the thing. Well Well, only Mark can I mean it th that brings back like, you know, we moved a lot of workloads and stuff to the cloud because we could spin up new stuff. We could we could ⁓ it was resilient, you know, ⁓ 'cause we got away from ⁓ if a switch fails in the closet in the office, now half the office is dark, ⁓ right. And everybody starts standing up and prairie dogging from their cubes, you know, and a mass exodus happens to go to the break room to go get coffee. Still happens. Shoot. Yeah. Somebody w we talked to somebody yesterday who was like the Wi Fi here in this office is Yeah. So so yes, right. Putting it in that, you know, you are adding so much additional overhead to say, I want to put it in my office, put it on this machine.

13You said, right, like you got I gotta make it so that not only Charles has access to it. We don't want the IBM mainframe to be down in the basement where it's only one operator who has access to it. Yeah, a hundred percent. And that's what frontier models kind of give you. ⁓ but these local models are good for privacy, etcetera. It's just how do we get these mid market companies to find the right balance between the two? I think a lot of this comes down to architecture. And that's I mean, that's really kind of everybody's domain now, whether they want it to be or not. You have to think a little bit in terms of how do I how do I design this new system that's supposed to do this cool thing and give me this great output? Right. So, okay, we've talked about what's in the news and what's, you know, what what's Veer reading when he gets up in the morning and sees stuff.

14But we talked a little bit about, you know, the fact that there's a lot of noise that is distracting to take people away from talking about how to use AI and spending time just talking about AI and saying that. That is the, you know, that's the pivot foot for the whole conversation, which instead, you know, it's about like shooting your shot. And so here we talk about ⁓ you know, yes, those the mechanics are important. They have pluses and minuses. We've got different ways that we can ⁓ implement these tools locally and in the cloud. The noise is loud about what is better or worse or smaller or faster. But we have been telling a lot of our clients, very specifically, you know, you need to come in Monday morning and you need to do something because you're already doing your job. And your job is not to go read a bunch of newsletters normally. you're you're you're right. So Pick something, right?

15Pick either a piece of work that is hard, inconsistent, repeatable, and and figure out is that something that would have a lot of value if you could make it either go away where it was not your fingers on keyboard doing the work? Or you know, is there ⁓ You know, speed to deliver, right? I mean, ⁓ would have value, speed to market to even get like the sales report out on Monday morning so that managers can call their employees and coach. Hundred percent. But this This is exactly what I've been trying to say. All the stuff that isn't that sexy. I mean, even stuff that is like deterministic, where you can use AI to build a system that will do the same thing over and over and over again, where you would have had to go and buy that, or you know, it was rife with human error or possibility of human error, I should say. All of those things are where I feel like people can spend their time. And we talked to ⁓ Justin Rae last week, and he mentioned, you know, getting we we've talked about this before, but he said it really well about you know, I've got to give my people reps.

16And then once they have that one good thing, they're gonna see all the other possibilities and be able to take on a little bit tougher problems. I think right now. What's stopping people is they think they have to learn a lot about everything that we've talked about. They think they need to go tackle the hardest problem at the company. But the best way to really do this is just like you said, pick one workflow and say, what do I want out of this at the end? What capability do I want to have that I don't have right now? And that could just be I want the capability to analyze this report. Not just put it together for three hours. So go through the workflow, write it down. Right. Well, and I mean, I I don't want folks to like like gloss over kind of what you said there. folks that are listening, your value as a leader or as a manager or as an executive is making decisions and then following through on those decisions. And in order to make those decisions, you sometimes have to synthesize information and and and ⁓ you know, read a chart, make some assumptions.

17play with it a little bit or or you know look for the trends. But the work to build that is not the value add that you bring to the table. And right, you you said, you know, so that you can do the analysis, not spend four hours. The and one would say the best four hours, right? The sharpest, most caffeine fueled, you know, moments of your day where you're you're you're moving fast and you're able to get good decisions made, but instead you're just grinding at a administrative part of your of today's job. And you don't have to do that. I mean, so perfect example, thirteen week cash flow forecast. Everybody does it. Right. Everybody do well. Everybody should do it. so, you know, when you bucket that thing, I mean you might have twenty, thirty lines in a mid market company that you can like kind of bucket the whole business into. I realize that if you're at a really complicated company with lots and lots of lines of business, there could be hundreds.

18But I'm saying when you summarize everything like a summary, you know, you're talking about ⁓ okay. Maybe fifteen, twenty lines. Inflows, outflows. Customers pay us money. We pay people money. What does that look like? That is a very easy thing. To over the next several weeks sit down and automate. You know, there's a I went through a whole process to do this. And then let's say it took me it when I started doing this, it took me 30 minutes to write a really good prompt. And that was with me talking it into existence, right? So using transcription, telling it exactly what I'm doing. It being the AI, telling that. And then it took the AI like two to five minutes to do all of it. And then now I have the three or four hours to just look at the numbers. Did the process go through correctly? Or hey, why did that deviate from what we thought? Why is the forecast accuracy not as good there? ⁓ I have time now to call Justin and ask him. Hey, did you guys do something different here?

19All those rich conversations where you really learn about the data and figure out your business, that's what you can do now. Well, and you mentioned like you spoke the answer into existence by kind of working the problem and having it the the tool join you. But right, the the setting it ⁓ you know, for for the for the for the concrete to set, right? The the end of that story is can this be automated? Can I can hey Chat GPT, hey Claude, can you make this some type of ⁓ you know, we use this word durable, right? It just means that it's something like a script, something that can run on your computer repeatedly, and you don't need the LLM to run it. Right. So using the LLM to build what you need that no longer requires the LLM is probably one of the best use cases I could think of for a lot of this not very sexy behind the scenes work. And I think as we talk to folks, folks are having a hard time wrapping their heads around that model, except if I replace the word LLM in that statement with programmer.

20Like you on this finance side, marketing side, you have asked me on the IT side this for years. You would ask me, I have a report, can a programmer make it so that the file is ready for me and sits in a directory when ⁓ you know I come in in the morning and then we we we we build stuff. You now have a personal programmer. Like if you can just kind of mentally leap from you don't need the programmer to do your work every day and have it have variance in the work that you do. You can have the programmer help you come up with what you have always asked IT for. Build me something that drops the file in my r in my ⁓ email every day so that I can do my job. Cause your job is interpreting, not building. As leaders, we really need to stop talking about the latest and greatest model and like what we can possibly do with AI and you know, having all these conversations where we dream and we need to just start using it to do some things. ⁓ people are getting there. It's just adoption is a lot slower than I I it's a slower than I thought it would be. hundred percent.

21And I I'm I talk to CIOs and technical leaders here ⁓ every week and they I am I am shocked at how few organizations are embracing it. Even the folks who are the the I would have expected like the more you know kind of technically savvy teams ⁓ And it's not an indictment, but it is a little bit, I mean, I it is surprising because I I would have expected my ⁓ my my ⁓ colleagues who are in protected environments ⁓ or domains, healthcare, finance, being ⁓ careful and and being more you know, conscious of the security side, ⁓ but I would have still expected them to be going down the road of trying to figure that out rather than maybe saying, It's just too hard of a question for us to deal with right now. And I realize everybody's busy, so it's not ⁓ a a c a a question of like whether or not someone's intelligent or not. But ⁓ it is it is surprising given what the the upside is. Yeah, but that's why you need people like us. That's why you need AI fluency training. You gotta show people the possibility. I did training earlier this week with a company. That's not my full-time job.

22That's not what I necessarily aspire to do in life, but helping people understand what's possible with AI is so much fun. Had a great session about how to use co-work, tons of really good feedback. Now we gotta get ⁓ hands on. We need to have some contests that allow them to use it. But that's that's really all it is. I think companies that, you know, give people time and space during the workday to do it, you know, are are gonna are gonna get there a little bit quicker. But yeah, gotta start doing it. Can't just talk about it. Right. And the it, if we can if we can stay out of the the kind of the news and the hype cycle about talking about AI, talking about the companies that build AI, the tools are there. And there's a lot of parody in them. Pick one. Don't worry about the cost right now. Not saying you can spend a million dollars, just saying most people, if you will play with it, you're you're not gonna break the bank.

23⁓ and ⁓ taking a Whether you sign up for a twenty dollar a month plan or a hundred dollar a month plan, ⁓ that is a small investment into professional development. You know, if you're just gonna be greedy about it, just be greedy about it and and and say that you're gonna put a little bit of money into professional development instead of, you know, buying a book or, you know, going to a conference this year. So ⁓ so yeah, do the work. Do the work. I like it. I think that's a good place. I think that's a good place ⁓ for us to to kind of wrap it here this week. So yeah. Yeah. Well, thank you so much for taking time out of your day to listen to to Veer and I talk about this. We're working on ⁓ some new guests. So hopefully over the next few weeks we'll have even more guests. I really enjoyed our conversation last week with Justin Rae from Cinch. And ⁓ we look forward to, you know, hearing some of your stories too. So please let us know. How are you using it? Like in real life, how how has it left the conversation and become a reality?

24That that's something that we'd love to know. So please drop that in the comments or email us, ⁓ hit us on LinkedIn. We're we're in all the places and we'd love to know more. Right, well this has been Veer. And I'm Jim, and we are executives. And ⁓ thank you again for listening. See you soon. Fox Executive Alright.