Using AI to Speed Up Development Without Losing Control

01Let's say that the units in which we pay for AI, let's say those triple next year. Have you guys experimented with local models? That's largely how we've solved that that QA side of the equation because there's aspects of that that we don't want to ship outside. I am Jim Veer. How are you today? I'm doing great. How about yourself? I'm doing fine. I closed alone yesterday, Evan. I closed alone yesterday. we'll introduce you in a second, but for now we we know you. but yeah, I mean, my it it took forever. I won't say the bank's name, but w wasn't a super complicated loan, but it took four or five months. To get through underwriting and everything and close and not the typical process. They had they had some turnover throughout and in the midst of your yeah. Yeah. Yeah. We actually went through underwriting twice because of a merger. So one bump bought another bank in the middle of our loan. So but i in the end it was great. I did leverage AI on my side for a whole lot of the stuff. So that's my big win right now. Today. Right.

02We're gonna be talking a little bit about with Evan Moser, CTO at iClass Pro. Right. Jim, you've known Evan a lot longer than I have. I have. I have. And we worked together at a company called Streamline Brands for a couple of years. And I think I'll leave it there. And Evan, why don't you why don't you tell us a little bit about your yourself, you know, your background and then What you do today at iClass Pro. And also tell us a little bit about iClass Pro because people will want to understand that software so they have a better idea of your point of view. Yeah, sure. I yeah, I'm I'm Evan Moser. Like Veer said, I'm the CTO at iClass Pro. I've been at iClass Pro about two years now. Prior to that, Jim and I worked together at a company called Streamline Brands. It was acquired by a company called Youth Enrichment Brands. I was at that organization for about five years running their technology. originally when I got brought on to Streamline, we had a homegrown platform that was that we wanted to take to market really, was kind of the pretext there. It wasn't ready to take to market.

03And so we had a small development team and what kind of a platform was it? It was a registration and billing platform specific to the the swim lessons that we were doing at Streamline Brands. So Streamline Brands was a swim school. franchisor. we owned a bunch of different swim school brands. and that platform we called SRS, the student registration system. I asked you that clarifying question, but for just all of our listeners, I feel the need to say as the CFO at Streamline Brands at the time, I knew what that software was. But anyway, all right. Yeah. I I just I actually never talked about I feel I feel like my credibility was gonna go down instantly if I didn't clarify. All right. Keep going. That's yeah. Jim and Jim was aware on the periphery of what SRS was. God. Perfect. Perfect. Perfect. Well done. no, but it was registration and billing platform. Ultimately we we ended up deciding that that taking that to market was gonna be something that was kinda outside of our core competencies. And so I actually shopped alternatives.

04iClass Pro was one of those alternatives, got to know that platform really well and ultimately I had a pretty seamless migration over to it, fortunately, and and coming over to iClass Pro about two years ago from from that migration. So it's been really good. Yeah, that's awesome. That is good stuff. So you know, we we've done a couple of we had a couple of guests on the show and and sometimes, you know, right we we lean away from the technical, sometimes we lean towards the technical. We're leading into the technical today. but yeah, but I but I'm still here. As a CFO. So don't freak out everybody. Like I'm gonna ask I we'll talk about some other things. It's not gonna be just ones and zeros from the nerds. So Yeah, I I know. You're you're you're very pretty, Jim. Okay, well I'm just we won't forget about I feel like I'm leading with my chin today, Evan. All right. So so the other day you know we were talking about a project that Evan that you guys had going ongoing initiative. and you know, kind of the topic here was around modernizing an API for a platform.

05I mean, I and we talked a little bit about the fact that like, yeah, pretty common, you know, kind of it's it's a space where technical debt grows as as a platform matures and as you add more features, as your user behavior changes, the the things that we built early that they they either become inefficient or insufficient for what our people want from us. and so then, right, sometimes we have to, you know, we can extend them to a certain point and then we have to maybe like do a a rewrite of of the software. You know, that's sometimes tough to justify or to to to work through. It's complicated. There's a lot of dependencies. So tell our tell our lit kind of listeners about kind of what you had going on. Sure. Yeah. It's so iClass Pro is Software as a service company, we have a bunch of different platforms that we write. Our flagship product is also called iClass Pro. It's that registration and billing platform that's focused primarily on on youth activities. So gymnastics schools, swim schools, cheer schools, those kinds of things. the platform's pretty generalized.

06It could be applied to any industry, but those are the industries that we we typically focus on. And in some cases, we build teachers directly for them. I think Veer you were being very polite, but the reality is we've been we've been in business for almost twenty years and the platforms really mature. And when you have a mature platform like that, you're gonna accumulate some tech debt. And so, one of the things that we're going through right now is a is a migration of stacks from kind of our legacy stack that very transparently we're actively outgrowing. And so we've we've been fortunate that we can kinda scale with the times. but at this point we've just got to make some changes. And so we've been working for the last couple of years on updating our APIs and greenfielding some new applications that would be supported by those APIs. that project initially started kind of pre-AI. and AI started really coming to its boom when we when we entered that project. And we had some pretty creative people that that tackled it in a pretty creative way.

07And so we've been pretty fortunate in in what they've been able to put together for us. well, like you like you said, you know, like for folks who are listening, I mean, like, you know, don't wanna gloss over the fact that you're like, yeah, it's mature. It's been going on for, you know, the project itself, like kind of started maybe a year ago. Like we're so used to, right, when we're watching social media and stuff and people are like, you know, I did I built this thing last night, right? I built a whole platform. Right. There is a growing consensus or or an expectation Right, of of what what is possible with AI and and and it's you know, overnight is is becoming a expectation for for people. And we've lived in both of us have lived in worlds where we've done twelve and eighteen month projects and and those were the you know, and and and we were we were booking it to get it done in twelve and eighteen months with with teams of people to work on stuff. so this one. You you'd had been working on it. You kind of mentioned it had a pretty long horizon left on it. Yeah. It did.

08And we a lot of the so I'll tell you a little bit about what we did. Ultimately what we did was we we have a pretty standard development cycle. We have a product team that scopes out our work for us, roughly what we want to accomplish. We've got a design team that helps us with the UI side of things if if that's applicable. In the case of this API, it really isn't applicable. Quickly, so the APIs, this is really foundational work. So it's kind of behind the scenes stuff that, you know, doesn't always get top billing. Yeah. Yeah. I think that's that's fair to say, right? Nobody nobody, myself included, when when we're releasing new things, wanna wants to wants to focus on things that aren't flashy, right? We want to get our customers some features that are meaningful to them and and tech debt generally. aren't aren't those things, right? They they set you up to do those things and they're absolutely critical. so fortunately with with AI and how we approach this, we can do both at the same time, which is a calculation that we just didn't make previously.

09So we would have built a foundation, then we would have started writing these applications on top of that foundation. And with the way we've accelerated our workflow with what we've done, we can we can do that. and I think it comes to like the The why of why we approach the problem. Like I said, I'll tell you what we did in a second. But we we pretty routinely plot our competitors on that Gartner magic quad quadrant. Are you guys familiar with that? Where the measure against ability to execute and completeness of vision. Well, when AI really started hitting its swing, everybody's ability to execute just exponentially increased, right? And so we've got new entrants that are entering the market that don't have tech debt. And so we we don't really have the luxury of of spending two years writing an API while not creating something new or or meeting the needs of our customers. And so we we got we got lucky with how this problem set came along and and how some of our engineers solved it. Yeah.

10And and I think you know to f to kind of add on to what Jim was kind of saying, you know, when I feel like for me, when I've gone to the CFO as the CIO, you know, the story And and and we go to the board, but the story is not, I need to do maintenance and I need to upgrade the the API. It's what you're saying. It's it's either a competitive hedge, right? The the API is not an oil change and an oil filter that will keep the car running. The API changes and stuff are additional potentially capacity for growth, right? Or right, so that it it has a sales component, a future looking component. Right. I I think that's a big piece of right kind of how you talk about stuff, Evan, how I talk about stuff, right? We're we're we're trying to make sure that we talk about why we're doing it, right? Not just it's the right thing or, you know, right, it's three thousand miles. That's what I need, you know, in my position. That's exactly what I need is the story behind it. Like what is the unlock or what is the you know, what does this enable? What what capability does this get us?

11So it's less about the ones and zeros and you've got to do you've got tech debt. It's more about, hey, if I do this, revenue can increase because I can unlock these five new features. Right. Yeah. And in our case too, there's a cost element of it as well, right? A legacy stack is gonna be more expensive. And so when our CFO and I were chatting about this, that you know, that won him over before I said any other words. So I think I think we we we had the opportunity to to dedicate some time to this project while also you know approaching those new prop platforms as a on a green field basis. And so yeah it it became a no brainer when AI really started hitting its stride. early on when we were tinkering with it, we were we were like everybody where we're just like, you know, it's not quite there yet. You it's not quite a junior developer. Their code it writes is acceptable, but doesn't really accelerate us. And then it just kept getting better and better and we just we just it feels like we just got lucky.

12that we had already dabbled in things to accelerate our workflow and we could just apply new models to it. So well, I mean, the other day you talk you were talking about this was a project that had like a like a a more than a year horizon on it that you were hoping to finish maybe the end of twenty seven. Yeah, it absolutely did. So we we accelerated that timeline pretty significantly. We're looking at finishing the project here in about the next month, which is which is really exciting. So it shaved about a year off of the the endeavor and the way we didn't I mean really think about that. That's that is bonkers. Yeah, it was it's material. And so it gave us the ability to to to apply those resources elsewhere too, which is like I said, why we could why we could do other applications at the same time that we otherwise wouldn't have. So so people will ask, right, okay, it accelerated it. So did you just you know dump it all into AI and a magic answer pops out the other side. Unfortunately no. Unfortunately no.

13So the the engineer and the the managers that that solved this problem for us, the way they approached it is that we didn't it was hard for us to get a scope of the work we were doing, right? We've got this legacy application. normally what like I was saying, our product team would kind of say what we're gonna do. The design team would tell us what it was going to look like. And then the developers would take that and run with it and they'd they'd create the application. Well, in this particular case, we're taking a legacy code base. We're trying not to repeat the sins of the past, and we're trying to convert that into a new code base that is you know, significantly faster. And so we didn't have the luxury of having product folks kind of involved in that process. And so the engineer that solved this problem, they they built that process with a series of AI agents using different models. And this was like really early on into AI adoption. So he was really stretching the technology and what it could do.

14But he found that if he had different agents responsible for different things, the scoping element of it was one process. he would give it general prompts, pretty detailed, but general prompts about what needed to happen. he'd have another agent investigate the legacy code base. He would have another agent writing tickets based on what it found. And then he would have agents that could potentially write the code for you and create a pull request and submit it to a developer for review. that last part's kind of optional based on what the what the developer wants. wants to do. but effectively it does all the product scoping, all the tech scoping, all the ticket writing, all the breaking up of work into smaller chunks, because as a developer, you don't want to, you know, have huge pull requests that that are impossible for anybody to read. That kind of Yeah, I'm gonna jump in there for just a second. Again, I'm the resident, you know, help all the non-tech people along. So in in layman's terms. Can you describe that pull request and maybe like some of the lead up to that?

15So everybody kind of understands, okay, we're developing something, but then there's some jargon that goes along with it. And I think it's actually really simple to understand if you if you don't, you know, use the the jargon and the GitHub terminology. Yeah. Yeah, absolutely. So yeah, ultimately it's a submission of code that that you're saying, hey, this is ready. Kind of pull it into our our main code branch and And ultimately deploy it. And so everything on the front end of that process, there's a lot of things that have nothing to do with code. It's just preparing to write the code. The code gets written and then it gets submitted for for review, basically. So really it's just a project. It sounds difficult. I think everybody's gonna need this skill, is why I'm kind of, you know, stopping on it. Like I and when I say everybody, I mean like finance, marketing. At some point, I feel like Unless they just the technology just changes and we call it something like, Hey, you wanna include this in your code now?

16But right now it's called a pull request and that's really just something that would go to somebody like you, the CTO, right? And then what would you do with it? Yeah, it it actually would go to other developers. So it depends on everybody's workflow, but you you kind of submit this code out there, you're saying, Hey, pull this code in to to that main that main branch of code. and what that does is it's kind of a stopping point, right? It's not it's all source control related. It's a stopping point that gives you the opportunity to say, hey, maybe another developer should look like look at this. you know, once another developer it's proper project management so you don't blow everything up. Yep, and and source control, right? So you don't you don't change your code until you know that everything's tested and ready to go. So it kind of can sit at that stage for a long time. in in some cases. But yeah, it's it's it's very much a gate. It's saying, I've done my part. I'm the developer. I've done my part. You guys take it from here kind of thing.

17And so you were talking you were talking about the you've set up kind of these processes, the things that that you're kind of the the lead engineer was able to to put together and y you were saying, Hey, all these things got done, right? And I think the the business kind of maybe question on top of that is is if we're gonna grade everything that was being done. Was everybody get like was every system, was every AI agent being given an A? Yeah, the the simple question, the answer to that is no, absolutely not. So especially in the early days of those models, right? And so, you know, i I I'm oversimplifying this gentleman's work. It was really impressive what he did. But ultimately what ended up happening was he kind of ended with that multiple agent approach. And I think he he started with single agents doing things and and doing things synchronously, and it slowly kind of blossomed into this individualistic approach for multiple agents on different models. and in the very early days of this, it worked well, but but it required a lot of refinement.

18You know, I'd say we got to a place where the code that it could generate and the product it could generate was maybe 80% correct. Unfortunately we've we've accelerated that. But initially it was it was really rocky. The the documents it would put together would routinely miss scope. the agents originally when it would look at the old code base to determine what what that looked like didn't have enough context to accurately do it because we've got this, you know, our legacy code base is this massive monolith. And so in the early days of those AI agents, they just couldn't get enough context to really answer the question appropriately. And so like that's that's the getting lucky that I was telling you about this gentleman had built this kind of before the AI systems were ready to handle the project that we had. And as they got better, he was just able to update his own processes. And now we're now we're really cruising with it. We can write API endpoints, you know, very, very quickly.

19Just yesterday I was I was talking to a old childhood friend of mine and and he threw a kind of an AI use case out on a on a call. And he kind of landed at a at a pretty common place that a lot of like, you know, people I think we all talk to, which is well. AI got it wrong. So it's not good. And you know, so yeah, I w you know, it's almost like and it's I'm never gonna use it again. Right. So I mean like so so so you know, here you talked about, right, here's somebody who was you know, Jim and I use the term often, right, intellectually curious, right? Like kind of, you know, wanted to see it work, was trying hard. H how did you get people to stick with it? Well, no, I mean everybody just adopted it. instantly, right? They wanted to use it. I that's always been my experience. Yeah. Maybe sarcasm for those that can't pick that up. We did meet some res resistance. And that there there are some individuals that are still resistant to it to some degree, right? Like there is a balance between the speed and the quality that we're trying to match and obviously cost as well.

20And so we're not willing to sacrifice on quality and so we dedicate experts to those projects downstream. So the developer is never out of the mix. We don't we don't have product folks writing code. you know, the developers are very much in the mix. And so we've we've addressed inequality issues by just making sure that those folks are still the experts in what they're doing. The the adoption thing is a really good question. and the way I'd answer that is we gave people the flexibility to decide how they wanted to use it is is really the answer. And We're still kind of in that phase of AI adoption. We we are letting folks use it the way they deem appropriate. And I think we're drilling down into to something that really works that we could kind of call the iClass Pro way of doing it. But we're not we're not quite there yet.

21We're it's just something that's that we're letting folks do and and and the flexibility I'm talking about is like some developers, honestly myself included, would would want to write the code maybe in an AI assisted way rather than have the AI produce all the code and then I go read it line by line. That's really difficult. You you you really need to understand the problem set. It it's tough to read somebody else's code, at least for me personally. especially if you don't agree with the syntax and the way they write it and all that stuff. So it can get kind of nit nitpicky. And so some of our developers would rather use Claude to supplement the things, you know, to to Defer tasks like they would to a junior developer, basically, and say, I wrote this chunk because it makes sense to me, but I, you know, this mundane code I don't need to write. And so they give it to Claude. And so that flexibility, I think, has earned us some credibility to say, you know, you guys are the experts, you find out a way to use it.

22but very transparently, nobody wanted to do the front end of that process, the tech scoping part and the product scoping part and the understanding the legacy code-based part. That that is tedious and and AI did that for them and generally does a pretty good job now. And so they that took a huge weight off their shoulders. Yeah. I feel like that's the part early in the AI story that we heard a lot of last year, or kind of the kind of general population pushback on AI was this music models and other things were coming up and people are like, but that's the stuff I love, right? Like I don't want to give up the piece like I I want I want AI to do my laundry, right? I don't want it to write the music, right? I want to do the art and I want it to do the the the the the this this like, you know, kind of mundane work, right? And and so I f it feels like your team was able maybe in s in some way to to to actually like make that happen, right? I mean, like not take away the the fun parts. Yeah, and and it's up to them. Right.

23Like they get to decide what the what the fun parts are or what the what the parts are that they want to do and I I'm curious, like, think about like the most difficult customer you had on the team. So the one that seemed the most resistant, that you won over. So not somebody who's still resisting maybe n you won over is not the right way of saying it, or that the technology you won over. Just if you could just kinda kinda think through that and Walk through the process. Like what what did that evolution for that person look like? I w I wish I could take credit for it, but it was really the progression of the models that that ultimately did it. And it's very similar to what Veer had just proposed, which is there are parts of folks' jobs that that are not fun, right? Like some people really write like writing the nitty-gritty code, but they do not like tech scoping it or looking at what the code does currently. and so th I think the way we approached that problem, because there's one holdout that I have in my in my mind. the progression of the models helped us, right?

24As they got better, they the this individual became more and more convinced that, okay, this this can actually write code the way I need it to be written. and the acceleration of workflow for doing things that that people don't like to do. I think like for example, unit testing. that's that's the notion that maybe the developer is testing their work before they ever create that PR or that pull request, like I said. And and so a lot of times in the development workflow that process gets skipped or it gets glossed over or we don't we don't unit test to the degree that we need to and we kind of defer it a QA to test everything. and and now we don't need to do that. Once we write really good code, we can have Claude look at the scoping documents and the code we've written and say, okay, I need unit tests that are just bulletproof. And it's really quite good at that. and so But it's that it's that alleviation of the mundane, I think, that has won those developers over. we still have some holdouts for the quality of the work, right?

25I I mean, some of our most senior developers can write better code than Claude can, right? But but for the average developer, Claude is Claude's right up there. And it it doesn't you know, it's very consistent with what it out outputs now. Yeah. So I I mean, I guess Are are junior developers going away? Yeah, I get asked that question a lot actually. And I'm I'm trying very hard to not have that happen, you know, because I think you have to start as junior to get to a staff engineer. And our staff engineers catch things that Claude does wrong all the time. I don't want to be naive and say that Claude's never gonna get to the point where they they've solved programming. I think one of the AI companies made that that claim and they're they I think there's a lot of truth to that. But from what we see in that conservative approach that you were alluding to at the top, Jim, I'm I'm really worried about just letting it run run wild. We're not one of those companies that is letting AI write the code and it gets pushed straight to production without a developer involved.

26Our developers are absolutely critical and we've we've hired more junior developers since since the inception of this really quality AI because we want to train them up on how we do it and And it's a really cool learning tool too, right? Like I I think that the path from junior developer to mid level developer is probably gonna shrink because they've got the tools and if they could really understand the code base, then so we talked to somebody last week who is in the recruiting space. And they were talking a little bit, and Veer, help me along here too, if if I need it, but they were talking that, you know, hey, it used to be that the old Deloitte kind of model was a triangle. It's like you had a bunch of people at the bottom, you had middle management and then you had a few people at the top and then now it's like all the expertise is what's needed. So it's more of a diamond. We're str we're shrinking that bottom cohort. And then the middle managers are asked to be a little bit more contributor. So more I see individual contributor than they used to be versus just admin management.

27Is that do you think we're headed there? Is that you aligned with that or yeah I can I could absolutely see that. I think the reason I think the reason I'm afraid to kind of remove the juniors from the equation is that somebody needs to be able to get there. And you can't just jump straight to that stage. And so we're trying to train them into that. So that because eventually everybody will become that. I think you're right. I I was at a a breakfast yesterday with a bunch of CIOs and and like almost the exact same statement by by one of the folks that was on a panel saying how that's the fear, right? The fear is that we won't develop the middle of that diamond because we're gonna age out in 10 years and and and and there's not somebody 10 years behind us or or 20 years behind us who are gonna who are gonna take that fat middle, right? I mean of the of the equation. So so I think it's wise right now to be able to and that you have the latitude to bring in junior developers. you know, I think the question for other folks is going to be, will they be able to justify it? Right.

28Like you know, to to to to build that feeder up up into the middle. Yeah, it's a really good question. I think the nature of our work too kind of lends itself to doing that, right? We're we're specifically a software company. We're our our growth engine, our culture is all built around progressing developers through this cycle and we're we might do it a little differently now. but I could see instances like I was kind of referring back to Streamline Brands where we were writing that software. We're we're not a software company. Then right, we we wrote software, but we weren't a software company. I don't know that we would have the you know, putting myself back in those shoes. I don't know that I would have taken the same approach because we, you know, our core competency was doing swim lessons and being a franchise or and so software development was tertiary to those things. And I think that that does change the script a little bit depending on what you're what you're ultimately trying to accomplish. What I'm seeing is small business is moving really fast.

29So you talked about just letting it run wild. And I think I'll stick with that. So if we let it run wild, we can actually build some pretty cool things. And small businesses are doing that in droves. And then when you get to the middle market, this more conservative approach starts to come in really heavily, especially in private equity, because folks are coming with frameworks from Fortune 500 companies and being hired into these private equity seats. Do you think that's slowing? progress. And do you think that is the pressure that, you know, you're feeling as well? Or I I shouldn't say you, but iClass Pro and other middle market, you know, kind of port co PE companies that are dealing with how we integrate all this AI into our processes? Yeah, I think there's certainly an element of that. And I think it goes back to that ability to execute. I really do think that anybody off the street could pick up AI. And if they got pretty good at it, they could probably write some pretty cool things in code. Right.

30And and if they've got a really cool vision of what they want to build and how they want to build it, even if it's just technology to supplement back office operations, you could you could really do a lot with AI with very little coding knowledge. So speaking of that, I because our and and actually you keep going, but I have another qu I have a follow up to that. I think the reason the reason that scares me is because what the examples I've seen that that Where that goes terribly awry. So our our CEO has somebody's in adjacent industries that have done that, right? And they've used AI entirely to greenfield applications and then they, you and they kinda like, Hey, check out the thing I built. Well, we pen tested one of those and we destroyed it. You know, like so so it had security gaps for days. And we could have, you know, you somebody with just a cursory knowledge of development could could really have done some damage to that application. and it was publicly available, right?

31And in this particular case, there was it was just a fun little application that somebody'd written not dedicated a lot of time to. But I think it's those scenarios where that's why I just think that developers need to be in the loop. We we've caught things that that Claude has has done that's just completely insane. and we're not sure where it got it, right? And and the developers with the expertise are able to catch things like that. So I think I think there's an element of what are you writing and what is your intent with it and and how much do you care whether or not it's could get absolutely breached in one second. So earlier I mentioned exactly. I I so you know, I'm aligned with you on that. I think because we do this and we talk about AI and we're excited about it, we get labeled, Veer and I get labeled sometimes. We know this because of comments we get. We get labeled sometimes as the people that don't care about security and hey, you don't know. You're just building this so almost everything you just said. I am actually working with a few clients where I am developing.

32Now it is those back office applications. I'm in their Atlassian stack and I'm submitting pull requests. Somebody has to review those though. They won't let they just won't let them through. How does that strike you? There's an element of that that freaks me out. And there's an element of that I think that is the is where we're headed. Right. It's I think the fact that they're I think there's a back office application, depending on the risk and of something going wrong, maybe there's there's some comfort that I could get with that process. The fact that somebody's reviewing the code is also very helpful. So I'm not I'm you know, I'm not a hundred percent opposed to it. In our particular use case, I I would cry tears if that was happening in our environment, you know. Are you talking about the customer facing though or Yeah, I think just in general for us, because well, all of our technology is customer facing in some cases, right? It's either it's either B2B customer, which is our business customers, or their ultimate customer, B2B to C.

33And so at some point somebody touches an application that we've written that's not a back office application. But what we have things going behind the scenes to help our operations. I could see a use case where that would be very helpful. But well what about so what about so F P and A systems. Have you ever had a CFO approach you about an FPNA system? Like Data Rails or, you know, Venna or Annaplan Adaptive, which, you know, work it it which is a work day now. I should I'm dating myself a little bit. Yeah. Yeah. I I yes is the answer. Yeah. So they they I would say that most of those implementations fail or those FPNA systems because they aren't nearly flexible. That's the type of back office stuff I'm talking about is Well, how do I create a a cash flow forecast linked directly to my ERP and then start building my financial model that I have an Excel in more of a web based view over time? Does that still give you the heebie jeebies? I I don't want to be naive and say that that's not where we're headed, right?

34And I think Claude does such a good job now at writing code that with the right controls in place, I could become comfortable with that. in in iClass Pro scenario, the controls, the risk outweighs the reward in that particular scenario. Right. I think that's I mean, like if there's something for folks to land on, it's what you just said, right? Like like you, your job, my job, Jim's job in in a in our respective roles is risk management. There's no morality to the choices we make, right? Like the these are not like evil and good like choices, right? These are these are these are risk based choices. I've worked a lot of my career in in healthcare, right? highly regulated, right? Like n very little room for and and and and an appetite for risk like what you said, right? I mean, before you said the like the the risk piece was if controls and you know things were, you know, set up, right? That you could get behind it. I think I could get behind it too, if there was, you know, maybe a little bit of like forward facing training for the gym possibles of the world, right?

35at a at a company or that the system would would enforce some level of of standards, right? Like they'd have to suck in a skill or something like that into their environment that would, you know, kind of, you know, drive and pre-audit. And then you know, I'm seeing with other folks, j Jim's all in on the on the anthropic claw ecosystem. I'm all in on the the open AI and and codex ecosystem. But they're really fun to to to put them kind of adversarial against each other, right? For for for this, right? Like it's the it's the like, you know, I mean it it's it's almost kind of fun because you feel like you're telling open AI, like, hey, go figure out what Claude did wrong, right? Like go pen test this app, you know, find the security flaws, and and and vice versa. So anyway, but Yeah, I think in in this example too, like I knowing Jim the way I know him. he has enough of that requisite experience to have that be successful. I think it's that that question that you posed, Veer is what what are those controls? Does that person have that requisite experience?

36Is there some is there some testing that's that's occurring downstream? Those those things would would make me ultimately feel somewhat comfortable with it. But I'm absolutely an outlier. I realize that. I'm and I'm not talking about I'm really more talking about the public at large. So I think I mean I I think you you have to answer the question with that lens. Yeah. Assuming assuming it's not the gems of the world, then then yeah, that that freaks me out. Well, okay. So so we we we we we kinda talked about like, you know, kinda how adoption has worked for you. We've talked about how where where the the the bumps were in along the way, I think your adoption story is a neat one. It's interesting because i in a way what you've to me what I hear here i is that you've created some space. Some you know, you've created you know, space for people to kind of like it's a tent and it's a big tent, right? And there are corners in the tent that you can go hang out in. And if you want to be in the, you know, in the wait and see side, like you've you've said it's okay, right?

37Rather than being very, very prescriptive for your team and and and it feels like that is driving adoption and And so, and you're creating capacity and you're and you're and you're and you're gaining momentum, you're going faster. So here's the the kind of the kind of a you know maybe something for us to kind of dig into here together is like so when the work gets faster, two things I think happen. These like they're not only two, but here's two. Two things happen. Either we figure out where there are new constraints, right? Like the right, the like like capacity. Development capacity grew, but QA capacity did can't keep up with development capacity. That could happen. But the other thing, and and so and and and w whether this is applicable at iClass Pro, but so there's this capacity and and where is the the new blockers, but also does it fundamentally change the sequence and the priority of what you're working on? That would be my qu as a CFO, that would be my question.

38Like You know, where where do we now allocate like what what's the most important and how do we get even more dialed in on that? So the the first thought of your the first part of your thought experiment there absolutely happened for us. So we drilled right in on the development operations. this engineering question accelerated our development two X at least. and we we got so excited and we're applying that to a bunch of different areas of our organization. So we're accelerating development throughout the entire organization. And you could call it for foresight on my part, but we immediately shifted a bottleneck to QA. We started throwing a ton of code at QA who just was not ready for that. Right. And and our processes at QA had not yet evolved. but you know, no fault of their own, but just we we significantly increased the output and our QA backlog just went through the roof.

39so fortunately, you know, given the benefit of hindsight, I would have probably tried to solve the application of AI to all of those areas of our organization before we went down this road, but that's that's kind of where that's kind of where we sat. we're applying to it to it now and those QA backlogs have come way down and there's still some question about how effective it is. That that QA area is absolutely critical to the quality of our output. And so having a human in the loop there is Is pretty important to make sure that we're not throwing out a bunch of code really quickly that that could cause a negative impact to our customer community. and so that that that aspect, it's it's funny that you bring that up because that absolutely happened to us. and so now we're trying to work through that. and then the second part of it is, you know, because of how we approach that problem, we we can do a lot of things in asynchronously or at the same time, right? We can We can build this greenfield front end application.

40We're we're actually building several green fielded front end applications to use these APIs behind the scene. And it changed the way our our workflow worked because we, you know, in some cases we don't have the benefit of work waiting for QA. So we've had to kind of make those APIs available just internally so that we can build those applications before QA has reviewed the APIs, for example. And so we we've revised our workflow a little bit. Before anything goes to production, it will be fully QA'd, but it's allowed us to work on things, Jim, that that we otherwise would have had to do in sequence. Yeah. Yeah. And you you can kind of do things on top of one another, but at some point there's still gonna be there's still gonna be a prioritization that needs to be done because people are gonna come up with ideas and features that customers would the customers would like and will drive revenue, improve profitability, the P&L, the whole thing, right? And you gotta try to find a way in my mind to prioritize those.

41I think that's it's probably a convers a larger conversation and probably for another day. But I think it's a I think what you said is really useful for a lot of people to hear who because I am positive based on the conversation I have with everybody, not everybody's doing what you're doing. I'm a hundred percent sure of it because I have these conversations all the time. Yeah. And some industries are much better than others. Yeah. It's it's let us keep our existing roadmap and add ambitious tech debt things to it. Right. Like we would have tackled this tech debt and done these things at the same time, fortunately, because we would have needed to bite off like I said, other we don't want to get behind. Fortunately, we're we're ahead in our space and I think we've we've got some leeway, but we don't want to lose it, right? And so our roadmap our roadmap for the next eighteen months was pretty well baked and we were going to execute against that. This just gives us the ability to to add things to it and and really hammer how we how we approach those problems.

42we've had a really aggressive year, what I would have called an aggressive year. We've had a we've had a year where we've released a lot of new features. and I can say confidently that without AI in in the mix the way we've applied it, we just we could not have produced what we produced and not had just a massive number of issues, right? We just can't we couldn't have moved that fast and produced the quality output that we could have without without AI's assistance. Yeah. I guess my point to both of you is you are finding a new level of capacity at some point. I mean, unless you go all in and take the wild, wild west approach, you will hit that ceiling again where the ideas outpace the speed at which you can produce. So I think you're just finding a new waterline. And then when you find it, the prioritization is still going to be the work. How do we allocate finite resource? I I just I think that's a tail as old as time. I think there's a cost element to that too, right now. Like right now we're spending I think the equivalent of two mid level developers salaries on AI.

43and for us that's that's pretty inexpensive. Right. Like a For the benefits we've gotten that that that financial calculation works for us. if AI becomes exponentially more expensive, that's going to change that ceiling again, right? Because we we might not invest to the degree that we're investing now, or we might we might not want to, or we might find other ways to be creative. But right now we've we've kind of reached the balance between what we're willing to spend and the output that we're getting, quality and and speed wise. I wasn't gonna ask this, but let let's say we triple the cost of tokens. Or, you know, the the the way that we price AI. Let's say that the units in which we pay for AI, these tokens, let's say those triple next year. Have you guys experimented with local models? They're getting a lot better. We we have, and that's that's largely how we've solved that that QA side of the equation 'cause there's aspects of that that we don't want to ship outside. and so I can't get too much into the details of what we're doing. That's a that's exactly what we've tried to do.

44We've tried to bring a lot of that in-house and we've we've found some success there without question. I was gonna say I just to to like like I think for for the technical folks who listen with us, I mean, I I think that is right where we're seeing the best opportunity for kind of local workflows because often the QA work is is not synchronous work, right? It's not a developer sitting and doing the work. This is things that you can queue up, give instructions, and run all night long. And and and so the the the speed of the chat kind of turns is is less of an issue when you've got something that's like that type of workload and therefore it it fits the model, like that fits the the it's the right tool for the right job. Exactly. And and with all the right work done up to that point, it it it meets those criteria really well. Right.

45If if testing instructions are there in the ticket and we've got the the code's really well executed and it's unit tested, so the developer has some confidence it's working the way it needs to work as it is, then you absolutely can hand off to one of those devices. And we're in a unique situation where those devices actually are pretty inexpensive still. Yeah. So So I I think if tokens exponentially increase in cost, I'm gonna guess those devices are gonna increase in cost too. So maybe there's some some question, but they're they're that pricing is materially less than than spending tokens on Fable, for example. We we haven't delved too heavily into Fable because you could really spend some money there. but it's quite capable. Yeah, I I think Salesforce was it la yeah, just last quarter they said they were gonna spend they were on pace to spend three hundred million this year with in anthropic tokens. And that's anthropic. They also spend on open they're testing all of the other flavors. Hey, so Evan, this has been an I mean, this has been an awesome conversation.

46If people don't listen to this and get value, I mean, especially people in your world, like IT professionals, if they're not listening to this and getting some value out of it, I I don't I don't know what they'll get value out of because this is a this is real life. I mean, like this is like exactly this is exactly what people need to hear. It's a balance I think it was a really balanced view of AI, which is probably pretty commensurate with the rest of the middle market. And I do know that there's a lot of people that aren't doing it. So truly thank you for your time and, you know, your thoughtfulness. to this whole to this whole process. And so I guess for a one of those IT leaders who listened, got value, and they're like, I'm not doing this. I want to try. What like immediately next week, because it's we're filming this on a Friday. So on Monday, what do they do? I think it depends on on how familiar they are with AI initially, but you can start a lot of the work that we started. just in asking Claude Code what to do next, right?

47If they've got some experience with Claude and some experience with Claude Code, then then you can dive right into this and Claude will help you solve this problem now. I think we were on the early side of it, but this is also things that folks are doing across the internet. Like folks have built pipelines like this. And it's it's pretty well discussed in a lot of blogs. and so we haven't published anything. we we can't decide what what side of the coin we want to be on, whether or not we're we're publishing out kind of how we how we do these things with some specificity. But I think step one would be getting familiar with Claude and Claude code if you're not already. I I'm I'm an anthropic guy because I really think that the output is is pretty great. I know Veer you said open AI and and Codex. I haven't played with those tools to the degree I've played with Claude, but getting familiarity there and then and then frankly, y you could ask Claude how to how to kind of what the next step is and how you'd host a pipeline like this, those kinds of things.

48And and anybody's obviously welcome to reach out to me on LinkedIn too. And I'd be happy to chat with folks one on one if they're interested. yeah. That's awesome. Thanks for offering that. Yeah. Well, I mean I think, you know, I think that's that's a good place for us to kind of like say, you know, thank you for for, you know, kind of the sharing, for making yourself available to us and to others. and having like a real world example. And I think you know, as we, you know, kind of reach out to our listeners, you know, if if you have used AI already, maybe in similar ways, that Evan and his team have, I mean, we'd love to hear that story. I think others, we're still early in the process where I think there is opportunity to help each other out and and and learn from each other without necessarily, you know, being in a position where we're we're giving away corporate secrets. I think that there's a little bit of just you know a rising tide, right? Raises lifts all boats.

49so I think there's opportunity out there for us to be a part of a community that shares our our processes and our wins and our and our and our losses and and where we've struggled and make it better for everybody. so I I guess you know as we as we kind of finish up today, you know, we'd love to hear from people. online, on LinkedIn, on YouTube, on Insta. and so this week do I get to do the closing Jim? Is that is that so so this week thank you again Evan and I am Jim. And I'm Veer and Jim and I are your functional executives.