Using AI to Speed Up Development Without Losing Control
Featuring special guest Evan Moser, CTO of iClassPro, Jim and Veer discuss how his team is using AI to update a mature software platform. Evan explains…
Listen & readFeaturing special guest Justin Rae, Jim and Veer continue a two-part conversation about the systems behind Cinch's marketing software. Justin describes his vision for more autonomous marketing and the work already underway to monitor AI conversations, watch incoming data and identify problems in the code.
He explains why Cinch uses local hardware for some continuous monitoring while turning to cloud models for other jobs. In one workflow, feedback from an AI interaction can lead to a proposed fix. They discuss which changes can proceed automatically, which need more planning and where a person should review the result.
That brings the discussion back to cost and capacity. Hardware decisions depend on the frequency and type of work a model will do, as well as the team's ability to operate it. Justin shares the choices his team is making; Jim and Veer consider what a smaller company needs to understand before making a similar investment.
Transcripts may contain errors. Check the audio for exact wording. Use the paragraph links to share a passage; timestamps are not available.
01The biggest thing that's gonna play out for us is the fact that we will have the ability to have full agent orchestration. One of the things that I've been preaching internally and we we gently do out ⁓ externally is self driving marketing. The idea that how can I make a marketer ten times more effective at one tenth the cost? If you used to have ten people that had to be able to do this, and now you have one who knows how to manage these agents and the agents are experts on what they do, ⁓ that's that's the future. Feel like th that opening line for every one of them is Yeah. Well, okay. So we've talked a little bit about how we're ⁓ you know, like how it's affecting the work and affecting the people and how to you can I you know and and and you know, J Jim continues to insult me with consultant speak about, you know, my past. And it's just I mean I I don't I don't have the same visceral reaction to change management. I just don't think people love to hear that word. It's scary. I just hurry.
02Well, and you know, let's let's just be honest, it's oversold. So yeah, yeah, you know, it's fair. But ⁓ like I I peace ⁓ one of the things that people, you know, hopefully are listening to us and and ⁓ wanting us to talk about ⁓ together is ⁓ you know, kind of the experimental kind of things. And and one of the conversations ⁓ Jim and I have joked about, like, you know, like when is the next deal gonna close where we're both gonna put you know, a spark on our desks and start playing with them. And I was talking to a client ⁓ a few weeks ago and and kind of coaching them as they start their AI journey to ⁓ because they kind of asked about like do we need to run local models and I'm like, no, because you don't know why you need to run a local model yet. You need to build fluency and ⁓ and you know and I talked to them about yeah, getting at bats and and and and getting in the gym. term a lot. It was that's why we both smiled to talk about at bats that that's what people need. Yeah.
03And so here, you know, like you know, and and one of the things I've told folks is one, like until you're ready, you shouldn't. And and also, you know, until you have a certain level of fluency, it it it's not a conversation you need to have today, where I think we we would say that we're all at the benefit where ⁓ the ⁓ anthropic and open AI and Google, right, they're still in the drug dealer model where like the first one's free. Right. Like like we're we're getting a lot of stuff for you know, for for low, low prices that is not sustainable necessarily. And but but that's fantastic. Take that and and run with it. ⁓ but then when you learn, it sounds like y'all have gotten to a place where, you know, you're making some choices of local inference and and putting some tools in place, like yeah, like how how's that like ⁓ you know, kind of what pushed you in that direction and kind of what kind of workloads, you know, are are you know, our local workloads. Yeah, no, that's a great question. ⁓ what pushed us that direction? I don't exactly recall. ⁓ what did?
04I just yeah, one of ⁓ one of our coworkers was just like, Hey, have you heard about the spark box? And I like, Yeah, I've seen a lot of these being advertised, like this AI super box and like there's there's a ton of them last fall. Right. And ⁓ And so at one point I decided, all right, I'm gonna I'm gonna give it a shot. I'll you know, NVIDIA's doing this one, like let's roll with NVIDIA's. We got the NVIDIA spark. And ⁓ just again, it's at bats of like trying to like have that experience. And honestly, NVIDIA did a fabulous job at putting that together with like, hey, here are all the things you can do. Here's how much time it's gonna take you to set it up. Here's all the instructions on how you get this set up and going. Right. And honestly, it made it a lot easier. The fact that Cloud Code. knew exactly how to execute that faster while I wasn't sitting at my desk. And so I could have it setting things up and getting getting ⁓ things rolling for me. And ⁓ and so I think that that that really again created a good experience from that. I actually now have ⁓ three of them.
05⁓ so I am ⁓ three that are actively being used and then we have another one sitting it well I guess it's actively used as well, but I'm not actively using it. So ⁓ we have ⁓ A lot of the local inferences is really going into the things that you're constantly monitoring, right? So if you need like ⁓ a momentary high power thing, we go for you know using something on Cloud or we'll use ⁓ open AI or Codex, right? But if I need something, ⁓ case in point, in all of the AI interactions that we have on with Cinch, we have a Spark box that is running entirely to just monitor all of those conversations. Look at those conversations, see how they're going. Are people thumbs uping, thumbs downing? And where we get a thumbs down, what do we need to do to fix that? Taking that information and then passing that off to Claude to go try to write a fix for it and see if we can make a PR that needs to go out, right? And so those typically tend to be things that just like, Hey, we need to tell the agent you can do this or you can't do this, right?
06Or you function like this, or you know, they're they're typically not very big. If they are bigger, then it needs to go through a little bit more planning process. But ⁓ but we have it running for that. We also have it ⁓ running just against our code base to constantly be looking to see, hey, are there places where we have security vulnerabilities? Are there places that we have ⁓ that we have just potential bugs and what type of bugs there are, right? And how those could be. ⁓ what else do we have monitoring? We have ⁓ integrations. Just anytime when you're from our side, data coming in is really key. And so being able to monitor, are we getting the data that we used to be getting? Is there ⁓ any inflection inflections there either up or down that we need to be aware of that maybe tell us an integration's broken? ⁓ and so those are the types of things they run 24-7, right? They're always trying to go deeper and deeper. And it would just cost an arm and the leg to try to pay Claude to be able to do that. Right.
07And so if you think about like I'll talk to the ⁓ CFO and Jim here a little bit, but if you think about Hey, how much would it cost to run tokens full time and to get that out of tokens? That would just be you know insane. Even if I was using haiku or something on the lower end of things, it would still be pretty expensive. And so that's when it makes sense to go ahead and say, hey, let's get a box, it's five grand, and let's go ahead and plug that in, right? I I'm pretty sure you'd burn through five grand in ⁓ easily a month, maybe even less, with some of the processes that we have going through. But then it allows you to run like, I mean, you can run some pretty powerful models, right? You can run GLM four point seven with you know, typically most people I would say you start with like the anywhere from 30 to maybe 70 billion. ⁓ but if you really want to quantize that and you want to make a few different ⁓ aspects to it, you can you can do some pretty cool things to move that up. ⁓ I've seen two billion work on a single spark. Not very great, but it can work a little bit.
08But then you can also, to your point, Veer, you can ⁓ cluster ⁓ You can put ⁓ together. So I've got sitting right here, the ⁓ I've got two of ⁓ connected together and it gives you two hundred and fifty six gigs of of and you can at that point run all kinds of things. Super cool. But you know, the the thing that I mean you went past it really quick, but I I want to make sure that folks catch this. This is what I heard, right? Is is one it's one thing that the the system is listening and you're being able to use you know, you're just paying for, you know, power, right? You're not paying for tokens and it's and it's queuing up issues or concerns. But but but you know, the the part that I feel like you kind of went past, which like I think is the more Like, like, ⁓ you guys are doing something. The fix is also being done and getting queued up. And really, Jim had talked to earlier about anthropics four Ds and you know, like you're still leaving it in queue for a human to do discernment. But but the fact that you're not putting people in a place to discern, is this important?
09Okay, now we should go do something about it. You're making them as a gate to say, yes, that's important. I like what got fixed, go and you know, like you've pulled you you you cut a whole cycle out. Yeah. Well think about how many times do you say like I I mean as a business owner, this has been my entire career is like trying to decide and prioritize, like, okay, yeah, that's important. And somebody else might ask me, Why didn't you do that? And it's like, Well, because I had this. And as soon as you tell ⁓ about this, they're like, ⁓ yeah, I get it. Makes sense, right? Or some people even are like, Well, you still should have done that. I'm like, Well, that was that unfortunate problem I had people. And and maybe I'd agree with you today, but I didn't then, right? I didn't know. And so the nice thing is with this, is you can start to like you don't have to worry about human trying to prioritize all of these things as long as you know you can handle the output that they do at the end, right? And and that's the key part, right?
10Is you can you you can trust it to go go fix a, you know, in in case like it's cinch, we have what we call cinch ops and cinch ops is able to I like I let it go fix anything that it finds in there and just tell me what it fixed. I let it go through the PR and even deploy those processes, right? ⁓ and in fact this morning ⁓ Jim and I were talking about a couple different things and I just asked it, ⁓ I gave it ⁓ jumped on, use good old whisper flow. ⁓ I've actually transitioned over to a tool called Peanut, which I actually like a little bit better because it'll read back to you as well. Just put that plug out there for Peanut. I'm a heavy whisper flow user, so I I th I and and I'm like, ⁓ that would be interesting to see. All right. No, it's great. Cause then you can like you can just be like, read this back to me and let it read. So I can still be pacing in my office, you know, getting some steps in or something while I'm thinking about whatever it's talking about.
11But ⁓ but I can just let it ⁓ in this case this morning, I just ⁓ pulled up Peanut, you know, hit my control start button and ⁓ told it what I wanted and gave it ⁓ gave it some context of some extra files and ran off with it, right? So it built it, it deployed it, put it all together and ran with it. And so those are the types of things where in certain realms, right, you gotta have some places where you can toy around with it, where you can be like, okay, you know what? If that breaks, I know I can just roll back to the lot like go back to my Git logs, roll back to it. It's not gonna ruin anything at the end of the day. Now SinCOps is actually becoming more valuable to where it could. ⁓ but ⁓ but it's those types of things, like having just that little project that you can work on that you can gain the trust and when you know how that trust works, right? I think one of the things I've expressed to our team is that ⁓ in fact yesterday I was on with our marketing team and I told them it's like guys look the future is you managing your own agents.
12You know, you I want you guys to leave this company one day, not next week, not next year, you know, maybe not in the next few years. I I like like our team, right? But I want you when you leave here to be the most versed in your trade with AI. than anybody else that's out there. So you can nail a great job. So Really glad you said that. I am not trying to interrupt your flow here. Intentionality and letting people know what you're doing, that's another key point. That's something that I've done when I've implemented AI at scale. certainly not to the level you have. Mine has been more of a how do I get it started as a process or as a thought in a company, but being intentional about telling the employees that you are actually building a skill set. for their career, for the long term that will set them apart. I think that's really important. Yeah. Yeah, it was actually really it yesterday, it was actually really nice because our copywriter, ⁓ her name's Erica, she had some really good questions.
13She's like, Okay, so help me understand, like, help me see how this works, right? Like, how would this function for me and how would that work with other other people on my team, right? And so as a copywriter, I told her it's like, Okay, what are the things in your job? That become a little bit more mundane, right? That you know they just take a lot of time. And if you could have them take less time, those are the types of things we need to do. So it could be like research. I need you to go do competitive research against, you know, whoever else has this product. I need you to also just go research internally. Like, what are the case studies we have? What are these things that we have that we could use? Let's say you have a researching agent, right? You maybe then have an agent that goes and looks at The tone that is used by a bigger company like Salesforce that maybe we're not fully competing with, but we want to emulate. And so we pull some of those in and explaining to her that your job is to really figure out how you can become the trainer of these agents. You need to train them.
14You need to then figure out how they work together. Maybe you then write as as a human, you continue to take the long form. You are going to be the one that writes out the actual case study. But You can have an agent that goes goes and does the analyst ⁓ aspect for you, brings back that data and what the case study was and how it were how it turned out. ⁓ you can have the communication going out to the customer to try to follow up with them about the case study. That can all be automated. But then you write the actual case study. You take the like core of how am I gonna bring the value? Maybe you do it by having a conversation with an agent, but you put that value together and then you say, now I'm gonna have another agent. Who's gonna turn this into short form? And it's gonna turn it into ⁓ a LinkedIn post, it's gonna turn it into an ad on Facebook, it's gonna turn it in, whatever it might be, right? And so she can take and manage this, she can be the manager of a series of agents that are orchestrated together, but she has to be the one that teaches those agents.
15Going back to the whole leveling up, right? It's like over time, she's gonna get better and better. She's gonna know how those agents need to work, she's gonna know the processes they're gonna go. She's gonna know how sonnet wor works versus Opus when she's using those. ⁓ I'm being able to dive into that. You can't tell we're really a clawed shop over here, but ⁓ but that's yeah, those are the types of things that I think are gonna really help people to understand like, hey, you can you can be a whole lot more effective. And I think for everybody this can start to 10x what they can actually do, right? At the least you should be able to to two X your outputs. Agree. I mean, like like so, okay, if we if we if we work towards kind of like a like land in the plane kind of stuff here, right? Two questions, right? First one is twelve months ago, right? What did what did you believe about AI a year ago that is, you know. It's exceeded your, you know, like you like I I didn't even think it could do this.
16And and but, you know, like ⁓ 'cause you've been using it for more than an you know, if you've been using it for several years, ⁓ in in in earnest, right? Like and the reason for asking that question, kind of the frame of of it is is how much has it changed in just a year? I wouldn't have done this a year ago. I can't believe it's here this year. Yeah, that's a solid question. ⁓ a year ago, I would say I I probably actually a solid one is Excel, right? I was still totally using XL for all kinds of things, right? ⁓ Claude, Skill, and XL made huge leaps and bounds. Maybe it was late Q4 last year, maybe the beginning of this year. But those that changed the game in a lot of ways for me. Like I I never I mean I I should have thought about it. Like all we got with a spreadsheet is a bunch of code behind it, right? But ⁓ the ability to be able to drop that in. I remember I dropped in a spreadsheet into Claw and I said, hey I need you to recreate something like this with our numbers. And here's access to our Snowflake database. Go recreate it. And it did. And I was impressed.
17Especially if somebody who I I I consider myself a very high power user of Excel, being able to I mean, I was doing machine learning regression analysis in Excel in 2010, right? Like, but I was just so thoroughly impressed. I was like, okay, this is this is getting to be game changing. And then I think the next one I I probably think was I didn't expect it to turn into being able to actually develop Code as fast as it really has. And so if I take an example, right? 2022 co-pilots out, and on a copilot in ⁓ Git GitHub, right? ⁓ it's not very great. It can help answer a few things, but not fabulous. And then ⁓ And then you had Chat GPT that came around. I got on Chat GPT way early. Like I had some fun with it, but there wasn't a lot of like business. Like it wouldn't solve the business things that I could do faster myself, right? ⁓ and so those kind of changed and came along, right? But I think it was really in when Claude launched their desktop app. And I gotta go back and remember when that was.
18I think it was February or February or March of this year, but That was like I think the biggest game changer that made things move a whole lot faster, right? Is now you had clawed code that didn't have to be accessible through an IDE and using in VS Code. ⁓ I did try anti-gravity for a little bit from from Google, but ⁓ all of a sudden it became so easy to use it in and get value with cowork or with clawed code in this desktop app. And then eventually it got to the point where honestly the biggest game changer for me was the remote access, right? When I could all of a sudden just be out on my phone at a kid's soccer game, and I'm like, Yep, no, he's out sitting on the bench. I'm just, you know, sitting here telling my agents to get back to work, right? And that's those were that that maybe like lines up the story of those changes all happened in the last year, right? Last six months. Yeah. I mean it's it it it it no, there's a little bit of you know, like the story's bananas when you when you start like looking at it.
19And so, you know, I I think you know, it sounds like you're also, you know, you got you got your finger on kind of the pulse of you know, changes that are coming, you know, playing with new toys that are are coming out. Jim and I are doing similar with you know, with different models or local inference or b or building things, but you know, for you You know, as you're doing strategic, you know, as a as an executive, looking at this, you know, as a tool set, but as this very ⁓ I mean, yeah, I we I I couldn't have imagined in July of last year what I would be doing in February of this year, much less what I'm building and putting together now. You know, but I'm more fluent and ⁓ and right, I've gotten a lot of at bats, right? And and and so now. you know, kind of what do you see, you know, kind of like the goal here? ⁓ you know, wha what's what's next for for cinch and, you know, how's this gonna like really play out for you guys? Yeah.
20No, I I mean I think the biggest thing that's gonna play out for us is the fact that we will have the ability to have full agent orchestration, right? ⁓ one of the things that I've been preaching internally and we we ⁓ gently do out ⁓ externally is self-driving marketing, right? The general idea, you know, we've talked about it a little bit here on how we're using it or how we've used it in different ways, but the idea that How can I make a marketer 10 times more effective at one tenth the cost? Right. If you used to have 10 people that had to be able to do this, and now you have one who knows how to manage these agents and the agents are experts on what they do, ⁓ that's that's the future. And that's where, you know, I I definitely am striving to get our team to, I'm striving to get our product to. But you know, you think about you know how much effort and time it takes to just go set up a campaign to run a digital ad campaign or building an email template, right? ⁓ to send that out. And those email templates within Cinch now are being dynamically created.
21They're an AI can go create them, put them together, design them where they want to go. And and that's where we're going, right? The idea is how do we get to ⁓ self driving marketing? ⁓ and I think the it's actually really cool to have you guys driven self driving cars, Tesla? Yep. Well, you know, it's funny, very, very quickly. So I have one. ⁓ I drive a Model Y and Veer visited. He came to town, spent a few days as we worked on podcasts and all other things that we work on. And he drove it home from the airport. So Veer, what did you think? I mean, it really was like right. As a technologist, like I've known about it for a long time. You know, I've been with folks in cars, right? ⁓ you know, right with Superdrive in a Cadillac and you know, you know, in rentals and stuff. and yeah, like like the it is jaw droppingly impressive, right? I mean, just you know, cause because you're keeping in your head the the the math right that's happening around you to make that possible. It is yeah. It it's impressive. It's you've been on a waiver of that.
22Yes. Yeah. Yeah. Yeah, right. The idea of the Waymo, right? No like the legal obligation is now on Waymo, right? They are like to make sure that that drive's safe, that it handles everything. Insane the amount of calculation that is going on, the input that is coming in to make those decisions, the decisions that are being made so quickly, right? ⁓ I mean, actually there's some Chinese ⁓ self driving vehicles. That are even more mind blowing, right? I have not experienced those in person, but I mean I'm all halfway tempted to fly to the other side of the world to get I think it would be amazing, right? Because like it there, you know, like, you know, that Waymo is fantastic, but it's got like all the stuff hanging off the sides of it. And we have them here in Nashville, ⁓ you know, where I live, and and I've watched them get stuck, you know, like like in in the gore in between two roads or you know, or where there's but but it it is, it's impressive. It's impressive how much data is getting crunched, it's impressive.
23The fact that like we're able to do something now, it did not take us this level of s we didn't need this to put men on the moon. If you're as long as one of the guys who doesn't believe that that wasn't a real thing and stuff. Yeah. World is round. Went to the moon. Yes. Yeah. Yeah. But right. I i it it is really impressive. So yeah, I mean, so yeah, what do you think? You know, I mean, so that you know, that you guys will be ⁓ you know, kind of working towards this self driving marketing, which I think is an interesting which It's a cool extension of of, you know, it's using the data and using it in ⁓ y it's an ex it's an extension of what you're already doing with email and text at cinch and then now taking that out to you're really paid media. Yeah. I mean, you think about it, the crazy thing about self driving vehicles, those take your life into your your hands, right? Like they can make a wrong decision and you're dead, right?
24Or, you know, you're gonna be, you know, costly decisions that might happen right there in your personal life, right? The thing that's really crazy is that we don't have self driving marketing yet. Because why? I mean, we have the data. It's all here, like. It's way easier to actually work with, and and all of my competitors that might ever watch this. Just, you know, it's harder for you. It's easy for us. But but the idea that well you know if you don't if you you're not gonna kill somebody if a decision's made wrong, right? In fact, half the time in marketing, you make a wrong decision, it gets more attention, right? And ⁓ and so The the concept here that we're talking about at Cinch is there's those levels of self-driving, right? You got cruise control by itself, level one, right? Okay, that's basically what marketing does today. Well, you know, you got MailChamp constant contact that do things like that. Let's send an email out. We're just gonna automatically handle this, like go at the pace we need to.
25But then you get level two, you start to introduce lane keep assists, you introduce the ⁓ adaptive cruise control. ⁓ automatic braking, some things along those lines, right? That becomes tools that have ⁓ that are automated marketing, right? I consider cinch in the space between level two and two plus right now. And and and so those are making certain decisions, but they're still following a lot of logic that is just defined by the user. You get to level two plus and now you're getting to the FSD from ⁓ Tesla, right? You're getting into some of the the Mercedes class side of things. And These these are places where now automation is making decisions. Liability is still on the marketer in this case. Like they have to ultimately make the decision, but they can let it go, be more automated, make deliveries and decisions for them. Then you get to level four, right? You're talking about the Waymo's, the the Zooks that are out there. And ⁓ and now we're s we're taking more liability away, but we're letting the engine take a lot more decisions.
26To get to all this, you have to take those steps, just like we've talked about the whole time. It's like You gotta have those levels that you're going through. And ⁓ those help make the decisions that help the agents know how do I make the right decisions? Because I've had human in the loop as I go each of these stages within the the bounds that I have. And so that's where we're going. Self-driving marketing. The goal is is to get to how we can make people far more effective, making better content, making better things that engage with customers and ⁓ and actually become valuable. We don't need to send millions of emails out every single week just to try to bring that revenue in, right? We can be more targeted. We know the destination. We know who's going to potentially ⁓ respond. And how do we bring that that business in, right? This is I I gotta tell you, i I mean, Veer, for a first guest on our podcast, I think we Don't flatter me.
27I think well, no, I well, no, I mean I okay, so truthfully, I'm inspired because a lot of the things that Veer and I have talked about, you've brought up. It's nice to find a kindred spirit, but it's nice to find somebody who's working in a real you know, Veer and I can talk about this and we can tinker a little bit with some of the clients that we work with to their tolerance. It is very interesting to talk face to face with somebody who's doing it. practically actually engaging with it and making it happen, you know, at scale. I think that's I think that's it everybody should be inspired by it because a lot of businesses can punch above their weight class right now. Yeah. If they start to work a little bit on this change management and get people get people to to start using this daily. This has been an absolute incredible conversation, Veer. Yeah. Definitely. No, I mean it so we you know we we appreciate you, appreciate the the time.
28I think we've we've ⁓ you know we we've been trying with the whole idea of functional executives to be able to have something that's of value to our peers, right ⁓ right, not just you know, just so we can talk a little bit about the the the the down and dirty functional, you know, nerdy pieces of it. But I think we've talked a lot about the strategy that you've been pursuing and ⁓ You know, I I don't want this to go unsaid. I think one thing that I heard from you consistently and 'cause I think it resonates with me and and Jim is ⁓ because I we're out talking with people out in the marketplace about this, is ⁓ you you consistently brought the ⁓ the person back into the conversation, right? This was not about I'm gonna s build a company that saves money.
29I'm not building a company that is just efficient and cold and and lifeless, but this is about investing in people, making people feel heard, both ⁓ from the people you serve as customers, the people that they serve, their customers, and then your team, you know, being able to say, This is about, you know, we're we're we are really, I mean, you know, trying to lift up everyone you know, in a way. And and the technology is is helping. Yep. Yeah, the other thing I wanted to do, ⁓ Justin, is just give you an opportunity because we'll try to in the show notes also put a link to cinch. ⁓ but why don't you just quickly say how people can find you? Are there any trade shows that you're gonna be at that people might wanna know about here ⁓ in the near future? We will definitely be at Pestworld coming up and then CEMA and Apex in November. So those are some of our bigger shows we will we'll be at. ⁓ through the rest of the summer, we've got a series of other shows that we'll be going to more regionally.
30So ⁓ any really any show we can get to in the home services or automotive services show ⁓ space, we try to get to those. otherwise check us out on on the internet. Cinch.io is our website. ⁓ our name is Cinch, but our brand or our ⁓ website is cinch.io. yeah. That's how you can find us. If you know drop us a a line if you wanna out and find out how we can help you with AI and your marketing and just ⁓ fill out a form to schedule time with our ⁓ sales team or you can even try out our AI voice agent right off of the the website and our we've got a self-serve motion you can sign up. In fact I'd encourage you both of you go check it out, try it out and ⁓ you can talk to ⁓ one of our voice agents.