Sidecar Sync

Snooze to Win: Why Digital Naptime Is AI’s Most Productive Hour | 83

• Amith Nagarajan and Mallory Mejias • Episode 83

Share episode

Send us a text

This week on Sidecar Sync, Amith Nagarajan and Mallory Mejias dive deep into two cutting-edge developments in the AI world. First up is the mind-bending concept of "sleep time compute"—how LLMs might learn and improve during their downtime, transforming into smarter, faster assistants overnight. Then, the duo breaks down OpenAI's $3 billion acquisition of Windsurf, the booming arena of AI coding tools, and what it means for developers and associations alike. From persistent memory to prototype-ready AI partners, this episode is packed with insights for both techies and the tech-curious.

🤖 Join the AI Mastermind:  https://sidecar.ai/association-ai-mastermind

🔎 Check out Sidecar's AI Learning Hub and get your Association AI Professional (AAiP) certification:
https://learn.sidecar.ai

📕 Download ‘Ascend 2nd Edition: Unlocking the Power of AI for Associations’ for FREE
https://sidecar.ai/ai

📅  Find out more digitalNow 2025 and register now:
https://digitalnow.sidecar.ai/ 

🎉 More from Today’s Sponsors:
CDS Global https://www.cds-global.com/
VideoRequest https://videorequest.io/

AI Tools and Resources Mentioned in This Episode:
Claude Code ➡ https://docs.anthropic.com/en/docs/claude-code/overview
Claude Desktop ➡ https://claude.ai/download
Windsurf ➡ https://windsurf.ai
GitHub Copilot ➡ https://github.com/features/copilot
Cursor ➡ https://www.cursor.so
OpenAI Codex ➡ https://openai.com/blog/openai-codex
ChatGPT ➡ https://chat.openai.com
Gemini by Google ➡ https://deepmind.google/technologies/gemini

📅  Find out more digitalNow 2025 and register now:
https://digitalnow.sidecar.ai/

🎉 More from Today’s Sponsors:
CDS Global https://www.cds-global.com/
VideoRequest https://videorequest.io/

🚀 Sidecar on LinkedIn
https://www.linkedin.com/company/sidecar-global/

👍 Like & Subscribe!
https://x.com/sidecarglobal
https://www.youtube.com/@SidecarSync
https://sidecar.ai/

Amith Nagarajan is the Chairman of Blue Cypress https://BlueCypress.io, a family of purpose-driven companies and proud practitioners of Conscious Capitalism. The Blue Cypress companies focus on helping associations, non-profits, and other purpose-driven organizations achieve long-term success. Amith is also an active early-stage investor in B2B SaaS companies. He’s had the good fortune of nearly three decades of success as an entrepreneur and enjoys helping others in their journey.

📣 Follow Amith:
https://linkedin.com/amithnagarajan

Mallory Mejias is the Manager at Sidecar, and she's passionate about creating opportunities for association professionals to learn, grow, and better serve their members using artificial intelligence. She enjoys blending creativity and innovation to produce fresh, meaningful content for the association space.

📣 Follow Mallory:
https://linkedin.com/mallorymejias

...
Speaker 1:

You know, we don't have technical strength, we don't have a thousand developers, we're not Amazon, we're not Netflix, but the field's leveling and you now have the ability to do this if you take the time just to go and experiment with this stuff. Welcome to Sidecar Sync, your weekly dose of innovation. If you're looking for the latest news, insights and developments in the association world, especially those driven by artificial intelligence, you're in the right place. We cut through the noise to bring you the most relevant updates, with a keen focus on how AI and other emerging technologies are shaping the future. No fluff, just facts and informed discussions. I'm Amit Nagarajan, chairman of Blue Cypress, and I'm your host. Greetings everybody, and welcome to the Sidecar Sync, your home for content at the intersection of artificial intelligence and all things associations. My name is Amit Nagarajan.

Speaker 2:

And my name is Mallory Mejiaz.

Speaker 1:

And we're your hosts, and today, hopefully, you will not be put to sleep, but we're going to be talking about some really interesting topics in the world of AI and you'll be hearing about it shortly. So very, very excited about this particular set of topics. It's going to be, I think, quite impactful and quite exciting for everyone. How are you doing, mallory?

Speaker 2:

I'm doing well, Amit. I feel like AI is getting more and more like humans than we know. I mean, even AI needs to sleep. We're going to talk about that in just a bit, but I thought that was interesting to preface the episode with.

Speaker 1:

Yeah, you know, I think AI is learning how to sleep.

Speaker 1:

AI doesn't necessarily need to sleep but when AI sleeps, interesting things start to happen, which is something we're going to be talking about in detail. I think that it's just yet another branch of opportunity and research in the world of AI, so I can't wait to talk about that in more detail. And meanwhile, at Sidecar, I know we've had some very exciting activities in terms of our AI learning content for association folks, evolving, brewing, developing not quite sleeping over the last several months, and we're about to roll out, and by the time all of you are listening to this or watching us on YouTube, by that point in time you will have all new content on the Sidecar AI Learning Hub. It is actually all AI-generated content, and I'll take a second to explain that. It is the content itself. The actual material is not AI generated. It's generated by a little bit of AI assistance, but primarily by us on Team Sidecar.

Speaker 1:

But what we do from there is we utilize an AI driven software system that we've built that essentially generates audio and video for the content, which allows us to much more rapidly change it, so we're super excited about it. The last time Mallory and I, along with our other colleagues on the Learning Hub, made big updates was in the fall and, as you all know, ai is changing so fast that the content from the fall is largely in many ways still good, but in many ways out of date, unfortunately. So that's the reason we're shifting to this model, so that we can make incremental updates constantly and push those updates every couple of weeks to the Sidecar Learning Hub. So we're extremely excited about this.

Speaker 2:

It's an incredible feat, amit, and I know you mentioned when you and I worked on the content in the fall. We used AI pretty heavily to help us generate slide decks and whatnot, but even so, it was quite a tedious process. We had to sit down at our computers and record everything slide by slide, which we were happy to do. But at that time I thought, wow, we are moving as fast as we possibly could. And then I think, amit, you had the idea. When, would you say, was the first time you thought, wait a minute, we could probably AI, or did you always think that? Is that the question?

Speaker 1:

Well, I mean the ideas came to mind at various points in time. To me, as AI video and AI audio generation got better and better that at some point in that curve there would be quality high enough or even better than the average human and it would be an opportunity, and so I started thinking about it, probably a year ago. But I'm a fundamentally very lazy person. I don't like doing things more than once.

Speaker 2:

I don't know, Amit, if I would call you lazy.

Speaker 1:

I'm extremely lazy and I have a very specific type of laziness.

Speaker 1:

I don't like repeating the same thing more than once, unless it's skiing. I do like skiing a lot of the same runs repeatedly, but at work I like doing new things all the time. So you can call me a spoiled brat, but I've had the good fortune, over quite a few years, of primarily been focused on like the what's new and what's interesting kind of work. Not all the time. We all have to do things we don't like, but the point is is that when I run into a task that I do not like, which is any task that's repetitive, I try to find a way to automate it. I've always been like that. I've been doing that since the beginning of my career, and now with AI, it's like being a kid in a candy store, because we can automate things that previously were totally outside of the realm of anything other than science fiction.

Speaker 1:

So we are living in interesting times, and this new content has been reviewed by a bunch of people internally and externally and gotten really positive feedback, so I'm really excited about it. We're going to always have a sliver of human content recording and generation in various aspects of the Sidecar Learning Hub. We think that's an element, that's an important addition. We're going to focus there on things that don't go out of date quite as quickly, to add personality and to add humanity to the Learning hub, and I think that's probably a good blend. We're going to experiment and learn and you know, for our association friends, it's an interesting thing to be talking about because, of course, you're interested in AI learning content, but also because you yourself are probably a prolific learning delivery organization.

Speaker 1:

Most associations have their hands in learning. Some generate the majority or the substantial majority of their revenue from learning, and so when you have the opportunity to consider new ways of accelerating the delivery of learning from idea to reality, it's an interesting thing to be thinking about. So we'll be sharing more and more about this with our mastermind group, which is a small, intimate group of very dedicated practitioners of association management who are on a learning journey together with us and with each other. This group meets once a month. We've talked about it in the past. It's an awesome group that's been together for about 18 months now a little bit longer than that and just the last meeting that we had there was a detailed discussion about how to actually do AI at this scale with your educational content. So we'll be sharing bits and pieces of that with Sidecar's listeners and possibly building a course on the Sidecar AI Learning Hub all about how to build an automated AI education pipeline. So really excited about it.

Speaker 2:

Yep, Quite fun too for us to be the guinea pigs and, exactly like you said, Amit, see what works, see what doesn't, and then share all those insights with all of you. So hopefully you can take like the next best step in that direction, Amit. I also have one more question for you. You said you don't like doing repetitive things. I would argue this podcast is quite repetitive. We're on now episode 83. Are you planning to automate the Sidecar Sync podcast?

Speaker 1:

Not at all. This is super fun to me. So if we recorded the same topic over and over, I'd find that quite boring, as would I believe our listeners and our viewers, but I think that it's super fun.

Speaker 1:

It's actually a great touch point each week for me where I know we're going to be talking about this stuff. It helps me reflect and put together thoughts on how I might want to frame certain topics with the association market to make them most helpful, and there's always new ideas that come out of this too. So it's actually it's a routine, but it's not repetitive.

Speaker 2:

Yes, all right. Well, you heard it here, then Amit and I are here to stay Today. We have two exciting topics lined up for all of you. We're talking about sleep time compute Hopefully that doesn't make you too sleepy, it's actually quite interesting and then we'll be talking about the open AI potential acquisition of Windsurf and some other coding tools that are out there.

Speaker 2:

So first, sleep time compute. Over the last year or so maybe a little more than that we've seen language models push to think harder by giving them extra test time. Compute seconds while users waited. Giving models more time to think allowed them to craft better responses, but every extra second increased latency and inference cost, aka the cost to actually run the model. And the model still forgot things between chats.

Speaker 2:

So a new research paper from Letta, which is a UC Berkeley spinout best known for its MGPT work, tackles that bottleneck with sleep time compute. The idea is pretty simple Keep the agent busy during downtime, so you have a heavyweight sleep agent that runs after hours, reorganizing knowledge and writing distilled insights into a persistent memory. Because that memory survives across sessions, the live primary agent can answer almost instantly the next morning without burning fresh GPU time. This persistent state architecture shrinks real-time compute by about five-fold and still boosts accuracy by up to 18% on tough reasoning tasks, according to Lettuce Benchmarks. The breakthrough matters because it turns an idle chatbot into a night shift analyst that keeps learning instead of starting every conversation from scratch.

Speaker 2:

So I worked with ChatGPT a bit to see how this might apply to associations, and it came up with two interesting examples. One might be a member service agent that digests a year's renewal FAQs overnight and can greet your members with a confident one-shot answer at 9 am the next morning. Or perhaps a regulatory watch agent that scans new rules overnight, stores key points in this memory and then delivers a curated briefing with your morning coffee. Sleep time compute shows that memory plus off-peak reasoning unlocks lower costs, faster replies and continuously improving service Exactly the mix that would benefit associations, of course, or, frankly, any business for that matter. So, amit, can you? There's really a lot to unpack here. Sleep time compute when you sent this to me I thought oh man, that's a great topic for the pod. I want to talk about test time compute a little bit, because I feel like it does relate to sleep time compute. Can you talk about both of those and how they relate to each other, or perhaps one solves something that the other can't.

Speaker 1:

Sure, let's zoom out a little bit and talk about some of the history behind the process of scaling AI. So some of you may have heard these terms scaling laws and a few years ago there was a lot of conversations about how scaling laws seem to continue to hold, meaning that as you increase the amount of computation that you threw at the training process for AI, the models became smarter. So that's basically what the so called AI scaling laws were to show and in fact, they did hold true for quite some time. They started to not hold true to the original benchmarks after a period of time, but there's still truth to the fact that if you throw more compute at training, you typically get a better model. Of course, being smarter about how you train and being more efficient in how you train is definitely an opportunity. Algorithmic improvements are an opportunity, but that was kind of the first dimension of compute scaling was training time, making the model smarter through better and more training. Now, test time compute was this concept. That is actually kind of an awkward term, which is, of course, very much the domain of AI folks and you know we're very good at that generally as an industry of coming up with weird acronyms and strange words that might mean something to the nerds but not a whole lot to everyone else. But test time, compute, test time, essentially, is when you use the model. So training time is when there's these massive computers creating the model from scratch, essentially. And then test time is when you use the model. So, mallory, when you type into cloud or chat GPT and you hit enter. As soon as you hit enter, that message is transmitted across the network and eventually gets to a computer where the model is running. We call that inference. Test time is another term. That basically means the same thing. So models have historically, for the history of neural networks, basically always been trained to respond as quickly as they can, meaning they infer from the input what the output should be. They're probabilistic machines, meaning that they'll say, hey, for this sequence of inputs what should be the outputs. That's basically what they've been doing. Now what's interesting is late last year there was the first release of a so-called reasoning model and we covered it in detail, first when it was called Strawberry from OpenAI, and then later when it became it was called O1. And since then lots of this happened in reasoning models. We've talked about that a lot and, by the way, there's actually a new lesson on reasoning models in the Sidecar Learning Hub with this update that we just were talking about.

Speaker 1:

But reasoning models essentially what they do is they invoke a new modality of thinking when you're querying them, when you're asking the model a question. So previously the models would only react as they just essentially react as quickly as they possibly could. They would not edit their response as they went, even if they potentially found a mistake. They didn't look back at all and they didn't really stop to think and say, hey, what is the nature of this problem? How can I best solve it? Let me break it down into pieces what's called chain of thought a lot of times today. So models were not able to do that. You could do those things in agents that sat on top of models, but models themselves didn't have the ability to do anything other than the instantaneous type of responses responsibly as possible.

Speaker 1:

So with test time compute, what we were saying is hey, if we give the model the opportunity to think longer, then the model might be smarter. It kind of makes sense, right, if the model has certain fundamental capabilities through training. But if we say, hey, model like you get, take 10 seconds to think about this, or take a minute to think about this, or take as long as you want to think about this. Like us, probably if I asked you a question and gave you you know zero time to respond, you'd have a harder time coming up with a great response versus some things you know. You'd probably want to step back and think about them and say, hey, like, what's the best way for me to solve this problem? You'd start working on it. Then you might go edit your response.

Speaker 1:

No-transcript. Extended thinking mode and we've seen dramatic improvements in really complex reasoning across domains like math and physics and biology and a number of other domains as well. So that's what test time compute is about, and you know it is really. Those are the two dimensions of scaling that we've had thus far. And what sleep time compute is about is this new third dimension of potential scaling where we can say, hey, what if we threw compute resources, not during training and not when Mallory asks a question of ChatGPT, but when perhaps Mallory is not asking a question of ChatGPT?

Speaker 2:

And what?

Speaker 1:

can the model learn from that and how can the model improve? That's essentially what this is about.

Speaker 2:

And what does that look like in practice, amit? So would you provide access to some large data source, initially that it can kind of study while it sleeps, quote, unquote, and then, when I ask it, questions?

Speaker 1:

you know the best models are like maybe originally they were high school graduates and then they were university graduates, then they were elite university graduates, now they're PhD graduates of the 80th percentile, right. So these models are really, really smart and really well-versed in a wide array of domains, which is cool. But models today are fixed in that moment in time, meaning when OpenAI releases O3, which they just did, the full O3, that model, that particular version of the model, will never get smarter. So it's like having this amazing, you know, multi-phd individual that knows all this great stuff, but every time you interact with that model, it doesn't remember anything about what it got right and what it got wrong, and so that model will never be better than the day it was born, so to speak, which is fortunately not true for us, because we are continually rewiring our brains based on our continuous experience loop, and so model architecture right now is still that way.

Speaker 1:

Models are essentially fixed in time as of the end of their training processes. Now you can do other things. You can do what's called fine tuning, you can do additional training through something called reinforcement learning. There's a lot of cool stuff you can do at the model architecture level, but they all require significant development processes and they're not things that you do on a continuous basis.

Speaker 1:

So models are kind of frozen in time, like that university grad who's brilliant but is incapable of remembering what you told them the day before when the next day comes around. You'd find that quite frustrating at work if you had a team member like that, right. So then the question is OK, well, how do we deal with that? How do we improve on it? And so a lot of different things have been done. You know people have been doing things like building scratch pads and trying to give models forms of memory. Memgpt which you referred to, which was from the same group of folks, attempted to do that, where it was trying to basically create like a scratch pad for memory to make it possible for models to quote unquote have persistent memory in an earlier version of this concept, and the idea, though, is that the model actually has no memory. It's just essentially like a separate component that the model has access to that has memory. That's what this is about.

Speaker 1:

So now sleep time compute. Coming back to that, so the key here is what are you trying to do when you sleep? Well, first of all, just individually, not thinking about the science behind it. You want to get rest. Why do you want to get rest?

Speaker 2:

So I can be better the next day, you know.

Speaker 1:

Yes, you'll feel better, You'll feel refreshed, you'll get started again anew and you'll perhaps, in that process too, something else is happening. You know, that's kind of quiet and not really something we think about a lot, but our long-term memories are being formed, some things are being pruned. We're kind of having like a cleaning process, both for emotions and for thoughts, and there's this distillation that's occurring where the brain is essentially saying oh this thing really was important. Today Mallory learned this really important thing, or had this experience that was really emotionally positive or negative, and then kind of lodges those into your memory and actually, in some cases, rewires the way your brain is actually working functionally. So it's quite interesting how that works.

Speaker 1:

Now, our AI architecture is ridiculously simplistic compared to the way a biological neural network works. We try to learn from this process, right? So the idea behind sleep-time compute primarily is to emulate what happens in a biological neural network, aka the brain, when we're sleeping. So I'll give you an example. We're actually implementing this concept and have been for about a year in one of our AI tools called Skip. So Skip, if you haven't heard me talk about it before, essentially is a data analyst agent. So what does Skip do? Skip is a conversational AI like ChatGPT or Cloud or Gemini, and you talk to Skip. Skip has private access to your data, so data that you consolidate into an AI data platform from your AMS, lms, whatever your systems are, and then Skip is able to talk to you about your business and also write reports. Basically, that's the primary function is to create analytics and reports, and Skip needs to understand quite a bit about you as a user, your organization overall and, of course, your data in order to be effective. So what have we historically done when implementing Skip for clients? We've tried to learn a lot about the organization and the data and put in a bunch of information into Skip's brain and make it possible for Skip to be quite effective, and that works pretty well. You know that gets us 80, 90 percent of the way there, sometimes 95 plus percent of the way there.

Speaker 1:

But users are constantly coming up with new ideas and having new questions, right, and so Skip may not have seen a particular request, or some users might use slightly different types of terms than others, and so Skip might fail at solving the user's problem. So I might say, hey, I want to run an analysis that shows me member retention, but I want to correlate that member retention with how long the member's been with us and also what their level of education is. So run a report, generate an analytics. You know an analytical kind of view of that. So that might be pretty straightforward sounding to us, but Skip might interpret that in different ways depending on how much prior experience he has had in solving problems like that. For you, right? So what if Skip gets it wrong? Well, skip gets it wrong and I say well, that's not quite right, you pulled the data from the wrong place, or it's really not what I was looking for. So I have this conversation with Skip where I'm giving feedback and Skip's like oh, okay, cool, and then Skip will be able to fix the problem and give you a revision and eventually you get what you want. Right, it might take two, three, four turns.

Speaker 1:

And we asked the question well, how can we make Skip just continually learn and also have transference of knowledge from conversations with one user to another across an organization? And so sleep time compute? We don't call it that. We call it a learning cycle, which is not nearly as cool as sleep time compute. We should have called it that. I was talking to Thomas Altman, who quite a few of our listeners know, and he and I were chatting about that. I was talking to Thomas Altman, who quite a few of our listeners know, and he and I were chatting about that. We're like, yeah, we totally should have called it that.

Speaker 2:

Drop the ball Done.

Speaker 1:

Yeah, we, you know, we tried to call it something a little bit more generic, but learning cycles. Essentially what happens is this it's very much what you described at the beginning of this segment where, essentially, outside of when a user's asking Skip for anything, skip on his own says every so often and this is typically actually done every hour or so, not overnight necessarily and Skip will say, hey, I had this long conversation with Mallory and I also talked to these 20 other users and in these conversations, what did I learn? Well, let's see, mallory really liked it when I did this, she really didn't like it when I did this. And so it's kind of like if you ever have done journaling, where maybe at the end of a day, you are in the practice of saying, hey, I'm going to write down some of the things, like some of my experiences, some of my thoughts, some of my feelings from the day, that can be both therapeutic and it can also be a very helpful way of learning. That's kind of what Skip's doing.

Speaker 1:

Skip has this journaling process where Skip's saying, hmm, that's interesting, what I learned and how does this compare to everything I've ever learned before? Because Skip's quote unquote journal is everything Skip's ever learned in these prior learning cycles. So Skip's saying, well, here's all the notes I've ever taken before. I've learned these things. And then in some cases, like, oh, what Mallory really means when she says ABC is what a Meath set means by something else, because I have a different terminology set. And so then in the future Skip becomes smarter, not only dealing with Mallory, but dealing with everyone. So that's the way these learning cycles work. This happens actually quite slowly, offline. In fact, you can utilize what are called batch APIs through all the major AI providers to get much cheaper rates. You just get much slower response times and in this process you get back your feedback. But if you get it back half an hour later or even a couple hours later, it didn't really matter that much. And so then that feedback essentially gets stored in this quote unquote journal.

Speaker 1:

Right, what I'm calling a journal is basically the scratch pad, but it's a distillation of knowledge using really high, high horsepower or high compute. So we're using like O4 and we're using cloud 3.7 and we're going to keep pushing the boundary of using really the most expensive, slowest models to do the distillation of knowledge, to say, hey, what are the key elements of insight that I need to glean from these 5,000 conversations I've had in the last day, and then how do I consolidate that with everything I've ever learned before? And then what happens is in the future. Every time future users come in and ask questions, that distillation of knowledge, that journal is immediately and instantly available to Skip to learn from, and so Skip will be able to utilize that to improve the quality of his responses With Skip. Specifically, we're very early in testing and rolling this out. We actually have not rolled out this capability to any users yet, but we will soon, like in the next 30 days, but our testing so far shows very positive results. It's a really exciting, you know, additional dimension of scaling.

Speaker 2:

Sleep time compute is a good name, but I will say you're on to something, Amit, with the AI journaling. I think that could. That could definitely get people interested. What I want to ask you is it seems like it's less about giving the model access to some repository of data to study beforehand and more about the ability to learn from previous experience and previous interactions. Is that correct?

Speaker 1:

I think you could use it in both ways. But in our use case for Skip, for Betty, for other products we develop learning from the interactions with users is really, really important, and until this innovation came along, it was really something that required a serious level technical sysadmin or developer even to go in and provide that additional knowledge to these tools. And now we're in this continuous learning loop, essentially, where, as you interact with these systems, they'll just feel smarter. Every day that you use them, they'll feel smarter, they'll be faster and they'll be better. So it's pretty exciting.

Speaker 1:

Now the underlying neural network, the underlying models we use, have not gotten any better in terms of their knowledge, but what we've essentially done is built an engineering solution on top of that basic layer to make the system smarter At some point.

Speaker 1:

If the neural networks become more liquid or more elastic in their nature, that will be great, but ultimately that also has some risk to it, because, you know, models tend to be shared across organizations, so do you really want other organizations' behavioral changes to affect the way your version works? There's a layering concept that people are working on, or what I just described won't be happening, but there's all these different things happening at the same time. This innovation I think my main takeaway that I'd share particularly with the non-technical leaders of associations is models themselves as they get smarter. It's great, it's exciting, but this innovation means things are going to happen faster. The capabilities of the system that you're using, whether it's an AI for you know, conversational intelligence, or if it's, or if it's a coding tool or whatever it is these tools are getting smarter and smarter and smarter, and if you think that we're about to slow down because we've had so much progress, I think it's quite the opposite it's going to continue to compound and drive progress forward at an even crazier pace. So pretty nice.

Speaker 2:

It's like can it get faster? I know it can. I know it can in theory, but it's crazy to me.

Speaker 1:

Yeah, the speed of progress. I mean the numbers are just math, right For our brains. We're already blowing up. But you know we'll, we'll see. But sleep time compute definitely is something I think people should at least be aware of at a minimum, because it's something that, if you're thinking about building one of the systems that you described earlier Mallory, you know, member services, agents or something else the capability to get smarter in this way is a fairly new concept.

Speaker 1:

And so being able to understand that that is possible. And so if you're working with your team whether it's an in-house development team, a third party or a product company many of them might not even know that this is a possibility. So you, as the non-technical individual in the room, can come in and say hey, have you heard of sleep time compute? You can start to weave that in. It might solve some of these problems that you're saying are unsolvable. So that'd be a fun conversation.

Speaker 2:

I think one day this podcast is going to turn me into a technical person. I don't know exactly how yet, but I'll just start saying you know, I don't know exactly how yet, but I'll just start saying you know, I'm pretty technical.

Speaker 1:

Yeah, I mean my last question. You're on your way.

Speaker 2:

My last question on this topic is kind of my gut reaction when you shared this information with me, which this sounds like it could be actually expensive running extra cycles while the model's quote unquote sleeping. Maybe it could have a negative impact on the environment as well. You said that wasn't exactly the case, so can you address that?

Speaker 1:

Yeah, I actually think this is going to help in all of those areas. So, first of all, most of these sleep cycles like you and I, you have the opportunity to run them offline at night, and so the power grid is less busy in the evenings. You know that's driven by a lot of factors. Obviously, people aren't doing as much stuff at night.

Speaker 1:

Also, you don't need as much power to cool things with air conditioning.

Speaker 1:

All that kind of stuff affects power consumption, so you tend to have both less expensive power and sometimes you have surplus power available in the evening, so that makes it more environmentally efficient in many cases to do what I'm describing.

Speaker 1:

The other thing is is that you don't really care so much about latency, so you can send your workloads basically anywhere, so you might have data centers that are too far away to be, you know, effective in terms of latency for a real-time application.

Speaker 1:

So that gives you another opportunity. And the other thing that's important to point to back from the research is that they found that because of the learning from the sleep time computer, or what we call learning cycles, because of the learnings, it actually decreases the use at inference time. So it makes the models faster because they can refer to this distillation of knowledge and solve a lot of problems that previously might have required both multiple turns of a conversation, which is, of course, very expensive in terms of both multiple turns of a conversation, which is, of course, very expensive in terms of GPU or LPU time and also with respect to environmental impact. So the net effect of this is use offline resources that are less expensive and less environmentally impactful to improve the efficiency of your online resources. So I think it's actually a really positive story in all those ways.

Speaker 2:

Mm. Hmm, well, perfect. It sounds like AI deserves a nap time just as much as we do, and it's beneficial for all of us, ai and humans alike. Next up, we're talking about the OpenAI acquisition of Windsurf, and we'll cover some other coding tools as well. So OpenAI has reached an agreement to acquire Windsurf, an AI-powered coding tool formerly known as Codium, for approximately just some small change $3 billion making its largest acquisition to date.

Speaker 2:

Windsurf is an advanced AI integrated development environment, or IDE, that leverages large language models and agentic AI to automate and enhance the coding process. It's recognized for features like Cascade agentic AI, which enables autonomous code generation and refactoring, local code-based indexing engine, which allows efficient context-aware code suggestions, and SuperComplete, which predicts developer intent and offers inline code completions. The acquisition is widely seen as a defensive and strategic move for OpenAI, which faces rising competition from Google, anthropic, microsoft and fast-growing startups like Cursor Speaking of. Beyond WinSurf, there are several powerful AI coding tools worth considering. Github Copilot is a widely used assistant that integrates directly into popular IDEs, offering real-time code completion, chat-based help and multi-file editing capabilities. Cursor, which I just mentioned, provides a full-featured AI-powered IDE experience with advanced multi-line autocomplete, chat-driven code edits and deep context awareness perfect for power users who want granular control over their coding workflow. Meanwhile, clog Code from Anthropic shines as a terminal-based agentic AI assistant designed for complex multi-step coding tasks, bug fixes and code-based exploration, catering especially to developers comfortable with command line interface or CLI environments.

Speaker 2:

Each of these tools brings unique strengths that complement different coding styles and project needs and, amit, I know you have some experience with maybe, for sure, probably all of them, if I had to guess. But I feel like my first question here is more of a declaration. We can always kind of follow the dollars, right, if we want to look at trend lines, if we want to look at where we're going in the next few years, follow the money. Obviously, openai making this $3 billion acquisition and when CERF makes you probably realize that this is a direction we need to focus in, is that shocking to you and me that we're going to be putting more dollars into AI-assisted code?

Speaker 1:

Not at all, and I think there's going to be, you know, continued investment in this area. And you know coding has been seen for some time now, for several years, as this killer app for the current generation of language models, and it continues to be the case. You know, windsurf is it's one of the tools in this space, as you mentioned, and I think, by the way, what you just shared shows that you are pretty technical. Going back to the earlier segment, you know, with all these AI coding assistants, they all do something in common, which is they build software for you. Right? That's what you're trying to do. You're trying to say, hey, what can I do to build software without being a coder, or to be a more powerful coder? And what I would point to in terms of the trend line is the ability for a so-called non-technical person to build software, to build applications to add to existing applications and do it in non-trivial ways.

Speaker 1:

So for a long time, we've had the ability, in a variety of different ways, to build very simple things. For example, there's a product called Airtable that came out years ago that made it possible for business users to create databases in the cloud. Came out years ago. That made it possible for business users to create databases in the cloud. It was not like SQL Server or Postgres or these other you know, developer-oriented databases. It was very, very simple for people to create apps. I mean, even if you rewind in time prior to that, we had Microsoft Access the last you know, 20 plus years. That allowed fairly non-technical people to build meaningful business applications. But beyond like declaring what type of information you wanted to store, beyond that, you kind of needed a coder to come in and like build things for you. And what's changing now is the ability to talk to an AI and say, hey, I want the app to do this and this and this and this. I want my membership application to work this way when the user comes into the website. I want my pricing to work this way. I want my you know, I want my functionality for abstract submission to work in these other ways. I'm using association examples intentionally because these custom you know, code-based things are way, way more accessible to everyone.

Speaker 1:

Now, but coming back to your broader point, mallory, about following the money, that's generally an interesting path to consider. I think it's true that it's oftentimes a line that gives you insight into where things are going. At the same time. Sometimes those insights might be directionally correct, but the timing and kind of the magnitude of the investments may be wrong. I actually think in this case the amount of money they're spending is trivial to them and a very you know it's kind of an irrelevant amount. It's more about them getting into the coding space and that's how big these dollars have gotten. In the world of AI that that represents less than 1% of OpenAI's market. Not market cap is what I was going to say, but their latest valuation. They're still a private company. So I would point out that.

Speaker 1:

You know, ultimately the model business meaning the business of building the underlying AI models like GPT-4 and CLOD 3.7, that is a race to the bottom. There is going to be really a very. It's going to be very, very difficult for companies to make significant money in building and selling models. There's free open source options available that are nearly as good as the commercial counterparts. Some argue that the open source market will at some point overtake the commercial market. We talked about that with DeepSeek R1 back at the beginning part of the year, where that model was as good as O1 from OpenAI, which is obviously a proprietary piece of software. So if models are becoming cheaper and cheaper and cheaper and eventually close to free.

Speaker 1:

How do you make money? You can't scale your way out of something that's approaching zero in terms of revenue and profit, so all these companies are heading to the application layer. So the application layer includes coding, it includes agents, it includes you know things that integrate into business applications. It includes research. It's all the utility that you get as a business user on top of the model. So you think about people that are starting to form opinions and loyalties even to certain tools, whether it's Clod or it's OpenAI or anything else. It's not really because of the model. The model is very, very similar between Clod and ChatGPT's latest underlying model.

Speaker 1:

But it's about user experience, it's about simplicity, it's about low friction. It's also about connectivity. So one of the things that has been going really well for the Claude team is they. I mean, they were the people who proposed the standard called MCP, which we've covered recently on the pod, and the model context protocol, or MCP, opens up AI systems to all sorts of connectivity with other tools, as we both covered and Mallory demonstrated in the podcast. It's really, really exciting, and Claude was the earliest adopter of this standard and now everyone else is following.

Speaker 1:

So you know that makes Claude more functionally valuable to me than ChatGPT, because ChatGPT is likely to very soon support model context protocol, maybe even by the time you're listening to this, but as of this moment in time, it does not. So it's those kinds of things that make the ecosystem better to lower friction and improve, like the business value, and coding is just one of those applications. In fact, just to kind of put an exclamation point on this, openai, in addition to their the Windsurf thing, they recently announced that they're hiring the former Instacart CEO, fiji Simo, who was and is on the OpenAI board, to be the CEO of not OpenAI but OpenAI's applications business, which I believe will include this as well, as you know, ChatGPT as a consumer product and a number of other things. So the applications business is clearly going to be where the money is at things.

Speaker 1:

So the applications business is clearly going to be where the money is at and you know clearly our thesis is that if you focus even more specifically within apps on particular verticals or particular highly you know specialized use cases, you can build something deeply meaningful for people and also have a path to you know sustainable business at the same time.

Speaker 2:

For our technical folks. Amit, I know you have experience with most of these tools. If there's someone technical I don't know if this would be the case, but someone technical listening to this podcast who has not experimented with any of the tools that I mentioned or AI assisted generating code, what would you say about the experience? One, using these tools, if you have any favorites, and then two, kind of what that experience is like developing software with AI versus without?

Speaker 1:

So we have a lot of people using Cloud Code. That's our hands down, our favorite coding tool. It's far more powerful than anything else. We've tried including WinServe Cursor Replit. You know Microsoft Visual Studio Code. It's not to say that it actually is used. Instead of those things, you still need what's called an IDE, which is this overall visual development environment where you can see your code and edit it and do things. But having a command line interface for developers is super powerful because it allows the tool to interact with your computer in ways that these other software tools really generally can't. It's also just much, much smarter in terms of where CloudShines is being able to deal with super complicated, long running processes. Where you want to, you know, go through an entire code base and make certain types of changes or check for problems, look for performance optimization opportunities or, in some cases, build entirely new apps completely from scratch. So if you consider yourself kind of an intermediate to advanced developer, I would say get Cloud Code. It runs on Mac. It will run on Windows using something called WSL, so it's pretty easy to install. Openai has a competitor product they announced fairly recently called Codex. I don't find it to be nearly as good as Cloud Code. So Cloud Code is, in my opinion, at the moment the king of the hill as far as IDEs are concerned.

Speaker 1:

I know a lot of people who love Cursor. We have some team members that use Windsurf. Most of our team still uses Visual Studio Code because they actually have within Visual Studio Code something that's equivalent to Cursor and WinSurf, which is called the Copilot Agent Mode, where Copilot, which is kind of the first AI that most people had experience with in the developer world. That tool now has gotten a lot more powerful, kind of quietly in the background, it doesn't have as much buzz around it as Cursor. My point of view is that, absent an acquisition from a major technology player like OpenAI, products like Cursor and Windsurf are going to have a really hard time because these products are going to I mean, it's really a commoditization unless you have enough scale. And the cool thing about Cloud Code is because it's produced by a model developer. They're closer to the metal, meaning that they are able to take advantage of the model in ways that I don't think these other vendors are able to do. Again, I'm not an expert in Cursor or Windsurf. I've used one of the two of those tools.

Speaker 1:

I just think that Cloud Code is worth checking out and a lot of people still haven't even tried it out. It's kind of maybe a little bit intimidating looking initially because it's a command line thing, but if you're a developer, check it out. I think you'll find it to be quite interesting. One thing you can do with Cloud Code that's super easy after you install it is just go in there, open up Cloud Code inside one of your projects and say explain this project to me and you'll see an interesting point of feedback and ask Cloud Code to perhaps solve an issue that you have in your backlog and give it the issue, a URL to your GitHub repository if you use GitHub or a description of the issue, and Cloud Code will go through every area of your code base necessary to solve a bug report or feature request and give you back a complete, you know, set of changes that you can review easily. So that's for the developers. For the non-developers and I'm talking to you, the CEOs, who declare yourself decidedly non-technical and you delegate all of your technology stuff to your IT folks I'm talking to you guys, as well as everyone else who's in the non-technical camp Get yourself access to Cloud and install the desktop version of Cloud, which works both on Mac and on PC. This does work. What I'm about to describe does work in the web version as well. It's just not nearly as good.

Speaker 1:

So, in the desktop version of Claude, go in there and have a conversation about some kind of an app that you want to build, or maybe you just built on your website. Like a common thing associations do is they put their prospective members through a whole lot of pain to sign up. So it's very typical that you'll have an e-commerce process where it's like oh, I have to go through this step and this step and this step. Or maybe you have a membership application that you paid tens or even hundreds of thousands of dollars to some developer to build, and that thing has been sitting out there for years and it's, you know, really crusty old software and it's, it's. It's maybe wasn't even great initially, and it's really not great now. And the problem is is that it's not the highest thing on your priority list, because it kind of works, but your members and prospective members do not like this thing, and so what I'd recommend you do is take a few screenshots of the current application on your website, you paste them into Claude, the desktop app.

Speaker 1:

And then what you do is you say hey, claude, this is my current membership application. It kind of sucks. I want your help improving it. Give me a prototype of what a new membership application will look like. Hit enter. What you'll see very quickly is Claude will start and, by the way, put Claude into extended thinking mode, which is this thing where Claude thinks more deeply, as we were talking about earlier, and Claude will very quickly come back to you with a fully interactive prototype. It won't be functional yet, but it'll show you. Hey, this is what I imagine you could do for your membership app. And you can go back and forth and say well, you know, right now it's like 18 steps in order to become a member. I want to really reduce that as much as possible. Can you take a look at the flow and give me some suggestions on how to improve it? So Claude puts on his UX UI hat and is deeply empathetic with your business. And, by the way, if you tell Claude, give Claude a URL to your website and say read functional in terms of connected to your data, but like visually functional prototype of a new membership application, or, if that's not your problem, maybe people wanting to sign up to be speakers, or people that are searching for volunteer opportunities.

Speaker 1:

Prototype these member-facing things that are giant pain points for you and ask Cloud Code to help you build a better way. Now you might say, well, that's really cool, this is really exciting, but now do I still have to hire a developer to take that prototype and make it real? That's where it gets really, really exciting, because you can say, hey, claude, thanks so much for this beautiful prototype. It's exactly what I want. I love this thing. Now I really want to go implement it. And, let's say, you're a little bit technical. At this point you could say, hey, claude, talk to your buddy, claude Code.

Speaker 1:

Claude Code has an MCP server, so your Claude desktop app can talk to Claude Code and say, hey, I want you to create this as a React project or an Angular project or a Vue project these are just different software development frameworks and go create it here and build it until it runs. Software development frameworks and go create it here and build it until it runs. And, by the way, here, use a local database for now. Just prototype the database and we'll have someone else later on securely connected to the data. And you could do that, and I know you actually have like a functioning true app, which then you can actually use cloud code directly to rewire it to connect to the real data source by API or whatever.

Speaker 1:

So you can go through this crazy fast iteration and build stuff and you can also say hey, claude, I want three different versions of a member application, right, which might take a human like three weeks to do you know, a week each or something and cost a lot of money. You're never going to do that. Well, you can do that. What about using Claude? But also go to Google, gemini or OpenAI at the same time and ask for the same thing and come up with the best answer for you.

Speaker 1:

So these AI tools can be used to solve business problems that are at this intersection of technology. That's normally your Achilles heel as an association. You say all the time you know we don't have technical strength, we don't have a thousand developers, we're not Amazon, we're not Netflix, but the field's leveling and you now have the ability to do this if you take the time just to go and experiment with this stuff. So for the non-techies, go check out Claude. I point to this one not because it's necessarily better at coding than the others. I mean it's one of the best, but it's just the easiest one to use. It's just so simple to use Claude's desktop app to see what they call an interactive artifact, and it's just pretty damn cool too.

Speaker 2:

Well, that's incredibly exciting and I feel it could also be a bit overwhelming if we have some association leaders here who have traditionally, like you said, outsourced dev work. They just don't have that strength within. This almost makes me think that you would need to reevaluate that whole part of your business, because is it worth maybe hiring a few technical-ish people and having like a really small team that can just iterate on projects like this all the time? What do you think, amit? Or is there still value in having some of that work outsourced?

Speaker 1:

If you have a dev team, you need to make it their mission in life to become experts at using these tools. If they tell you they don't like AI or they use AI but they're still slow in developing responses for you or building things, they're not using AI the way I'm talking about using AI, there's ways to help your dev team come up to speed on this stuff if they're willing. I do know some folks that are deeply technical but are, frankly, giant skeptics of AI and say no, no, no, it's not going to be perfect, it's not going to be as good as my code. And the reality is is that may have been true three years ago and it's not true now. So if you have a dev team, you have to push them and pull them If you need to demand from them that they become truly native AI developers. It's critical because your velocity is going to go up by probably a factor of 5X, maybe even 50X. That's what we've seen in our dev teams. It's just ridiculous how much productivity you have. So if you have a dev team, get them up to speed on this stuff. By the way, we're thinking of building an entire series of software development courses specifically tuned for the association world in the Sidecar Learning Hub. If you think that's an interesting idea, we'd love to hear from you. So please drop us a line. We have a feedback loop through the pod that Mallory can explain in a bit, because I forgot how that works. But we also have, obviously, email, or you can do hello at sidecarai and you can also hit us on LinkedIn. We'd love to hear from you if you think that's a useful idea. We'll probably put some public-facing videos on YouTube as well that give little snippets of things like what I just described with using Cloud Desktop.

Speaker 1:

Now, if you don't have a dev team and you outsource this stuff which is very, very typical just make sure the team that you outsource to is up to speed, because if you're spending tens or hundreds of thousands of dollars to get these little micro features out of people and it takes weeks, months or forever to get them and they aren't that great when you get them which is unfortunately the common experience with custom software developers or people who do configuration or customizations of package software go demand more, you know now that you can get more and you should expect more. And finally, if you don't fall into either of those buckets where you don't have your own dev team and you don't traditionally outsource software development work and you just kind of don't do anything with software. You just kind of use out-of-the-box software and you kind of make your members pay the price in terms of high friction. Just go try this and see what happens and you can see that it is actually quite possible for you to do almost all of it yourself.

Speaker 1:

Maybe you go hire a freelancer on Upwork or you hire a team that knows associations really well. We obviously know people who do that kind of stuff, but the point is there are ways to do this that are dramatically different than what you think is the way to do this. So it's not only lower cost, but faster, higher quality, more reliable. It's just an exciting time. We can do things for our members, for our audiences, that there's no way we would have been able to do. Even the largest of associations with the largest budgets and the largest technical teams cannot do what now the very smallest association with the smallest budget can do, literally in days.

Speaker 2:

And even all my non-techies out there the fact that we can go to Claude and create an interactive prototype of an idea that we have. I mean that just wasn't even possible, quite literally, a year ago. So to think that, like any idea we have as it pertains to your work, even if you're non-technical, you can potentially build or start to build, that's very exciting. And what Amit just mentioned is in the show notes. If you're listening audio only right at the top of the show notes there's a send a text button and you can text Amit and me in the Sidecar Sync podcast. And let us know if you think that software development course would be interesting. I think, personally, amit, maybe we could do a route for, like technical folks and non-techn interesting. I think, personally, amit, maybe we could do a route for, like, technical folks and non-technical. I think that would be a really cool way to get everybody involved in that conversation. And, yes, let us know if that's something you'd be interested in.

Speaker 1:

We've covered a lot of ground here, and if Sleep Time Compute didn't put you to sleep, maybe the technical conversation did, but I hope neither did. And it's a great time to be alive, it's a great time to be an association leader, it's a great time to explore and experiment.

Speaker 2:

Absolutely Everybody. We will see you all next week after some good sleep, hopefully.

Speaker 1:

Thanks for tuning into Sidecar Sync this week. Looking to dive deeper? Download your free copy of our new book Ascend Unlocking the Power of AI for Associations at ascendbookorg. It's packed with insights to power your association's journey with AI. And remember, sidecar is here with more resources, from webinars to boot camps, to help you stay ahead in the association world. We'll catch you in the next episode. Until then, keep learning, keep growing and keep disrupting.