{"id":8812,"date":"2026-07-08T20:00:06","date_gmt":"2026-07-09T01:00:06","guid":{"rendered":"https:\/\/bluecatnetworks.com\/?p=988993"},"modified":"2026-07-08T20:00:06","modified_gmt":"2026-07-09T01:00:06","slug":"podcast-how-ai-is-changing-enterprise-software-development-packet-pushers","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2026\/07\/08\/podcast-how-ai-is-changing-enterprise-software-development-packet-pushers\/","title":{"rendered":"Podcast: How AI is changing enterprise software development \u2013 Packet Pushers"},"content":{"rendered":"

HS136 \u2013 Sponsored Bluecat<\/strong><\/p>\n

[John Burke] (0:00 \u2013 0:05)<\/p>\n

Hi, I\u2019m John Burke, CTO of Nemertes, here with my co-host.<\/p>\n

[Scott Robohn] (0:05 \u2013 0:14)<\/p>\n

 I\u2019m Scott Robohn, CEO of Solutional and host of Total Network Operations, a sibling show here on Packet Pushers.<\/p>\n

[John Burke] (0:16 \u2013 0:28)<\/p>\n

And you\u2019re listening to Heavy Strategy, the show that tries to ask the right questions, not give the right answers. Joining us today is Andrew Wertkin, Chief Strategy Officer at BlueCat Networks. Andrew, thanks for joining us.<\/p>\n

[Andrew Wertkin] (0:28 \u2013 0:30)<\/p>\n

Thank you for having me. I\u2019m looking forward to it.<\/p>\n

[John Burke] (0:30 \u2013 0:52)<\/p>\n

And on today\u2019s show, we\u2019re going to be talking about just what a network management company is going to talk about, the challenges of using AI for enterprise network automation, because we all know IT folks are bending AI to the task of speeding up network automation efforts. Why isn\u2019t that as simple as it sounds? Why can\u2019t we just take Claude\u2019s word for it 99% of the time?<\/p>\n

[Andrew Wertkin] (0:54 \u2013 2:08)<\/p>\n

Yeah, just put it in a permissive mode and just click go. Done. Yeah, you know, it\u2019s interesting because, you know, I\u2019ve been in, well, I\u2019ve been developing software since I can remember.  You know, there\u2019s a reason I can\u2019t write. I\u2019ve been typing my entire life. And I\u2019ve enjoyed the craft very, very much. But you know, it\u2019s been interesting on the network side because the teams have been doing a really good job. And by the teams, I just mean network engineers out there driving automation, like, you know, pre-code generation, pre-LM, and really trying to sort of, you know, mature the methodology of developing software akin to how software development companies develop software. And with AI and with LM-driven or generated code, without those best practices, it\u2019s kind of scary to think about what might happen on the network. So, you know, it comes down to probably a further transformation in those teams to sort of more sophisticated processes around software development, because the faster stuff can be created, the more stuff can break, probably.<\/p>\n

[Scott Robohn] (2:09 \u2013 3:24)<\/p>\n

You have a thesis that really resonates with me, right? I would say, you know, one of the other hats I wear is a co-founder of an organization called the Network Automation Forum. And as of recording today, I\u2019m coming off our last meeting, AutoCon 5 in Munich, Germany.<\/p>\n

And, you know, we started that discussion on the attenuation against the adoption of network automation before AI entered the room, right? And so there\u2019s always been a certain level of resistance to letting automation go full throttle. And now, you know, the bots and the agents are walking into the room and, you know, the eye rolls are happening and the skepticism is strong. I would love to use that as part of our conversation here to say, I agree with you. We need to see great best practices develop in software engineering and software development, because a lot of what we do in network operations is downstream from that. So I know I\u2019m throwing a little maybe taking a little too much time here up front, but just that\u2019s my perspective. And I\u2019m really interested to have this dialogue with you.<\/p>\n

[John Burke] (3:24 \u2013 3:42)<\/p>\n

Yes, super. And I think you\u2019re spot on. The distrust issue of automation in the network shop runs 25 years deep. I mean, we have been hurt. We have been hurt badly by trusting automation too far, too fast, many times.<\/p>\n

[Andrew Wertkin] (3:43 \u2013 6:00)<\/p>\n

Yeah, what I always hear is trust, trust more so for repeatable, templated actions. Yep. If it\u2019s outside of sort of simple, repeatable, templated, then, yeah, that\u2019s where that sort of lack of trust starts, starts coming in pretty heavily. And, you know, as we know, our brethren, a vendor coming by saying just point and click, simple, easy automation is probably the worst way to gain trust with end customer. You know, the more sort of opaque it is, the more concern there usually is, which, yeah, is, you know, drives, it just sort of accelerates potential that distrust in the world of AI, you know, and I don\u2019t know, well, we can talk about ways to sort of approach it and drive more trust, but, you know, it\u2019s not just in networking and network, you know, it\u2019s obviously across the board. You know, many of us have lived in environments where source code control wasn\u2019t being used, where, you know, reviews weren\u2019t done appropriately, where code was copy and pasted from somewhere else without being scrutinized for current environment, where I could just go on and on and on.<\/p>\n

And you can trip like, you know, 10 of those major issues and like within a day if you\u2019re if you\u2019ve only accelerated it, but it does bring a tremendous amount of promise. And but it\u2019s not just it\u2019s not just software development, right? It\u2019s debugging what\u2019s going on and trying to figure out what\u2019s going on in live environments and why it\u2019s not working in some of those areas. As long as you stay, you have somebody with the right expertise and you stay critical of what you\u2019re being told, the amount of work that be done in a short amount of time is just it\u2019s it\u2019s amazing, just absolutely amazing. As long as you\u2019re not just like, oh, you said it\u2019s that it\u2019s that, OK, I should do this, I should do this. That\u2019s where the concern is, because we know the LM\u2019s, you know, tend to as hard as you prompt them otherwise tend to get pretty black and white about, you know, they think they found the root cause.<\/p>\n

[John Burke] (6:01 \u2013 6:43)<\/p>\n

I like with the idea of accelerating automation, the idea that you\u2019re going to be able to help people accelerate better practices if they\u2019re using better practices with AI, they\u2019ll get to the end of their automation project more frequently. They won\u2019t leave it half finished or three quarters finished, never fully debugged, never fully documented. Right.<\/p>\n

And then move on to the next emergency. There\u2019s more chance that they\u2019ll be able to in a single day or two days or a week or two weeks get all the way through and leave something behind that they can actually use again confidently as opposed to, you know, with some fear and trembling.<\/p>\n

[Andrew Wertkin] (6:44 \u2013 6:58) <\/p>\n

Yeah, yeah. It\u2019s a lot of it\u2019s just about architecture. A lot of it\u2019s about. Well, one is that, you know, the appropriate developer mindset, which is I\u2019m writing this for the people that will maintain in the future, so let me keep that in mind.<\/p>\n

[Scott Robohn] (6:58 \u2013 7:02)<\/p>\n

Which is automatic, right? Every developer thinks about that from day one.<\/p>\n

[Andrew Wertkin] (7:02 \u2013 9:15)<\/p>\n

Yeah, yeah, yeah, yeah, yeah. Especially when it started with, oh, you know, we need a quick script to try to solve this issue. And and, you know, and then somebody finds that six months later or. So a lot of it comes down to architecture, you know, like a lot of what is going to be in that automation will be repeated, right? Like how do we authenticate or what are standards around logging or how do we want to log or, you know, just how are we going to interface with with our devices? How are password faulted and how do we get those out of the vaults and, you know, how do we want to test or how should we think about backing out changes or whatever the case might be? Almost everything I just listed off is something that is repeatable and in this appropriate methodology. Well, now you just need to go sort of add your business logic, not necessarily all these other pieces. And and therefore all the code stays fresh and it\u2019s being tested and on and on. But, you know, it\u2019s it\u2019s it\u2019s really easy to build something quickly. And everything I just listed off also sounds like it\u2019s going to take longer than just having eight people reel off a different, fully functional, you know, A to Z scripts, applications, you know, whatever we\u2019re building. And so it\u2019s just there\u2019s there\u2019s, you know, it\u2019s probably more important than ever to sort of start with the right foot forward and start thinking about how we\u2019re going to make this sustainable and and, you know, where humans will be in the loop and where they, you know, it\u2019s critical they\u2019re in the loop.<\/p>\n

And, you know, what what sort of practices are we going to bring forward to do this safely? All of these things are just, you know, it\u2019s many of these are the same things we\u2019ve always considered or we\u2019re concerned about as building software over, you know, the last several decades. Right. Some of it\u2019s just how rapidly it can be done now. And and it\u2019s not like just people copying and pasting stuff off of Stack Exchange.<\/p>\n

[Scott Robohn] (9:16 \u2013 10:33)<\/p>\n

Like anybody can have code generated that works in that environment, you know, like that\u2019s easy to do, it\u2019s, you know, and that it was not the case two years ago, like I think like we you were in the middle of this fast changing and experimental time, right, where if you go back just a couple of years, you know, people had their their initial experiences with chat based tools and got widely varying results. You know, the whole nondeterminism of LLMs thing that was that was our first experiences. And then, you know, we see we see the advent of vibe coding and what an enabler it is for prototyping and but what a spaghetti mess it can produce for an enterprise code base to now emergent methodologies like spec driven development and test driven development. Like, I think we\u2019re seeing the emergence of, OK, here\u2019s how I use the tools, you know, and here\u2019s how I can actually build trust in the technologies versus my first, you know, chat GPT three dot X experience. Right. What are you seeing on those fronts, you know, in the emergence of these strategies and repeatable tools that are coming up?<\/p>\n

[Andrew Wertkin] (10:34 \u2013 11:59)<\/p>\n

Yeah, no, we\u2019re we\u2019re we\u2019re seeing them for sure and have been experimenting and adopting a couple. I think we\u2019re we\u2019re. We my company and also me personally have had the most success is interestingly, like the part I spent probably the least time about in my career, not the company, but me personally, which is that upfront spec development and working with the LM on trade offs, and that\u2019s where I can bring in my experience and how should we approach this and spending that time up front, breaking it down into reasonable chunks and and then, you know, putting the appropriate guardrails in place and, you know, that that sort of stuff has been transformative and versus, you know, trying to type out a prompt describing exactly what you want, just the sort of iterative spec and test development that really starts with, OK, I want to create a high speed TCEBF logger. Right. I\u2019ve never done that before. Right. And and so I don\u2019t even know what the trade offs are. And so, you know, I\u2019m not I\u2019m not, you know, so how much can I trust, you know, whatever? But I can start at a very high level. I want to do this. What don\u2019t I know?<\/p>\n

[John Burke] (11:59 \u2013 12:47)<\/p>\n

Intent. You got it. Yeah. I think there\u2019s a really important point, sort of one level underneath that, which is that stating your intent and need very clearly is crucial because when you\u2019re having software twiddle the settings on hardware out in your network, twiddling the wrong settings shuts down the traffic flow or overburdens the processors so much that you get, you know, terrible performance drops as it as it maybe devotes all of its time to generating logs and sending them. So, you know, you\u2019re in a space where infrastructural mistakes can happen and can happen at automation speed and have really huge consequences.<\/p>\n

[Andrew Wertkin] (12:47 \u2013 14:57)<\/p>\n

Yeah. Yeah. I mean, in our world of DNS, I mean, just on the public side, if you look at the last several major outages that were announced by huge public corporations, I think, you know, 80 percent of them were automation driven change to DNS that took everything down, you know, and and now that I\u2019m not going to remotely blame an LLM for that. That\u2019s just but but you just, you\u2019re well more likely the faster you can come up with the solutions and the more especially the more certain your your coding partner or your debugging partner, your performance partner, whatever the use case might be, the more certain they are, the more likely you\u2019re going to be like, OK, you know, and then and then, you know, they\u2019re everybody suffers from permission fatigue and. You just your fingers over enter and you just keep pressing it, you know, and so, you know, that comes down to also like, how do we how do we appropriately supervise the behavior and make sure those things don\u2019t happen?<\/p>\n

But, yeah, and, you know, so the more successful that sort of more successful express intent includes those things. You know, what do you know could potentially go wrong? What are you concerned about? What do you make sure is actually tested? What does load actually mean here? How should it behave? Now, you don\u2019t need that stuff up front, but having the right experience to know what sort of the categories of things that you should probably be considering are good, you know, but what what you don\u2019t need to do, and I think this is really matured in both tooling, but also process over the last six months, insanely small period of time, is just that you don\u2019t need to start by writing a four page prompt on exactly what to deliver, right? You you do if you\u2019re using like a, you know, less expensive, more run rate type model, like not a frontier model, but but just have a frontier model, write the spec for the run rate model. You know, you don\u2019t need you\u2019ll never be as good as an LLM in generating prompts.<\/p>\n

[Scott Robohn] (14:58 \u2013 15:55)<\/p>\n

Right. Well, I I so I\u2019ve layered in a technique where I basically ask the LLM to interview me where I\u2019ll bring, you know, a page, page and a half of intent that\u2019s reasonably detailed. And if you\u2019re using it, using the process to develop all the all the all the right markdown files for SDD, number one, it\u2019s it\u2019s cheap. I know there\u2019s token cost here, but, you know, I\u2019m creating human readable documents that I can validate and verify after I\u2019ve gone through an interview process. And I heard a great comment last week. You know, the code is free in a sense where, you know, we\u2019re used to thinking about us, you know, laboring to get the code just right. But now we\u2019re asking agents to write the code and we can iterate and we can throw away stuff that looks like garbage. And you still have to have enough domain knowledge to know that it\u2019s actually garbage, right?<\/p>\n

[Andrew Wertkin] (15:56 \u2013 16:32)<\/p>\n

Exactly. And that last point is, well, on the personal side, sort of my fear, what everything looks like in 10 years, but but also the most critical thing now without the domain knowledge and the sort of experience and expertise of how things can fail and how things should work and how they should gracefully fail and on and on, then then you\u2019re you\u2019re asking for trouble. But but yeah, I like that sort of and I do that to probably less regiment than you do. But that sort of like, you know, what what am I not thinking about here? Exactly. What else could go wrong here?<\/p>\n

[John Burke] (16:33 \u2013 16:53)<\/p>\n

And when the LLMs are just as happy to bring you the, you know, deadly poisonous mushroom as the tasty and edible mushroom, and they look the same to a casual inspection. Yeah, that kind of domain expertise becomes irreplaceable and absolutely necessary.<\/p>\n

[Andrew Wertkin] (16:54 \u2013 20:08)<\/p>\n

Yeah. And here\u2019s what happens and why process is so important here is people sit down and they think they\u2019re on the same page with the LLM. And, you know, over a couple of weeks, it seems to be true, like it\u2019s seemingly remember stuff. And then, you know, on day seven, you start a new session and seemingly it forgot everything it ever knew. And it\u2019s been sort of degrading over the week because you haven\u2019t been documenting this. You haven\u2019t been having, you know, saving the appropriate memories and making sure you\u2019re updating your prompts and everything else. And then it makes a decision that is unbelievably naive. And you, because we humanize these things, have this thought of like, you know, you\u2019re you\u2019re working with a doppelganger or who is this person I\u2019m working with? Like, how is it possible they forgot this? And so there\u2019s a lot of errors that occur in those moments when operator, developer, network operator, whoever\u2019s using it, is unclear of the state of the memory of the session and assumes some not necessarily something deterministic, but something at least informed by things we\u2019ve done in the past is about to happen. And then, you know, surprise, you know, and and what do we call that? Like, you know, part of it is rediscovery tax. So, you know, I\u2019m going to spend a lot of money as in tokens or credits or whatever in rediscovering the same thing over and over and over again because it\u2019s not appropriately documented. And and these are things that the LMs in general, but, you know, depending on the coding tool. But, you know, I tend to use Claude a lot, but but regardless of you, several. But, you know, they\u2019re really good at, you know, every time I end a session, like, what did we do? What did you have to rediscover and what that you had to rediscover? Should we should we appropriately document so you have to rediscover it again? And that\u2019s not everything, because, you know, experimentation works. Sometimes you learn a ton when you make mistakes, you know, and maybe going back to the personal side is still professional side of sort of hardened software, like software that works, as we say, like works in anger, doesn\u2019t just work in the lab. It doesn\u2019t just work in the happy cases like it works, put pressure on it and still works. And, you know, getting getting to that point is, you know, it takes takes experience of having done it, but specifically having failed at doing it like it\u2019s like, you know, something smells bad because you once made that error before, you know, or you around when that happened. And that kind of learning, which is, you know, very specific to the very large context sizes of our brains, you know, like is like you don\u2019t even have to go review your notes. You\u2019ll spot that pattern immediately if you\u2019ve had the experience of failing, it just it\u2019s wired in and those are things I just don\u2019t know how they\u2019re going to happen, you know, without purpose, without purpose.<\/p>\n

[John Burke] (20:09 \u2013 21:10) And I sometimes hear people saying it won\u2019t be too long now when, you know, we won\u2019t even be having AI develop these scripts or these programs for us. We\u2019ll just tell it what we want to have happen. It\u2019ll spin up a program in the background, send it off and get the thing done. And the next time we need it done, it\u2019ll generate a program and send it off to get executed and we\u2019ll be done. So it\u2019s like it will be creating automation for us. It\u2019ll be the automation. Sure. And for me, I think what you\u2019re saying right now is the pushback on that. It\u2019s like, OK, but the second time it sees this problem and it\u2019s time to generate a program to solve it, is it going to remember how it solved it the last time?<\/p>\n

Is it going to remember what went wrong and how it got fixed? Maybe, maybe not. I think until that problem is resolved and resolved by stuff that works in anger, as you say, like no matter how mad I am when I type the prompt to the AI, it\u2019s still going to give me the right answer and the good answer. Then it\u2019s not solved.<\/p>\n

[Andrew Wertkin] (21:11 \u2013 24:34)<\/p>\n

And yeah, then it\u2019s not solved. And the only thing I was going to add to that is you run into the opposite problem of, of, um, you know, cause the elements are matching patterns basically. Uh, and so something looks like it fits this pattern. It doesn\u2019t mean that\u2019s the answer. And ironically, like if you only have a couple of cases of this sort of like, you know, something failed, here\u2019s why I know the pattern, like then the more likely something\u2019s going to get sort of, you know, force fit into that, like, you know, pound the, the, the round peg into the square hole if it almost fits, you know? And, uh, and it\u2019s just, I think there\u2019s a, still a bit of a chasm between, um, especially in the world of Brownfield, but, but, uh, between, right, I\u2019m just going to state my intent and the scripts will be written, stuff will be done. I\u2019ll never have to look at this twice. Um, for that actually being done in an enterprise network where, um, you know, you don\u2019t have all the tools today to get that done. Um, you don\u2019t necessarily have the budget today to get that done. Um, and you have history. And that history includes things that are not good in the world of AI, which are things like tribal knowledge and one-offs. And, you know, this was done for a very specific reason, but there\u2019s like yellow tape around it. Nobody knows why, but we just know if we change it, everything breaks. So, so we\u2019re not changing anymore. You know, it just, this was never documented that wasn\u2019t documented. So, um, you know, I, I just go back to, um, one of the sort of, um, really well-known, um, foundation books in software development that, that, you know, tools and tactics have changed, but this book is as true as it ever been was, uh, Martin Fowler\u2019s refactoring, which, which I think in the forward, like the first, in the first couple of paragraphs, it says, if you can\u2019t test, put this book back on the shelf, um, paraphrasing something, something close to that, like you can\u2019t change it unless you can test it. And in this world of like, um, you know, tribal knowledge and one-offs and not everything following a pattern, um, how, how are you going to, how are you going to approach that, um, especially if there\u2019s nothing to test against your lab, doesn\u2019t have those one-offs, your lab, you know, it\u2019s how do we actually test this appropriately? And, and, and that\u2019s what causes all of this, you know, longer change management process. They know the organization\u2019s learned if we touch that, it\u2019s going to break. So that\u2019s only going to happen on like this type of change window where we have three and a half hours and nothing else can be scheduled during that change window and like they\u2019ve, they\u2019ve never fixed the underlying problem, but they\u2019ve, they\u2019ve wrapped it around process. And so, you know, companies also enterprise is going to have to look at, you know, where to start with this stuff. Um, and, and stating the obvious, um, uh, net new is, uh, you know, build this way to begin with great, um, changing something exists, whether that\u2019s software or a network architecture, um, that\u2019s way harder for humans and LLMs, uh, but on the L at least the humans hopefully have some of the tribal knowledge.<\/p>\n

[Scott Robohn] (24:35 \u2013 26:08)<\/p>\n

One of the, so another exciting thing that I think is in front of us here is, you know, to your point of everything you\u2019ve just laid out. Um, you know, if I have an LLM, everything looks like a prompt, right. And we\u2019re, I think many of us are figuring out that, you know, not everything is a problem for an LLM. And, and we have, we have this really interesting and potentially useful set of new computational techniques in front of us, and we haven\u2019t gotten rid of all the others. Like machine learning is still really useful for other things in the, in the AI space, statistical regression last I checked doesn\u2019t really cost me any tokens. Right.<\/p>\n

Right. So I\u2019m adding to the tools in my toolbox and the really fun puzzle that\u2019s going to be in front of us all is like using the right tools for the right job here. LLMs where they make sense, ML where it makes sense, regression where it makes sense, um, maybe for the, the, the problem that you called out of, you know, reinventing the wheel every time the question\u2019s asked, maybe part of my constraints are, okay, I\u2019ve solved a problem and I\u2019ve put it into my IT service catalog. And if there\u2019s something in my service catalog that is a 90% or better match for this prompt, use what\u2019s in the service catalog and don\u2019t spend tokens on creating another solution for it. And I\u2019m, I\u2019m riffing here, right? But I think these are all like approaches we need to put on the table as we collectively all figure this stuff out.<\/p>\n

And I\u2019m, I\u2019m much more positive about it than I am cynical about it. Um, check in with me in a year and see if I say I feel the same way.<\/p>\n

[Andrew Wertkin] (26:08 \u2013 30:11)<\/p>\n

So, no, it\u2019s interesting on that sort of service catalog side. I mean, it\u2019s in the good news is just have the LM check to see if you solve the problem before, you know, and, uh, and they\u2019re again, super good at matching patterns, they\u2019ll find something. And that\u2019s actually like of all the sort of techniques I\u2019ve developed, uh, over the last year, let\u2019s say, um, that sort of tight link between, uh, documentation of what I did and what I actually did is, um, is, is, is super both from the perspective of, you know, it\u2019s, it\u2019s, it\u2019s quick story. I years and years ago, I was CTO of a company in the application lifecycle management space. So we built software to help build software and we specifically, um, like part of our, like ideal customer was in, uh, embedded software that was safety relevant where, um, a defect could cause harm to the operator, for instance, um, like automotive or, you know, you name it. And so, um, there in, in that world, there\u2019s all sorts of standards like ISO 26000 and, and, uh, automotive SPICE and on and on. And part of the standards around, like there must be traceability between requirements and code. Right. Um, and if the requirements change or the code changes, that trace becomes, um, you know, potentially problematic. And I\u2019m going to go in and look for any of those impacts and I\u2019m going to have to say in this change, by the way, I changed the code, the requirement still, you know, is correct. The test case is still correct. The architecture is still correct.I\u2019m going to have to go do that for all these suspect traces. Um, and that like, you know, you\u2019re not going to convince somebody building, um, B2B software that they need that level of traceability. Right. But like I do that when I\u2019m doing LLM based development, cause it\u2019s amazing, you know, one and mostly, well, I enjoy it, but, but also because the LLM then goes and traces. Um, and the LLM can quickly figure out why we did something the way we did it, because it can trace back to the documentation and, and, and the LLMs writing that documentation and the LLM knows to write the documentation for itself, which is something that, you know, like, um, uh, even if I, sometimes I will say, write this for a human, sometimes I\u2019ll write it like, you know, write this document for a customer that may or may not be aware of software development, how software development works, or may never have installed Python before, whatever the case might be. Um, but oftentimes with those docs, I\u2019ll be like, um, either include an addendum just for an LLM or just write this for an LLM and, um, and one, it\u2019s more terse, but two, like, you know, LLMs are pretty good at writing documents for them to read and, uh, and, um, you know, both from an efficiency and cost standpoint, reading it, right. You know, like, um, the human version is, is more expensive, but regardless of that, um, that also like goes to this, um, okay, now I\u2019ve included in the right software for the next person to maintain to also write software for the next LLM to maintain. Um, and given it, you know, the capability obviously to pull in and parse a lot of text rapidly, um, yeah, I\u2019ve never had better documentation in my code or outside of my code. Um, would never have done this much documentation as a human. Um, ever, uh, you know, with the assumption, like most humans have, like while you\u2019re writing it, uh, yeah, I\u2019ll remember this. Yeah, this is pretty easy to read. Like I\u2019ll just read the code, like it\u2019s obvious or anybody would understand what I meant by that. Yeah, exactly. Yeah. I said, sorting, I\u2019m sorting here, you know, and, um, and that, that those things lead to a tremendous amount of defects, but so does, um, uh, that, you know, even worse sometimes, I mean, what\u2019s the old adage? Like the only thing worse than, um, no internet connectivity is crappy internet connectivity. That\u2019s right.<\/p>\n

Yes. Yeah. It same thing with documentation.<\/p>\n

[John Burke] (30:11 \u2013 30:48)<\/p>\n

So in a way, um, you\u2019re both mentoring your AI to improve its domain expertise. You\u2019re teaching it more about what it means to run a physical network of physical devices and what the risks are and, uh, what the limits of experimentation are and so forth. Uh, and at the same time, you\u2019re mentoring it on being a good team member by doing things like documenting its code and explaining itself for sure.<\/p>\n

[Andrew Wertkin] (30:48 \u2013 32:30)<\/p>\n

No, it\u2019s a good, it\u2019s a good analogy because, um, that\u2019s what I usually, uh, advise people is like, um, yeah, especially if they\u2019re, they\u2019re senior, like, um, yeah, just imagine you\u2019ve got four junior to intermediate software developers working with you and, um, and that are pretty self assured, you know, like that, that are convinced they\u2019re correct, you know? And, and so if one of them said to you, no, no, this is the best way to do it. You would say, why, what evidence do you have? Do we have enough evidence? You know, like what alternatives did you pursue before you came to the solution? And if you think about your interactions like that, then, then you\u2019ll always think about what the right question, even though from a ability to generate code that works rapidly, you know, this is not a junior or intermediate software developer, but in terms of thought process, um, yeah, in many cases, intermediate software developers is, is, is, you know, is, is better thought process, you know, in other words, cause it\u2019s thought process, it\u2019s not, it\u2019s not pattern matching and, and, you know, the other magic of LMs. Um, and so like that, I think that\u2019s the biggest mistake, uh, people, you know, whether it\u2019s, it\u2019s bias, you know, confirmation bias or otherwise, like, um, uh, people are looking for, um, that sort of positive, this is how we\u2019re going to do it. And, uh, and if they say, Hey, I was thinking about this, this is a good idea. We all know. Um, that\u2019s the worst way to interact with an LLM because it\u2019s going to happily come back and tell you brilliant. Wow.<\/p>\n

[Scott Robohn] (32:31 \u2013 32:39)<\/p>\n

Yeah. You are so smart. That\u2019s the best way I\u2019ve ever heard. Yeah. Just, just include no sycophancy in every one of your problems.<\/p>\n

[Andrew Wertkin] (32:39 \u2013 33:53)<\/p>\n

Yeah, no, a hundred percent do that. Cause I, I, I don\u2019t need that from my peers, my employees and my parents. I definitely don\u2019t need from an LLM, but, um, I do like it for my wife every once in a while. Um, but, but regardless, and my kids definitely, but regardless of that, yeah, no, it\u2019s, but that idea that you\u2019re like, you are the expert here and you\u2019re working with, um, uh, something with less knowledge than you do of what you want to create and how it should work and what success looks like and how it\u2019s failed in the past. And just keep that mindset, keep that mindset, keep that mindset. And then at some point you feel like, this is like, I\u2019ve got the four best interns in the world working for me because that to me with the LLM, maybe five, sometimes seven, but like, it\u2019s, you know, um, uh, the, the acceleration of being able to get this done tonight, please launch 10 parallel agents.<\/p>\n

I wouldn\u2019t say please, by the way, launch 10 parallel agents and run through our blitz testing protocol. And then I wake up in the morning and a whole team has been doing work all night and, uh, I, I don\u2019t feel guilty. I don\u2019t feel, you know, I, I have my coffee and read through the results and very powerful.<\/p>\n

[John Burke] (33:53 \u2013 34:32)<\/p>\n

And you\u2019re sort of, I think by implication saying what vendors like you can do to support this kind of work in enterprise network shops, and that is, you know, basically building the power tools that those AIs can wield with appropriate safety features. So, you know, the, the, uh, circular saw has the automatic shutoff, so you can\u2019t cut your own fingers off as easily as you used to be able to things like that. Uh, you want to help them not crash the network by failing to do the obvious things with your power tools.<\/p>\n

[Andrew Wertkin] (34:32 \u2013 39:51)<\/p>\n

Yeah. No, it\u2019s interesting. Cause like, you know, like, um, most, uh, software vendors today, one of the things that we, we produce our, our MCP servers for our backend products.<\/p>\n

And we have several backend products and, and, um, and I think like a lot of vendors out there, like initially that was just, okay, let\u2019s wrap up our open API tools and this new protocol. And we\u2019ll just expose that and, you know, uh, LM will figure it out. And you realize pretty quickly that, um, that\u2019s not the right approach at all. You\u2019re just going to waste a ton of tokens. It can be a ton of guesswork. You just sit there watching the LM guess and guess and guess. And, um, uh, you know, you\u2019re yes, no, I\u2019m can pretty quickly figure out how to use a rest API. And if it\u2019s wrapped in, in, uh, MCP tools even faster, but that doesn\u2019t mean they understand your domain. And yes, especially the frontier models have been trained on not just your domain, but probably understand more about your product than you might realize. Um, but, but none of them, um, like part of, part of what we deliver, it\u2019s, it\u2019s not just, um, you know, uh, more, uh, domain specific tools than an open API. Like, you know, have more conversation between the MCP server and our backend server than the LM has to see. Don\u2019t don\u2019t send the, uh, there\u2019s a packet capture example. Like there\u2019s no reason to send the LM a 20 megabyte packet capture that that\u2019s, that ain\u2019t going to do anything or send it a whole time series that you should be sending to, uh, ML, not, not in AI, you know, like, and so how do I get the data needs summarized data, maybe even opinionated data, because we know our systems well, so, you know, with, with sort of opinions built in. What does the LM need to answer the user\u2019s question, which might\u2019ve been like, you know, why doesn\u2019t this work and think through that stuff as opposed to just, it\u2019s a bit, I\u2019m going to use the word lazy, but it\u2019s a naive. I think it\u2019s the better word. It\u2019s naive just to assume the LMs will figure this stuff out well. And especially if you want this stuff to be remotely repeatable. Um, so yeah, so we do that, but it\u2019s way more than that. Like, so now, you know, we just went through software development stuff. And, and so like, yeah, our, our customers can now pump out tons of automation scripts and otherwise against our products, super. How do we encode the best practices for doing that? So our customers don\u2019t, you know, automate themselves into, um, downtime. So that, uh, so that they know, and that comes out in things like, you know, uh, network advisors and other things that, that, um, whether through prompts or, uh, specific code or capabilities or training material, like, uh, help our customers succeed at automation, because in our industry, like on the DDI side of our industry, on the, in the world of DNS and IPAM, especially, um, the tailwind of this industry has always been automation. You have to change the stuff faster. And so, um, it, we want our customers to be successful. It, it pains us. Um, we had a customer a few years ago, well before LMs, and it just going back to an example of like, um, the horrors that can happen if there\u2019s no processes around the way you develop software was sort of teaching himself how to script against our product. And he had an administrative user and he wrote a script and went to test it and deleted the entire DNS environment inside this company, which immediately locked him out of systems and active director was down and they\u2019re using that for LDAP and yada, yada, yada glass had to break. And it was, and he\u2019s sitting there, um, like I can imagine I\u2019ve, I\u2019ve had that experience before, you know, like, um, I think once in university, I wrote an endless loop in an application running on a like old IBM PC where there was no, like you couldn\u2019t control seed. You couldn\u2019t like, you know, three hours of work. The only thing I can do is reboot the computer. You know, I know that sort of, that\u2019s one of several examples, some being professional, that sort of sinking feeling of, oh crap. Yeah, we, we got, and I can imagine what was going through that guy\u2019s head, you know? And, um, and so like, yeah, obviously that\u2019s an extreme example, but my point is, um, the more we like, um, we always get asked in software development, like, uh, about metrics and measurements and somebody always goes, well, why don\u2019t you guys just use like lines of code or something? And that\u2019s like, yeah, like the idea for anybody in software engineering that the number of lines of code they wrote, the more, the better is a appropriate metric is, is insanity, right? So we don\u2019t do that and nor do we suggest it. But my point is like, when you start thinking about the right metrics, well, certainly in the world of SAS, but now also in the world of like these scripts, it\u2019s just not about like, the question is, did it work the way we expected it to? Was this a successful automation? Um, and, and what did we learn that didn\u2019t succeed? And, and how do we sort of bring that back into a feedback loop? And like, it doesn\u2019t, um, you know, I mean, I was like, uh, the, the least number of lines of code to solve a problem, the better, but, but mostly did it actually solve the problem?<\/p>\n

[Scott Robohn] (39:52 \u2013 40:36)<\/p>\n

Well, in going to, you know, least lines of code as, as the other, um, could cause other problems, right? And like, again, one of the things that really jazzed about this whole environment and conversation is like the art of specifying our constraints is becoming much more important, um, probably than it ever has in our careers, right? You know, we\u2019ve got power tools, where are we going to put the right blade guards on? And there were grounding plug. Um, my grandfather made a power tools for Porter cable. And I\u2019ve actually have some of his old prototypes that I would never use to build a deck because there are pieces missing that keep losing digits. Right. So the system\u2019s thinking, um, and there\u2019s tribal knowledge.<\/p>\n

[Andrew Wertkin] (40:36 \u2013 40:39)<\/p>\n

Cause yeah. Somebody else can, uh, that was right.<\/p>\n

[Scott Robohn] (40:39 \u2013 40:44)<\/p>\n

For sure, John, this could go so many ways. Like, where do you want to take this?<\/p>\n

[John Burke] (40:44 \u2013 41:20)<\/p>\n

Well, I think just to, to round it out, I want to follow through on this idea that, uh, folks in the business now need to be thinking about their software portfolio as power tools that will ultimately be wielded by robot hands, not human hands. And, um, how that\u2019s going to change the enterprise software landscape. You know, we\u2019ve talked a bit about how it changes process inside the enterprise, uh, for using it, but what about the rest of it?<\/p>\n

Like, does it change licensing? Does it change, uh, you know, how else does things for sure.<\/p>\n

[Andrew Wertkin] (41:20 \u2013 44:13)<\/p>\n

And I think, I mean, you know, part of it is sort of what you\u2019ve seen in what the, you in, in the private markets in general with, um, with the devaluation of some software companies, which, uh, like, you know, we, well, we all should know the stock market doesn\u2019t is, is it\u2019s own thing. It\u2019s his own thing. Yeah. Uh, but, but outside of that, um, it, for sure, it\u2019s going to change licensing models. Uh, like, you know, if you\u2019re only, if your licensing model is just based on per user and I\u2019m not saying cause everybody\u2019s in a fire, everybody, I\u2019m just going to say, okay, what\u2019s an agent is an agent user. And, and, um, uh, you know, it, it, it, yeah.<\/p>\n

Companies are going to have to figure out, um, you know, what, what the right metric is going to be. And, you know, a lot of companies are switching to, okay, it\u2019s going to be credits. And that\u2019s the real value is, you know, how, how much you\u2019ve pushed through this LM or how much, and, and their, you know, enterprises don\u2019t like surprising bills every month.<\/p>\n

Um, they, they like consistency, you know? And so, um, I, I think, uh, you know, early companies, especially the, the LM companies themselves, or the AI companies themselves, obviously are very focused on a sort of an opaque credit type, um, model where we all know the price is going to keep going up and up. We won\u2019t be able to do the networking vendors.<\/p>\n

Won\u2019t be able to go in there and go up. Sorry, your bill this month is, um, yeah, it\u2019s twice as much this month because yeah, we\u2019ve changed the ratio between, um, credit and token. And, uh, and by the way, uh, we improved our stuff. So it\u2019s better. So you\u2019re going to pay more. Like it doesn\u2019t work that way in our world. And so, yeah, I think licensing models are going to have to change. And, um, uh, uh, but so much, so much more than that, I think there\u2019s going to be, I think we now have the opportunity, you know, how, how we always sort of go back and forth between platform and best of breed and platform and best of breed. I think we\u2019re going to be heading back towards best of breed because sort of interoperability between these vent a lot between different products has become, um, much simpler to solve. Um, especially with things like MCP, like I don\u2019t necessarily think I need a specific integration to, uh, ITSM systems anymore because all the ITSM systems, uh, have the ability to speak MCP. And so, um, that becomes easier, uh, picking a platform versus best of breed, but, but so, so in certain areas, it\u2019s going to be disruptive to businesses where, you know, if their only goal was put as much stuff in our platform as possible, uh, and nobody will be able to compete with us, uh, that\u2019s gonna change. Um, you know, and, uh, and I think should I launch in this? Yeah, I\u2019ll do the 60 second, 30 second version.<\/p>\n

[Scott Robohn] (44:13 \u2013 44:13)<\/p>\n

30 second.<\/p>\n

[Andrew Wertkin] (44:14 \u2013 44:29)<\/p>\n

Yeah. Yeah. It used to be like in the world of big, ugly central systems that nobody can change, like the big ERP systems and stuff like that, like where companies would have to, like in their earnings report would be, we missed it this quarter because of a bad upgrade to this massive internal system.<\/p>\n

[Scott Robohn] (44:29 \u2013 44:31)<\/p>\n

That wasn\u2019t that long ago. <\/p>\n

[Andrew Wertkin] (44:31 \u2013 45:58)<\/p>\n

Yeah, it wasn\u2019t that long ago. Yeah. And that was like, that was like the whole Salesforce marketing campaign initially with the no software. And we got that for you and all this greatness about, uh, SAS, um, uh, in that world, how they started getting disrupted was all these like new companies were bringing SAS based systems that were easy to use. They\u2019re still the system of record, but here\u2019s this new system engagement for HR, for travel management, for purchasing, for whatever, any part of that system. And then of course, those large companies in that space would just go buy those companies one after the next, after the next, after the next. And so to some extent, I, outside of the, I\u2019m not quite sure acquisition is going to happen at, at, uh, there can be way too much to acquire.<\/p>\n

Um, you\u2019re going to see a lot of companies that, um, are coming up with system of engagement, like value propositions for existing platforms out there to slowly start taking user engagement or system engagement, uh, away. And, um, and that\u2019s going to happen way faster than, than, uh, travel management and, and, uh, you know, goals on the HR side or purchasing or whatever these system engagements were that sort of started doing this with a large internal system. So in that world where it\u2019s simpler to disrupt or at least do better a small part of an existing platform with integration almost laid out for you, I think that\u2019s going to sort of drive this, you know, us back to best of breed.<\/p>\n

[John Burke] (45:59 \u2013 46:42)<\/p>\n

And I can even, you know, envision taking it, uh, really far down that path. Uh, only the AI\u2019s that work for us really understand what the software portfolio currently is because a new service pops up, they see that if they can do half of what they need to do in functional area X better, faster, cheaper. So now I\u2019ve got two things in that area and it\u2019s the one managing what work goes to which one. And, you know, it\u2019s like a redundant array of software vendors kind of an approach. Uh, I\u2019ll just arbitrage wherever it works best against the criteria that my AI, uh, understands for me matter the most saving time, saving money, better performance, whatever. Yeah. Yeah.<\/p>\n

[Andrew Wertkin] (46:43 \u2013 50:10)<\/p>\n

Yeah. Cause I, I see like go to some expo floor in tech these days, especially in networking, but, um, you know, pretty much any, any tech related area. And you\u2019re going to see company after company after company, uh, selling your platform for AI, you know, no matter where they came from, they\u2019re now have a platform for AI that\u2019s going to be multi-vendor. Why not? Everybody\u2019s got MCP servers. And, and so what are you going to have as an enterprise, multiple platforms for agentic operations, uh, a single platform, um, uh, you know, one meta agent that manages a bunch of other agent platforms. What if I put the wrong one here? And, uh, and, and how quickly can I change? And, uh, um, and, and we seemingly, and I don\u2019t understand this part cause it\u2019s not the way I work, but people do have easy bias as well.<\/p>\n

Oh. Oh yeah. They said they can handle this all for me. So I\u2019m just going to buy that, you know? And, uh, and it comes down to like this theme. It\u2019s funny, you know, earlier I said, I gave a real example. I had never written a EBPF, like, uh, you know, uh, internal space, high-speed logger. And, uh, and now I have, I still couldn\u2019t write one. Um, right.<\/p>\n

I use that as an experiment of what if I follow this process in an area that yeah, I\u2019m well aware of how this stuff works, but I\u2019ve never actually written software there. Like what will I learn? What I learned was how rapidly I can do something that I didn\u2019t know how to do ahead of time. Um, and just with establishing the things we\u2019re talking about before, like, you know, criteria and exit criteria and things I\u2019m concerned about, I\u2019m worried about, I have something that works. I don\u2019t know if it\u2019s the best way to solve the problem I started with. I don\u2019t know if like, uh, you know, all I know is it\u2019s maintainable code that, that works.<\/p>\n

I mean, you know, I know more than that. That\u2019s really well documented. So, so I guess my point is, um, uh, just like people still need to learn this stuff, you know, to, to sort of bring that experience forward. And that, I mean, like as much as I appreciate this technology, that\u2019s my biggest fear is just, um, the worst mistakes always start when somebody believes they\u2019re right. And people are either too intimidated or too starstruck or too odd or just too lazy to question person who for sure knows how to get something done. And the end result are things that could have been prevented that cause, um, problems. And we, we see that across like, you know, everything from tech to, uh, social environments to everywhere else, you know, and, uh, groupthink or, but, but, you know, it\u2019s, um, that\u2019s what I rely on my experts for, uh, you know, even if they don\u2019t know, even if it sounds right In fact, the more right it sounds to them, the less likely they\u2019ll take that at face value, uh, because there, there must be something wrong with this. What is it? It\u2019s a puzzle. These are, you know, engineers in many cases. And, and, um, um, so as much as I appreciate the technology, use the technology, pushing forward with the tech, like creating value with the tech, uh, there\u2019s part of it that just makes me uncomfortable.<\/p>\n

[John Burke] (50:12 \u2013 50:35)<\/p>\n

Uh, a solidly cautionary note on which to end the conversation. Yeah. Thank you, Andrew, for joining us today.  This has been fantastically interesting and lots for the enterprise strategist to chew on as they think about how things go next, uh, in their it shops. Uh, Scott, thank you so much for, for joining us today as well. And as always, thanks to everyone watching.<\/p>\n

BlueCat Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

HS136 \u2013 Sponsored Bluecat [John Burke] (0:00 \u2013 0:05) Hi,<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[90],"tags":[91],"class_list":["post-8812","post","type-post","status-publish","format-standard","hentry","category-resources","tag-resources"],"featured_image_urls":{"full":"","thumbnail":"","medium":"","medium_large":"","large":"","1536x1536":"","2048x2048":"","chromenews-featured":"","chromenews-large":"","chromenews-medium":""},"author_info":{"display_name":"Blue Cat","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/bluecat\/"},"category_info":"Resources<\/a>","tag_info":"Resources","comment_count":"0","jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/8812","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=8812"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/8812\/revisions"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=8812"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=8812"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=8812"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}