Latest / AI for HumanIT / Ep003 Hire It Like a Human: Running a Company on Agents

Transcript

Ep003 Hire It Like a Human: Running a Company on Agents · AI for HumanIT

• Bruce Cullen: Hello and welcome to AI for Humanity. I'm Bruce Cullen and if you're looking for an AI expert who's got all of the answers, you've probably clicked on the wrong podcast. I'm a product guy, a community leader, a dad, and someone who's genuinely fascinated by how AI is changing the way we work, build, and solve. And like a lot of people, I spent the last couple of years experimenting with AI and sometimes it's been fantastic, but there are other times it's been completely useless. And for me personally, the answers often lie in iteration. In each episode, we explore AI one prompt at a time. We'll talk to people who are using AI for something interesting, from AI adoption in the enterprise to AI governance. Together, we'll figure out what's useful, what's hype, and where humans still need to stay firmly in the loop. No magic, no pretending we've got it all figured out, just real conversations with real people. So grab a coffee and open your favourite prompt machine and let's learn together. This is AI for Humanity. Let's get started. So, welcome to the show, Janek. Good to have you with me. How are you? Jannik MVP: Hey, Prus, happy to be here. Yeah, I'm good. I hope you all as well. Bruce Cullen: Thank you, all is well here, thank you. So tell listeners, tell the people out there who you are, a little bit about yourself please. Jannik MVP: Yeah. Yeah. As mentioned, my name is Janik. I'm from Germany. I'm a Microsoft MVP since I think it's my fifth year in the program. I started with Intune topics. It means I worked in a world of device management, but I shifted step by step more and more in the AI area. I started to... convert to AI before we had the big hype, where OpenAI released Chagybti and all these amazing models some years ago. This means I already collected experience with traditional AI, with anomaly detection, with pattern detection, stuff like this. This means a lot of funny things. yeah, as of today, AI keeps me busy. A lot of cool projects, lot of cool initiatives where I can work on in a great community. And I always try to share as much as possible knowledge with people out in the field. Bruce Cullen: Sounds great. So you got involved in AI before it was cool, so to speak. How did that kind of come to be? What were your sort of earlier AI experiences like? Jannik MVP: Yeah, I think here's a big difference. In the past, you have different models, you have different algorithms for smaller tasks. For example, if you have data from your devices, log data, yeah, everything, also utilization of CPU or RAM, you can build... Some specialized models, some specialized ways to figure out when something went wrong, to get really early aware of some issues. But today with large language models with, GenAI, you have a general way to fulfill stuff. This means you don't have to pre-define how things works. The models are smart enough to help you with your issues. some cases are better, some cases are worser, but overall, I think you can solve more or less all. task what you have, all requirements very easily. And yeah, if you have some needs to more building customized ways, they also help you on a very easy way to build your custom application without writing line by line as it was in the past. Bruce Cullen: So that makes sense. One thing that is really interesting in that for me is so language models designed for something really specific. Now, clearly with large language models, we're moving away from that. But I think there's still something in that. There's something in a small language model that's dedicated to something really specific. Have you seen in your experience the ability to get better results out of a smaller model that's targeted to something than a general purpose larger language model? Jannik MVP: Yeah, that's a good point. What I see is that people use large language models for everything. And that's not always the right approach. I see it for automations. I see it for also AI use cases. But in some cases, the other standard way to build automation, which has a clear step-by-step approach, and not to use the large language model where you have a kind of randomness. What's the outcome if it always proceed on the same way? That's not a solution for everything. Also, when we talk about, I don't know, anomaly detection clustering, the models can do it on a very good way. And I'm sure with current generations, you have 90 % of the cases which works. But when you compare it to to the classical anomaly detection, you know, when it works with your test data, it's, you can be very sure that it also Bruce Cullen: Hmm. Jannik MVP: works in a, in a real case. When you test it for multiple cases, for multiple months, and it performs, then it will also performs in the future. And with large language models, you'd still have a kind of randomness in it. And yeah, to building classical processes, I would stick, to the classical ways and use, example, a chain AI to trigger it when you need it. It's Bruce Cullen: Hmm. Jannik MVP: always depending how you design everything, but it's not a solution for everything. Also, if it can solve everything. Bruce Cullen: This is true. So how do you know, how do you go about selecting the right language model for what you're trying to do, especially thinking of the smaller language models? Jannik MVP: Yeah. I think you have multiple aspects what you should consider here. One is pricing. One is also latency and one is the quality. In any case, you find the model which has the lowest price for the specific task for sure. The quality and the prices are a bit in balance. This means as higher the price is more the quality. It's not always true with open source models. We, we see also kind of opposite. but in general, yeah, it's the case. means if you have an application, what you build, try to go as cheap as possible that it fulfills your work, but yeah, it's not an overkill. Don't use Fable or Opus or Astra for everything. In some cases also Sonnet is. Very good. And Sonnet is very fast or the new GPT is six, one soul. Same, same here. It's fast. It's very good. Or also go to open source possibilities. The Chinese models are also very, very good and very cheap. This means not always the most expensive or the biggest one is the best decision. And then if you. Bruce Cullen: Hmm. Jannik MVP: Yeah. Select one. If you try one, then it's also kind of experimenting with it. Make your tests, build a kind of test data set, see how it performs. And if you see, only 50 % of my cases performs well, then you maybe should go and step, step up. Use, use a better model or use a different model. Try how it performs here, but not directly start with a, with overkill. Bruce Cullen: That makes sense. I feel it's a step that's often skipped. How do you get about sort of building the tests to figure out what what good enough looks like from a model when selecting one? Jannik MVP: Yeah, always depends how you can do it. think in Microsoft Foundry, have good ways to, in the end, a Foundry can make tracing for every request what you have in Foundry. You can also make an evaluation to see how good your agent performs. That's an important point. Also, when you build applications for your customers, work with experts. Sometimes you build applications where you don't understand if results are good or bad, then you have to bring experts out in the field who at least, yeah, validate it, say, Hey, that's good. This we have to optimize or Hey, it's completely useless. And you have to change something on a technical side. Or yeah, you, build your evaluation data set. which you can get provided from a customer and run it through automatic tests also here. Foundry had good possibilities to run these tests. And if something don't fill the expectation, it can automatically make optimizations on prompt on other stuff to improve the quality here as well. Bruce Cullen: It'd good to get into that at some point myself. The thing I was thinking about when you talking, so you mentioned getting effectively test data from a customer. How is this a sort of thing you need lots of, like training data? Or are we talking about sort of test data for a specific more well-defined test? Jannik MVP: Yeah. it's not a classical training data. it's more or less a kind of, proof that what you build, fulfill the expectation. for example, when you build a very specific, app, on an area where you don't have a clue, I'm an IT guy and I'm not the experts in finance or whatever. then I, I at least need some, yeah, golden questions. This means, some, some questions, what a customer provide and also some answers to understand what should be the content out of it. What does he expect? Because I am not on a level that I can approve every, every kind of, of cases. Or for sure the easier way is to give it to a customer. The customer should evaluate it, build a POC, customer evaluate it, say, Hey, that's good. That's not good. Make thumbs up, thumbs down. And also with a signal, you can also use AI to auto-approve it, your application based on the feedback of the customer. But here it's a good collaboration what you need between Requesta, the business, and the developer. Bruce Cullen: me through that a bit then. you'd have AI auto-approve its own outputs but you would still need something from the customer then it sounds like some kind of customer hitting yes or no kind of approach. Jannik MVP: Yeah, for sure. When you have some subs down from a human, you can use the signal to give it to an AI. AI approved the application, approved maybe the code, approved how it was built, generate a pull request, you review the pull request. And if it makes sense, you can push a new version, can give it back to the customer. He can evaluate it. If it's in proof, it's good. If it doesn't improve, You have to go through the same loop or jump in and think about does my architecture make sense? Does my model make sense? do I have, do I need additional connections? It's always depending what the issue is for sure. Yes. yeah. A developer as technical guy should be on top of everything. This means I should steer it. I should Bruce Cullen: Hmm. Jannik MVP: approve everything, not the AI should do it by their own. am. I'm in charge, I'm responsible for it, but the AI can do the lazy work and I can only say, make sense. Hey, go another step. This, this goes in the wrong direction. Bruce Cullen: That makes sense. At the end of the day, you can't hold an AI accountable. I'd be interested to get into what you've built with AI. Tell us about the scale and the breadth of solutions you've built with AI. Jannik MVP: Yeah. In the end, I built a lot of different solutions. I think the biggest one was used by 120,000 customers or users. It was a framework which automates IT and HR support. means users who has an issue can ask AI, AI help it. AI auto-remediate issues. AI can support in every IT process from incident change, ordering, whatever. This means it's a dramatically reduce the need of individual persons. That's one point in my, my current shop. I built also a big AI or agent orchestration suite. This means we have a full organization Bruce Cullen: Hmm. Jannik MVP: of agent. Every part of our company, sales, marketing, IT has a complete agent organization with sub agents, everything. And we can hand over as much as possible task to these agents, which yeah, partly make things autonomous, partly make things guidance guided, but it saves us unlimited time or also improves the productivity a lot. This we use in turn, but we also provide it to our customers. And beside of this, I had a lot of individual customer projects from Yeah. Document search and getting out some inputs from, from documents really on a high level quality. It's not like copilot go on a chat and search something in SharePoint and you get something out where you cannot really trust. It's really on, on a high quality level. build things where you can dynamically chat with data, what you have in every kind of databases, fabric. yeah, whatever, on every data store and agent discover the schema and, you can ask every question you have. builds dashboards and graphics out of it. I built a lot of community projects, what you can also find on, on my GitHub around Intune around also other stuff. yeah, in the end I have, hundreds of different AI projects, which I already, yeah, built. Bruce Cullen: lots there to dig into. I'm going to start with, you talked about the sort of aegentic swarm, aegentic orchestrator kind of approach that you're using internally. What's that sort of, what do you typically use it for? What are some of the things your teams are using it for internally? Jannik MVP: You mean the use cases? Bruce Cullen: Yeah, yeah. Jannik MVP: In the end, I don't know. Why? I don't know. In the end, we handle it the same as an employee. Every agent has a CV, is described, it has a profile picture, and it is onboarded like every other employee as well. It has access to the different tools, it has different skills. And in the end, you can handle it as your colleagues. This means in every area in our company, we have different use cases. Some use it to produce campaigns. Some use it to, yeah, make our whole sales process. Some use it, yeah, to get support on customer projects. This means every area has different use cases, but it's completely general purpose and all our tools are integrated. It's our central way. And we build our processes around of it. And that's the issue. What I see in a lot of companies, they use AI, they use their default processes, what they have since hundreds of years and try to bring AI Bruce Cullen: Yeah. Jannik MVP: into it. And this will fail. I think you have to change the way how you work. You bring AI in the, in the center of everything and build your processes around. And this is exactly what we did with an, yeah. general purpose way, we handle it like a normal employee. has the same permissions and the same skills what an employee has and we can delegate or can hand over complete tasks to it and it will fulfill it without interaction from a human. Bruce Cullen: Hmm, so the process, there's so much to pick into there. I'm gonna start with the process point though. So I completely agree that if you use all the processes you've had for decades, you're going to fail with AI. What specifically needs to change? Is it the way you tackle a sort of task? Is it to do with data governance? it to do with, yeah, what is it? Jannik MVP: yeah, I think for, for this question, there's a long answer, but let's, maybe split it a bit down. The first thing is, you have to work on identity level. As of now you have user identities, maybe also identities for your applications or manager identities, but you also have to think about identities for, for your agents. That's one thing to secure them. Also build a conditional access policies, whatever. to treat them as a, as a normal person. The second thing is to work on your persons in your company. This means if you're as CEO say, Hey, from today to tomorrow, we make AI and you have to shift all the work away from you. This will fail. That's not the right way. You have to do this process with your people, educate them, skill them up. Bring them into the process to figure out how you can bring out the much as much as possible value with AI together with all your employees. Let them understand what, what you shift out and let them build together with them to get here and good success. That's important thing. And also, I will not say start from scratch, but question everything what you have in the end. When you do a specific process, day by day, think about is this really needed or should I give it to an AI? I'm also sure there are a lot of Bruce Cullen: Mm. Jannik MVP: things which are complicated because you have a strange tool which has no interface, which is legacy. And sometimes you also have to go, there are dramatic steps to say, hey, we kick out this tool and search for a more modern way, which is ready for AI. This means you have here a lot of different Bruce Cullen: Mm-hmm. Jannik MVP: puzzle pieces, which has to came together. also, yeah, do not set your goal to make it from one day to another day. It's a journey and the journey has to grow. And it doesn't mean that from day one AI is the center of everything. You can start with POCs, which are targeted in specific areas, but set the end goal to Bruce Cullen: Hmm. Jannik MVP: question everything and figure out how you have to change things that AI. can do it efficiently and AI is not buying a co-pilot license. How many companies do it? AI is to work on strategy, work on your people and figure out what's the best way Bruce Cullen: Hmm Jannik MVP: to fulfill a specific task. Bruce Cullen: So that's really refreshing to hear because it's a very people-centric answer and it comes down to all the things I learnt when I was a project manager and basically change management. It's a different form of change you bring into the business which is really... refreshing on one hand, but also it's not a new set of techniques, which is kind cool. I'm going to focus on the new techniques a little bit there. As far as selecting or building even a framework, an orchestrator for a multi-agent approach, how does you even go about doing that? they off the shelf? I mean, there are off the shelf options. Did you go with one or did you end up building something yourself? Jannik MVP: there are different ways to do it. not every way is the perfect way for everyone. I can tell you what's my way. My way is to use Microsoft Foundry. I deploy the models in Foundry. Why? You have a secure way. You can be sure that you have all the, yeah, guidelines and agreements from Microsoft that not your data will be used to train models, whatever you have all Bruce Cullen: Yeah. Jannik MVP: the security stack. What Microsoft provides you, you have, you have content safety. This micro means Microsoft provides you a lot to build good application. That's one point. Then you have the possibility use SDKs what Microsoft provides you or to build everything by your own with API. I go the second way. I would not recommend it for everyone because the second way. Is the way where you have really to everything by your own. should know what you do, but you have much more freedom. How you can do things. don't face bugs and where you have to wait until Microsoft fix it. Bruce Cullen: Mm-hmm. Jannik MVP: This means that's my preferred way. But if you start with everything, work with SDKs or also look at, yeah, some agent builders like co-pilot studio, like Foundry. Also here you have your eyes. how you can build agents to make it very easy for you to start with. But as soon as you go in a more customized and more advanced mode, you're very quickly at code. And then, yeah, you Bruce Cullen: Mm. Jannik MVP: should know what you do when you build something like this. Bruce Cullen: So what does that mean for the role of stuff like, so you mentioned CoPilot Studio there, which is a great low, no-co approach to building agents, but clearly has lots of restrictions as to what you could do. What does that mean for those sorts of tools when we're talking about enterprise level scale? Jannik MVP: I think co-pilot studios, in my point of view, nothing, for, for it, it's something for end users that they can click together their use case, chat, what I often see chat with SharePoint or build some crazy, crazy flows. good to start with to, yeah, making quick, quick implementation of your use case. And for sure you are in the Microsoft universe. This means everything what you do when you have a bit of governance built around is more or less secure and it's good on enterprise level. But I think you cannot build serious AI solutions with Co-Pilot. For me, it's very close to collaboration, write better emails, get files from SharePoint for sure. can also connect your tools also here. you get, get good connections and you can, yes, build good agents also with co-pilot studio. But as soon as it gets more personalized, if you have more, yeah, requirements and quality, more requirements on, possibilities, I'm, I think you're very sure on other solution or build something by your own. Bruce Cullen: Hmm. Hmm. That makes sense. It sounds like what you're describing is macros in Excel as an alternative, you know, historic way that people from accounts or whatever would have tackled building something. And this is the Excel macros option where the alternative sounds like frown G is hire a software developer to build something. So Jannik MVP: Yeah. Bruce Cullen: is that fair to say? Jannik MVP: Yeah, maybe also to be open. I'm not the biggest fan of copilot. I'm not happy of the quality. I not use it really regular when I use it. I'm always frustrated. In the weekend I had in case I save all my, my mails and in the archive folder. And I only have a simple use case. So this mails in my archive folder in my sub folders, which are below the archive. It's yeah, with cowork, it was not possible to do it. I do the same with my own application, which I built one minute later. Everything was perfect. And yeah, sometimes question myself, what happened here? Who built this product and why? Yeah. Has it still this quality for this price tag? it brings? That's my, my open opinion. But I'm sure. for end users, it's a lot of valid cases. You cannot give them open AI or Claude or stuff like this to make research, to find documents, to write emails for this. It's a great product, but as soon as it's a bit more in detail, more advanced, I'm always frustrated. Bruce Cullen: I have similar expectations and Jannik MVP: Thank Bruce Cullen: experiences to you then, it sounds like. So tell me about building. You've talked about a high code solution, effectively on top of Foundry, where an end user can come in and effectively upload a CV for an agent. And that agent is sort of part of their team and they'll define their own use cases and hand off work to them. It sounds like if I were, you know, whatever, Bob from accounts building an accounts agent that files, you know, whatever invoices or something, they'd be able to do that themselves within a framework that you've built there. Is that fair to say? Is that how it works? Jannik MVP: you, you, sorry, can you, can you repeat it? You have frameworks to build it. what, what I can Bruce Cullen: Yeah, sure. Jannik MVP: recommend, how, how you can do it. yeah. In the end, you have a lot of, standards what you can use. I think there are skills, are MCLIs, there are MCPs, there are records possibility, and all this, kinds you can use. This means everything what you can fulfill with the standards makes things much easier. And I think with this, can fulfill 90 % of your use cases. I see some disadvantages. If you use native MCP, it consumes a lot of tokens. It consumes a lot of context. Bruce Cullen: you Jannik MVP: It's very static. This means, for example, when you have to do the same action a hundred times, it calls a hundred times the MCP call, response, context. call, response, more context. Bruce Cullen: Mmm. Jannik MVP: And that's not very efficient here. I like to build my own way who use kinds of a standard, but improve it. This means here you can tune a bit more. For sure, it's important to have good context engineering, good prompt engineering, to have clear targets, to have clear gut rails. stuff like this, should always consider and for sure building good governance around it that the agent can only do what he is expected to do. And, if not that he is at least in, in, yeah, in a, in a good, good, good shell that he cannot break out. Bruce Cullen: So talk me through governance and guardrails then when you've got end users defining what they want these agents to build. It sounds like they're doing the building within a framework you have designed. Talk us through that. Yeah. Jannik MVP: Yeah. Maybe start with a point of security. I think from security point of view, it's good to give them, for example, a sandbox that they, when they run code, it's only in a sandbox and not somewhere else on the PC or in another container. It's good to make it in a sandbox, make here a good management, which URL it can reach. It's also good to work here with. Microsoft content safety, helps you to detect stuff like prompt injection, whatever. It's good to use agent identity to give them these privileges. It's good to work with conditional access. You can also bring defender into the game to figure out what your agent can do. You should have a good purview concept that the agent can only access what they should do. This means the basic, before you start with AI overall, you should have a good governance on enterprise level for, for the classical stuff, SharePoint teams, Azure, everything around it. When this works, you can think about how can I put a level on top that I also get in governance for, for my AI, but, agents should handle very similar, like a normal, normal person. Also when you're on board someone new. You will not give him directly access to everything. This means you should make it step by step and really use case based. And also you should do similar also for agents. It's also important that you have a kind of registry to see who builds agents, what they do with it. It's important to see which tools user use, which MCP servers they have connected that you also build kind of registry for it. or gateways, Bruce Cullen: Mmm. Jannik MVP: also possibility that you don't, yeah, do things which are not expected or where it's also from a regular, perspective, it's not allowed. This means you cannot go and say, Hey, my, my personal data, let's go give it to an AI. Things like this is also, things that you have to, clarify with, with contracts, with terms of use, whatever, to be here also Bruce Cullen: Mm-hmm. Jannik MVP: on. on the legal side of things. Yeah, clean. Bruce Cullen: So in your company, so you talk about building a registry of agents which is makes total sense, speaks to all of my history around CMDB, ITAM, SAM, all those kinds of good stuff but in the context of agents. How have you implemented that given you've got users building their own things? Jannik MVP: the easiest thing is to give Microsoft more money. use agents 365 by seven, for example, it's a great product. I like it. The agents registry is good. It's not there yet, that you can say, Hey, that's the central place for all your AI activities, but it's on a, on a good way. I like it. The discovery, the registry is good. They are open. This means you can also connect. other tools like when you run something on AWS, Petrock, or if you build cloud, cloud, cloud, cloud agents, you can also connect it to this. You can block MCPs, you can block agents. This means it gives you a lot of possibilities, what you can configure here. And as said, with a complete Microsoft stack with Defender, with global secure access where you can have also. possibilities to bring governance into the game, also outside of the Microsoft universe, across all AI tools, Per View, Defender, Intune. This means the complete Microsoft stake, everything provides you good features to build a good governance across your company, across the tools, and can make sure, independent of what the user uses, that you have a solid base. Bruce Cullen: That makes sense. I've looked at agent 365 a little bit, but only a little bit. I didn't know it had any form of you know, registry kind of capabilities in it. So that's really cool to know. And what sorts of information does it sort of hold effectively on each of the agents that your end users are building? Jannik MVP: Sorry, can you repeat your question? Bruce Cullen: Yeah, what kinds of information does Agent 365 hold on the agents that it holds the registry data for? Jannik MVP: in the end, in agents 365, you have a kind of agent card. This means every agent has an owner, has a sponsor. You see the access level of your agents. You'll see the tools, what they use. You also have activity logs. If you make an open telemetry forwarding, you see really what your agents did. You have an agent ID, which you can then steer also via, via Entra. This means here a lot of great things. what, what you see, see, here, on, on the agent case. Bruce Cullen: And if you're allowing users to build their own agents, how do you force them? How do you basically make them use the Agent 365 feature sets and capabilities? Jannik MVP: In the end, you open it for end user, if a user are licensed, in the end, it's automatically discovered. It Bruce Cullen: Huh. Jannik MVP: means if they build a copilot agents, the user don't have to consider it. It's automatically discovered. Foundry also forward the information or register the agent automatically in agents 365 asset, also third party tools. There's a connection here. The only risky thing is when you build really something outside of this universe, then you have to build an agent blueprint, have to register the agent, create an agent identity, federated. this means it's a lot of more manual work. I, I think today or maybe next week I publish a YouTube video, how this works on, on code side. but on, yeah, using the standard tools, it's, automatically. discovery, this means you don't have to do anything. Bruce Cullen: even easier. We'll share that video when it exists, it sounds like very worth a watch. Jannik MVP: Yep. Bruce Cullen: I'm interested to dig into some of the architecture of what you've built. So this agentic framework that you've built, what's the general architecture of it outside of the Microsoft pieces? Is it all Microsoft? Yeah. Jannik MVP: In the end, the whole intelligence, the whole code is self-built, but everything runs on a Microsoft stack. means in most cases I use Azure App Service and bigger applications. You can also think about to use Azure Kubernetes, but it's very soon and overkill. This means for the most cases an app service is enough. Built-in container, you can very well validate if it works locally, if it runs locally. You can deploy it to an Azure for sure building a strong, strong Azure landing zone where you have policies that what you deploy is on, on the first hand, very secure use, get our pipelines, use your def, use your product environment to, validate things to then publish it to prod. but as I said, the most applications run on the app service and then building a service around like. Azure AI search, search, which is very good using application insights to have telemetry data of your agents. And whatever you need, Keywall for sure, storage account, databases. It's always individual depending on the Bruce Cullen: Hmm. Jannik MVP: app, you do. But Microsoft has a complete full, maybe, yeah, over full toolbox where you can find everything what you need. and it helps you a lot to build secure applications for sure. Everything what you do on code side, you should know what you do. You should also build good CI, way of scanings, way of testing. in the case, yeah, use Bruce Cullen: Mm-hmm. Jannik MVP: multiple AI tools to also make security checks. but on, on, Bruce Cullen: Mm-hmm. Jannik MVP: on a general purpose, Microsoft helps you a lot to shift, to ship, secure apps. Bruce Cullen: Hmm, makes total sense. What sorts of challenges did you face then given, as you mentioned, there's loads of tools in the Azure toolbox and knowing which one to select, which ones to not select. What sorts of challenges did you face when building on top of Azure? Jannik MVP: Sometimes bugs, sometimes outages, sometimes, yeah, the complete crowded setup of Foundry. think in one region you get this model and another region you get another model, but this is not available here. Then is this data center overloaded? Then you have request access to deploy. another model, then you have this all only global available, but this is not allowed when you build the app in, in Europe. this means the full, the way how foundry is structured and the cover, capacities is not as good as it could be to make my life easier. But in general, yeah, they are not, clear challenges, from, a building perspective, but, what is very important. When you build an app, you should have good Azure knowledge. Don't trust the model. You should always validate and always own the architecture of your code, of your infrastructure, not trust what the model do. And, yeah, I would not say, Hey, you, you never develop an app. Now you can do it with AI because this is wrong. It should go. Sorry. Bruce Cullen: Why? Why should you not trust the model? Jannik MVP: the models are good. The models also better than I am in development in everything, maybe better as everyone on, on this planet. But, the models sometimes think too, too, too bright. The model sometimes takes decisions, which goes in the wrong direction. This model sometimes, misunderstand you. The models sometimes have outdated information and this is the point where you should. Be on the top of it. You should understand what it do. You should understand how Azure works. You should understand which resources are here and what's the correct way to build this application. And then you can consult with it. You can say, Hey, I will do it on this way. Does it make sense? Or do you have a better idea? Hey, I run in, in a performance issues. How can we improve it? And then you get answers and then you can say, Hey, this is a good answer. Let's do it. Or hey, this runs in the Bruce Cullen: Mm-hmm. Jannik MVP: wrong direction. Let's do it on the other way. But you are, as multiple times said today, you are in charge. You should decide. But the model is a smarter part, which helps you. But you should took this ownership. Bruce Cullen: Makes total sense. I do not disagree at all. So moving on little bit, what do you hope to see from AI in the future? Thinking of like trends and future design patterns. Jannik MVP: I will not say it to improve a couple of capabilities because, every generation, what you have, you build something and then you see at least some limits where you see, that's an issue. Not really solve it. Then there came another generation. It was easy. One second it works and you spend hours before this means every generation, brings you on another level, also improve your productivity. Today I do, I don't know, six, seven, eight tasks in parallel because I can delegate it. But the biggest hope what I have is that AI lets the humans survive. I'm a bit skeptical how things move forward because maybe we are already on a point that the models are smarter than the humans. And, if a human don't understand what AI do and has not the controls to, yeah, have it in real guard trails. And, yeah, that's a bit the point where Bruce Cullen: Hmm. Jannik MVP: I'm a bit skeptical or a bit scared about how things move forward. But in general, I'm, I'm happy it improves my, my productivity 500X or maybe more. it's crazy. but I'm. Bruce Cullen: Hmm. Jannik MVP: I'm a bit scary how things move forward. Bruce Cullen: I share your skepticism. I was reading this morning about OpenAI models, the frontier models, like Astra effectively, you know, going outside of its guardrails. I was interested to get your take on that. Is this something that you think will get ironed out or is this something you think we should really be worried about? Jannik MVP: I think on the current face, I'm not so super worried. The models are still limited, but they are super smart. And I think with all of this, press releases with, open AI hacks, hacking face and this company hacks this Bruce Cullen: See you Jannik MVP: and then Google will also hack someone. Then they also bring out some news. I'm not sure if this is always, out of, randomness. Or if it's really an expected behavior because you see it goes to the press of my, wow, crazy model. I need it to Bruce Cullen: It's PR. Jannik MVP: fix my application. I'm not sure if this is also used for promotions, but for sure you see Bruce Cullen: Yes. Yes. Jannik MVP: when you read how they work, they are so smart that they, yeah, that they cheat persons. Bruce Cullen: Hmm. Jannik MVP: On this case, it's quite crazy. if the capabilities or the knowledge will grow further and further, we are in a case that we don't understand what the model do and the model do some bad Bruce Cullen: Hmm. Jannik MVP: stuff. And that's, yeah, a bit crazy. Bruce Cullen: I share that fear. share that fear. So I was also interested to ask, are there any guests you think I should get on the show in the future? Is there any person you think I should speak to? Jannik MVP: that's a good question. depending, yeah, what topic you, you, want to, fulfill, but I think there's a big, MVP crowd out in the field also in copilot topics, which are much near on the adaption side of things, which can also give you real cases. what, yeah, is the hardest part to bring AI to the end users. I think there are a lot of people also out in the field who works on security on all this, these things. And that's also a very interesting part. What you can do to secure on one hand, the company, that the company is not, attack or in, in, in the front of, attack wire AI. I think also, yeah, this we will see more often that, companies get hacked from. from AI attackers, but also how you Bruce Cullen: Hmm. Jannik MVP: can secure your agents, your AI to make sure that, yeah, they are on a secure environment and in a secure phase. Bruce Cullen: Hmm, hmm, makes sense, makes sense, sounds good to me. And then I think my final question is, keep you an eye on the time. What's the one thing that existed before AI that we should either bring back or we should not let disappear? Jannik MVP: I think a bit more the humidity, I think when you read posts, when you read mails, when you read teams messages, just rarely see some spelling issues. Everything is perfect. You have the same wording, everything. this is a bit of the point where I see, it was cool to see a boring post because someone did it by his own and It's not a blog post, which is 100 hours of work or it goes in in a death, was never possible, but in the end, no word is written by, by himself. For sure. I also use AI for, for a lot of things. also use AI to formulate emails or, to make my blogs better. But I think this is something I miss. also the full speed, what you have today. Bruce Cullen: you Jannik MVP: This means everyone is in. Go to next, go to next, go to next, make more projects, make more, things. And it's not so, Bruce Cullen: Hmm. Jannik MVP: Hey, you have a project, what you will do today in one hour. was weeks and it was full teams who work on it. And now it's lazy. No one Bruce Cullen: Yes. Jannik MVP: recognized what it is and how much effort it was before. And this is a bit, Bruce Cullen: Hmm. Jannik MVP: yeah. And change of a world, which is not Bruce Cullen: Definitely. Jannik MVP: so nice. Bruce Cullen: Yeah, I remember the first products I built as a product manager defining what it should look like. And I think I said this on the last episode I recorded. We called them paint frames because I designed them in Microsoft Paint and it was a screenshot with lines and squiggles and stuff drawn on them, super crude. And I built Flow, I'm using air quotes, using PowerPoint and it worked, you know, to communicate the flow, but it was ugly. And then we had real- real designers, UX designers that did a far better job than anything I can cook up with AI, but it did take weeks and months versus today where it's a morning's work and I've got something that's 80 % of the way there. It's not as good, but it's close enough. Yeah, I agree. The craft is Jannik MVP: Yeah, Bruce Cullen: being lost. Jannik MVP: that's, that's the full, yeah, the individuals who have their style, who have the way how they do things. think this is really lost today and it's a bit, yeah, and development, which I not so Bruce Cullen: So I quite like the podcast format. Jannik MVP: like so much. Bruce Cullen: Yeah, yeah definitely that's definitely true. Everything kind of looks the same, it's got that same AI style. Yeah that's why I quite like the podcast format. I mean obviously you can replace podcasts with AI but there is a human component to them. They are time consuming to record and edit and all of those things but the content is real. So that's one thing I'm going to try and hold on to. Jannik MVP: Definitely. Bruce Cullen: So that brings us to the end of this episode of AI for Humanity. I would say if there's one thing I've learned on this journey is that none of us have all of the answers. AI is evolving so fast and the best way to keep up with it is not by pretending that we're all experts, it's by staying curious. You can connect with the show on LinkedIn or with me personally, just search for AI for Humanity or me, Bruce Cullen. As the show evolves, I'm planning to introduce round table discussions to think more humans there, effectively, no bots allowed. everyone has something to contribute to the conversation. I want to hear what's really happening with AI in the real world, not just the scary headlines that we've been discussing in this episode, but also the lessons, the successes, the mistakes, and ultimately the discoveries from those people who are using it every day. I should also mention that AI for Humanity is proudly sponsored by Workspace DNA. This is the company that I work for. At Workspace DNA, we help organisations understand the ripple effect of change before they hit deploy. So whether you're patching a single application or validating complex multi-application workflows, our platform helps you identify issues early on, reduce risk and avoid those dreaded production surprises. And when everything is tested and validated and good to go, we can help you deploy those changes with confidence too. So that's it for this episode. I don't have a guest planned for my next episode, so honestly if there are any suggestions out there please do send them my way. There's much to learn. I'm thinking about an episode on the Chinese models in particular because I'm really intrigued by them. I feel there's lots to learn in what they can do and especially thinking about, Yannick you were mentioning earlier, how you evaluate a model. Cost was one of the levers as well as sort of quality. I'd be really interested to hear how the Chinese models stack up. So all that's left to say is if you enjoyed this episode, please subscribe, leave a review and share it with somebody who's trying to figure out AI just like the rest of us. And until then, stay curious, stay human and I'll see you next time, one prompt at a time. And thank you for joining me, Jannik Jannik MVP: Thanks.