Latest / What the Hoot! / Modernizing Mission-Critical Messaging Without Disruption | Marcel Bil

Transcript

Modernizing Mission-Critical Messaging Without Disruption | Marcel Bil · What the Hoot!

• 0:10
  SPEAKER_00: Welcome, welcome, welcome everyone to What the Hood, the podcast where we get behind the technology powering the enterprise and talk with people that actually build it, that actually run it, that actually troubleshoot it and keep it moving.
• 0:22
  SPEAKER_00: I'm your host, Michael Jo Kay.
• 0:24
  SPEAKER_00: Today we're diving into a part of enterprise IT that most people never think about when it's working, but everyone notices when it isn't.
• 0:32
  SPEAKER_00: Integration and middleware.
• 0:34
  SPEAKER_00: Joining me today is Marcel Bill, an integration and middleware specialist with nearly two decades of experience across enterprise infrastructure, security, IBM Q, managed file transfer, integration platforms, and most recently, Apache, ActiveMQ, and Artemis.
• 0:47
  SPEAKER_00: Marcel has worked inside some incredibly complex enterprise and government environments.
• 0:52
  SPEAKER_00: So today we're going to get his perspective on what it really takes to modernize mission-critical infrastructure without disrupting the services that depend on it.
• 1:00
  SPEAKER_00: We'll talk about the continued importance of IBM Q, the coexistence of legacy middleware, messaging, and newer technologies like Kafka, the unique challenges facing government agencies and invisibility, and where AI and automation take or may take middleware operations next.
• 1:17
  SPEAKER_00: And Marcel, I think it's the best place to start is at the beginning, really.
• 1:21
  SPEAKER_00: So I would love to, you know, for you to walk us through your technical journey.
• 1:24
  SPEAKER_00: How did you first end up in the world of integration and enterprise middleware?
• 1:28
  SPEAKER_01: Well, uh, Mike, uh, thank you uh for your introduction.
• 1:32
  SPEAKER_01: Now let's go back uh about 20 years ago.
• 1:34
  SPEAKER_01: And not uh I didn't start immediately with integration middleware.
• 1:38
  SPEAKER_01: First, I had to build a foundation of infrastructure, uh, platform administration support and operations.
• 1:43
  SPEAKER_01: In the meantime, alongside my work, I pursued and obtained a bachelor's degree in IT infrastructure design and security.
• 1:48
  SPEAKER_01: Here I learned the language that architects uh speak, together with gaining more knowledge about the infrastructure, networking, and security as well.
• 1:56
  SPEAKER_01: About the step to middleware, therefore I have to go back to uh around 2011.
• 2:01
  SPEAKER_01: I worked at the second largest uh banks with uh within the Netherlands, a very large team of about uh 20 people.
• 2:08
  SPEAKER_01: And I worked at a uh platform administration team.
• 2:11
  SPEAKER_01: Well, the platform uh had to service various agents and tooling, including middleware, to name a few of those projects uh IBMMQ, Student Commerce, Connect Direct, also known as Node Device Manager, NEM on COS, and uh the Tifli monitoring agent to uh monitor IBM MQ on the platform.
• 2:29
  SPEAKER_01: My colleagues found these products abstract and had the odd ones out, and yeah, they didn't like to I like them, they were a bit strange to them.
• 2:39
  SPEAKER_01: So uh yeah, but I on the other hand, I found them uh yeah rather interesting.
• 2:44
  SPEAKER_01: So I became a product specialist on those uh products within the platform management and uh yeah, were responsible for the for the maintenance operation of those agents, upgrade mechanisms and documentation uh yeah of these uh yeah middleware tooling.
• 2:58
  SPEAKER_01: It's only within the scope of uh platform administration.
• 3:01
  SPEAKER_01: So yes, I get them to work, but and yeah, didn't really learn uh what we what they really were doing.
• 3:07
  SPEAKER_01: And so I didn't know about messaging uh back then, but that changed in 2013.
• 3:13
  SPEAKER_01: I transferred uh to the middleware team of that bank.
• 3:16
  SPEAKER_01: There I really learned IBM MQ, version 7 at uh at the time, and also a little bit of connector for uh managed file transfer between Sterling File Base, right?
• 3:26
  SPEAKER_01: So really in the art of integration within a bank.
• 3:29
  SPEAKER_01: Super uh fun uh to be in that team.
• 3:32
  SPEAKER_01: Uh I've learned a lot uh also about the application landscape, uh, what's really happening within uh within the bank, and because you have to work with uh with middleware and all those uh and other platforms it's what it's running on.
• 3:45
  SPEAKER_01: So that will be the C US mainframe, uh yeah, Linux, AIX for me, uh HP non-stop, uh quite exotic as well.
• 3:53
  SPEAKER_01: Um yeah, there really was, but it was uh nice to get to know of.
• 3:58
  SPEAKER_01: Yeah, I quickly broadened uh my knowledge uh there.
• 4:00
  SPEAKER_01: And from then on, middleware and integration became my uh my primary focus.
• 4:05
  SPEAKER_01: Uh I've worked for several other enterprises with IBM Q.
• 4:09
  SPEAKER_01: For example, I helped automating it at uh another large bank within uh Netherlands.
• 4:14
  SPEAKER_01: Uh also worked on IIB and the integration booths, uh now known as uh Ace.
• 4:19
  SPEAKER_01: Yeah, and also other uh managed well transfer products, X-ray uh B2BI and three different organizations.
• 4:25
  SPEAKER_01: So and uh yeah, now I'm uh self-employed.
• 4:28
  SPEAKER_01: Yeah, I've been a technical lead on IBM Q for a few years and responsible for the technical implementation of the whole MQ solution.
• 4:37
  SPEAKER_01: So that services business, constantly uh constant uh improving the service uh in conjunction with architects, in conjunction with all the DevOps teams, uh, what they really uh what they were missing, introducing new functionality, etc.
• 4:51
  SPEAKER_01: Now uh yeah, as a freelancer, I offer my knowledge about all the things I just mentioned.
• 4:55
  SPEAKER_01: Uh so my knowledge of integration and middleware together with uh yeah, my passion for infrastructure, security, and package, and I'm helping organizations to move forward and uh to let the uh the middleware perform better, uh, improve security, uh, high availability, etc.
• 5:12
  SPEAKER_01: I am also broading my knowledge now with uh other MQ techniques and techniques, more in the open source uh world.
• 5:19
  SPEAKER_01: And like uh Mike mentioned, uh patchy active MQ, very active MQ Artemis.
• 5:24
  SPEAKER_01: The visited MQ Simage was very uh yeah enlightening, enlightened got to know of uh a different and several other MQ products, uh but also supporting products like uh like MASH IQ are really great.
• 5:39
  SPEAKER_01: So yeah, to support the organizations, uh yeah, I also professionalized my home lab.
• 5:45
  SPEAKER_01: I've got three uh really uh data center servers here and uh with an hypervisor with different types of storage to different products, so I can uh play out uh scenarios for uh for businesses.
• 5:57
  SPEAKER_00: Yeah, you you've you've had a very interesting vantage point because you've seen the space from infrastructure to security all the way through actually leading middleware environments.
• 6:06
  SPEAKER_00: So, with those nearly two decades, 19 years of that perspective, I'm curious how much you know you think the architecture itself has changed.
• 6:14
  SPEAKER_00: So, over the course of your career, you know, what is what is the biggest shift that you've seen in how large organizations approach messaging architectures?
• 6:23
  SPEAKER_01: Uh well, in many organizations, I found developments regarding MQ to be very stable and will always be a place uh for point-to-point messaging.
• 6:32
  SPEAKER_01: Uh, I did have worked extensively on standardization and automation, and security aspects of uh aspects have become more important.
• 6:39
  SPEAKER_01: Not really a shift, but yeah, it uh gets had more attention.
• 6:43
  SPEAKER_01: Uh event-driven architecture has uh became more prominent, right?
• 6:48
  SPEAKER_01: And also streaming.
• 6:49
  SPEAKER_01: Um but it's a different form of match messaging, a different interaction pattern.
• 6:54
  SPEAKER_01: So uh yeah, it's it has other business requirements, and uh yeah, both uh forms of messaging uh just perform really well.
• 7:02
  SPEAKER_01: Uh really think about the shift.
• 7:04
  SPEAKER_01: I think uh the biggest change in this area is uh more to a synchronous model instead of an asynchronous model.
• 7:11
  SPEAKER_01: Uh so uh yeah, for example, using REST or an API gateway, uh yeah, synchronous uh uh interaction uh yeah in the front, but nevertheless, the back end can still contain uh asynchronous elements.
• 7:22
  SPEAKER_01: So we'll uh still be with uh with queuing, for example.
• 7:26
  SPEAKER_00: I agree, and I think that that distinction is very important.
• 7:28
  SPEAKER_00: It isn't necessarily that one architecture comes along and replaces everything before it, right?
• 7:34
  SPEAKER_00: Instead, organizations keep adding new patterns, and now they have to make all those worlds work together.
• 7:40
  SPEAKER_00: So, like I imagine that complexity is part of what keeps the work interesting.
• 7:46
  SPEAKER_00: So when you look at your current focus, what keeps you most passionate about solving complex infrastructure problems every day?
• 7:55
  SPEAKER_01: Yeah, that's a good question.
• 7:57
  SPEAKER_01: Well, uh, I start with a great quote from one of the services, uh, service managers I worked uh for.
• 8:02
  SPEAKER_01: And this that is the silence is or applause because integration is like water.
• 8:07
  SPEAKER_01: Just like many other elements in infrastructure, every uh one simple uh simply assumes it works.
• 8:13
  SPEAKER_01: As long as the business is silent, we are doing good.
• 8:16
  SPEAKER_01: So I find constantly uh improving metaverse solutions while minimizing their impact to be one of the most enjoyable challenges.
• 8:23
  SPEAKER_00: Yeah, I I I really love that phrase, so I'll uh we'll come back.
• 8:26
  SPEAKER_00: Public sector and government agency landscapes are notorious for having unique constraints.
• 8:32
  SPEAKER_00: What makes integration and government distinct from private sector enterprises?
• 8:37
  SPEAKER_01: Uh challenge uh government uh uh especially is facing uh yeah is uh sovereignty.
• 8:43
  SPEAKER_01: It's really uh it's really a hot uh topic at the moment.
• 8:46
  SPEAKER_01: Questions like where do we store our data?
• 8:48
  SPEAKER_01: Uh are we owner of the data?
• 8:50
  SPEAKER_01: Uh is it uh easy to migrate with between uh suppliers if necessary?
• 8:54
  SPEAKER_01: So uh for example, with uh US vendors, nothing wrong with the US, of course, but the uh GTP, no, but uh the GP GDPR, for example, or the Cloud Act, uh yeah, really have some risk about that uh the data ownership.
• 9:09
  SPEAKER_01: And also from the European side, to now have the NIS uh too, uh and IS too, and uh Dora as well for the financial sector to audit uh organizations on the uh yeah on those topics.
• 9:22
  SPEAKER_01: We really have to think uh about it.
• 9:24
  SPEAKER_01: And also another thing at the governance right now is that open source is really preferred preferred.
• 9:30
  SPEAKER_01: So we really have to uh think uh about open source solutions as well.
• 9:34
  SPEAKER_00: Right, we're seeing a big push uh in every direction and every industry for open source, but so there are really several forces pulling in different directions.
• 9:43
  SPEAKER_00: You may want to modernize, but you're also dealing with systems that have been there for years.
• 9:48
  SPEAKER_00: Sovereignty require requirements, uh supplier considerations and governance that creates a very different modernization equation.
• 9:56
  SPEAKER_00: So, how do you balance the pressure to adopt modern real-time architectures with strict security, um compliance, and even you know, legacy continuity, um, continuity, excuse me, requirements government agencies face?
• 10:12
  SPEAKER_01: Um I can have uh some uh examples from uh from practice.
• 10:18
  SPEAKER_01: Uh one of to start with at my last employer, where I was a technical lead on the MQ solution.
• 10:24
  SPEAKER_01: I faced the challenge of implementing improvements, but yeah, struggled with the extensive division uh between the the uh the different disciplines within the organization.
• 10:33
  SPEAKER_01: So then the reason uh this resulted in many barriers and challenges uh in certain collaborations because it's very divided.
• 10:40
  SPEAKER_01: And yeah, one of the yeah major challenges in this uh yeah is uh it's it's the uh just like networking, but it's the darker part of the organization, almost everything uses uh everyone uses it.
• 10:53
  SPEAKER_01: So if you want to improve, if you want to change, uh yeah, you have to uh collaborate with almost everyone, and uh on a functional level but also on a technical level, that makes it very hard.
• 11:05
  SPEAKER_01: Yeah, is it the pressure?
• 11:06
  SPEAKER_01: No, not uh I do not experience uh any direct pressure, yeah, because uh especially government works with long-term uh contracts, up to seven years sometimes, yeah.
• 11:17
  SPEAKER_01: Uh and also so broad contracts can be extended, meaning the old situation remains in place.
• 11:22
  SPEAKER_01: Yeah, and due to the obligation of European tendering, a change is only possible once every few years.
• 11:28
  SPEAKER_01: And those are the moments when the requirements uh are tested and uh yeah to check whether new functionality is needed or more secure solution.
• 11:36
  SPEAKER_01: Another assignment I worked for one of the largest conf uh government organizations in the Netherlands, that was the Dutcher Tax Administration.
• 11:44
  SPEAKER_01: This organization also has a very challenging landscape.
• 11:47
  SPEAKER_01: The IT organization is enormous, and the separation of roles and governance is also one of the major challenges there.
• 11:52
  SPEAKER_01: Uh, as a technical lead at uh my former employee, I designed I designed a solution based on hardware appliances.
• 12:01
  SPEAKER_01: And uh yeah, that really worked out for that uh particular organization because it was a uh collectively used environment, like often based on an I hypervisor or Kubernetes uh like uh environment.
• 12:15
  SPEAKER_01: Yeah, but uh that environment really had uh some uh yeah noisy neighbors.
• 12:20
  SPEAKER_01: And uh if there's anything middleware doesn't like, then it's noisy neighbors.
• 12:24
  SPEAKER_01: It's not really CPU intensive or memory intensive, but really I/O intensive.
• 12:29
  SPEAKER_01: So if the collectively the storage or the network is used too uh too often, then uh you get problems in a way within your middleware.
• 12:38
  SPEAKER_01: You can also have a snowball effect because if the middleware uh yeah is running uh uh slowly, uh it might uh yeah, fight errors on the application level.
• 12:47
  SPEAKER_01: Uh when there are errors, there's more logging.
• 12:49
  SPEAKER_01: When there's more logging, more storage and network will be required, and as a result, uh the middleware is uh starting to go uh even more sluggishly if you uh if you don't uh watch out.
• 12:59
  SPEAKER_01: So yeah, that's one of the reasons I advise to use the appliance because then you've got everything in a box, no more, no more noise enables, and it that really worked well uh for the organization uh while with using the total cost of ownership uh of the solution.
• 13:15
  SPEAKER_01: And yeah, it did me good to find the same MQ solution based uh on those appliances uh at the other organization as well.
• 13:23
  SPEAKER_01: Uh also uh I consider my current client to be quite modern.
• 13:27
  SPEAKER_01: It has very uh little legacy, uh, has a yeah, nice Kubernetes uh private cloud uh environment with a CICD approach, and uh yeah, which is also used for middleware.
• 13:39
  SPEAKER_01: They work intensively with the cloud principles.
• 13:42
  SPEAKER_01: So uh yeah, the MQ solution as well, it's a pod container-based uh HI solution and up to date.
• 13:47
  SPEAKER_01: Uh the services are uh there we can do the messaging through the service mesh.
• 13:52
  SPEAKER_01: Uh, services uh yeah, the most of the services are running in uh within the Kubernetes environment, so very little accuracy, and uh the integration is handled via Apache Camel, for example.
• 14:02
  SPEAKER_01: Also, the uh LCM on the FedNet driven products is also uh to date.
• 14:06
  SPEAKER_01: So, yeah, not really Russia uh there, but the difference uh uh between government uh the ministerias, uh the the they're not all alike.
• 14:17
  SPEAKER_00: Well, I think that's the most interesting part there.
• 14:19
  SPEAKER_00: It for me it's it's extremely interesting that they're all is that you know modernization clearly doesn't have one definition.
• 14:26
  SPEAKER_00: In one environment, the right answer might be an appliance that gives you greater isolation and reliability somewhere else, it might be containers, K8s, you know, Kubernetes uh CICD and Apache Camel, but I think the common denominator seems to be choosing what works without jeopardizing the service.
• 14:43
  SPEAKER_00: So, you know, in an environment where systems can't can't simply be ripped and replaced, what does a successful modernization with Dell disruption strategy really look like for a major agency?
• 14:56
  SPEAKER_01: Yeah, I've had the opportunity to execute the migration on several occasions.
• 15:00
  SPEAKER_01: And uh yeah, by using load balancer or a proxy within a service mesh, and you can uh migrate relatively easily without major impact.
• 15:09
  SPEAKER_01: It's really uh something to have.
• 15:11
  SPEAKER_01: In uh one case this was already in place, so then you have just have to uh to alter the uh the the pool members or the backside, but uh there also was a customer there uh there I had to introduce this technique.
• 15:22
  SPEAKER_01: Yeah, this allowed the modernized facial to be set up parallel to the accessing solution, and then uh yeah, in this case it's important, of course, that testing is uh to and complete.
• 15:33
  SPEAKER_01: So once you uh turn on the switch and to migrate to use the new solution, everything runs fine.
• 15:39
  SPEAKER_01: Another uh also positive, uh, if it's not fine, then you can easily roll back to the old solution.
• 15:44
  SPEAKER_01: Yeah.
• 15:44
  SPEAKER_01: So if approved writing traffic to the new environment via the load balancer or proxy is relatively simple, and at the same time, yeah, rollback is in place.
• 15:52
  SPEAKER_01: Once communication has resumed, only the remaining data needs to be transferred to the old environment to ensure there is no data loss.
• 15:58
  SPEAKER_01: And this has already proven successful a few times.
• 16:01
  SPEAKER_01: Case of migration to another product standardization is uh very important.
• 16:05
  SPEAKER_01: And the same use of protocols, same yeah, message standard, for example.
• 16:10
  SPEAKER_00: Yeah, no, I I think that that ability to run old and new in parallel, you know, test uh thoroughly to redirect traffic and still retain a rollback path really emphasizes how much these teams are trying to remove risk from that transition.
• 16:25
  SPEAKER_00: And there's always a reason.
• 16:26
  SPEAKER_00: They have to be careful because the the consequence of failure goes well beyond the middleware team.
• 16:31
  SPEAKER_00: So it makes me want to ask when when critical government data flow or interagency message stalls, what what are the broader operation risks behind just you know an IT ticket?
• 16:43
  SPEAKER_01: Yeah, that's a good one.
• 16:45
  SPEAKER_01: Always remember the government exists for the citizens.
• 16:48
  SPEAKER_01: So if intercommunication between government uh government uh parties fails, the service for the country's uh inhabitants will not function or they'll not function fully.
• 16:56
  SPEAKER_01: Right.
• 16:56
  SPEAKER_01: This this causes inconvenience, uh inconvenience and often receive immediate the uh media attention as well.
• 17:04
  SPEAKER_01: Yeah, another type of risk you that want to prevent.
• 17:06
  SPEAKER_01: I can provide some examples if you like, from uh with another uh yeah, government.
• 17:11
  SPEAKER_01: Well, think about the retrieve retrieval and updating of citizen data from the civil registry.
• 17:16
  SPEAKER_01: Yeah, that's a really common uh service.
• 17:18
  SPEAKER_01: Uh the decision regarding a tax assessment, if you have to pay or not, whether or not a benefit or allowance is paid out, uh the payments or uh yeah, the debit orders to the bank.
• 17:29
  SPEAKER_01: Yeah, I'm sure the list goes on.
• 17:30
  SPEAKER_01: Yeah, yeah, the collection of possible traffic violations, also uh one, and the print commands for traffic fine, retrieval of a diploma from the national diploma registry, the seizing of uh uh or issuing of drivering uh driving licenses, the suspension of vehicles, so along uh yeah, I mean not to mention the exchange of data with and uh between the police and military is also oh boy, very important one.
• 17:53
  SPEAKER_00: Yeah, yeah, things I think people take for granted, right?
• 17:56
  SPEAKER_00: Exactly.
• 17:57
  SPEAKER_00: And these are these and these examples, they they they make the stakes much more tangible, at least for me.
• 18:02
  SPEAKER_00: We're talking about citizen data, taxes, tickets, being able to drive, right?
• 18:06
  SPEAKER_00: Uh, we're not just talking about you know q depth or or an error code anymore.
• 18:10
  SPEAKER_00: We're talking about whether someone receives a benefit, they're getting paid, or you know, whether the the government uh can exchange the information and and reliability at that level helps explain why some technologies remain so deeply embedded.
• 18:23
  SPEAKER_00: I I think that IBM Q remains a massive core backbone across public sector.
• 18:28
  SPEAKER_00: IT infrastructure.
• 18:29
  SPEAKER_00: I you know, I why do you think that MQ has maintained such staying power in these mission-critical environments?
• 18:38
  SPEAKER_01: All the MQ products I've worked with, uh yeah, IBM MQ has the longest lifespan.
• 18:42
  SPEAKER_01: It has undergone the necessary evolution to reach its current position.
• 18:46
  SPEAKER_01: It's it's constantly developing, uh, but it has a really uh yeah robust uh yeah foundation.
• 18:52
  SPEAKER_01: It just works, ranges between places where the highest reliability is required.
• 18:56
  SPEAKER_00: Love it, love it.
• 18:57
  SPEAKER_00: Thank you.
• 18:58
  SPEAKER_00: Um I I think that that reliability explains why organizations aren't simply throwing MQ when Kafka or any new technology enters the picture, more often they are they they end up running both, right?
• 19:10
  SPEAKER_00: Which introduces an entirely new operational challenge.
• 19:14
  SPEAKER_00: So, what are what are the biggest operational headaches or um or even friction points that teams face when they're managing IBM Q at scale alongside newer streaming platforms like Apache Kafka?
• 19:26
  SPEAKER_01: I think uh that maintaining standards is one of the biggest challenges.
• 19:29
  SPEAKER_01: The message standards are often present within an organization, uh, for example, the other information conformed to XSD or to ensure the routing.
• 19:38
  SPEAKER_01: Message uh routing works differently, uh differently for events, uh, for HCraft case users, for example.
• 19:43
  SPEAKER_01: But even here, you need uh the standards to ensure an ambiguous uh communication between services.
• 19:49
  SPEAKER_01: And messages can also trigger an event or vice versa.
• 19:52
  SPEAKER_01: So yeah, you are then talking about the chain.
• 19:54
  SPEAKER_01: How is uh data classification handled in that case?
• 19:58
  SPEAKER_01: Because yeah, and if you keep uh checking end-to-end uh messaging, uh then every component uh needs to have the same level to handle the data.
• 20:08
  SPEAKER_01: Uh one type of messaging uh yeah contains more sensitive data than the other.
• 20:12
  SPEAKER_01: So how do we ensure that the same level of security measures is guaranteed uh yeah within the same chain?
• 20:17
  SPEAKER_01: And how do we manage these types of flows?
• 20:19
  SPEAKER_01: Uh, do both products uh fall under the same team that performs the management?
• 20:23
  SPEAKER_01: What about the access rights?
• 20:25
  SPEAKER_01: Who's allowed to do what?
• 20:27
  SPEAKER_01: I've uh yeah, I know an organization, then it's really divided, and even the MQ team is divided between mainframe and uh the rest, right?
• 20:35
  SPEAKER_01: And Kafka is very distributed, indeed, indeed.
• 20:40
  SPEAKER_01: Literally, literally, yeah.
• 20:42
  SPEAKER_01: They also found organizations that just uh say yeah, it's all integration.
• 20:46
  SPEAKER_01: So yeah, there's one team that manages uh it all.
• 20:49
  SPEAKER_00: Right.
• 20:50
  SPEAKER_00: Well, and and once you know a transaction crosses those boundaries, ownerships can become just as complicated as the technology.
• 20:57
  SPEAKER_00: You may have different teams, different access models, different standards, but the application team just knows that something isn't working.
• 21:06
  SPEAKER_00: So in complex environments, applications teams often see an issue, but middleware teams have to find that root cause.
• 21:14
  SPEAKER_00: You have to you have to bridge it, but how do you bridge that troubleshooting gap without creating ticket bottlenecks?
• 21:21
  SPEAKER_01: Yeah, you really see it everywhere in the world.
• 21:24
  SPEAKER_01: Yeah, always the fault of MQ.
• 21:26
  SPEAKER_01: And in fact, it's uh it's often different.
• 21:29
  SPEAKER_00: The blame sponge, you know.
• 21:31
  SPEAKER_01: Yeah, that's yeah, I've noticed uh at the latest uh elastic conversations I've worked for that there is uh more openness regarding available logging, uh logging.
• 21:39
  SPEAKER_01: So, for example, an Elastic or Splunk, so you can confidently ask the team what have you figured out yourself?
• 21:45
  SPEAKER_01: Do you have an error code for me?
• 21:46
  SPEAKER_01: Uh, did you uh uh yeah check uh the stack phase yourself?
• 21:50
  SPEAKER_01: In the case of IBM MQ, do you have an MQ reason code for me?
• 21:53
  SPEAKER_01: And uh yeah, together with making documentation available on uh, for example, conference pages, uh you can use uh that to use.
• 22:00
  SPEAKER_01: To educate the application teams a bit, and also with the Advent of AI, it is quite possible that an application team has already done a necessary uh preliminary research before the issue reaches the integration team.
• 22:11
  SPEAKER_01: Uh, but honest uh organization I work for, you seem to have a mix of the forward uh thinking uh teams, it really works that way, and the teams that are absolutely not uh they don't handle change.
• 22:22
  SPEAKER_01: Well, sometimes you get the nickname dinosaurs because yeah, yeah, like they never learn.
• 22:28
  SPEAKER_01: Yeah, you always have to play somewhere.
• 22:30
  SPEAKER_01: So really educating the teams is uh is also a part of the job if you are in an integration and middleware team.
• 22:37
  SPEAKER_00: You know, I I I I suspect that a lot of middleware engineers listening can relate to that.
• 22:41
  SPEAKER_00: It's always MQ's fault.
• 22:43
  SPEAKER_00: But what are you describing is really about giving application teams enough information to the first level of investigation themselves, and then the next challenge is making sure everyone is looking at the same transactions or the same transaction as it moves through the environment.
• 22:59
  SPEAKER_00: So, how are government integration teams maintaining that end-to-end visibility and transaction continue uh continuity when messages traverse mainframe distributed in cloud environments?
• 23:10
  SPEAKER_01: I've seen uh different solutions within different government parties.
• 23:13
  SPEAKER_01: Uh yeah, at one organization uh a custom logger uh logger was built.
• 23:19
  SPEAKER_01: It uh creates unique organization message ID within a custom header, and those are subsequently logged and forwarded uh to Elastic in this case, and then you can follow it uh yeah throughout the chain.
• 23:30
  SPEAKER_01: Yeah, the custom ID is inherited at every step, so yeah, that makes traceability uh yeah one possibility, but it's custom built.
• 23:38
  SPEAKER_01: Uh now uh the use of a more standard approach, yeah, like the the W3C uh trace context is researched.
• 23:45
  SPEAKER_01: Um, so yeah, really looking forward when that will be in place.
• 23:50
  SPEAKER_01: And another organization uh have also observed a focus on uh open hotel, uh open telemetry, which works really fine with IBM Q, uh, I know.
• 24:00
  SPEAKER_01: Uh so that's a technique.
• 24:02
  SPEAKER_01: And uh also another one had Dynatrace in place.
• 24:05
  SPEAKER_01: And the Dynatice agent also has some uh measures that go across platforms to follow the uh the messages.
• 24:12
  SPEAKER_00: I think that that evolution from you know custom correlation IDs into standards like like you said, W3C trace context and and open telemetry, it's very fascinating because it creates much more context around what is actually happening across the transaction.
• 24:27
  SPEAKER_00: And once organizations have that kind of telemetry available, the obviously the obvious next question is what they can automate with it, right?
• 24:36
  SPEAKER_00: Looking over the next few years, do you see AI or a genetic automation impacting the way middleware integration environments are operated?
• 24:46
  SPEAKER_01: I see that various uh middle defenders, such as IBM, but also X-Vay, who I work a lot with, are focusing on the usage of uh AI.
• 24:53
  SPEAKER_01: And this based on what I saw during the demonstrations, it's uh yeah, primary uh primary adds an abstraction layer that queries the underlying database or retrieves uh information from the API of the of the product.
• 25:05
  SPEAKER_01: And currently it seems to be quite an art, yeah, an art to ask the right questions and provide the right context before before the answer that gives for uh proves uh useful.
• 25:15
  SPEAKER_01: So uh yeah, what I've seen, uh yeah, I can do it uh yeah a lot quicker with uh yeah when uh consuming less resources because you have to up and go uh yeah and bounce a bit.
• 25:28
  SPEAKER_01: Um at the yeah, so at the moment it only creates extra overhead against what uh costs, because if it's token-based, then you're uh losing a lot of tokens uh with uh yeah, the incorrect questions.
• 25:41
  SPEAKER_01: But that's uh what I've seen now.
• 25:43
  SPEAKER_01: Uh you are asking me to say something over the uh the following years.
• 25:48
  SPEAKER_01: I'm sure it's it's yeah, development uh is is that there's a lot of development on AI.
• 25:53
  SPEAKER_01: Yeah, I'm sure yeah, I think it gets uh smarter.
• 25:56
  SPEAKER_01: So in the future and truly large in a large environments, I think first line support in particular will benefit from this, or the end customer that can already ask questions regarding, for example, a lost many message.
• 26:07
  SPEAKER_01: So uh also at the traceability, they can if they have the message ID, then they can ask an AI agent uh where the message is, so that helps uh support.
• 26:17
  SPEAKER_01: Also, in the business with business, it may be uh already be possible to automatically provide a root cause in the solution proposed by the AI, because the II can uh see the error message and there's a little bit of of help uh in it, but then I see it more of a kind of a glower five time monitoring agent, to be honest.
• 26:35
  SPEAKER_01: Uh also depends on the amount of security boundaries the A agent will get.
• 26:39
  SPEAKER_01: I think uh yeah, take a while uh uh in a few years that custom years, yeah.
• 26:45
  SPEAKER_01: When when it's allowed that agents also between organizations have contact with each other, then they can do self-resolvement, self-rehealing, uh starting internally in the organization.
• 26:55
  SPEAKER_01: Yeah, that is a future, uh it depends really on how to manage it, uh, the security boundaries of the agent, what is the agent allowed to do, a certain level of trust in those agents.
• 27:05
  SPEAKER_01: Right.
• 27:06
  SPEAKER_01: So yeah.
• 27:07
  SPEAKER_00: Right.
• 27:08
  SPEAKER_00: I I can appreciate that perspective, uh, because it's it's much more pragmatic than simply saying AI is going to automate everything, which I think everybody thinks is gonna happen, but uh I think technology will improve.
• 27:18
  SPEAKER_00: It we've seen it, it happens, it's it's a natural uh you know phase of life.
• 27:22
  SPEAKER_00: But I think that organizations still need the architecture, the the governments, the the expertise and and the operating model model underneath it, right?
• 27:31
  SPEAKER_00: So you know I'm the same with the cloud.
• 27:33
  SPEAKER_00: The cloud everything had to go into the cloud.
• 27:35
  SPEAKER_01: Yeah, the cloud was everything, yeah.
• 27:37
  SPEAKER_01: And now you see some organizations uh moving a little bit back with the main thing.
• 27:44
  SPEAKER_01: Exactly.
• 27:45
  SPEAKER_01: Also the sovereignty case from oh maybe it's uh not so smart to have everything over uh there in the cloud, and let's take some back to us.
• 27:53
  SPEAKER_00: So who didn't who didn't who told who didn't tell us that the cloud cost us much?
• 27:57
  SPEAKER_00: Let's go roll it back, roll it back.
• 27:59
  SPEAKER_00: Exactly.
• 28:00
  SPEAKER_00: Yeah, um, so you know, I would love to finish uh by bringing this back to the people responsible for making those architectural decisions today.
• 28:08
  SPEAKER_00: Um, and if you could give one piece of advice to a lead architect or an IT director that's just struggling to modernize a heavily legacy-dependent government agency, what would it be?
• 28:20
  SPEAKER_01: I can say, do not underestimate the power of middleware and MQ.
• 28:24
  SPEAKER_01: Use integration, use uh those products.
• 28:27
  SPEAKER_01: In several places I have been, there are infrastructure architects, application architects, illusion architects, all types of architects, but I we rarely see anyone who includes integration and middleware in their portfolio.
• 28:38
  SPEAKER_01: So it's it's a little bit in between.
• 28:40
  SPEAKER_01: So and me as an MQ Tech lead, two times already, I often have to uh yeah, initiate the proposals of improvement within the architecture board or uh to comment communicate major changes, uh yeah, or my my long-term vision for the MQ integration solution.
• 28:57
  SPEAKER_01: Yeah, does that make me an architect?
• 28:59
  SPEAKER_01: No.
• 29:00
  SPEAKER_01: Middleware and integration uh products uh bridge uh bridging the gap between the old and the new.
• 29:05
  SPEAKER_01: So the legacy uh uh services and uh what's uh happening within the cloud, for example.
• 29:10
  SPEAKER_01: Through MQ and uh universal interaction patterns uh within, for example, a broker, old and new applications can exchange information with each other.
• 29:19
  SPEAKER_01: This is regardless of the platform programming language or the code page that is being used.
• 29:24
  SPEAKER_01: So that makes you very uh flexible, but it's often underestimated.
• 29:29
  SPEAKER_01: And yes, that may entail some costs, but it pays for itself many times over, as this layer facilitates the entire application landscape.
• 29:37
  SPEAKER_01: I experienced the cutbacks towards open source software, which works fine on paper, but without vendor support, it settles the department responsible for managing the solution with an enormous amount of extra work.
• 29:48
  SPEAKER_01: I'm not saying that it's bad, because I'm now in a very nice organization working with Apache Artemis, and it's doing quite great, but it still is community-driven.
• 29:58
  SPEAKER_01: So if there is really a critical uh yeah failure uh within uh Artemis, then uh it has some challenges uh compared to like uh IBMMQ.
• 30:08
  SPEAKER_00: Yeah, I I I think that middleware may not always be part of the architecture that gets the most attention, but it's it's as you described today, it's often the layer that makes everything else possible, connecting legacy and modern applications while maintaining the reliability the business depends on.
• 30:23
  SPEAKER_00: You know, Marcel, thank you so much for sharing both your technical and perspective and your real world experience behind it.
• 30:30
  SPEAKER_00: It's been a fantastic conversation.
• 30:31
  SPEAKER_00: And again, I just want to thank you for sharing your experience, giving us such a practical look at what it takes to operate in modernized complex middleware environments.
• 30:39
  SPEAKER_00: I think one thing that really one theme that's really stood out to me today is that middleware may often sit behind the scenes, right?
• 30:46
  SPEAKER_00: Uh, I think of uh the Wizard of Oz, right?
• 30:48
  SPEAKER_00: Oz behind the curtain.
• 30:50
  SPEAKER_00: Uh, but it plays a critical role in connecting the old and the new.
• 30:53
  SPEAKER_00: It supports mission critical services and keeping information moving reliably across increasingly complex architectures.
• 31:01
  SPEAKER_00: And as you pointed out, modernization doesn't always mean replacing what came before.
• 31:05
  SPEAKER_00: Often it means finding smarter ways to connect, to standardize, to secure, to observe, and to evolve it.
• 31:11
  SPEAKER_00: We covered a lot today from IBM Q to Kafka to government infrastructure, troubleshooting, observability, and the potential role of AI and automation in the years ahead.
• 31:21
  SPEAKER_00: Marcel, again, thank you so much for joining us and sharing your perspective.
• 31:25
  SPEAKER_00: To everyone listening and watching, thank you so much for tuning in to What the Hood.
• 31:29
  SPEAKER_00: If you enjoyed today's conversation, be sure to follow the podcast so you don't miss any future episodes.
• 31:34
  SPEAKER_00: But until next time, keep asking questions, keep challenging complexity, and we'll see you on the next episode.
• 31:40
  SPEAKER_00: What's the hood?
• 31:41
  SPEAKER_00: Again, Marcel, appreciate you.
• 31:42
  SPEAKER_00: So great to have a new friend.
• 31:44
  SPEAKER_00: You're most welcome, Mike.