Latest / Broadcast2Post by Key Code Media / On-Prem vs. Cloud AI for Creative Teams: Costs, Security, and Workflows

Transcript

On-Prem vs. Cloud AI for Creative Teams: Costs, Security, and Workflows · Broadcast2Post by Key Code Media

• 0:00
  Speaker A: Foreign.
• 0:11
  Speaker B: In case you've missed it, AI is very quickly becoming part of media production. But creative teams are looking for more than just content generation. They want practical tools that help them transcribe, translate, search, understand and even organize existing media without sacrificing security, predictable costs, or of course, creative control. The discussion I just had with SNS's Steve McKenna explores what media professionals actually want from AI and not just what we assume they want. And why. Transcription and translation are the leading use cases. Plus where cloud based tools can create challenges around cost, data security and of course, your internal company policies. We'll also examine the role of on premises AI, price and models that work for real production environments, and how the SNS AI suite brings AI assisted workflows to television, film, broadcast and post production teams. So, Steve, let's jump right in. First, what I really like about what SNS has done is that so many companies have just rushed to market with AI, right? There hasn't been a lot of market research, it's just been, ooh, here, here's a new AI Shiny. And what I liked is that you spent a lot of time collecting feedback from customers across, you know, several NAB shows. So through all this research and through all this, shall we say, market research, when media teams say they want AI, what are they actually asking for?
• 1:33
  Speaker A: So that's been a fascinating answer because two years ago, you know, when we started this quest, that's exactly what they said, we want AI. And then we said, what do you want to do with them? They said, well, I'm not exactly sure yet, but we need it. So that really led us to the, to embark on this, this quest of discovery, right? And it's not just NAB, it's IBCs, it's Computex in Asia, it's Broadcast Asia, it's global, right? This is a global phenomenon, certainly. And what we really discovered was they wanted practical tools, they wanted to not have to manually tag things, they wanted to be able to find clips faster, they wanted to be able to jump through and get through huge libraries that predate their employment, wherever they were, to be able to find things in context. The other big one that we got was transcription. And these were more for, you know, broadcasters and things like that. So at a high level we kind of broke it down into they want to save time, they want to be more efficient, we all don't have enough time, we all want to be more efficient. They want to quickly find things they may not know exist, things that weren't tagged, and most important, they want to Stay in control. So the concept of intellectual property rights and what is the AI doing with my video? Exactly. That became a hot topic as well as we progressed. So the answers in year one were different and evolved into year two as we went through it. And we kind of took all those and, and roll it into a solution.
• 3:12
  Speaker B: So what I want to make sure that everyone understands is that, you know, there's. Of the multitude of AI types that are out there, I think the kind of dividing line is the generative AI versus the kind of analytical or utility AI. And so to be very clear, the tools that people are requesting from manufacturers out there and to SNS specifically have always been analytical.
• 3:33
  Speaker A: Right.
• 3:33
  Speaker B: You're now, people aren't asking you to generate cartoons to do style transfers. They're saying, look, we have all this content, we, we want to have actionable data downwind of that content to make decisions. Would you say that's about right?
• 3:48
  Speaker A: Yeah, absolutely. I mean, I think a lot of the generative stuff will be left to the NLE manufacturers, to the Adobes of the world and things like that. So with sns, with Share Browser, with asset management, we've always been the data repository that enables video workflow. Right. It's more than just storage storage, it's share browser asset management, it's Slingshot automation and Transcode. It's all of these different backend functions that help us process video, you know, from the production stage all the way through to the delivery stage and do so in a collaboratively and organized way. So it's enhancements to those functions that people are really looking for.
• 4:26
  Speaker B: Can you dive in a little bit more to your survey results? Obviously it's analytical versus generative, but could you kind of go into a little bit more granular detail as to what your survey found as far as the top tools?
• 4:39
  Speaker A: Yeah, so we got a lot of. Can it summarize things? A transcript is great. I don't want to read the transcript to be able to do that. I don't have time to watch every minute of video. Can it transcribe, you know, like YouTube? Yeah, I do want that. I want that to be searchable. Can it translate things? We deal in 90 plus countries now, so the ability to do language localization, the ability to adapt to those markets was huge. And then can it recognize people in places? Can it figure things out and add some tags without a gps and can it make everything searchable? Now, in true SNS fashion, we took that feedback back and we went a bit of a step farther on it. So instead of just going with the top three or four, we decided to do 12. Because much like in AI in general, we also inferred some of the things that people want. And we started to turn internally some. Some things maybe they hadn't even thought about, you know, in their answers. And so that's where things like not just transcription, but smart transcription come in. That's that context. We heard that a lot. You know, can I get a summary? Facial recognition is great, but then we looked at our numerous, numerous documentary clients and realized that there's no face on the screen. So voice recognition became really important. And as we move forward from there, it was, well, if we got voice recognition, can voice recognition provide context? And yes, it can. We can do semantic segmentation. So instead of just summarizing an entire transcript, we can go in and say, all right, well, topics changed here. Let's separate those topics out to make those searchable. Named entity extraction is a big one. People love this because it's something that we're talking about, something on screen. So now the AI knows what the object is on the screen because we've described it. So it's context aware. It's, where can we get all these different vectors? Where can we get all this data? Where's it coming from? Do I need to know that someone scored a goal in a World cup match? Or can I trust the announcer to screen goal at the top of his lungs and use that to get me where I want to go? So all of those things got built in. And I think that's important for customers to understand. With AI too, it's not just, we want AI, it has to be, what do you want AI to do? Because all these different models, all these different trainings, all these different vectors affect what the AI does. You know, if you get I want AI and you get facial recognition, and then you realize you don't often have people on screen, well, not going to help you.
• 7:26
  Speaker B: What this reminds me a lot of is when you start to have conversations with folks regarding asset management, right? Because the same things pop up, which is, we need asset management. Okay, that's great, but what do you want asset management to do? And there's the find things. And you're like, that's great, but that's only one bullet point, right? So I kind of like this because it also kind of follows the kind of SNS Share browser methodology, which is, what are the things you actually need? And let's promote those and then apply that same logic to the AI. So I appreciate the fact that you went methodical about this. Something else we run into quite a bit when we're discussing AI. Again, generative versus analytical is something you mentioned earlier, which is control and that is are we going to put this in the cloud or is it going to run on premises? And so with how powerful cloud AI tools are, kind of what were the biggest challenges that many organizations run into with cloud based models, especially around cost and tokens, security and of course the almighty internal policies.
• 8:29
  Speaker A: Sometimes innovation outpaces internal policies. Sometimes we learn a little later on that side. I mean, a lot of the same standard things apply. Just like you mentioned, AI is bringing in to the fore a lot of the conversations we feel like we used to have about asset management and we still do. Right? And it's important to understand those two things are contextually different types of search in general, both are valid. You know, AI is not a full replacement for a well organized, well tagged file system. Nor is a well organized file system a replacement for AI where I can chat with the thing and say, show me all the clips we have of our CEO talking about said topic. So both have validity in a workflow. The use case, the user profile for who's going to use each one might be a little bit different, but one is not a replacement for the other inherently. But to the point about what you're saying, we were just doing some Googling the other day and I've seen an inordinate number of headlines recently for what did that cost? So if we flashback six months to the beginning of the year, every company was we have to use AI. AI is going to make us more efficient. Let's use AI. We're going to embrace AI. And that was good. It trained AI. It trained people on how to interact with it. Now the bill is coming due, AI costs are going up. AI firms are looking for ways to be profitable on these enterprise accounts. A trillion dollars in OPEX is being used to build data centers right now that's going to have to get paid for. So with that, what we need to see is we need to understand cost structures in order to inform internal policies. So many, many headlines have been up recently. This is from a Reddit thread. I was just reading that a large auto manufacturer ran through its AI token budget for the month on day four of the month. So now it's much like cloud. It's not cloud or no cloud. Right. It's hybrid on that side. So it's not always cloud first, it's cloud appropriate. What is the application? And I think the same is true and going to be true for AI. But the way we've architected this at SNS is to run on prem because we deal in a very unique space, media and entertainment, right? It's not we're going to protect engineering design, right? It's not we're going to protect patents or just internal communications like that. The, the video assets a lot of our customers generate are the intellectual property. They are the product of that company. So whether that be a large feature film house, you know, many of them close to you out in Burbank, that's the stuff that we need to figure out how is that being used? So our AI is going to run on prem on customer hardware. So the customer can invest in the hardware, the customer can update the GPUs, the customer has the control. And that provides back to our share browser model something that's very important to us. No per seat license fees, no usage fees. You're not going to run out of tokens four days in. You own the software license, so use it as much as you want. Because I know you've been deep into the AI world the last few years. Michael I think the other thing that a lot of people fail to appreciate is AI is not super intelligent. It doesn't have unlimited context, it doesn't have unlimited memory. So as a model gets better as it learns things, as it learns a new face, we reprocess what we've already processed in order to reanalyze and apply what the model has learned. When we do that in a consumption model over thousands of hours of video and we have to reanalyze and rewatch it every time we get a new, a significant new upgrade or a new face gets learned or something like that, that can become incredibly expensive. So cost control is important. Cloud AI is certainly fast and can scale and is great for large coding tasks, big data analytics and things like that. But much like cloud storage for giant data sets, they aren't transitory. A lot of the promise of cloud or cloud AI as an extension is the look. It can scale up, it can scale down when you're not using it, it doesn't cost you money. The problem with video data sets is they don't scale down a website. Sure, not as many people hit my website. I'm busier during Christmas than I am in February for a retail website. That's not true for video data sets. We make more video, we repurpose more video in our experience. I never had a client go, you know, I got too much storage now.
• 13:25
  Speaker B: So some of the other things we've run into as well, and I'd love your input on this is what a lot of folks don't realize about utilizing AI, and especially when you put it in the cloud, is that if you're doing a lot of video processing, so facial analysis and you know, pulling speech to text and finding locations and things like that is each one of those is a separate process. So you're paying for that video to be processed 5, 6, 7 times over depending on what model you're using. And then when a new model is out next week, next month, next year, okay, we need to reprocess that again. So it's this never ending chase of being able to process this media. And what we've also found is that when you start Getting into enterprise AI accounts like OpenAI for example, is that when you start getting these enterprise accounts, you're getting X amount of tokens per enterprise and then each person can use a subset of that in comparison to a standalone, where maybe you get a pro license, where you get a ton of tokens per week and then it resets and there is no congruency, there is no apples to apples comparison between what a individual's AI license or LLM license and how many tokens they get to the flat token count you get on enterprise. So there becomes this fight internally of you can use this AI, but only in a limited fashion, which kind of limits the entire use case if I can only use it sometimes. Right. So I love the fact that this is deployed on prem, which means all you're paying for is electricity at that point. And granted electricity isn't cheap, but it's a lot less expensive than paying per token.
• 15:06
  Speaker A: Also, at least I know what the cost of a kilowatt hour is. Right, Right. If we really want to get down into that level of detail. Yeah, I'm with you. I think that's something that a lot of people don't understand. That's where these 12 models. Right. If I were to do the same thing in the cloud, I'm running 12 models, I'm analyzing one minute of video is 12 minutes of video. In this regard, it's the same thing with, you know, when we do production for multicam, how many hours of video? Well, we're going to shoot for eight hours, but I got 10 cameras, so that's 80 hours of video, not eight hours of video. So, you know, I think when we put it in those terms, people understand it. But you're right, there's a lot of opacity and there's a look, there's a reason that AI companies are garnering massive valuations. People who understand the pricing model, understand the revenue potential of everything going into there. And again, it's great. It's going to be used. But just like the cloud, we're going to figure out where to use it. And I think those who have waited a little bit to really go full in on AI will reap the benefits of that. Because it's not just the consumer understanding it better over time, it's the companies themselves understanding the use cases better over time.
• 16:20
  Speaker B: And I think something else that we shouldn't neglect and not bring up is that when we're talking about M and E and Media Entertainment specifically is that there's kind of unique challenges that Media Entertainment has versus every other vertical out there. So maybe you can speak to our M and E friends about why on prem is kind of a better fit specifically for M and E. I mean,
• 16:44
  Speaker A: it comes down to the same reason we have specialized servers at its core. We have bigger data sets than most people. The files themselves, the individual files are gigantic. I'm not just going and reading inside of a PDF. I'm scanning thousands of hours of video, hundreds of terabytes. I mean we have probably a hundred clients well in excess of a petabyte of storage at this point. There's no office standard business use case, data use case that approaches that outside of maybe geospatial modeling, you know, satellite imagery, subterranean radar, things like that. So our, and it's not like the cameras are doing us any favors either. You know, resolutions continue to get larger, with more detail, with more things to analyze. And the more that continues to improve, the more AI can do with computer vision and transcription and listening. So it's a, it's an ever growing cycle on that front. But the problems, it's funny, the problems don't change, the challenges don't change. Whether that be asset management or you know, the visual nature of what we do. The. We layer some new challenges on, you know, the legal ramifications of, you know, letting an entire legacy media library that movies we've watched as children go to a third party company to be analyzed and things like that. But the challenges themselves remain across all models. So we always have to be a little careful when we get really excited about a new technology. And I think we saw that. You know, Milo, you and I have talked about it numerous times with, with cloud in the beginning, you know, especially
• 18:26
  Speaker B: with COVID in, in addition to the, you know, the, the kind of Buzzword of AI slop. There's also been the, the concept of AI grifters and there's definitely who are building AI and then saying, well, we're going to march into Media Entertainment and I'm going to let loose my AI of them and tell them that's the AI they need. But what I think they don't realize is that M and E very differently than other verticals is a lot of companies who are making this content, it's not. They're the, they're doing VFX for someone else's content, they're doing sound effect for someone else's content. And because of that, they have contracts with the studios or whoever is licensing that, which means a lot of times in their contracts and writers, they say, you can't use the cloud, right? So that means you have to do it on prem, whether you like it or not. And what we're also seeing is that some studios, some big VOD studios are slowly saying, you can use AI, but you can only use these approved tools. And if you use these approved tools, we expect a discount. So it becomes less altruistic, it becomes less, we're keeping up with technology, we're keeping up with demand, and more, we'll let you use it as long as it benefits us financially. So I think we're at a really real inflection point right here, which is these contracts need to start relaxing the mandate to not use the cloud but also say you can use AI. But it should be, in my estimation for the creative process more than just to save a few bucks. Which kind of goes into our next topic, which is when you and I have spoken before, you've mentioned that creatives don't necessarily want AI to create for them. Right. They want it to assist them. So how does that distinction look in a real workflow? Because there definitely is some bleed over between those two sides of the house.
• 20:21
  Speaker A: I mean, there is and it's. Any transformative technology I think gets into that. And I would, I would amend that a little bit as we've gotten more data. It's not that creatives don't want help creating, but AI is not craft editorial, you know, as far as like an LLM going in and saying, hey, can I find some shots that are funny? Can you string them together into an EDL and help me with a rough cut and help me review it much the way two or three years ago, like, hey, can you check my email? Or can you write this for me? And I'm going to go edit this Down. I think we've all seen, you know, social media posts or things where people have forgot to check what the AI said before they just posted it and it's asking for the next task that it can do for you at the bottom of someone's LinkedIn post. I think creatives don't want that. You know, I think they want Human in the loop, I guess, is what it would be called in an AI vernacular. But as far as helping them organize data more quickly, anything that's going to save them time and help them be more efficient is something I think they embrace. So if it's. I don't want to have. I don't have time against this deadline to go out and shoot this thing. Like maybe I can get a few frames or work on a transition or something. I think that's well within the wheelhouse of what they want to see. But again, that will leave that to the creative tools themselves. For the generative AI stuff. For us, it's going to be not just the analytical, but the organizational, the assistant kind of thing. Because our advanced AI suite, so there's kind of two models, now also includes an LLM. Right. So that goes beyond just a traditional search that goes into. Let's use standard language to converse with this. And this is super helpful, I think, for creatives too, for executives to get involved. Right now I can just ask my senior VP of marketing, who may have come from a photo background, may not have video background, may not need to understand video production terminology, but they can certainly find, hey, what are all the product shots that we have in the woods? That's something that I think AI provides a lot of promise, and that's true in a lot of those pro tiers and things as well. I think a lot of people gravitate to the LLMs more than the CLI stuff. But the enterprise and developers and people that go to the cli. Then we get into that token challenge. Yeah, well, we're going to run it. I don't know exactly how many tokens it's going to take. We're going to find out. Oh, that was a lot. It's that kind of thing.
• 23:07
  Speaker B: So my last look at the AI suite was back at nab, so in April, a few months ago. Would you be able to kind of show us what it looks like today and how a creative team would use the AI suite for transcription and translation or even media search?
• 23:23
  Speaker A: Yeah, I mean, we can get into it a little bit. I've got a bit of a video here, not a live environment. I encourage all your clients, key codes clients to reach out. We do these 30 minute demos kind of all day, every day where we can actually interface and interact and really get you through it. But it's pretty, pretty straightforward to understand. So let me go ahead and get this popped up. There we go. And we can kind of talk through it as it goes. So we can see kind of this left column over here. And this is going to show us keyframes, which is another big advantage over kind of a traditional mam. Right. Every scene change is going to generate a jump to point basically automatically because we can use computer vision to see that. Going to get that contextual summary down below. And then in the middle we're going to see that transcript at the time code where it's talking. And then to the right we're going to see the models. So as we kind of go through here, we'll see it's picking up faces right now. We can go through these key frames. We can click into a transcript, identify a word which will take us right to timecode in the video player. We continue to kind of come through here. We can see that summary. Now what I really like about this too is you can also edit that summary. So if it's not what you want, that's okay. We can see content type. So it's going to provide a high level tag. This is about education is where this video sits. Content format. You know, what is it? It's a special. And that's all derived without any human intervention. Now we can add to it certainly as we come through here, if we do just a more of a generic search, it's going to show me everywhere in the transcript that word or whatever may show up. As we continue to kind of scroll through here a little bit and it jumps around. You'll see in the upper right there's a language of the transcript. So here it's seeing a female. It doesn't know who it is yet, but we're just going to go ahead and flip that to Spanish and English. And so it's just that simple. And this supports 90 different languages. So this is a large amount of things. It's not just the big three or four that we think of here in the United States. Right. This can be truly global. Over here we'll see keyword search. So this is more traditional namm, except it's catching some of these keywords and deciding what they are based on inflection. So we come down to faces. So these faces are already programmed in. There's a Training module to a training UI that you can pop over to and it'll look at a video to say, look, I found these 10 faces. You can upload specific faces. If you upload them at different angles, it's going to be better at finding them because not everyone is always center on frame the way we do in a two up interview as we're doing today. Once those are identified, it'll go back and tag everybody with that. If it misses one, you can quickly add it here and say no, no, no, this is this person if it does miss something. So it can continue to get smarter as you use it. As we come down a little bit farther here, semantic segmentation. So this is the one I was talking about kind of at the outset. Notice now it's got three pills here for what are they talking about? This is independent of the transcript. So if you and I change topics from what is AI for instance to hey, let's talk about the advantages from on prem to cloud AI, those are going to appear here as pills and I'm going to be able to quickly jump to those transitions and subdivide out what I need to do. Kind of continuing on through ocr. This is a huge one and I think we have a question coming up maybe about this. So OCR reads words on screen, right? So now with computer vision, it's not just about the face, it's not just about the voice and the voice print, it's about what does their shirt say. Now this is tricky because this is different than logo recognition down below, right? We can also train a logo. If I'm a corporation, then it's different than image description if I'm a corporation. And I want to say find my logo in every major league sports broadcast because I spent a lot of money to sponsor that league and I want to know what my ROI is. I can do that. And that's going to be logo, which is discrete from words, which is discrete from just an image description. Image descriptions you'll see here, like participating in the monitor program, right? What, what is this? So we've got kind of a, a two person interview in a classroom. It could be, oh, I see it in a coffee shop, they're outside in a park. So that's going to describe that a little bit more. More like scene recognition would be. Same thing down here with scene types. It's a meeting, it's a social gathering, these kind of contextual cues. And this again informs the LLM for common language. Show me every group setting we have. You know, maybe not the company Christmas party, something like that. And then we also have objects and these are named objects, past scene types here. So like I said before, do we need to see and know it's a goal or can we trust the announcer? Can I say this is this thing, this is this building and have the AI pick that up? And I think this is fascinating too. I think this part's really important for production because I think it's going to affect how production works as well. We can produce in a way that the AI is going to be able to use.
• 29:16
  Speaker B: That brings up a really good point, Steve. Is that because AI is becoming part of the workflow, what should production and post teams change today, right, from what happens on set to how editors approach the edit. What would be your advice?
• 29:32
  Speaker A: Yeah, I mean, I think knowing what models you're going to use to catalog this stuff is really important to production. So this is where our corporate marketing agencies and things like that have a bit of an advantage. Right, because they control production and post. It's not quite the same as, you know, feature film or third party marketing agency where post may be different than production. That said so. But knowing what models are available and what tools are approved can inform you. So if I have ocr, for instance, good reason to bring back my favorite video production tool, the clapboard, write it on something so the AI can read it. When you're doing B roll shots of buildings and things like that, or background scenes, if the building has a placard on it with the building name and address, make sure to get that in the shot. Before you go to a wide, consider that the AI can now read what's on screen. If you're doing a site location scout, talk about where you are. Because it can listen to your voice and it can infer the context from what you're saying. So we might do, instead of an old 3, 2, 1 tape leader in to get it right, we might do a hey, let's provide some context and voiceover here before we even start the scene. And then the AI will be able to separate that. So I think it's hugely, it can hugely affect production. And it doesn't take that long on set to do these things to really make sure that we get the most out of the AI on the back end.
• 31:05
  Speaker B: I've heard of many productions doing things like that and saying things like after the best take to say that's the take. And then when that gets to post for logging, there's the E. Well, sure, we have all these circle takes, but this is the one that in the moment the the onset folks said this is the shot we want. So I think you're completely right. Being able to to add that narration to kind of help guide AI in post for what your workflow is is probably the way to go. And I and I think what the overarching takeaway from our chat today is that AI and media production is not just about generating new content, although that is kind of what the new sexy is right now. But for many teams, the real value is helping people find, understand, translate and organize the content they already have, while of course keeping the creatives in control and understanding where the cost centers are. So you can have, as you mentioned earlier, the best ROI you can for AI. So Steve, I want to thank you for joining us today and showing us what's new in the AI suite for sns.
• 32:10
  Speaker A: Yeah, well, thanks for having me, Michael, and always a pleasure to talk to you and I look forward to seeing you in person soon.
• 32:16
  Speaker B: Thanks for watching Broadcast to Post don't forget to follow Keycode Media on social and contact us about your projects@keycodemedia.com.