Voices of Video
Explore the inner workings of video technology with Voices of Video: Inside the Tech. This podcast gathers industry experts and innovators to examine every facet of video technology, from decoding and encoding processes to the latest advancements in hardware versus software processing and codecs. Alongside these technical insights, we dive into practical techniques, emerging trends, and industry-shaping facts that define the future of video.
Ideal for engineers, developers, and tech enthusiasts, each episode offers hands-on advice and the in-depth knowledge you need to excel in today’s fast-evolving video landscape. Join us to master the tools, technologies, and trends driving the future of digital video.
Voices of Video
Are ASICs the Future of Video Processing?
What happens when the relentless demand for video processing slams into the hard walls of data center power and space limitations? Dennis Mungai, heading R&D at Cires21, takes us on a fascinating journey through the evolution of encoding hardware that's reshaping how broadcast giants like the BBC deliver content.
"Density, density, density" emerges as the driving force behind Cires21's technological evolution. Starting with flexible but resource-intensive CPU-based encoding, Dennis reveals how their Madrid-based team methodically explored GPU acceleration before discovering the game-changing potential of Video Processing Units (VPUs). The conversation demystifies why purpose-built ASICs are upending conventional wisdom about the necessary tradeoffs between quality, power consumption, and channel capacity.
Most revealing is Cires21's extensive codec comparison study, where they tested approximately 1,500 samples across CPU, GPU, and VPU implementations. Their findings challenge long-held assumptions: NETINT's VPUs delivered visual quality comparable to software encoders but at a fraction of the power cost. Perhaps most surprising was the discovery that these specialized processors performed "extremely competitively" even at lower resolutions where traditional hardware solutions typically struggle.
For streaming providers facing the reality that "we have run out of power in the data center," this technological progression couldn't be more timely. The ability to fit hundreds of broadcast-quality channels into a single rack unit represents a fundamental shift in video infrastructure economics. As Dennis eloquently puts it, this evolution "is either going to find you, or you're going to find yourself buying into these solutions and you will be behind time."
Stay tuned for more in-depth insights on video technology, trends, and practical applications. Subscribe to Voices of Video: Inside the Tech for exclusive, hands-on knowledge from the experts. For more resources, visit Voices of Video.
Voices of Video. Voices of Video. Voices of Video.
VoV 43 - Dennis Mungai, Cires21:Voices of Video.
Mark Donnigan, NETINT Technologies:Hey, welcome to another super exciting episode of Voices of Video. I am Mark Donegan and today I am joined by Dennis Moonguy from Cirrus21. Dennis, thank you for joining us.
VoV 43 - Dennis Mungai, Cires21:Thank you, Mark, and thank you for the warm introductions.
Mark Donnigan, NETINT Technologies:Awesome. Well, I'm really excited for our conversation. We have spent a little bit of time talking about what all you guys are building, and so we're going to you know we're going to dive in, but why don't we start by telling you know, the audience, who you are, specifically what you do at Cirrus 21, and then give a brief intro of Cirrus 21.
VoV 43 - Dennis Mungai, Cires21:Absolutely, that would be a pleasure. I am Denis Mungai and my current role at Cirrus21 is on the research and development of the video encoding, packaging and handling infrastructure that we have built over the years at CirES21. The role mostly encompasses on a lot of research and near constant iteration, because this field is a moving target. It's always been a moving target yes, it is.
VoV 43 - Dennis Mungai, Cires21:And CIRES21 in particular, has had very deep investments into R&D efforts in both video processing, audio and also up and coming with a lot of AI integrations. So this is going to be a mainstay of what we do towards not only improving on the product stack that we have, but also contributing our work towards making better tooling for video.
Mark Donnigan, NETINT Technologies:Yeah, amazing. Now where is the company based?
VoV 43 - Dennis Mungai, Cires21:The company is based in Madrid, Spain.
Mark Donnigan, NETINT Technologies:Madrid. Okay, yeah, beautiful, I've been to Barcelona, I've not been to Madrid. It's definitely on my list. So, yeah, yeah, amazing, and I know that I don't think you're in Madrid. So are you the only one that works, you know, outside of the office, or is it a distributed team as well?
VoV 43 - Dennis Mungai, Cires21:It's a distributed team.
Mark Donnigan, NETINT Technologies:It is a distributed team Amazing. Yes, yeah, yeah. That's the way of technology today, right.
VoV 43 - Dennis Mungai, Cires21:And that is similar.
Mark Donnigan, NETINT Technologies:I mean we I mean we definitely have, you know, three primary offices, but you know, there's team members located around the world. So, yeah, incredible. Well, why don't you start? That was. That was a great introduction and great introduction and I'm really excited. One of the things that we're focused on here on Voices of Video is I really want to begin hosting conversations in the area of R&D and I really want to talk about one of the studies that you have recently completed, but we're going to get to that later, so I don't want to jump there yet. But why don't you tell us what is the core product that Cirrus 21 offers to the market and who are your customers, what are the customer profiles for this product?
VoV 43 - Dennis Mungai, Cires21:For this product range that we work on at Cirrus 21,. Our main market is broadcast, primarily broadcast companies, and we have had multiple partnerships with the likes of BBC, among others, the likes of BBC Prisa, among others, and our tooling has been validated to work with a lot of their workflows, some of them meeting very specific edge cases that are well, I wouldn't say hard to describe in this episode, but, yeah, it's in that direction and we mostly focus on video. So a lot of our products will be handling video broadcast workflows, will be handling video broadcast workflows. This includes the C21 live encoder, the CDN endpoints and everything else.
Mark Donnigan, NETINT Technologies:Okay. Okay, so you referenced the Cirrus 21 live encoder that is correct. You have CDN endpoints, I guess, basically an origin server solution. That is also correct. Yeah, what else have you guys built?
VoV 43 - Dennis Mungai, Cires21:We have built, apart from the C-RES 21 live encoder, we've also built the C-21 live origin, which is for protocol and transport adaptation and contribution. We also have the C21 Live Editor, which can be used for clipping and editing. And on the distribution side of things, we have integrations with C21 Live Captions, which handles real-time subtitling and translations. Subtitling and translations we also have. We also do a bit of integrations with C21s on Intel AI sub-platforms and this again feeds back into live captioning and other expanding works on transcription and similar workflows Interesting.
VoV 43 - Dennis Mungai, Cires21:So, from the orchestration from the source to the distribution to the audience, we have also C21 Live Control, which handles the orchestration processes, and then we also have the C21 Live cloud with integrations to Google Cloud, aws, azure, ovh Cloud, kubernetes and others.
Mark Donnigan, NETINT Technologies:Wow, okay, so I know that you've got some slides that we can go through and I definitely want to get to those, but just to understand why clients choose to work with you. So you mentioned the BBC, and that's very impressive. I think we all know well that the BBC is a very demanding organization. So is it that you are building things that they're just not able to buy, In other words, you're building solutions that are not available, or are you, you know, sort of you know customizing existing solutions, bringing you know a few features that are missing? Or maybe you can characterize what it is exactly that you know why your clients work with you, why do your clients work with you.
VoV 43 - Dennis Mungai, Cires21:Our clients work with us primarily from the very strong reputation and partnerships we have with Broadcast and specifically because the solutions we build for them are typically handcrafted for the kind of appliances and budgets that they have to fit into. So we cannot simply quote, unquote, repackage something that has worked elsewhere and then just assume that the product will work in another deployment. So, like I said, being centered on the R&D front of doing things allows us to handcraft solutions that fit the specific client demands that we have at any given moment in time.
Mark Donnigan, NETINT Technologies:Yeah, yeah, I think it's a really good positioning.
Mark Donnigan, NETINT Technologies:You know the requirement of and you know this better than anybody. You know the requirement of each organization is sufficiently diverse, and when I say requirement, I mean the technical requirements but also the business you know, and of course those things are related, right, but you know they're sufficiently diverse that it's a challenge that all product companies have is that you know you can have 80% of what any one client would need, but it's that extra 20% that either can be very difficult to deliver or very expensive, or just you know it's always a challenge because you know client A, it needs a different 20% than client B, which means that the work you do for client A may not really be usable across other you know other applications or other clients. So, yeah, it's an interesting place. Why don't we go back to your live encoder? I would like to, you know, I'd like to start there, because we are going to have an opportunity to talk about a recent codex study that you published. So I'm sort of teasing that out for the listeners, don't?
Mark Donnigan, NETINT Technologies:click off yet because you're definitely going to want to hear what Cirrus 21 has found regarding CPU, gpu and then VPU. So explain your live encoder, talk to us about what you've built, what some of the advantages are, some of the unique features.
VoV 43 - Dennis Mungai, Cires21:So our live encoder was initially built around being able to be as platform agnostic as possible.
VoV 43 - Dennis Mungai, Cires21:Any CPU, any platform that we can run containerized solutions on our encoders could run on.
VoV 43 - Dennis Mungai, Cires21:And CPUs are awesome because it is very trivial to tune an encoder profile and presets to the type of workload and the report you want to run on.
VoV 43 - Dennis Mungai, Cires21:And this has been the basis of the encoder for years.
VoV 43 - Dennis Mungai, Cires21:And with recent developments and demands on functions to do with both transport density and also efficiency do with both transport density and also efficiency we had the need to essentially integrate GPU-based processing and offloading for some of the workflows that we do, and also extending the same encoder capabilities to GPU support, and here I'm referring to the commonly known H.264 slash AVC.
VoV 43 - Dennis Mungai, Cires21:We also have support for HEVC and the up and coming AV1 codec, and it's been quite an interesting interplay between the type of quality you want to deliver and the kind of design compromises and constraints you have to take into account when you are adding discrete GPUs into the mix, and the reason being that they also have their own specific software and platform level requirements that you cannot always abstract away, no matter how well you do your containerization and orchestration. So it's been the necessary upgrade but one that often comes with a lot of R&D into that front to make sure that we are delivering what the market wants and what it demands in terms of both compatibility and retaining the quality that we have delivered consistently over time with CPUs.
VoV 43 - Dennis Mungai, Cires21:So that's the most for GPU support.
Mark Donnigan, NETINT Technologies:yeah, so can I assume that your CPU-based solution is built around? You know open source, x264, x265,.
VoV 43 - Dennis Mungai, Cires21:SVTAV1?. Some of the components are yes.
Mark Donnigan, NETINT Technologies:Yeah, okay, okay, did you actually build your own encoder or are you using some commercial solutions?
VoV 43 - Dennis Mungai, Cires21:We have at some point integrated some of these open source encoders and we also have quite a bit of middleware, where applicable, to offload specific functions and for utility and also for resource monitoring.
Mark Donnigan, NETINT Technologies:Yes, yeah, yeah, exactly yeah, that, exactly yeah. That totally makes sense. So you know, you absolutely said it right. Software is great when it's running on x86 and it's running in a container. You effectively can run it anywhere, right, which gives a lot of advantages.
Mark Donnigan, NETINT Technologies:Yeah, I'm curious if you're seeing, you know what, what we're seeing and not seeing. We're hearing very plainly from the largest platforms in the world, almost universally, in fact, I don't think there's any that hasn't said. You know the words, we are, you know, and people are in different phases of this, but basically people want to get off the public cloud. Now, it's not that they want to get off the cloud, you know. So it's not that you know everybody, but just the, the notion of, hey, I'm just going to pay for usage, and you know I'm not going to single anybody out, but we know who.
Mark Donnigan, NETINT Technologies:The three or four, you know largest platforms are right. You know folks are starting to do the math and they're starting to realize wow, you know like I can go do this, either on my own, spinning up my own data centers or going into colos, or you know there's different strategies. There's the hybrid. You know where you build your infrastructure, but maybe you build it for 80% capacity and then you flex into the public cloud. You know different approaches. I'm curious are you hearing and seeing the same things with your clients and your customers?
VoV 43 - Dennis Mungai, Cires21:That has come up multiple times and it's something that our product has had to evolve around. The thing about market demands is that they are not bottlenecks. They are more of opportunities to not only realize the next steps that need to be taken, but also the nudge towards building flexible solutions. It is true that the public cloud has never been. The definition for it is that it's not a stopgap to just deploying things you can automate and seeing the results. No, it has implications on billing, availability and also the kind of resources you can quote, unquote, borrow from a given instance at any given amount of time, so for the customer to then be able to spec out a build on their own. In terms of some of the appliances we support, it's important.
Mark Donnigan, NETINT Technologies:It's important.
VoV 43 - Dennis Mungai, Cires21:The public cloud is not an all-in-one solution.
Mark Donnigan, NETINT Technologies:There are places where going hybrid and also retaining compatibility, for the edge will matter. Yeah, the goal is flexibility. I'm going to steal that phrase. You know it's not a bottleneck, it's an opportunity, you know, in terms of, you know, like you said, changing market demand or changing behaviors, and I think it's.
Mark Donnigan, NETINT Technologies:You know, as a as a vendor, you know we have to think of these things too. You know the market starts to move in a different direction or starts to show signs of adopting, you know, maybe a different set of technologies than we have contemplated. You know there's always that first reaction of like, oh no, you know, like, okay, you know we're kind of going in this direction. It looks like we need to go in the other direction, but there always is the opportunity. You know that that that goes with it, and I love that perspective, you know it's, it's a good way to think about it. So so this is so.
Mark Donnigan, NETINT Technologies:This is interesting because it strikes me that you're out in the market, you're competing against some very well-entrenched competitors who have broadcast solutions. You know, again, I won't name them, but you know there's not too many and you know everybody knows who they are. So why is it then that someone would choose, you know, to work with you to adopt one of your solutions, and maybe even products. It sounds like you productize, like your live encoder, for example, so why would they choose that instead of just pulling something off the shelf from one of the established brands that are out there?
VoV 43 - Dennis Mungai, Cires21:That's a good question, and I believe the key point to what you just described is something I prefer to call it a fit. We are not here to reinvent the wheel right. The goal towards the continuous R&D efforts that we make to building these products are that we are also consumers of the products. We make the concept in software engineering that we term as dogfooding. These are products that we have used in our own workflows to the point that we refine them as a broadcaster world. So we are shipping products to engineering and production teams that not only work like the way we do, but have the exact same problems that we have had to solve over iterations to solve and to also address, and a growing concern here being not only a function of compatibility but also being able to be what I now refer to as being cloud agnostic. Cloud agnostic Our solutions will run on any publicly available cloud that exists out there and whatever hardware you have, be it CPU, be it GPU, be it even the NetInt GPU stack we support that now.
VoV 43 - Dennis Mungai, Cires21:Yeah.
Mark Donnigan, NETINT Technologies:Yeah, exactly.
VoV 43 - Dennis Mungai, Cires21:It's not about being what you call trending towards being vendor locked and everything. No, we are going to work with some of the biggest cloud providers there are to make sure that whatever resources they have on those instances, we can use them and we can also create products that fit not only into the budgets but into the flows for the people that use them.
Mark Donnigan, NETINT Technologies:Yeah, for sure, for sure, For sure. Yeah, it's a good, it is the right way and you answered the question beautifully because I think you know sometimes it's easy, depending on where someone is in the ecosystem, what their focus is. So, for example, if an engineer is, you know, primarily been trained in video codec development or codec optimization, or, you know, or they're just really, really heavily inclined towards encoding, you know they see everything as like an encoding challenge. You know, and it can. Sometimes, you know, decisions can get weighted down into areas that are important, but they're important in a narrow band and when you look at the whole, when you kind of step back and look at the whole workflow, you know there's optimizations across the entire workflow. You know that can be done which, when you add all those up, can deliver a much better result. So, yeah, it's interesting.
Mark Donnigan, NETINT Technologies:Well, here's what I'd like to hear. You know, I'd like you to talk about your journey from moving from and you know we're focused again on encoding in this particular discussion, although feel free to talk about other areas of the pipeline, of the workflow of your product that you've transitioned from CPU to hardware and the journey being CPU. I know that you adopted GPU, you know, for greater efficiency, higher densities. You know some of the obvious reasons, even codec support. You know, like live, hevc initially was really hard to do in software. It required a massive machine and you only got a handful of channels. You know Now the situation has changed a lot over the years as X265 has been optimized and you know there's, you know, other encoders out there that are quite efficient for HEVC. But now we look at like AV1, like AV1 in software or live, and to get good performance and good efficiency with reasonable quality is still very, very, very hard. So talk to us about this journey, tell us what you've been doing, you know. Uh, yeah, walk us through cpu, gpu to vpu definitely so.
VoV 43 - Dennis Mungai, Cires21:Initially, as stated earlier, our work started by implementing codec support in the most vendor-neutral dispatch methods that we had, and that is on the CPU, and of course this grew from adopting H.264, hevc, and then, at some point, we had workflows that also required VP9. Yes, vp9 in software, which is by all means a pain to set up, but that's a topic we will definitely revert back into, because VP9 never really got quote unquote it never really got the traction that its successor EV1, did, and the reason being, again, platform adaptability, for it still remained quite limited, even in areas where you could have done specialty workflows for the likes of WebRTC and Core. And even by the time we had chips that could handle hardware-axiomated decoding for it, the likes of HEVC and even H.264 were still relevant. Now, a challenge we have faced, as everyone else that develops video products will tell you, is that CPUs are pretty awesome for the flexibility they give you. You can push in almost anything and get the output you want, but you are limited, and I mean hard limited by the actual physical resources any given chip can give you, regardless of its size or memory constraints or the storage fabric that you have, because there are goals that require a given amount of density and also a given amount of throughput to be handled at any given time, and this is doubly so when you are doing multiple EBR style ladders. This is average bitrate and multiple bitrate ladders, and this is where GPUs come in.
VoV 43 - Dennis Mungai, Cires21:With GPUs, it became then possible to not only drastically increase the footprints of our appliances versus the outputs we can handle, but we could also offload more of these pipelines into the GPUs. I'm talking of scaling, color conversion, hdr to SDR we call that tone mapping in this field and everything else. We can even handle custom payloads now because, again, doing R&D for this kind of work, you'll often find yourself solving problems long before they come knocking at your door. And GPUs opened those platforms so that we can not only ship products that are lighter in weight and in chassis, build and everything else, but for the type of throughput they can give you, you also get a higher return on your investment over time and again, maintaining compatibility with cloud solutions that have these GPU provision. Sure, sure, and I assume.
Mark Donnigan, NETINT Technologies:Sorry to interrupt, but just for a point of clarity when you say GPU, you mean NVIDIA. When we're talking about using the encoder and the GPU, you mean NVInc, right?
VoV 43 - Dennis Mungai, Cires21:That is one of them. That is one of them. Our product stack is quite extensive. We even do support Intel's OneVPL platform. Initially they called it QuickSync. We also support the open source standard called VAPI, the video application API. We implement that and the beauty of these dispatch methods is that with VAPI, anything that has a MESA-compliant driver that can do the offloading we can support. We also extend a bit of this processing to the open-source Vulkan API and in Vulkan we can also do video encoding for H.264, hevc and even for some of these very edge-based codecs such as the archival format called FFV1. This has been developed by FFM Peglabs. It's lossless, but we do have an open source encoder for that that works for that codec.
VoV 43 - Dennis Mungai, Cires21:It's an edge case, but it's there and it's implemented.
Mark Donnigan, NETINT Technologies:Yeah, it's there. Yeah, yeah. Now are you building workflows? It sounds like it's a mix of contribution and distribution. Sure, Is that correct? Or are you kind of more focused on one or the other use cases?
VoV 43 - Dennis Mungai, Cires21:Both All of these use cases are correct because again, the whole point of GPUs comes down to utility. How much more can I do with a smaller footprint on both power draw and?
Mark Donnigan, NETINT Technologies:rack density. And that's where.
VoV 43 - Dennis Mungai, Cires21:GPUs come in.
Mark Donnigan, NETINT Technologies:Yeah, yeah, yeah, yeah, do you find? So we're going to get to VPUs, but it's really interesting because, just as an aside, I think some people obviously those who use VPUs and have spent time working with us and using the product know that, yes, on some level, a GPU is a competitor to the VPU. In other words, there could be a scenario where someone would choose to use some NVIDIA card and use the video IP instead of a NetEnt Quadra, for example. It almost never happens for reasons that maybe we'll get into later, but it is possible. But it is possible. However, it's interesting. People assume, like well, but you know, aren't you sort of like, you know, at war with the GPU? You know, and it's like no, no, actually there's some scenarios. We have a very, very large customer, you know, they're in the cloud gaming space, operate a massive platform, but we actually did some very special work with their GPU vendor to be able to pass the frame directly to our VPU without going onto the NVMe bus. It's in memory, it's in the DMA.
VoV 43 - Dennis Mungai, Cires21:That's possible. Dma peer-to-peer transfer.
Mark Donnigan, NETINT Technologies:And one of the reasons that we're able to get the densities for this particular platform that nobody can believe initially, nobody could believe initially until they really understood what we did is because of this. You know, we're able to keep everything in memory and so we're very friendly, you know, with the GPU vendor and, in fact, the other big GPU vendor who also wanted the business. You know, started calling and how can we work with you? And you know we're like wait a second, aren't? They're like no, no, you know like right now we're friends. You know like we want to work with you. So, yeah, so, so I'm.
Mark Donnigan, NETINT Technologies:I'm curious, though are you finding that the GPUs that you want to use or because GPUs are sort of universal and you know NVIDIA has done a wonderful job of building software and supporting software across the ecosystem, but you know it's not true that it's just I can just grab any NVIDIA GPU and just use it. So do you find, on the public clouds, that there's sufficient availability? You know that the customers basically can get it whenever they want or have you had to kind of know like, hey, you know, platform A is a little bit better than the other, platform A, and they're slightly better than the other platform G.
VoV 43 - Dennis Mungai, Cires21:That actually does happen, and I will not even have to mention names. It's one of these things where we have what we now call super vendors, big enough to not only have possibly the largest share in not only AI but also GPU availability in the public cloud, and then also having the tooling to segment these GPUs further into smaller instances. I believe the technology they have for that is called MIG, the multi-GPU split. This is not just GPU virtualization, it's a form of negotiation where the driver splits the GPU into multiple partitions, and it is true that if you do not use the tooling as well as you should, you may get partitions that may not have some of these slices enabled. Intel, I believe, had a similar solution back in the day. They used to call it Intel GVTG, and then it was discontinued. It's a bloody mess. It was just dropped dead in the water, and NVIDIA's advantage, I believe, lies in their tooling. But now us adding the assumption that all this tooling will work in the same way in all these public clouds is another topic of research, exactly.
Mark Donnigan, NETINT Technologies:Yeah, yeah, yeah, yeah, it's you know. And look um, in all fairness, you know we deal with um. We don't have as much of a tooling issue. In fact, we really have zero um tooling issue because, um, um, you know just the way that you would use our product and our APIs and everything that we make available. Also, you know our product's not general purpose, like a GPU is, so you know. So our issue is not as much tooling, but we do very much have an access challenge. You know that is at this point.
Mark Donnigan, NETINT Technologies:We're still being built out in a lot of the public-facing clouds. So, with the exception of Akamai, Akamai is one of the most recent clouds or edge providers that we can talk about publicly. That's building out. We can talk about publicly this building out. Our challenge also and it's a challenge of all hardware is where can I get it? Where is it available? Because it's a physical thing. This, again, is why software is so great on one level, but then when you begin to scale and build these massive systems, it just doesn't scale. Cpus are just too power hungry, they're too expensive. There's a lot of reasons why they don't scale. You know CPUs are just too power hungry, they're too expensive. There's a lot of reasons why they don't work, you know. So. Okay, so we're going to get to VPUs.
Mark Donnigan, NETINT Technologies:But I'm curious was this move to hardware, ie GPU? Was it driven just more from like a? Hey, the only way we can do, you know, like 422, color conversion and you know, do some of these more, you know deinterlacing at very high quality and some of these more you know, is with GPU. In other words, I'm just curious if it was driven more from just a technical reason. The only way we can do it is with GPU. It's just not really feasible on CPU. Or was it cost? Or was it a combination? Did your customers pull you into it? Talk to me about the drivers behind this transition.
VoV 43 - Dennis Mungai, Cires21:Absolutely, and I believe the greatest driver here has been a term you might have heard me drop a lot in this podcast and that term is density, density, density, because at no point in time are you going to sell someone a very high-capacity server and then just give them a number. Hey, for the type of renditions you want and the inputs and outputs, you can only give you like 10 instances of this and there's only so much you can do on silicon so much power and so much rack space right, True.
VoV 43 - Dennis Mungai, Cires21:Before you start running into and I'm not just talking about memory bottlenecks, but also throughput A lot of software encoders excel at the reference level. They are good at producing excellent bit streams as per the specifications. Even without dropping names, we know who they are. Now, gpus excel at offloading some of these functions in a way that you can also tune out the quality offsets for this and in a way that is acceptable for downstream consumption. And these are measurements we have done over time, because your customer will demand a given amount of visual fidelity and visual quality before something becomes viable as a product on the line. And it is true you simply put, having to offload some of these processes isn't something we developers would want just to wake up and say you have to do it, it's a necessity, it is evolution. It's either going to find you or you're going to find yourself buying into these solutions and you will be behind time.
Mark Donnigan, NETINT Technologies:Yeah yeah yeah, yeah, I can. So relate to what you're saying, uh, to you know, to what you're saying, and I'm really happy to hear that you have taken the stance of leading um, this um. You know, I I don't even want to say trend because it's really requirement and, you know, early in the development so we've been shipping our VPUs. We're, you know it's well-known we're in our second generation, um generation VPU, and that one has been. Now we're coming into the third year of Quadra, so it's really established. But we've been in the market for, you know, five years now and you know there was a good probably first three, even four years.
Mark Donnigan, NETINT Technologies:The situation only changed in maybe the last 12 to 18 months when it started this sort of but where people would evaluate our VPUs and they would evaluate it against the very best software implementations. And I guess on one level the rationale was hey, you know, why would I compare it to something that's you know, just sort of call it average? You know why would I compare it to something that's you know, just sort of call it average? You know whatever that means. And everybody has different definitions, of course, of like what you know good and you know okay and bad is, you know. But you know, just call it average. So they would naturally sort of say, hey, I'm going to compare it against the very best. We know that hardware always performs differently. So they would come back and they would usually say, wow, the quality is actually really good, like much better than we expected. But then they would proceed to say, oh, but software is so much better and beats it here and beats it there, and beats it there, and this would happen time and time and time again, and it took a couple of years for the market to kind of wake up and go. But wait, no, in a one RU server I can get 320 live channels, even with AV1 or ATVC, at quality that is better than what I would typically be doing in software. You know, for live, now again for live, and I would be getting just a handful of channels and software. You know for live, now again for live, and I would be getting just a handful of channels and software. You know, and, and so it took like a little while for the market to sort of like figure out that.
Mark Donnigan, NETINT Technologies:Wait a second. You know, you've got levers. You can dial up quality and bit rate, efficiency, compactness, and you're going to dial way back density, right. Right, you can, and I'm speaking of in software, and we know all the levers. You know, everyone you know works in video knows this. You know. I can also then make choices to dial up throughput so I can get more channels. But guess what? My quality is going to go down. In software, it's going to go way down. My bit rate efficiency is going to go way down. You know, and I have these levers.
Mark Donnigan, NETINT Technologies:But with hardware, you know, your scaling is based on physical. How many U.2 or PCIe slots do I have? You know how many? You know machines, can I fit in the rack? You know it's very, very different. And this is a massive, massive, massive advantage. Uh, as you're moving through gpu and then to vpu, you know, is that? Um? Yes, software is still always going to give you the ultimate flexibility. You can wring every last bit of quality out of it. You can, and for certain applications it is the right solution.
Mark Donnigan, NETINT Technologies:But more and more, we are seeing that people who traditionally held this, shall we say, very, very high bar and if you didn't meet it or even exceed it, like they didn't want to talk to you, you know, they're now approaching us and saying we've seen the light of day, you know, like, like, look, our costs cannot continue in the direction they are.
Mark Donnigan, NETINT Technologies:We have run out of power in the data center. We have no more power, we have no more rack space, and it's unacceptable to answer to the CTO's office, you know, or my CEO, and say sorry, we just can't expand the service because you know we have this high bar for quality and we're not going to let anything go above it or go below it. You know, and so this is, I think, a really important trend. And for those listening, you know, if you're wrestling in your own organization, listen really carefully. You know to what is happening, because it's not that these tradeoffs mean that you're delivering, you know, lower quality. No, in a lot of cases you're delivering higher quality. It's that, this mythical high bar that the industry often would shoot for, and yet no one was actually delivering. It is now gone, and if you can't deliver hundreds of channels inside a single chassis, it's gonna be really hard to compete, you know, with with those service providers who can. So it sounds like that resonates with you what I was just saying it does, it does.
VoV 43 - Dennis Mungai, Cires21:It absolutely does.
Mark Donnigan, NETINT Technologies:Yeah, yeah. So with that, then, how did you, how did you find NetEnt? You know I'm curious and you know what. What's that journey been from? You know, moving through GPU and then, you know, to VPU, from moving through GPU and then to VPU.
VoV 43 - Dennis Mungai, Cires21:I understand that the VPUs are essentially what an engineer would call an ASIC, so they are in a very separate microcosm of their own, because it's not a GPU. It will never be a GPU, even the way that ASICs implement support for the processing they kind of do. The device behaves like an NVMe or some other specialist bus and then returns the result to you, but it's not a GPU. There are no quote-unquote host-side drivers or special hooks you have to attach to get it to work. It's simply slash, dev, nvme, 01234, as many as you have, and as long as you have the namespace initialized, then boom, you have the technology stack working up and down and that's it. Now, my introduction to the NetIn stuff was at the IBC 2023. And this is where I met one of your most brilliant engineers. Unfortunately, it's not a name I can pull out right now. That would be Kenneth Robinson. I believe Kenneth Robinson.
Mark Donnigan, NETINT Technologies:Kenneth, yeah, kenneth was there he had this open rack.
VoV 43 - Dennis Mungai, Cires21:I think by then the layout they had taken was that they have this open rack showing you how these multiple modules can fit on the storage backplane Mark. This isn't the PCI stuff. I'm talking of the NVMe storage backplane. We had a super micro. We had a 1RU Mark. This isn't the PCI stuff I'm talking of, like the NVMe storage.
Mark Donnigan, NETINT Technologies:We had a super micro. Yeah, we had a 1RU, real run-of-the-mill super micro chassis Nothing fancy, very affordable. You know this was not an expensive machine.
VoV 43 - Dennis Mungai, Cires21:Yes, and the module that they were showing then this would be the Codensity G5.
Mark Donnigan, NETINT Technologies:T1U is what we were showing yeah, T1U it looks like an SSD, you know. For listeners who haven't seen it, sorry, I'm just describing for those who aren't familiar it literally looks like an SSD. It's the exact same form factor, has really cool, you know extruded aluminum case and all. I mean it looks really cool just pops in the front to an nvme slot. You pop the latch and voila, it works.
VoV 43 - Dennis Mungai, Cires21:And I am here, and I'm here just um. I'm here looking at this hardware and I'm like I can count at least 10 slots.
Mark Donnigan, NETINT Technologies:Because the board that Nathan had in his hand this isn't like the flagship.
VoV 43 - Dennis Mungai, Cires21:This isn't a flagship motherboard. No, this is literally off-the-shelf rack, something that I could possibly put up in my own homeland. And I'm here doing the math on power, like these cards are supposed to draw in, where between, I think, seven watts give or take yeah, um so the.
Mark Donnigan, NETINT Technologies:So the t1u draws about uh 20, 27 watts something.
VoV 43 - Dennis Mungai, Cires21:You know something like that sure but you're here doing the math on the kind of stream density you can get versus, yeah, the actual form factor you are looking at and that piqued my interest to the point that I had to drag our city over and I'm like, hey, yeah, you need, you need to put your eyes on this, because this is the next frontier. You, I need your eyes looking at this, just take a look. And we picked up a bunch of brushes then. And then this is the time I went and started actually looking into the kind of product portfolio that NetInt had by then, and then I understood that there was an earlier generation called named Logan. And then I understood that there was an earlier generation called named Logan. Now I have never had the chance to use Logan, but I have seen some of the documentation and I think what caught me off guard initially at first is that I thought that NetInt was like any other VPU grid company, because AMD and even Amazon themselves have their own lineup based on the Xilinx hardware stuff.
Mark Donnigan, NETINT Technologies:So I wasn't expecting anything that's mind-blowing.
VoV 43 - Dennis Mungai, Cires21:I was simply looking at hmm, let's see what these guys have done and the problems they have addressed, and then I go through the Logan. I think that the Logan is the first generation, the one that does H.264.
Mark Donnigan, NETINT Technologies:Yeah, Logan is exactly. It's H.264 HEVC, Exactly.
VoV 43 - Dennis Mungai, Cires21:And then on the second day of attending that event on the second day of attending that event, I remember holding the brochure side by side and I'm like I'm looking at the generational jump between the first generation and the second generation and I'm like, yeah, this guy's cooked. Whoever worked on this is someone that understands the kind of problems we are not just trying to solve, but they too have run into this. They are looking at the kind of workflows that how you call it they have solved problems in the present that someone else will run to in the future and they'll be like yes, that is the product that you want. That's right, like the generational leap between the Logan and the Codency.
Mark Donnigan, NETINT Technologies:G5 is.
VoV 43 - Dennis Mungai, Cires21:It's not easy to describe because I have worked with GPUs for years and I have seen how the likes of Nvidia typically jump from a generation to a generation, but it does not necessarily imply anything significant. As far as tunability, rate control, performance is concerned. Of course they do have their jumps, but Logan, the Logan versus the Codensity G5 jump to me was like okay, this is interesting and that's what piqued my interest. The difference in the feature set, like looking at a product and you can like see the thoughts of the engineer that worked on the problem and came up with a product. This is someone that has run into a problem so unique that you can only solve it adequately on a VPU, on an ASIC, and we've had the chance to try this out in our labs. We even have an encoder version that fully supports it and it's mind-blowing the kind of throughput you can push through, something that is sipping power, and I'm talking of literally sipping power.
Mark Donnigan, NETINT Technologies:Yeah, yeah, well, you know. Again, just to throw out some numbers, I'm glad you're talking about the power and the dense. You know, of course, density, but power because it is a big deal. Usually when we start talking to a new company, a new potential customer, you can tell right away are they primarily being driven by cost reduction you know they have budget pressures or is that power? And there can be a few other things, but it falls really into about three or four buckets as to what their main motivation is.
Mark Donnigan, NETINT Technologies:The beautiful thing is is that reducing power, reducing OPEX, also cuts your capex, also reduces expenses. In other words, I sometimes will make the point, even when someone's like yeah, of course we care about cost, but we're really motivated by power, in other words, increasing density. And I'll joke and say well, the beautiful thing is is I'm going to save you a lot of money at the same time, because you can take nine out of 10 servers either out of the rack, literally decommission them I'm speaking if you're running software to do, you know or you can repurpose them, you know. The point is is that they're now available for other things, or you just literally, you know, take them out, you decommission them, you know. So it's a beautiful thing. Well, so you have built your own solution. Are you primarily using FFmpeg? Are you using GStreamer? Have you built your own transcoder?
VoV 43 - Dennis Mungai, Cires21:What does that look like? We have evaluated all these pipelines, all the way from FFmpeg to the GStreamer implementation, and what we have specifically for NetInt is a solution that's in between One pipeline it's good enough for handling on the fly live transcoding. Another handles filtering and filter chains in a much more robust fashion. And I remember you mentioning that some of these deployment challenges, with NetInt not having the tooling to do some things, the fancy stuff the likes of NVIDIA do and I believe this is where I can interject to the comment that we've actually found it to be the opposite. We have found NetIn's minimalism towards how they approach the handling of, specifically, the NVMe namespace to be possibly a blessing, because our containerized solutions do not have to go through an intermediary runner to access the hardware. It's NVMe, as long as the namespace is initialized, you have your stack at the very moment that transcoding command begins and it is valid for as long as that container instance or handle is open, and that's the kind of simplicity that you cannot out-engineer.
Mark Donnigan, NETINT Technologies:You understand.
VoV 43 - Dennis Mungai, Cires21:So for us, that minimalism has actually been an advantage, because even the appliances that you have to build to support net, their containers, are a fraction of the size for what you have to build to support an equivalent GPU solution. And we are not dialing back on rate control. We are not dialing back on quality. We are not dialing back on rate control. We are not dialing back on quality. We are not dialing back on throughput.
Mark Donnigan, NETINT Technologies:Yeah, yeah. Which in software you would have to do Again. Going back to my levers, you know example? Yeah, Wow.
Mark Donnigan, NETINT Technologies:Yeah, thank you. Thank you for pointing that out, because that is true. And that scaling sometimes we don't talk about it enough or you know it's not, yeah, okay, well, dennis, this has been an amazing conversation, but I don't want to end yet because you're about to release a blog post and, in fact, at the time that we publish this episode it'll be live. We'll make sure to put a link to the blog post, but I would like for you to give a high level overview of you know, some of the results, because you know, let's face it, there can be cost advantages, power advantages, but you know, if the encoder fundamentally can't get the job done in the best way, then you know it still may not be usable or it's certainly not usable by everyone. But I think you found a different, you know a different result. So, yeah, why don't you talk to us about the study that you did?
VoV 43 - Dennis Mungai, Cires21:the study that you did. Absolutely so, for the most typical flow that we have in terms of ensuring our CBR style play out a low latency, gop preset, a fixed keyframe interval and also being able to signal a stream that is highly resilient against packet losses In this case we have to signal HRBs. These are hypothetical reference decoder signaling. We also have to handle repeated headers. We also have to handle access unit delimiters for H.264 and HEVC and also in EV1, your encoders implement support for AV1 error-resilient encoding mode and in our extensive testing it has no performance impact. And for quality retention. We also have access to tuning knobs for long-term reference frames we call them RTRs, and these are enabled in all the tests. And we also have compatibility on some of these restricted encoding profiles for the, let's say, baseline in H.264 and core, where you can then limit the length of the long-term reference frames to one instead of entirely disabling the feature set. And it is based on this flexibility in being able to tune this encoder that these results were generated.
VoV 43 - Dennis Mungai, Cires21:And what immediately stands out here is that this is a NASIC, so we cannot therefore do direct VMUF to VMUF scoring for workloads that have had scaling applied to them.
VoV 43 - Dennis Mungai, Cires21:By scaling I mean you have taken a source like 4K and downscaled it down to something like 1080p and core and then assume that an ASIC will then give you a similar result to a discrete GPU solution that has all these silicon blocks for scaling On the ASIC.
VoV 43 - Dennis Mungai, Cires21:I understand that it's implemented differently, but regardless of the exact numbers we have here and this is something we have done with roughly 1,500 samples.
VoV 43 - Dennis Mungai, Cires21:We have an entire lab set for these samples the visual quality scores and the BD rate curve for what NetLink does is consistent. There is virtually no perceivable quality loss from any distance or from any device that has been used to view the content that has been encoded and downscaled and encoded with these VPUs compared to the GPUs, regardless of the minor point scores compared to a GPU implementation versus the VPU, because the delta here between the GPU and the CPU for especially the 1080p, that can seem a bit skewed because we have a difference points of about seven. And then again when it comes to 720p. Now this is where the VPUs absolutely shine. They blow everything out of the water, absolutely shine. They blow everything out of the water. And going straight to the score summaries for HEVC, we observed the best performance at about 720p, which gave us an aggregated mean score of 81 points, followed closely by 1080p at 79.58. This is again explained by the scalar implementation that NetInt does and again at that power capex that they operate at.
VoV 43 - Dennis Mungai, Cires21:Now as expected and also which is also conformant with other platforms we've tested on, the quality degradation, even at extremely low resolutions, remains acceptable and is also comparable to software encoding. And of course, as expected, the lowest tire ladder, at 360p variance scores about 55.15 points and then for HEVC it mirrors about the same performance number trends as HEVC. I think a different score is just minus one for both ladders. Again, it has minimal quality degradation, even at extremely low resolutions compared to H.265 HEVC, at extremely low resolutions compared to H.265 HEVC, and it's again comparable to the equivalent software encoder, that's LibX264. And, as expected, the 360p variant, which is the lowest ladder, we have scored the lowest which is about 54.
VoV 43 - Dennis Mungai, Cires21:And again, now we go to everyone, and this is where the ASIC shines. This consistently delivered the best scores across resolutions and again tops out at about 720p with a good 81 points and a 1080p scoring slightly shy of 80, 79.65. Scoring slightly shy of 80, 79.65. And then we also have extremely strong performance at the very low tires 576p and 360p compared to the other codecs. And one of the biggest highlights about these VPUs, as related to encoding demands and efficiency, is that, compared to the software equivalents and at extremely low resolutions, where someone would have possibly wanted to switch to a software encoder, they perform extremely competitively. Now, this is a result I did not expect, because the GPUs here, absolutely they fail. I wouldn't call it like a fail, but it's much, much lower than their software gearing, but the VPUs hold out on their own here.
VoV 43 - Dennis Mungai, Cires21:So for the guys that are saying hey, we want density at a scale that's targeted for, say, mobile content and I also believe this is related to the companies that would possibly do a lot of these live events and gaming and stuff where you have bandwidth constraints that need to be taken into effect. These VPUs are performing at a level equivalent to the software encoders and at the lowest solutions.
Mark Donnigan, NETINT Technologies:That's right.
VoV 43 - Dennis Mungai, Cires21:We also have access to the CUDA-axialated VM of scoring engine developed by Netflix scoring engine developed by Netflix and we have been able to also, just for an experiment, run it against the NVIDIA's proprietary scaling algorithms to see the kind of quality degradation the different scoring methods would likely impact. What we observed with NetInt and the scalars that NetInt uses actually meet the kind of scaling that the VMAF scoring expects, because the scores again across multiple runs, are identical. The differences are in dot, what do you call it? They are in the second decimal place, so it's virtually identical across multiple runs.
VoV 43 - Dennis Mungai, Cires21:And again, the direct implication here is that the scalars that the NetInt VPUs have are going toe-to-toe with a GPU-based scalar for a fraction of the power. Now, that is not a result I can explain right now, because it's something I still have to go to the lab and continue running this test, but that's an observation I have made.
Mark Donnigan, NETINT Technologies:Yeah, yeah for sure. Now, one question I had is what was your anchor quality target? Maybe you said it and I missed, but when you were comparing CPU, gpu, vpu and the library of 1500, you know, a lot of times people will choose like maybe medium or slow or fast. You know, they'll kind of choose some quality target to anchor. Did you choose a quality target?
VoV 43 - Dennis Mungai, Cires21:Absolutely. But again, when it comes to video encoding, video encoding is a craft. I would not trust a preset to dictate the kind of settings I want preserved on a given bitstream and that's why, when I started a conversation about the VMUF scoring that we have here for the specific encoders, I mentioned the knobs that we touched and what we didn't touch. And I typically do not consider presets to be a good starting point when you are tuning footage. I guess my question yeah.
Mark Donnigan, NETINT Technologies:And I and, by the way, I completely agree. But I guess my question is slightly different. It's just, you know, like for point of reference when somebody's looking at this data. Would that be comparable to, you know, like an X265 medium?
VoV 43 - Dennis Mungai, Cires21:you know roughly, I mean roughly or X265 slow or is it fast or faster? Yeah, you are looking at medium, but with a restricted look ahead. You have to set a given number of look ahead frames in software Because, again, the software encoders can be very placebo at anything below medium. At that point you're just wasting CPU power.
Mark Donnigan, NETINT Technologies:Yeah, that's right.
VoV 43 - Dennis Mungai, Cires21:So for the question that you raised yes, if you are comparing this against a software encoder, without even touching the settings, the VPUs are holding out their own.
Mark Donnigan, NETINT Technologies:Yeah yeah absolutely. Well, that's good. Well, yeah, thank you for sharing this and you know, we will definitely link up to the blog post and, of course, we'll be promoting it Also. I you know I would be really remiss if I didn't point out that you are actually going to be in our booth at NAB.
Mark Donnigan, NETINT Technologies:And we're yeah, we're doing something very special this year. You're one of the first five inaugural companies to join us. I think this is going to become a regular part of our trade show presence moving forward. But, yeah, we're really pleased that you're going to have a kiosk. You'll be showing off your solutions and obviously in the context of NetEnt, so I do want to invite all the listeners. If you're coming to NAB, odds are you'll stop by and see us, but make sure you say hi to Dennis and make sure you talk to Cirrus21 as well and you can learn more about what they're doing. Absolutely, yeah, super Well. Dennis, I know that. You know we could easily extend this into a two-part and I know you have more to share and I actually think maybe we should follow up with a part two to this. But we've gone over and I guess we probably should wrap it up. We've gone over and I I guess we probably should wrap it up. So so, yeah, thank you again for joining us on Voices of Video. It's been a wonderful conversation.
VoV 43 - Dennis Mungai, Cires21:Thank you so much for hosting me. It's also been an extremely wonderful and, I dare say, eye opening for me too.
Mark Donnigan, NETINT Technologies:Excellent.
VoV 43 - Dennis Mungai, Cires21:Excellent, well, good, well. Thank you so much. This episode of Voices of Video is brought to you by NetInt Technologies. If you are looking for cutting-edge video encoding solutions, check out NetInt's products at netintcom.
