Video codecs: the next episode in the SEP drama series
On the train to work, I watch episodes of Bridgerton. In those quiet moments, I can get through the period dramas that I secretly like, but don’t want to admit to. And I am watching them in resolutions far higher than even a few years ago.
The difference between Bridgerton (2022) and Poldark (2015) is striking enough. But if I watch a 1990s television series on my tablet today, it looks as if I have forgotten to put on my reading glasses. Everything is blurred and muted, whereas today’s shows are pin-sharp and vibrant.
The problem with video has always been the amount of data that it uses. A picture requires a lot of data. My laptop screen is 1920x1080 pixels (a format known as HD). If it has 256 possible colours per pixel, each picture on the screen requires seventeen million pieces of information.
To watch an episode on my laptop without my eyes noticing the flicker of changing images, I need at least 24 frames per second, which means my laptop needs to receive a billion pieces of information every two seconds. It is not long ago that our computers were connected to the internet by cables and modems which, at their best, could handle 14 million pieces of information per second.
That seems a lot, but it would take 2 days to download each episode of Bridgerton (I am getting through the series rather faster than that).
This problem, extracting a lot of information from a narrow pipe, is at the heart of most mobile phone technology development. Mobile phones have always been constrained by how little data they can receive. Radio spectrum is a finite resource, and surprisingly little of it is allocated to mobile telecommunications. Voice, emails, pictures and websites all need to be extracted from what can be sent over a radio link.
A feat of genius
If video calls were ever to become a reality, mobile phone engineers had to find a way to compress the extreme amounts of data that video requires into small enough packages that could be sent, and then extract those at the other end to display the image. It was not enough to use legacy television techniques: Four channels of television consumed much of the UK’s higher frequency spectrum in the 1990s, and we were all receiving the same terrible picture at the same time. With mobile phones, we all receive different videos at different times. We all need our own TV channel.
The same problem was being tackled within the more conventional television industry. Our increasing demand for channels, and the rise of video on demand, all needed ways to extract huge amounts of video data from the fibre optic cable that came into our living room set-top boxes.
The telecommunications and television industries collaborated to develop what became called “video codecs” (a contraction of “coding” and “decoding”).
The two codecs that you will hear about most are H.264 and H.265, also known as Advanced Video Codec (“AVC”) and High Efficiency Video Codec (“HEVC”). These were standardised by the International Telecommunications Union’s Video Coding Experts Group, in collaboration with the International Organisation for Standardisation’s Moving Picture Expert Group.
The development of these video codecs has required feats of engineering genius. Most patent litigators (and indeed judges) prefer to steer clear of mobile phone patents if they can because the patents are technically dry and complex.
Medical devices are more accessible: a patent for a cardiovascular stent or a heart valve is more understandable to most of us. Mobile phone patents, with their complex signal processing and algorithms, are not an easy read, and video patents are harder still. But they do amazing things.
The fact remains that, even with 5G, or WiFi 6, we cannot receive anything like a billion pieces of information in two seconds over a radio link. Serious ingenuity is needed to give us that amount of video resolution on our devices. An H.264 decoder can extract it from an input of just 6 million bits per second (6Mbps). An H.265 decoder could extract it from just 3Mbps.
Motorola takes on Microsoft
The first pure video patent case that really hit the headlines was between Motorola (who held a portfolio of video patents) and Microsoft, who was using that technology in the X-box gaming console. They couldn’t agree on price, litigation ensued, Motorola obtained some injunctions, and Microsoft sued for a licence in the Western District of Washington. The court set a price for video codecs in gaming consoles.
In the last few years, another revolution has occurred in how we watch television. I barely use my cable set top box any more. It is all Amazon and Netflix, high definition content streamed over radio links: Wi-Fi to my laptop or smart TV, and the cellular network to my tablet.
Those services all rely on the H.264 and H.265 video codecs to extract high resolution video images from packages small enough to be sent over radio. And this change has forced the media and cellular industry participants back together, but this time they are on opposite sides of the table as they try to work out how to license the patents in this new television model.
The traditional licensing approach of dollars per device doesn’t readily translate to companies whose product isn’t a physical device, but a service: the delivery of emotionally-charged televisual data to a commuter on the 07.54 to London.
Nokia launches multiple lawsuits
The Nokia v Amazon and HP litigation is about exactly this problem. Unable to agree on the terms on which they should license Nokia’s video codec portfolio, the parties have engaged in litigation. As one might expect with defendants Amazon and HP, the centre of gravity is in the US.
Nokia has asserted patents in the ITC in relation to imported devices, and in Delaware in relation to devices and services. There are parallel actions outside the US, and most notably in the Unified Patent Court (UPC). Nokia is not the first to use the UPC for standards-essential patent claims (Panasonic took pole position with its assertions against Oppo and Xiaomi), but it is one of the first.
As is often the case when large portfolios are engaged, the real dispute is unlikely to be about whether any of the patents are valid and infringed. Some, inevitably, will be, and some may not, but those are mere skirmishes on the way to the real battle: the royalties payable.
So this litigation is likely, at some point, to move on to a dispute about the rate for the portfolio.
In Nokia’s case against HP, the main products are laptops, so this will be a more straightforward argument about what HP should pay per laptop. The most interesting part will be how to address Amazon Video, or Amazon Prime—Amazon’s video-on-demand services.
As we are only at the start of the litigation, it may be a long time before we get to that.
Richard Vary is a partner at Bird & Bird. He can be contacted at: richard.vary@twobirds.com
