Augmented and virtual reality: A fuzzy picture for VR

After the flurry of activity with Facebook’s $3 billion purchase of Oculus Rift in 2014, the application of augmented reality and virtual reality (AR and VR) has been slow. The lag has also been felt on the IP side, with little legislation available for companies implementing the technology and looking to protect their IP.

While active cases are thin on the ground, trademark, copyright and patent lawyers have made some predictions on how courts may soon treat these technologies.

AR and AI

One legal issue surrounding AR and VR in the US, a market leader in the technology, comes from biometric privacy laws instituted by some states.

As Kirk Soderquist, a partner at Perkins Coie in Seattle explains, these laws have begun play a role due to the combination of AR, artificial intelligence (AI) and machine learning. Soderquist paints a picture of AR technology that might become commonplace in homes and offices of the future.

“One of the interesting places that AI comes into play in respect to AR is in voice and facial recognition,” he says, giving the example of an innovation allowing users to view a projection of an avatar into the room in which they are standing.

“When you look at the room through the camera on your phone, it looks as if there is a character in your room,” says Soderquist.

This innovation also collects a user’s reaction by studying and recording their facial responses, allowing the user to communicate with the avatar.

"The courts don’t find that these are infringements because there is no confusion as to what goods and services the trademarks are associated with." - Alexia Bedat, Klaris IP

The avatar uses the information collected by the user’s facial responses to react to and engage with the user—creating an interesting legal issue.

“In order to teach the algorithm how to react to your facial responses, it needs to be taught what different faces look like,” Soderquist says.

To do this, developers are using large databases of images to teach applications the ability to read different facial expressions.

“For example, the computer has to learn what a black person looks like versus an Asian person, or how it looks when a person with a beard smiles versus someone without a beard,” he says.

These image databases may be open-source, but at the time they were created, it wasn’t anticipated how the images contained within them would be used.

In the US, some states have introduced specific laws around the collection and use of biometric information.

“Illinois, Texas and Washington all have statutes around the use of biometric information. Let’s say I was in Illinois and had my picture taken and put into a database; Illinois law says companies have to get specific consent from me to collect and use that biometric information,”  Soderquist explains.

But, at the time the photo was taken, these laws didn’t exist.

“Going forward, if a company is going to use that database to teach an algorithm how to recognise different facial expressions, it is left with the question of how to get consent,” he says.

Transformative use

Alexia Bedat, an associate at Klaris IP in New York City, says that when it comes to AR and VR technologies, many of the laws which already exist for IP protection will apply, but there are certain nuances.

One hurdle surrounding the use of copyright and people’s likenesses in VR and AR is the problem of “transformative use”.

“It will be interesting to see how the transformative test will be applied in VR,” Bedat says.

In many countries, “transformative use” of a rights owner’s IP takes the original IP and transforms its appearance and nature to such a high degree that the use no longer qualifies as infringing.

“With both technologies, the whole point is to seem as realistic as possible. You want to simulate reality as much as you can, but by doing that you cause confusion,” she says.

“For something to be fair use and not an infringement, you have to have transformed the IP in some way. But, if you include a copyright-protected work in your VR experience, it leads to a difficult crossroads where the more time you spend making something look realistic, the less transformative it might end up being,” she explains.

Michael Brown, a partner at Wiggin in London, says the issue of copyright also arises when creating VR landscapes.

For example, when VR designers try to create a realistic representation of London, they may want to include a key landmark such as Trafalgar Square. But, as Brown explains, the sculptures or architecture surrounding such landmarks may be protected under the Copyright Act.

He adds that certain moral rights exist around such landmarks, which protect them against being subject to derogatory treatment, such as being able to spray graffiti on a building during the VR experience.

"If a company is going to use that database to teach an algorithm how to recognise different facial expressions, it is left with the question of how to get consent." - Kirk Soderquist, Perkins Coie

“One would have to be careful as to whether that might infringe the moral rights of the creator of that work,” he says.

Another issue is the recent uptake of the registration of building shapes as trademarks, Brown says.

“We are increasingly seeing the commercialisation of building shapes. For example, The Shard’s primary purpose is to provide residential and office space but it also has a commercial arm whereby people visit as part of an experience,” he explains.

As part of that commercialisation, developers and designers may seek to protect the look or shape of the building by the way of a trademark.

“This raises an interesting, open question: whether the representation of a trademark-protected building could be subject to trademark litigation,” he adds.

One statute which may protect VR creators is largely known as the “freedom of panorama”. This is a provision of the copyright law which permits the creation of images of buildings and sculptures which are in a public place.

Doing the right thing

UK-based Apache, which creates VR experiences for brands such as Disney, Marvel, Martell, Mercedes-Benz, and Bulmers, transforms real-life environments in order to protect itself and its clients from trademark infringement suits.

While much of the ownership of the IP related to these experiences lies with the client, CEO Adam Vahed explains that Apache retains the rights to any methodologies used to build the experiences. Additionally, it has its own code library.

“In terms of the final product, the client would own that, but we retain the IP that went into creating it,” he explains.

“We tend not to be able to use much in the way of generic imagery,” Vahed says. In most cases, Apache’s clients will provide material or images that the Apache team converts into the desired format for the VR experience.

As Apache must create the environment, this can have both trademark and copyright implications. A job for Disney that involved creating downtown Hong Kong is one example.

Vahed recalls that although Apache needed to ensure the experience and the environment were as realistic as possible, several third-party brands would have needed to approve the experience if it had been made completely realistic.

“In that case, we would have made changes. For example we would ‘debrand’ a McDonald’s store front into a generic burger store front,” he explains, a task which is not without its own challenges.

“Debranding is a step we take. It has to be done conscientiously and we also have to make sure we are not having a negative impact on third-party brands.”

Right of publicity

Another area where transformative use is key is in the right of publicity.

This exists in the US, where an individual has the right to control the commercial use of his or her name, image or likeness.

“The right of publicity is interesting in VR and AR. For example, if in the VR experience you visit a virtual museum, and you have a representation of a famous scientist there, it brings up the question: how does right of publicity work?” Bedat  asks.

This has already proved to be a difficult area in video games. Bedat refers to a 2013 case in which athletes from the National Collegiate Athletic Association in the US sued Electronic Arts over the use of their likenesses in a video game. Ultimately, the courts sided with the sports stars, finding that their appearance in the games was not transformative enough.

“I think the same thing will apply in VR, where the more time you have spent to make the game a more realistic VR experience, the less likely it is that it will be found to be transformative,” she says.

Additionally, the right to publicity in the US is further complicated by the fact that it is a state law, and different states have different tests for deciding whether something violates a person’s likeness.

“Some states recognise the right of publicity after death; others do not,” Bedat says. For example, in California a person’s right of publicity exists for 70 years after death, but in New York, once you die, your right of publicity dies with you, she explains.

Reality bites

Advertising is an area where AR may play a part in the future, and it is already being used by some brands.

For example, a Burger King campaign in Brazil in March used AR technology to encourage users to “burn” its competitors’ adverts, as an incentive for getting a free burger.

Kimberly Culp, a director at Carr McClellan in San Francisco, says that although there haven’t yet been any lawsuits in this area, the issue doesn’t fit perfectly within current case law.

“It will depend on the use. If there is an economic incentive to bring an infringement lawsuit to stop the augmentation  of a trademark which is being used negatively by a competitor, brands might take this option,” she says.

“The industry hasn’t matured enough to the point where we are seeing trademarks being augmented all the time, but once that volume increases, we will certainly see actors who will push the line,” she predicts.

Similarly, Bedat says, the use of trademarks in VR experiences, and whether the use constitutes infringement, have yet to be clarified. But, she predicts, the courts will treat it similarly to current trademark law and look for evidence of confusion.

For example, if a VR experience depicts a food court with lots of different trademarks on display, that’s far less likely to cause confusion than a billboard at the entrance to a food court with just one brand, such as Coca-Cola.

“It’s possible this could cause the impression that Coca-Cola is endorsing the experience. But generally, with trademarks, courts have been very reluctant to find evidence of confusion and I think that will be the same with VR,” she says.

“The courts don’t find that these are infringements because there is no confusion as to what goods and services the trademarks are associated with,” she adds.

While several industry players are experimenting with these technologies, AR and VR have not so far tested the courts. One explanation for this, as offered by Brown, is that the technologies have yet to hit the mass market.

“At the moment we don’t have a market-leading, extremely accessible AR/VR option,” he says.

“Once it reaches a point where it is part of a genuine consumer product, then cases might bubble to the surface.”

Brown adds that with new technologies, it often takes the law a while to catch up with what is going on in the marketplace, and that the absence of court cases doesn’t mean litigation in this field isn’t happening.

“It takes a while for cases to get to court and often there’s opportunity for settling along the way, but it takes the right combination of parties who are prepared to litigate to that extent,” he says.

“The courts will have to grapple with these issues at some point, but the technology and commercial landscape are not there quite yet,” he adds.