Many IP offices are investing in artificial intelligence technologies such as machine learning and neural networks to develop innovative services for users. James Nurton looks at some of the new tools.
If you visited the World Intellectual Property Organization’s (WIPO) exhibition booth at INTA’s Annual Meeting in Boston, Massachusetts in May 2019, you might remember seeing a demonstration of the new image search tool in the Global Brand Database (GBD), which was launched just a month before.
Using machine learning, GBD’s image search tool identifies concepts, such as a dog or a tree, within an image, and then finds similar marks that are already registered. The search results are then presented in order of their similarity to the original image.
Now, 18 months later, the tool is being used throughout the global trademark community. The biggest demand has been from China, which accounted for about 35 percent of users in the first year.
“There are a lot of recurring users; people start to use it and keep on,” said Christophe Mazenc, WIPO’s Director of the Global Databases Division.
He said the image search tool covers all the countries in the GBD (55 at present), which includes over 22 million images.
It is also being used in combination with other search tools, such as the Vienna Classification and Nice Classification. One of the advantages is that the GBD’s image search can be used with any combination of search criteria.
What AI Offers
The GBD’s image search tool is just one example of how intellectual property offices (IPOs) worldwide are investing in digital technology to provide more information and services to users, improve efficiency, and make greater use of their data collections.
Increasingly, this technology involves some form of artificial intelligence (AI).
“Trademark offices worldwide are engaging with innovators in AI to promote the predictability and reliability of IP rights relating to AI technology,” according to Julie Katz, Owner at Katz Group LLC (US).
A report filed by INTA’s Emerging Issues Committee in October 2019, “Use of AI by IP Registries,” revealed that most of the registries consulted for the report were already in the early stages of adopting AI solutions.
The report suggested that AI is likely to increase the confidence that trademark professionals have in a trademark office, resulting in an improvement in the relationship between IPOs and their stakeholders.
It predicted: “Consequently, just as the public welcomes the development of a market for any highly coveted, useful new item, so too will the public be impacted positively by, and adapt favorably to, the new technology.”
AI tools already launched or under development in the IPO sphere include:
- IP Australia’s tools such as Trade Mark Assist, Smart Trade Mark, IP GAIN, and Transactional Digital Services;
- Helpdesk Services at the Canadian Intellectual Property Office, based on the IBM Watson suite of tools, and similar tools at offices such as those in Singapore and Uruguay;
- Image search at the National Institute of Industrial Property in Chile, based on an algorithm developed by the Engineering School of the University of Chile;
- Trademark classification and image search at offices including the China National Intellectual Property Administration (CNIPA) and the Japan Patent Office (JPO);
- The use of a machine translation tool called Babelscape at the European Union Intellectual Property Office (EUIPO); and
- Trade Marks Distinctiveness Checker and Trade Marks Class Recommendations Tool, both being developed by the IP Office of Singapore (IPOS) in association with local research institute A*STAR.
Conferences to discuss IPOs’ use of AI have been held by the EUIPO, the European Patent Office (EPO), and WIPO. In October last year, the United States Patent and Trademark Office (USPTO) advertised for the newly created post of “senior level AI technical expert,” saying that the new position will support the Office’s “continuing initiatives to develop AI tools for the examination of patent and trademark applications.”
“Trademark offices worldwide are engaging with innovators in AI to promote the predictability and reliability of IP rights relating to AI technology.” - Julie Katz, Katz Group LLC
It is not only IPOs that are benefiting from new technologies.
“The development and adoption of technological systems and tools has completely upended the practice of IP law in recent years,” according to Colm Dobbyn, General Counsel of IP at Mastercard (US), whose team uses a number of technological tools in its patent and trademark practices.
“IP information that was previously unavailable can now be readily accessed and evaluated in a myriad of ways to provide valuable business insights,” he said.
Ms. Katz noted that blockchain is another innovative technology benefiting practitioners and IP rights holders alike.
“Blockchain apps enable brand holders to access proof of continued use over time when the need arises to prove priority or acquired distinctiveness in litigation or other challenges to trademark rights, saving time and money,” she said.
“Comparatively, digging through company records and scanning documents is very time-consuming and, thus, expensive.”
Trademarks offer many opportunities for the application of AI tools such as machine learning or deep learning, given the importance of concepts such as similarity and distinctiveness. Assessing the similarity of images has been particularly challenging, as traditional computer systems are very good at finding similar words or letters but not images—yet it is estimated that 40 percent of the world’s filed trademarks contain images.
GBD’s image search tool was developed by WIPO’s Advanced Technology Application Center, which also created WIPO Translate. Mr. Mazenc said it works using “convolutional neural networks.”
He explained that the neural network has been trained to classify images according to the U.S. design code classification. The system can find estimations from more than 1,300 codes. There is a large vector of codes describing images according to the classification.
For example, if a user does a search for “dog,” a classical approach to searching based on classification might generate thousands of results. Those results would have to be examined manually to assess which ones are most similar. With the image search tool, the results are sorted visually, meaning there is a better chance that the desired result will rise to the top of the list.
This will not remove the jobs of trademark attorneys or examiners, but it will make them more efficient and productive, suggested Mona Remlawi, Head of Brand & Design Database Section at WIPO.
“You can never guarantee a perfect search, as there is no definitive answer. But we know that the image search can do in a few seconds what might have taken many hours to do with the Vienna Classification,” she said.
“We can improve the training sets so that the system can find more concepts in an image.” - Christophe Mazenc, WIPO Global Databases Division
Lessons from IPOS and IP Australia
Image searching is not the only task that can be enhanced by using AI. Many IPOs see the potential to use similar tools for various services, from chatbots on helpdesks to assisting examiners better assess distinctiveness. Two IPOs that have invested heavily in digital technology are IPOS and IP Australia.
IPOS has launched two initiatives, IP2SG and IPOS GO, as part of its goal to create a conducive ecosystem for IP development in the region. The former was unveiled in 2014 as the culmination of the Office’s efforts to fully digitize Singapore’s IP services. The latter, a mobile app launched in August 2019, brings the same services to a user’s smartphone.
The IPOS GO app integrates AI technology to enhance image search capability for similar trademarks.
“With trademarks being the ‘meat and potatoes’ of the IP world, IPOS GO brings great convenience to businesses by shortening the average time taken for simple trademark applications from 45 minutes to less than 10 minutes,” said Isabelle Tan, Director, Registry of Trademarks, at IPOS.
She added: “The greater ease of filing and resulting cost and time savings will encourage more local businesses and entrepreneurs to properly protect their IP through registrations, which will minimize downstream business disruptions and costs arising from inadequate IP protection.”
In 2019, IP2SG handled more than 99.9 percent of the IP Registry’s transactions electronically.
In August, IPOS announced that IPOS GO had been downloaded more than 3,500 times and used for more than 500 trademark applications in the year since its launch. According to the press release, IPOS will be extending IPOS GO’s renewal feature to patents and registered designs, enabling users to manage all their registered IP in one mobile platform.
At IP Australia, one of the services that incorporates AI is Trade Mark Assist, an education tool to help businesses navigate trademark prosecution. Since Trade Mark Assist was launched in 2017, it has been used by more than 50,000 customers and has a 90 percent satisfaction rating; those who use it are 18 percent more likely to be successful in receiving an acceptance report, according to the Office.
IP Australia Acting Chief Information Officer Graeme Jones said services such as this are not designed to be a substitute for lawyers or attorneys.
“Trade Mark Assist is not providing legal advice. It’s designed to help make the system work as well as possible and to assist people to make decisions early on,” he said.
He added that research indicates that machines and humans work best when they can complement each other.
“Putting the strengths together means we can augment quality and efficiency. Machines can eliminate the lower-value work and free you up to focus on higher-quality outcomes,” he said.
Mr. Jones said IP Australia was “an early adopter of digital,” and has a small team of specialists working on machine learning.
“The development and adoption of technological systems and tools has completely upended the practice of IP law in recent years.” - Colm Dobbyn, Mastercard
In September 2019, IP Australia launched IP GAIN, a simple, secure, cloud-hosted service to share AI and machine learning tools with other IPOs.
“That’s the way the IP system is going to move forward—with collaboration and working together,” Mr. Jones noted.
While IPOS and IP Australia are leading the way in their use of AI in trademark services, the five largest IPOs—which between them handle approximately 85 percent of the world’s patent applications—have formed a joint Task Force on New Emerging Technologies and Artificial Intelligence, to assess the impact that innovations are having on the patent system.
In January, the CNIPA, EPO, JPO, Korean IP Office, and USPTO held their inaugural meeting to explore the legal, technical, and policy aspects of new technologies, including AI.
Involvement in the Task Force also aligns with the EPO’s Strategic Plan 2023, under which the EPO intends to develop its Berlin office into a center of expertise on AI.
One of the key characteristics of the new digital tools is that they are constantly evolving in response to technological developments and user needs.
At WIPO, developers have enhanced the image search in the GBD in response to users’ needs, said Mr. Mazenc.
“We have tuned the image search system for speed, and you can now see up to 100 images at a time,” he said. “We can improve the training sets so that the system can find more concepts in an image: some concepts are more difficult than others.”
Ms. Remlawi added: “The next big challenge is to do something similar for industrial designs. That is more complex because you have several different views of the same image, and they may be photos or drawings, black and white or color. But we are confident that we will find a way to teach the machine similarity.”
AI also offers the opportunity to create completely new services. Last year, IP Australia launched a trial of its Smart Trade Mark, a digital representation of trademarks using blockchain technology that can help fight counterfeits, provide online validation, and offer additional data such as images and video. It won in three categories at last year’s Canberra iAwards, which promote excellence in digital innovation in Australia.
Mr. Jones said Smart Trade Mark can be adapted for many different applications—for example, to help protect indigenous artworks.
The growing use of AI tools such as machine learning and deep learning, as well as blockchain and related technologies, reflects the needs of users for greater transparency and efficiency and provides opportunities to make better use of IPOs’ structured data.
As noted in “The Intellectual Property Office of the Future Report” released by INTA’s IPO of the Future Think Tank earlier this week, the world has become increasingly technologically dependent and IP systems must adapt to the changing nature of innovation.
“The IP ecosystem is not immune to the disruption of emerging technologies,” the report said, predicting, “These changes will test the strength of the IP system, the value of IP rights, and the confidence in IPOs.”
INTA 2020, IP offices, artificial intelligence, machine learning, neural networks, innovation, WIPO, technology, trademarks, stakeholders, patents, EUIPO