Security cameras are everywhere, part of our everyday life: in the stores we visit, in the buildings where we work, in the transportation systems we use to move around. Their video data is not used only for monitoring purposes; it also improves work efficiency, helps to solve problems, and enriches our lives. But as the number of cameras grows, and with them their data, it takes longer and longer to check and analyze the content. And that is where AI steps up.

Companies large and small, and many of them, are grappling with video analysis AI. One of them is Toshiba, which is developing “SATLYS Video Analytics AI.” Drawing on over 50 years of experience in AI R&D, and through dialogues with its customers, Toshiba aims to maximize the power of data. What are the strengths of the system it has developed and continues to refine? We interviewed the leaders of technology development and commercialization to find out the secret of how AI can be quickly, easily and widely introduced to the market.

Requirements for video analysis AI that really solves problems

Ensuring that self-driving cars can detect people, checking for defective products, picking goods in warehouses, detecting lesions in patients—all areas where video analysis AI has improved accuracy and won significant labor savings. Yuuji Irimoto, the technical development leader for SATLYS video analysis AI, is well aware of this: “In the last 10 years or so, video analysis AI has advanced by leaps and bounds, and we are starting to see its use in business and daily life.”

“In this third wave of AI, which began in the 2000s, driven by deep learning, demand for video analysis AI has exploded,” says Irimoto. “A particularly important factor is the improvement we have seen in computer performance in recent years, including graphic processor units. These are arithmetic units dedicated to video processing, and their ability to perform enormous calculation has been a major catalyst. In about 2019, we started receiving more inquiries asking us, “Can we solve this problem with video analysis AI?”

Yuuji Irimoto, Expert, Software & AI Technology Center, Toshiba Digital Solutions Corporation

In this current wave, companies have been joined by universities and other research institutes in developing video analysis AIs capable of highly accurate object detection, and making them widely available. Many startups have emerged to take advantage of these advances. However, as Irimoto is quick to note: “General-purpose object detection AI alone will not deliver solutions to the broader problems faced by companies and local governments.” For that, we need to dig deeper. “The reality is, real problems can be solved only by combining an AI that detects objects with one that recognizes people’s faces and actions, plus the ability to process low quality video to improve detection accuracy, and above all, to adjust the AI’s learning and inference parameters. This is what we mean by offering video analysis AI as a solution.”

This understanding lies behind the SATLYS Video Analysis AI that Irimoto helped to develop. It is an AI analysis service build on the knowledge Toshiba has gained from manufacturing, and it analyzes image and sensor data to support system optimization and autonomous operation.

“With SATLYS, we have enhanced our AI lineup with measures to tackle a wide range of problems” says Masayuki Ashikawa, the man who lead commercialization of the AI system. He was engaged in the introduction of the video analysis AI, and in its management and maintenance once deployed.

“Initially, with SATLYS, we listened very closely to customer requests, analyzed video data, and developed and introduced custom-made AI to support business optimization and autonomous operation. As we did this, we came to realize that there were issues common to many customers, and they would really appreciate it if we had AI that could be quickly and easily introduce as a solution.

“Acting on this, we put together 10 types of ready-made AI that have been trained with basic data and that are frequently used, and on which we have received a lot of customer feedback. That is the current suite of products that make up SATLYS video analysis AI.”

Masayuki Ashikawa, AID, AIS, DEC, Toshiba Digital Solutions Corporation

SATLYS Video Analysis AI can be quickly, easily and widely used

The 10 trained AI currently available in SATLYS are: 1. object detection and tracking; 2. pose estimation; 3. face recognition; 4. cross-camera tracking; 5. crowd density estimation; 6. action recognition; 7. suspicious behavior detection; 8. object counting; 9. age and gender detection; and 10. people counting in area.

“SATLYS Video Analysis AI” Trained AI

This large number of trained AI gives SATLYS strength in depth, making it possible to introduce AI quickly and easily, and in a lot of situations. There are many companies that can provide video analysis AI specialized for a certain purpose, but Toshiba can meet customer needs by combining multiple AI. Toshiba operates a wide range of businesses centered on manufacturing and knows the field well, which allows it to specialize in making meaningful proposals to customers that are not just about AI, but grounded in dialogues that lead to real solutions.

Another positive of the system is that it is not tied to a particular model of camera. Generally, video analysis AI proposals are often accompanied by a requirement to use this or that camera to record data, but SATLYS Video Analysis AI uses whatever camera the customer uses.

Another factor is the way customers use AI, either in the cloud or installed on a PC at the customer’s premises.

Ashikawa explain how this works. “We can use the cloud to meet requests like, ‘We want to start using it as soon as possible,’ ‘We want to start small’ or ‘We want to avoid maintenance.’ In other cases, where customers have a vast amount of video data, have to pay data transmission fees to go online, or don’t want their confidential data in the cloud, we can incorporate the AI into their work environment. Another thing we can do is first of all trial the AI in the cloud, and if that goes well, we can introduce it on-site. We see being able to make choices based on the customer’s situation as a big advantage.”

Features of SATLYS Video Analysis AI

Supporting society, business, and daily life

SATLYS Image Analysis AI offers a range of features, but at the core of it all are Toshiba’s technological strengths. For example, in facial recognition technology benchmark testing sponsored by the United States’ National Institute of Standards and Technology, in 2021 Toshiba ranked among the world leaders, and first among Japanese companies in three categories, including “12-year change in facial images.”

“There are three points to bear in mind,” says Ashikawa as he explains SATLYS’ technological strengths. “First is the sheer volume of learning data. We use domestic and international publicly available data, and our own original data collected through Toshiba’s business. However, it’s not just a case of the more data the better, the AI has to be precisely put together to meet the purpose at hand. Toshiba has the experience and knowledge needed to do that.

“Second is the ability to develop the algorithms that are the foundation of the AI. Even if you collect good data, it all means nothing if the algorithm that reads it is inaccurate. Toshiba has developed a wide variety of highly accurate algorithms in-house, making full use of knowledge built up over the years, and of the cutting-edge technologies it has made available to the public.

“Third is adjustment of the settings that determine how the data and algorithms are combined, and how they run. This adjustment is in the domain of data science, and Toshiba has many highly skilled data scientists. By combining these three factors we are able to achieve highly accurate video analysis.”

Strengths of Toshiba’s Video Analysis AI

Irimoto adds another observation: “Because of these capabilities, we are good at handling what is called wild data, such as grainy or badly angled images.” This advantage makes it possible to analyze video data from security cameras that have been in use for many years.

“Toshiba has a history of over 50 years of AI research and development, and ranks third in the world in the number of AI patents and first in Japan^*1,” says Irimoto. “That’s why we can create innovative technologies and solutions that solve problems. Improvements in computer performance and the release of algorithms into the public have certainly lowered barriers to entry into AI, but it’s a different story when it comes down to mastering AI. To do that, it is necessary to have real on-site knowledge and a lot of practice. Those are the factors that underpin Toshiba’s strengths.”

*1 World Intellectual Property Organization 2019 report

The objective with SATLYS video analytics AI is to be a supporter of social infrastructure, business, and daily life. There are various ways it can be applied to infrastructure, such as monitoring the condition of rolling stock and track, and river and dam maintenance; in business, it can be applied to security in business premises, behavioral analysis in factories, and ensuring safety through object detection; in daily life it can detect lost items and dangerous objects, report suspicious persons, and search for lost children.

Targets for SATLYS Video Analysis AI

As a coda to this article, we asked Ashikawa and Irimoto to share their views on the future.

At the moment, we offer 10 kinds of AI, but we will expand the lineup to meet customer needs,” says Ashikawa. “Toshiba’s management philosophy, “Committed to People, Committed to the Future.,” is about being useful to society. All of Toshiba’s products and services exist for this purpose, and I would like AI to play a part in this, and to create value by combining AI with those products and services. I would like people to say, “Toshiba is all about AI.” said Ashikawa.

It is important to show customers who intend to introduce AI what SATLYS Video Analysis AI can do for them,” says Irimoto. “What customers are looking for is not technologies, it is actual solutions, so we need to increase the number of practical examples so that customers can easily visualize what we can do. To this end, we will continue hold to hold dialogues with customers and listen to them carefully as we develop technologies, always aware that we need to produce technologies and value suited to solving their problems.“

SATLYS Video Analysis AI frees customers from the need for the advanced knowledge and extensive preparation required to introduce AI, and will be applied as a solution that uses video data to help solve problems facing Toshiba’s customers and society. It will evolve as a supporter that protect society, business, and daily life.