Imagine an e-book that tracks your reading speed, highlighted quotes and even pages where you stopped reading. This is a simple example where data has become increasingly ubiquitous, instantaneous and interconnected. And companies need to leverage data they gather to make intelligent decisions to maintain a competitive edge and thrive in a landscape that has been affected by the COVID-19 pandemic. Data is king— one particular sector of interest is social infrastructure and it needs to create real value to make lives safer and more comfortable. The question is, how do companies collect and analyze data, and how well do they utilize this data in the real world?

Toshiba promotes an organizational structure resilient to changes in the business landscape—one where automation, backed by a combination of data and AI technologies, handles the automatable processes, and where the people are free to express the creativity that makes them human. The company is evolving, innovating upon its more than 140-year manufacturing history. We spoke to those at the front lines of this process of innovation, to see what the future will bring.

Expert Techniques Applied to Al through Data

In recent years, people and objects are generating ever-increasing amounts of data. Room temperature monitors, fitness trackers, machine monitoring devices… The sheer amount of this data has changed the way companies operate, and how they make themselves competitive. How can companies utilize this vast array of data and turn them into something of value? The answer, it seems, is innovation. More specifically, companies need to collect large troves of diverse data efficiently, analyze the data through artificial intelligence (AI), and use the analyses to help solve social issues.

The current darling in the world of AI is deep learning, made famous perhaps by the computer program AlphaGo, which defeated a professional human player in a game of Go. In deep learning, multi-layered networks conduct machine learning on a large amount of data, allowing the program to analyze complex phenomena. The world’s major companies have used this combination of data and AI to create new value in many ways.

An example is the manufacturing industry. Toshiba enables businesses to bring various parts of the manufacturing process, from planning/sales to development/design, all the way to processes including installation and maintenance, into the same data network. This data will then be analyzed through AI to create new value, including optimizing the balance between product quality, delivery dates, and costs. The competitive edge for the manufacturing industry now extends to the entire value chain, not just the finished product. As optimization and automation are increasingly integrated into the manufacturing process, we will be able to focus on creative work and create new value.

Optimization and automation also make value chains more resilient, even against world-altering situations such as the COVID-19 pandemic. By allowing companies to maintain a stable level of production for a steady supply of products, these processes enable people to focus on resolving the key issues on hand. For example, when the Japan government announced an emergency declaration during the COVID-19 outbreak in April 2020, Toshiba’s response was to impose general travel restrictions for employees, including those at manufacturing sites, while allowing the work needed for maintenance with social measures in place that resulted in minimal disruptions to overall work processes. This efficiency is attributed greatly to Toshiba’s early implementation of advanced AI technologies capable of making higher-level decisions and its extended automation of the production line. We asked two young engineers about their work which involves in research to further accelerate the division of responsibility between human beings and machines.

“As part of the inspection process to find defective items in a production line, a sensory evaluation where human senses, such as vision, is carried out to evaluate the quality of a product. However, there has been an on-site labor shortage for sensory evaluations, and we’ve struggled with the challenge of transferring the knowledge gained from experienced evaluators. So we’re researching on how we can use AI deep learning to apply the tacit knowledge of these experts on these machines. By using AI for these sensory evaluations, we’re trying to identify defective items efficiently, and to ensure that inspections are conducted consistently with quality standards. We want to divert our human resources to focus on more high-level tasks —for example, analyzing the defective items that the machines have discovered, or developing and implementing ideas to promote consistent quality.” says Yusuke Hirose, Research Lead at Toshiba’s Corporate Manufacturing Engineering Center.

Because different people have different levels of skills and sensory abilities, there will always be some degree of variance in the evaluation standards for sensory evaluations. This is especially true for items that have been put through processing—for example, welding or molding. In these cases, it may be difficult for non-experts to be able to evaluate the quality of the finished item. This is where the research comes in. AI can take image data of processed items deemed high-quality by experienced engineers, and extract a model of a high-quality item to serve as the standard for the evaluations. By conducting the inspections against this model, a stable inspection system can be established with very little variance. But for this to work well, the image data used to create this model can’t just be collected at random. There are complex variations in what makes a high-quality item—the shape and appearance of processed items can differ very slightly according to the specific unit of equipment used to process it, and the items may be affected by variations in different materials. High-quality assembled items tend to look very similar, a product of an averaged-out set of specifications can serve as a model for AI. But for processed items, this process will not create an acceptable model. Hirose’s team is currently researching ways to overcome this obstacle, experimenting on large collections of image data (images of high-quality items) under various conditions, and applying deep learning to create an appropriate model for processed items as well.

This data collection process is helped greatly with the experience acquired by the company through the years. Toshiba’s experienced engineers know intuitively how an item will turn out based on the process conditions including the materials and machines used. Hirose and his team members visited factories many times to conduct interviews with these engineers, gathering the data that they needed on sites. Many expert advice, including how to capture stable, high-quality images (consider images to be taken at various points and conditions in a manufacturing line), also emerge based on the company’s long-standing experience in manufacturing. After the AI-developed models were made, the team went to assess the suitability of the models, carefully and methodically reconciling their models with the engineers’ expert knowledge.

Another challenge that Hirose’s team is working to overcome is the “black box” element of deep learning. In the deep learning approach, an image data of high-quality items is entered into a multi-layered network, after which the system automatically “learns” the characteristics of a high-quality item. However, the system is not able to explain why an item is deemed high-quality or defective, which renders the system as a system with unknown inner workings (a black box). To overcome this obstacle, Hirose’s team has implemented a technology that visualizes the area that AI focused on to make its judgment. By giving insights into the AI’s reasoning, the team is working to make the system more useful.

The ability to weed out defective items efficiently will not only improve a company’s productivity—it will also help minimize other issues, including complaints and recalls/repairs of defective products. This will be a significant AI-powered innovation in the manufacturing industry.

A Self-Optimizing Welding Device, Fueled by Its Own Data

Welding is one area in the production line where skilled professions are required. These welders use their vision, hearing, and tactile senses to manage the welding process and make necessary adjustments to the welding conditions. New welders undergo rigorous training and gain experience over time. In Japan, there has been an increasing burden on these experts due to a labor shortage because of the declining population. Construction period tends to revolve around them, and can be delayed based on their availability.

But what if the welding process is handled by a robot? What if a robot is able to control the welding, always carrying out the process under the most optimal of conditions? Working to find the answers to these questions and more is Tetsuo Sakai of Toshiba’s Corporate Manufacturing Engineering Center.

“Germany’s Industry 4.0 project, which started in 2011, places its focus on technologies such as automatic control systems and the interconnection of devices and systems. Here at Toshiba, we have a production concept called ‘Digital Manufacturing,’ proposed in 2000, that aims to make maximum use of IT technologies, and that we’ve continued to innovate in the area of manufacturing. This is the landscape in which we’re conducting this research—to apply AI to welding technologies, which haven’t been automated in a way like other processes, and enable a robot to conduct welding autonomously. This will help companies save on labor costs for processing and allow people to focus on developing technologies and systems that will create innovation and new value. And this, in turn, will help drive sustainable increases in corporate value. I think that having such high-level manufacturing technologies will help companies contribute to society, and make life richer and meaningful.”

The general idea of Sakai’s research, using laser welding as an example, is as follows. First, collect clear, real-time image data of the areas to be processed, so that the researchers can see how the robot is performing the welding works. AI is then used to extract certain characteristics from this image data, for example, the spots where the laser hits and the spots where the heat of the laser melts the metal. The data extracted is hosted on an online platform, where they are compared to a pre-established welding states database, through methods such as simulations and experiments. The system then selects the optimal laser welding conditions based on the state of the next welding spot. These optimal conditions are then fed back to the robot in the physical space, so that it can conduct the appropriate welding process. This process, repeated quickly many times, is what the team is using to develop an autonomous welding system—one that can maintain optimal laser welding conditions at all times.

Sakai’s research was a first for Toshiba in the area of welding. He started by forming a small team together—a representative on the business side to manage the on-site welders, and young members of the research lab. Sakai’s motivation was simple. He believed, firmly, that there’d be great value in combining Toshiba’s manufacturing expertise, accumulated over years and years of investment, with its digital analysis technologies, cultivated through over 50 years of AI research. Implementing AI-based knowledge and expertise into manufacturing would, he believed, also result in a boost for the Japan’s manufacturing industry, and this belief fueled the progression of the research. Later, when they were given the indication that their research would be reflected in the manufacturing process, they transitioned into a more collaborative team, one that coordinated with each of the relevant divisions. The actual development of the AI technology that would extract the characteristics of the welding state was handled by Hirose’s team (in the previous story), and the welding state simulations were developed specifically for this research by young engineers. One engineer’s passion for innovation had lit a fire in his teammates’ hearts, driven them to put their heads together and create new value.

Sakai’s research is an example of what’s called a cyber-physical system (CPS)—a system that creates value by sending real-world data into cyberspace, analyzing it, and feeding it back to the real world as easily-digestible information and/or knowledge. Toshiba is currently utilizing CPS technologies to become an infrastructure service company, and is applying these technologies to a wide variety of areas, including manufacturing sites. Toshiba engineers like Hirose and Sakai work to refine their technologies, always with the knowledge that these technologies will one day have real-world applications.

One example of such a real-world application is the project Toshiba is conducting with Gestamp, the world’s largest manufacturer of automobile press components. Drawing on the rich manufacturing expertise of Toshiba, the companies have developed an IoT solution that applies Toshiba’s AI analysis technology to data from internal welding quality detection by AE* sensors, along with surface welding quality detection based on camera image data.
*AE (acoustic emission) sensors: Sensors used to check bridges and large buildings for vulnerabilities. The sensors detect the characteristics of acoustic waves in the high frequency band.

Using AI to automate processes such as sensory evaluations and laser welding, people are left free to focus on the kind of work only human beings can do, the kind that creates new value. This kind of development is made possible by Toshiba’s tremendous assets—the combination of the Toshiba’s more than 50-year experience in AI research, and its over 140 years in manufacturing. And the results of these types of research don’t just end at the production line, or even just in factories. They’re already being extended to areas closer to us, such as karaoke and TV, and even areas such as medicine. Toshiba’s AI technologies, intriguing on their own, gained even more importance when combined with the Toshiba Group’s various troves of expertise. And powered-up in this manner, the technologies (and the people who develop them) are ready to turn on the promise of a new day.