Machine learning models teach each other to identify molecular properties

Biomedical engineers at Duke University have developed a new method to improve the efficiency of machine learning models. By combining two machine learning models, one to collect data and the other to analyze it, researchers can work around the technology’s limitations without sacrificing accuracy.

This new technique could make it easier for researchers to use machine learning algorithms to identify and characterize molecules for use in potential new treatments or other materials.

The research is published in the journal Artificial intelligence in life sciences.

In traditional machine learning models, a researcher will input a set of data and the model will use that information to make predictions. While this is often effective, the capabilities of these tools are limited by the datasets used to train them, which can often miss key information or include too much of one data type, thereby introducing bias into the model.

Instead, researchers developed a technique known as active machine learning, in which the model is able to ask questions or request more information if it detects a gap in the data. This questioning ability makes the model more accurate and effective than its passive counterpart.

Although active learning is very effective for machine learning models, the technique faces serious limitations when applied to more complex deep neural networks. Designed to mimic the human brain, these deep learning models require much more data – and computing power – than is often available, limiting their accuracy and effectiveness.

Reker and his team wanted to determine whether an educational concept known as “yolked learning” could be applied to the world of machine learning to improve these systems.

During group learning, a student will actively learn the content. They can ask questions and check different textbooks for missing information. A second student will then be responsible for reviewing the information that the first student deemed important to understanding the lesson. While studies of paired learning have shown that the second student often fails to learn concepts and retain knowledge as effectively as the student who is actively learning, Reker believes that this technique shows promise in learning automatic.

“An active machine learning model knows how to go through a data set and not only identify important information, but also ask for any missing information that it deems important,” Reker said. “We wanted to see if we could use this active machine learning model to ‘teach’ another model using the data that the active learning model deems important.”

To understand how machine learning relates to active machine learning, the team used an active learning model to identify different characteristics of molecular compounds that are important to its success as a therapeutic, such as potential toxicity of the molecule and the metabolism of the molecule. An associated system was developed by combining different “teaching” machine learning models with different “student” machine learning models to identify the same features based on data selected by the “teaching” model .

The team found that although active machine learning was more accurate than the associated system in most cases, the associated models were very effective under certain parameters.

“We found that the performance of the teaching model was very important to the student models,” Reker explained. “Just like in real life, an ineffective teacher means the student is not prepared to succeed. If the teaching model has not identified useful data, the student’s model has not succeeded to decipher them.”

These results motivated Reker and his team to test combined learning with a deep neural network model called “student,” dubbed Yoked Deep Learning, or YoDeL. Unlike an active deep learning model, in which the deep neural network itself is responsible for data selection, YoDeL has another active machine learning algorithm that acts as a “teacher” to guide actively acquiring data for the “student” of the deep neural network.

In several comparative studies using various models, the team found that their YoDeL technique was either more efficient or just as accurate as an active deep learning system when identifying different molecular features. They also found that YoDeL was much faster, often taking only minutes to complete a task, whereas deep active learning took hours or even days.

The team has filed a provisional patent on the YoDeL technique, but they already plan to continue testing and improving the model’s parameters as well as using it in the real world.

“There are many different models of machine learning and deep neural networks, so we want to figure out which pairs are a really good match for this integrated learning,” Reker said. “YoDeL’s ability to leverage the strengths of classic machine learning models to improve the efficiency of deep neural networks makes it a very interesting tool in a constantly evolving field. We are optimistic that we and other scientists will be able to use this tool in the near future. to help discover new drugs and drug delivery solutions.