Generative AI tackles clinical predictions in cancer

A study published in Nature Artificial Intelligence presents an advanced artificial intelligence (AI) model capable of creating virtual stains of cancer tissue. The study, co-led by scientists from the universities of Lausanne and Bern, represents a major step forward in improving pathological analysis and cancer diagnosis.

Using a combination of innovative computational techniques, a team of computer scientists, biologists and clinicians led by Marianna Rapsomaniki of the University of Lausanne and Marianna Kruithof-de Julio of the University of Bern has developed a new approach to analyzing cancer tissue.

Driven by the motivation to overcome the lack of experimental data, a challenge researchers often face when working with limited patient tissues, the scientists created the “VirtualMultiplexer”: an artificial intelligence (AI) model that generates virtual images of diagnostic tissue stains.

Virtual coloring: a new frontier in cancer research

Using generative artificial intelligence, the tool creates precise and detailed images of cancer tissue that mimic what its staining would look like for a given cell marker. These specific stains can provide important information about the state of a patient’s cancer and play a major role in diagnosis.

“The idea is that you only need one real stain of a tissue, done in the lab as part of a routine pathology, and then simulate which cells in that tissue would be positive for several other more specific markers,” says Rapsomaniki, a computer scientist and AI expert at the Center for Biomedical Data Science at the University of Lausanne and the University Hospital of Lausanne, and co-corresponding author of the study.

This technology reduces the need for resource-intensive laboratory analyses and aims to complement the information obtained from experiments. “Our model can be very useful when available tissue material is limited or when experimental stainings cannot be performed for other reasons,” adds Pushpak Pati, first author of the study.

Understanding the method: unpaired contrastive translation

To understand the underlying methodology called “unpaired contrastive translation,” one can imagine a mobile phone app that predicts what a young person would look like at an older age.

From a current photo, the app produces a virtual image that simulates how a person will look in the future. It does this by processing information from thousands of photos of other, unrelated older people. As the algorithm learns “what an older person looks like,” it can apply this transformation to any photo.

Similarly, the VirtualMultiplexer transforms a photo of a stain that broadly distinguishes different regions of a cancer tissue into images representing the cells in that tissue that stain positively for a given marker molecule. This is made possible by training the AI ​​model on many images of other tissues, to which these dyes have been experimentally applied.

Once the logic defining a real dyed image is learned, the VirtualMultiplexer is able to apply the same style to a given fabric image and generate a virtual version of the desired dye.

Preventing hallucinations: ensuring performance and clinical relevance

The scientists applied a rigorous validation process to ensure that the virtual images were clinically meaningful and not just AI-generated results that sound plausible but are actually false inventions, called “hallucinations.” They tested how well the artificial images predicted clinical outcomes, such as patient survival or disease progression, compared to existing data from real stained tissues.

The comparison confirmed that the virtual dyes are not only accurate but also clinically useful, showing that the model is reliable and trustworthy.

To go further, the researchers subjected the VirtualMultiplexer to the Turing test. Named after the founding father of modern AI, Alan Turing, this test determines whether an AI can produce results that are indistinguishable from those created by humans.

By asking expert pathologists to distinguish between traditional stained images and AI-generated stains, the authors found that the artificial creations were perceived as nearly identical to real images, demonstrating the effectiveness of their model.

Multi-scale approach: a major advance

One of the key advances that sets the VirtualMultiplexer apart is its multi-scale approach. Traditional models often focus on examining tissue at the microscopic (cellular) or macroscopic (whole tissue) scale.

The model proposed by the Lausanne and Bern team takes into account three different scales of the structure of a cancer tissue: its appearance and overall architecture, the relationships between neighboring cells, and the detailed characteristics of individual cells. This holistic approach allows for a more accurate representation of the tissue image.

Implications for cancer research and beyond

This study marks a significant advance in oncology research, complementing existing experimental data. By generating high-quality simulated stainings, the VirtualMultiplexer can help experts formulate hypotheses, prioritize experiments, and deepen their understanding of cancer biology.

Marianna Kruithof-de Julio, director of the Urology Research Laboratory at the University of Bern and co-author of the study, sees great potential for future applications. “We developed our tool from tissues of people with prostate cancer. In the article, we also showed that it works just as well for pancreatic tumors, which gives us confidence that it can be useful for many other types of diseases.”

This innovative approach also has the potential to support so-called baseline AI models in biological studies. The power of these models lies in their ability to learn by processing vast amounts of data in a self-supervised manner, allowing them to understand the logic behind complex structures and gain the ability to perform different types of tasks.

“Available data on rare tissues are scarce. The VirtualMultiplexer can fill these gaps by generating realistic images quickly and for free, and thus help future fundamental models to analyze and describe tissue characteristics in different ways. This will pave the way for new discoveries in research and diagnostics,” Rapsomaniki concludes.