2026-04-20-am-ai-revolution-in-cancer-research

Date: 2026-04-20 Primary topic: agentic-ai Cross-referenced from: agentic-ai, bioinfo-tools, virtual-cells Word count: ~15,915

Auto-generated captions

This transcript is Vimeo auto-generated English. Expect misheard names and technical terms. Use the ripgrep/search approach for exact quotes.

Our program is about to begin. The AACR Annual Meeting is the focal point of the cancer research community, where attendees gather to share the latest advances in cancer science and medicine. With the theme of Precision. Partnership. Purpose.: Advancing Cancer Science to Save Lives Globally, the AACR Annual Meeting highlights the work of the best minds in cancer research from around the world. Welcome to San Diego. Good morning. Welcome everybody to our early morning plenary session on artificial intelligence. Thank you very much for coming. Please join us. There's plenty of free space over here. My name is Jakov Catsa. I'm an oncologist and I'm doing AI research in Germany, and I was part of this year's AACR scientific committee, responsible for some of the AI content. We've had tremendous artificial intelligence content at this conference, and this is our crown jewel. This session here is where we are presenting some of the most exciting advancements in artificial intelligence. So before I call the first speaker here on the stage to join me, I'm going to present you with a very short introduction on the topics that we are going to cover here today. These are my disclosures, and I just have one slide here to outline the evolution of the artificial intelligence techniques as they are applied to cancer research, as they have evolved over the last decades. Many years ago, in the 2000s, we already had methods that we now call classical machine learning methods. These methods allow us to analyze structured data, tabular data, features that we extracted from images. But for complex data such as image data, we needed more capable methods. So in the 2010s, deep learning came along and allowed us to really extract patterns from more complex data. And this was supercharged by the advent of another technology, foundation models, just a couple of years ago, whereby you have a large data set, you can train a big model one time on this large heterogeneous data set, and then you can repurpose it for many downstream tasks. Another type of foundation model is large language model, the technology that powers all of the chatbots that everybody's using. But one of the most exciting applications of large language models is that they can reason about complex procedures, and this enables us to build AI agents around them that can solve complex procedures. So the two technologies that we will focus on in this session from a technological point of view is foundation models and AI agents. And here's just a bit of terminology on these. Foundation models are trained on large amounts of data, and their purpose is to provide you with embeddings of the data, so compressed representations of the data by which you can solve many downstream tasks. Agents can be single-agent systems or multi-agent systems that help us solve complex multi-step procedures. And of course, you can put both of these technologies together, and have agents that use foundation models as tools that can really do great things, as we will see in the upcoming talks. So thank you very much. This was my short introduction, and now I'm very pleased to introduce our first speaker. Our first speaker is Dr. Jure Leskovec. He's a computer scientist and a professor at Stanford University. His team was a pioneer about AI agents applied to biomedicine, specifically in cancer research. Jure's team published Bi Omni, which was the first and the most widely used collection of tools for AI agents in biology, and these tools are where the magic happens and where agents get the capabilities to solve problems in cancer research. So without further ado, let me welcome Dr. Leskovec here on the stage. Thank you very much. Good morning. So I'm here today very excited to talk to you about what's new in AI and in particular, what is new in agentic AI. What I'll talk about is how do we go from AI as chat agents like ChatGPT that everyone here knows, but really think about developing copilots and co-scientists that allow us to do science better, faster, and more reliably. If we think about scientific discovery, it really drives the civilization forward. Not only drug diagnosis and clinical diagnosis and drug discovery, but also agriculture, manufacturing, health, and so on. And all these are extremely complex processes that require super advanced and sophisticated human teams. And really, if you think about it, the pace of discovery is limited by our human bandwidth. There are petabytes of new data that is left unanalyzed. Scientists have many ideas, but it takes them time to test them out. So really, we are limited by our ability to analyze this data, act on our hypothesis, and test them out. So the question is: how can we close this gap? How can we fulfill this gap that is opening between the demand for discovery and the supply of human experts that we are able to provide?So the goal for this talk is for me to walk you through our research journey on developing an AI agent that helps us to simulate and scale the scientific discovery process. So this is about building a co-scientist, a partner to a human researcher that allows the human scientists to make progress faster and speed up the rate of discovery. What I want to contrast this with is the advanced AI tools that have been recently developed. So we are not talking perhaps about a model or a tool that somebody can use, but we are actually talking about an agent, a co-scientist, an active partner in research that helps us plan, act, debug our experiments, verify them, and also iterate. That we go from observations to hypothesis to experimental validation of those hypotheses. And we close this loop and we can run it faster, more reliably, and with greater transparency. So the goal is to speed up scientific discovery dramatically. Create a virtual scientist that helps us automate scientific research, suggest and explore novel hypothesis, perform tasks much faster, and importantly, tackle many tasks at the same time. So really parallelize and speed up the entire discovery loop. And if we look at how scientists do research and discovery today, how we work, it's actually extremely fragmented. We read research papers, we collect and process data, we run experiments and log them. We order some materials, perhaps design some molecules, write some data analysis code, analyze the data, draft a protocol. We talk to our colleagues, and only then after all these kind of small steps, we come up with a result. So the research tasks are manual, slow, and very costly to run today. So the goal is to develop an environment in which scientists are able to do their work, where all these small steps have a collaborator so that they can be run in synchrony, they can be run in a coordinated way, and this hopping between different tools and between different systems is eliminated. This is very important because the impact of scaling biomedical expertise this way would be humongous. First important consequence of this is that we can democratize expertise, in a sense that there are only few places in the world where we really have the critical mass of scientists to do the groundbreaking research. But with AI-based co-scientists, anyone in the earth gets now access to the critical mass of expertise to do their research. So wet lab scientist has access to a programmer, a programmer has access to a wet lab scientist. It also will lead to dramatic time savings because tasks once took months, now they can take minutes. And for example, drugs or therapeutics can be brought to market earlier. And what is also important is the enablement of novel findings, because these systems are semi-autonomous, they can propose and validate hypothesis at large scale, so that human scientists, for example, can only focus on the most impactful or the most interesting ones. So what we built last summer in 2025, in my research group at Stanford, is called Bi Omni, which is a general purpose biomedical agent. It's an open-source project, and it's the first general-purpose biomedical agent that helps scientists in biomedicine speed up, automate, and amplify their research. And what I want to do next is actually tell you about how we build this agent and what kind of capabilities it has and what scientists all over the world are using it for. And if you'd like to play with it, you can go to biomni.stanford.edu, and register there. As I mentioned earlier, biomedical workflows are complex, diverse, and abundant. Questions might be simple, but they require coordination and collaboration between complex teams, to work together, talk to each other, use complex tools from knowledge bases, ontologies, to all kinds of bioinformatics tools, a lot of reasoning on top, wet lab experiments, and several iterations of this, where the result of the experiment feeds back to refine the hypothesis, and only through this we come up with a discovery. One problem that is actually very important is that if you go and use ChatGPT or Claude or whichever is your favorite large language model, is the key limitation that they have, which is that they cannot perform biomedical actions. You can kind of talk to them and they will talk to you back, but they won't do any work for you. And what an agent is, the key capability of the agent is that it does work. And for the agent to do work, it needs an environment. So an environment is a technical term that says that this agent, this large language model, has an ability to take actions and see the result of those actions. And if you think of ChatGPT, ChatGPT cannot take any actions. It can just talk to you, you talk to it, and you are talking back and forth.But for an agent to be truly useful, it needs ability to take actions, to take steps by itself. So what we have built is actually we have built an environment in which such an agent operates. So this agent has access to all kinds of biomedical databases, tools, bioinformatics tools, different software packages, literature, web, even robots in the wet lab so that it can talk to them and get work done by executing tasks and delegating tasks to them. So the key is that we need an environment in which the agent will take actions. The question, of course, is where does the environment come from, and what should be in that environment? So how we solve this is that we actually built another agent who is reading biomedical literature in real time, every day and analyzing what kind of tools are people using, what kind of tools have scientists developed and found useful. And based on our real-time reading of all those research papers, we can truly cover the entire biomedical science, basically every paper on bioRxiv, and make sure that the agent has access to the tools needed in some sense to reproduce those papers. So through this, we build an environment automatically by reading papers and understanding what are the tools that human scientists are using. So this means now that the agent has access to a diverse set of tools, software tools, databases and knows how to use them. So the agent has the skill to use AlphaFold, to reason over PDB, to know how to use ScaMPi, how to use AutoDock, or whatever is the tool that is important to complete a given task. And this environment is large and very complex. We have over 100 different specialized software tools in there, different biomedical databases, ontologies, and knowledge bases, as well as other types of tools that are needed in everyday research. So what this now gives is the ability of the agent not only to think and talk to you, but actually go and take an action. And that's very important because a lot of scientists, what we are doing a lot of the time is downloading databases, setting up software packages, fighting with different Python versions and things like that. And now none of that is needed because the agent can automatically use all these different tools by just us telling the agent what to do. Another important component of the agent is that first it has the environment, but now it needs to reason. So the way the agent operates is that the human operator talks to it and gives it a task, uploads the data, could be terabytes of data, and then talks to it what it wants to do. And the agent, what it's going to do is actually first going to enter in a planning stage, where together with the human scientist, they are planning out the steps to execute or complete that task. So it's a collaborative process about what are the steps that need to be taken to potentially complete the task. And this is where the collaboration between human and the agent is happening so that the agent is not going off the rails and do something strange, but it's actually a collaborative effort for the human scientist and the agent to decide on the plan about how to complete the task. And then as the agent goes and executes on the plan, the plan is tracked along the entire process. And also the agent can go back to the human scientist and say, "Hey, based on my results, I want to enter into a replanning stage." And again, it's a collaborative effort to do that replanning step. And the reason I'm mentioning this, because when we were designing Biomi, it was very important for us, for the human scientist to be at the steering wheel. We wanted to design the system in such a way that it's truly a collaborative experience. And the important aspect of why we want to do it this way is because large language models are notoriously unreliable, non-deterministic, and they hallucinate all the time. Right? You can ask the same question multiple times, and you are going to get different answers each time. So the way we solve this is that Biomi actually asks clarification questions, brainstorms with the human scientists with follow-ups, and then iterates and aligns on the plan for each it executes. What is also important that as the human and the Biomi agent collaborate, the Biomi agent continually learns from the scientist, that it extracts scientists' preferences, scientists' intent, and just the ways how the human scientist likes to run certain processes. So this means that over time, each scientist builds their own personalized virtual research collaborator that actually knows them, knows their preference, knows their ways of doing work. So this notion of scientific rigor and trust is really important to us because in Biomi, every agent step is fully transparent and reproducible. The agent self-reviews the intermediate steps and validates them. And we have a number of deterministic guardrail systems that take care of the safety and prevent hallucination.What is also important that whenever the agent is making steps and conclusions, it actually analyzes the literature so that it can factually ground its inferences and give references and citations to the original sources. And that is very important when we want to go and validate the work completed by the agent. So what does this mean? This means that we have basically developed an integrated development environment or an IDE, which is something that computer programmers use. But now we have an IDE for scientists. So we call it an IBE, integrated biology environment. Which is basically this user interface in which a scientist can operate, upload the data, talk to the agent, brainstorm to the agent, and the agent can then take actions, visualize the data, and be a partner in the scientific workflow. What is important is that Biomi omits discovery, both at the level of hypothesis generation, as well as hypothesis validation. And in particular, for hypothesis validation, there is basically this inner loop where it's all about automating day-to-day research tasks, keeping track of them, making sure nothing gets lost, and nothing gets forgotten. We have released the agent into the open source. We are making it freely available for the scientific community. And just in the three months, we had more than 10,000 people sign up and use it. We have automated 60,000, 70,000 different research tasks just in the short amount of time. We have people using it every day for all kinds of tasks. And the top user basically used it every day in this three-month period. And we find that also in, let's say, in hospitals, in pharma organizations, and so on, it's very useful because now it's a single tool that a scientist can use, interact with to get daily work done. What is interesting is through our validation, we ask ourselves, "Okay, how accurate will this approach be and what kind of tasks can it solve?" So when we benchmark this on a number of different tasks from, let's say, variant prioritization, causal GWAS gene discovery, perturbation screen design, single-cell RNA-seq cell analysis and annotation, drug repurposing-type tasks, microbiome disease taxa analysis, rare disease diagnosis, patient gene prioritization, and so on. Even though the agent has never been told or engineered to do each of these tasks by themselves, it's actually able to solve them quite successfully. I want to show you a few examples of what the agent was able to do, and these were our initial case studies with our collaborators at Stanford. So this is a very interesting case study where we worked with Le Cong's lab at Stanford around designing wet lab experimental protocol design. And you can see here the task that the agent gets. So it's about I'm having this particular plasmid, and I would like to design a cloning protocol. A very simple task description. Here you can see the seven steps that the agent has planned out that it's going to take to execute this task. And about 10 minutes later and after executing this series of seven steps, here is the protocol, the cloning protocol, that the agent produced. It also visualized the plasmid map and all that. And when they tried it in the lab, the protocol was actually correct. And after the trainee did the experiment, we talked to him. And he was saying, "This molecular cloning is super hard. It takes about three years of training, and even then, biologists need to spend hours navigating different tools just to design one construct. But here with the agent, I was able to do it in 10 minutes, all in one environment, no tool hopping, and no wasted time." Another example I want to show you is that this is also very useful for exploratory data analysis. So here we worked with Mike Snyder's lab, also at Stanford, where they had very interesting data about 30 humans. They had their continuous glucose monitoring data, plus their physical activity, food intake, and a bunch of other measurements. And in the end, this data was in 458 different Excel files sitting in some folder, and it was a nightmare to even open that folder with all those files. So the scientists in Snyder Lab basically used the agent to say, "Hey, please analyze this data for me, figure out what it is, and come up with some interesting observations and hypotheses." And here the agent planned this 10-step data analysis procedure outlined here, and then actually autonomously analyzed the data, visualized the data, and came up with some very plausible and good scientific hypotheses outlined in this box. And again, when we talked to the senior post-doc in the lab, basically she was saying, "This task was complex. It was huge data mapping task of different layers of information between participant ID and device ID and different timestamps and food types and things like that." And she estimated it would take her about three weeks working two, three hours per day to complete this, but the agent was able to complete the same task in about 35 minutes.Right? So it just says how effective these types of tools can be for everyday life of a scientist in biomedicine. We've also applied this agent to a number of different use cases, for example, to automate massive transcriptomics and cell painting data analysis where two people would need to work for several weeks, but here a single person with a few hours is able to complete the same work. We are applying this agent to CAR T-cell therapy for cancer to personalize it and kind of understand, based on the immune cells of a patient, whether the therapy is going to be successful or not. And again, the agent is able to automate large parts of that part of analysis, integrate disparate data connected to literature, and do work really effectively. And maybe the last thing I'm going to mention about the agent is that we've also been using it to try to solve or help with the Alzheimer's disease. And we are collaborating with Mount Sinai to curate and integrate 60-plus expert-curated databases, as well as a number of different patient cohorts and EHR data, and let the agent analyze it to help us make progress on this important disease. And actually, the Biomi agent, we call it Biomi AD, for the Alzheimer's disease, just won a one million Nobel Prize competition for helping with the Alzheimer's disease. And I showed you a few examples of how this can be done, but there is a lot of other tasks or other analysis that the agent just knows to be able to do because the agent was trained basically on all the latest research that is coming out and can basically generalize to many different tasks and has access to the tools and knows how to use them to complete those tasks well. So the way we think of this is that basically this is now the new operating system for biomedicine, where all different databases, all different tools, all different protocols, designs, are all integrated in a single environment where the scientists can do their work. So it's not just about literature review, but it's also about taking actions autonomously. One thing I want to emphasize is that what is very important is that these agents actually increase reproducibility of science. If you think about it, because scientific workflows are so fragmented, it's practically impossible today to trace a scientific workflow. Right? And even running scientific analysis consistently is a huge problem. Right? So today, science is suffering from this traceability problem where, I know a postdoc or somebody produces the result, and then it's very hard to actually work back what exactly happened for us to get that result. And also that consistency of scientific analysis is not at the level where we'd like it to be. And the reason for this is because of these diffused, complex steps that scientists are doing, hopping between tools, collaborating with each other. A lot of this just doesn't get captured, doesn't get digitized. It's written in notebooks, it's in Excel spreadsheets, and it just gets forgotten. But now, for the first time, if we have this integrated environment in which scientists can operate, if it's actually the agents who are taking forward the actions on behalf of the scientists, it's all digital. It's all digitized. So it means it's traceable, and it also means it's verifiable. Right? Every step is documented, every step can be checked, every step can be verified, and mistakes and errors are much easier to be discovered, and the entire process is much more traceable and much more transparent. What is also important is the notion of consistency, because today's large language models, they are non-deterministic. You ask them a question, they're going to come back with a different answer almost, like, every time. So we were very careful when designing Biomi to solve for consistency, to make sure that answers are deterministic in a sense that no matter who asks the question, the answer and the analysis is always the same. And this is, I think, a huge step forward from this fragmented science that we are doing today that is non-traceable to actually having these traceable digital workflows. I'll show you in two slides why this is very important. So the first reason this is important is because our experiments show that now we are basically able to speed up science by about 10X. Agents are able to write about 10 times more code than humans. They are able to run about 2.5 times more experiments or tests in the same amount of time and just bring down the total time needed to complete the task. What we also see is that through these interactions between the human and the agent, we can actually use them to train even better agents. And what I'm showing here is that we were able to take a very small language model and train it with these interaction patterns between human and the scientist to actually surpass the accuracy of the best models out there, like GPT-5 or Claude Sonnet. So it means that as this process gets digitized, there is so much more signal in that process that we can use to build even better and more reliable co-scientists that is going to even further accelerate our progress.So that's why this transparency and the digital trace of the human's scientific workflow is so important, because it generates future training data for these types of agents. One important aspect is also actually closing the wet/dry lab, let's say, schism, and we have actually connected the agent to the robotic or robot-operated wet labs, and we see great success. So we can actually move towards automating the entire discovery loop, including the wet lab as well. And this is collaboration with Adaptive and PyLab Robot. So, to conclude, I really want to convey here that the future of science is agentic, and that virtual AI scientists are going to help us speed up biomedical research and help scientists in everyday research. It's essentially augmenting every scientist to get the impact of the entire specialized team. It's making biomedical research universally accessible, reducing cost barriers. It's making it traceable, verifiable, and much more transparent. And the point is that this is not about AI displacing or replacing scientists, but it's about amplifying and extending their potential. With this, I'm done. Thank you very much. Wow, yeah. The future of science is agentic. I think we've heard a pretty compelling argument for that. I want to introduce the next speaker, who is going to tackle the other technology that is transforming cancer research, and that is foundation models. It's my great pleasure to introduce Dr. Bo Wang from University of Toronto. His team has published the first and most widely used foundation models for single-cell data. They've extended this to single-cell spatial data, and I think we're going to see how foundation models really capture the essence of cancer biology and can then be used as tools in agentic workflows. Dr. Wang, please join me here on the stage. Hello, everyone. Today I will talk about some of the research around foundation model and the reasoning model in single-cell domains and other biological modalities. All right, so, what is foundation model? We probably heard this term quite a lot, particularly post-ChatGPT era. You've probably seen this term in many talks, in many social media. But on academic terms, the first paper that coined the term foundation model is actually shown on this left side of the slide titled, "On the Opportunities and Risks of Foundation Model." There are 100 co-authors from Stanford University, and in that paper, they first coined the term foundation model and really illustrated the essence of foundation model, which is shown on the right of the slide. The idea of foundation model is that you can train this bulky foundation model, most of the time language models, using massive amount of internet data such as text, images, speech, et cetera. And once this foundation model is trained, you can adapt this foundation model to many downstream tasks. However, that paper has 300 pages. Too long. Don't read. I summarize the essence of foundation model as A, B, C, D here. So here, A, algorithms. Most of foundation model are based on a specific deep learning architecture called transformer, which were invented by Google in 2018. Although there's many other architectures that build foundation model, but most of industry-level foundation model are still based on transformer-based deep learning architectures. B, business. Really the idea of foundation model is that one foundation model can enable multiple tasks or applications by transfer learning. And throughout this talk, you will see how we can apply transfer learning to adapt foundation model to multiple tasks. And C, computing. Most of foundation model requires massive amount of computing, particularly specialized hardware such as GPUs from NVIDIA or TPUs from Google. Last, D, data. Data is the new oil for AI, but it's particularly true for foundation model. Most of foundation model require massive amount of data for pre-training and sometimes fine-tuning as well. And throughout the talk, hopefully I can again illustrate the importance of data to build foundation models. In our lab focusing on single-cell research, I started single-cell research even when I was a PhD at Stanford. And I was asking myself, "Can we build a foundation model for single-cell?" This is exactly what we did three years ago. We published the first foundation model for single-cell data set. We call it scGBT, which were published in "Nature Methods" in 2024. And since its publication, it quickly become the most popular, most widely used foundation model for single-cell RNA-seq. And then I quickly noticed that what's next for foundation model for single-cell data? And build upon the scGBT and other foundation model for single-cell, the whole field came up with a new term, which we call virtual cell. But the idea is to build a powerful AI system to simulate how cell respond to different interventions. And the steps I outlined for building virtual cell are really kind of the three steps. The first... generational foundational models such as SCGBD are built on descriptive datasets, such as gene expression for different tissues, different cell types. Particularly, our SCGBT was trained on cell by gene census, which has 30 or even now 50 million different single cells. And the next step to me is we need to introduce causal biology. One of the causal biology datasets, I believe, is perturbative datasets. This is what I'm doing at Zara Therapeutic, with the company I'm working right now, which I will briefly talk about the work in the next few slides. And eventually, I hope virtual cell model can generalize to more complicated biological system, such as animal, organoid, or eventually patient datasets. So in the next few slides, I will talk about a new work I published at Zara Therapeutic called XCell, which is the first virtual cell model at Zara Therapeutic. And really, the key innovation we built XCell is to learn on massive causal datasets and predict on non-scalable models. And we really hope to bring this kind of causal learning concept into foundation model because the first generation of foundation models such as SCGBT rely on descriptive data. There's no causal concept inherited in that model. And we really want the new virtual cell model, XCell, to start to learn how causal data or causal biology is embedded in the model. And eventually, as I mentioned, we want to scale from a simple model such as cellular model, where you can generate massive amount of data, to more complicated biological systems, where it's a lot harder to generate a large amount of dataset, and eventually we want to scale to human systems. So in March this year, we published this bioRxiv paper, XCell, and it has been widely accepted by the community, and there's lots of very positive comments from the field. And I really want to talk about the foundation of this model. But first, nothing works without datasets, so I want to talk about the dataset we use to train XCell. We call it Pisces, X Atlas Pisces. And essentially, Pisces consists of almost twenty-six million perturbed single-cell transcriptomics across seven screens, and overall we have 16 biological contexts. This is probably the largest high-quality genome-wide perturbation datasets out there. And we draw different UMAPs and trying to inspect the quality, the diversity of the datasets. And here I really want to emphasize the importance of scaling here. Not only just the number of cells, but more importantly, it's the number of contexts we have in the pre-training datasets. And really the key improvements to work on this XCell is more biological context and a lot more size in terms of number of cells. And we also find that as we increase the number of contexts, the Pisces datasets start to show really high-quality datasets and recover known biology. And we also even just look at the PPI networks, we find that as we increase the number of screens and we increase the number of contexts, the datasets start to recover lots of known biological PPI interaction. This really give us hope that with such high quality, diverse dataset, our model will eventually learn some causal biology. All right. So let's also look at the AI architecture. So different from SCGB, which is the first generation of virtual cell, XCell use a very different training paradigm. SCGBT, from its name, you know it uses GPT-like model, which is essentially next token predictions. We use the gene token from previous gene lists to predict the gene tokens in the next gene list. And we find that autoregressive training works perfectly for languages, because languages inherently have order. For example, "I like coffee" is very different from "Coffee like I." So the order of words naturally matter for understanding languages. However, we find that when it comes to gene expression datasets, the order of genes doesn't really matter that much. So to fix this limitation in XCell, we use a very different training paradigm. We call it diffusion language models. To help you better understand what's the difference between autoregressive training versus diffusion language model training, is that you can think of autoregressive training as typing. You're typing from left to right. However, for diffusion language model, it's more or less like editing. You generate the first round of very coarse prediction of perturbed signals, and then you iteratively refine your predictions until a certain threshold is reached. We also injected lots of prior knowledge into the model, such as literatures, as simple as you can ask ChatGPT, "Tell me everything about this gene," and we can use that output to help inform the models. We also injected prior knowledge such as PPI networks from STRING, also some other morphology features from TempAD. And essentially what we are doing here is that we're trying to teach the model some known biology so that we can ground the predictions based on certain known biologies. Based on these two innovations, we really build the first state-of-art virtual cell models. So we compare XCell with other models such as State, which is a virtual cell model from Arc Institute.We also compare cell to sentence from Google. And across the board, we find that XCell outperforms these industry-level virtual cell models at different metrics. And then we also show very beautiful scaling law of virtual cells when it comes to the number of parameters. So the first virtual cell model, scGBD, has 50 million parameters, and the biggest XCell model, we call it XCell Ultra, has as big as 5 billion parameters. That is almost 100x increase of parameter size. And we also find that as we increase the number of parameters, if we train on very diverse perturbed seq datasets, the training loss keeps decreasing and the accuracy on test set keeps increasing. So this is really one of the first comprehensive scaling law experiment we did in virtual cells. And then I want to kind of switch gears that to talk about another work, we call it BioReason. And foundation model is great, but essentially it's still a prediction machine. And what we want is a next level of understanding what specific biology this foundation model is teaching us, and that requires reasoning. Reasoning essentially is basically to decompose a task into different steps, and you can reason through different steps to get to your answer. So the first reasoning model we published was in NeurIPS, and called BioReason, Enhancing Multimodal Biological Reasoning with DNA Large Language Models. So the idea is that we already know frontier language models can reason through other modalities such as images, audios. Can we teach language models to reason through other biology data such as DNA sequences? So this is the first paper it's experimenting about. The architecture of BioReason is quite straightforward. So it consists of a few parts. First, we have biological foundation models. Here, we use Evo2, which is a state-of-the-art foundation model for DNA sequences. And then we need a reasoning language model. For our BioReason, we're specifically using Quanta3 4 billion parameters because it's open source, it's also state-of-the-art reasoning language models. And the idea is to teach language model about the biological sequences. Because we know the current frontier language model, they don't understand DNA sequences because it's never trained on DNA sequences. So what we did in BioReason is that we freeze the DNA sequence foundation model Evo2 here, and we align the DNA sequence tokens to language tokens so that we can use reinforced learning to teach what DNA sequence looks like in the context of languages. So that the frontier language models can understand what DNA sequence looks like. So this is a very bold experiment, and turns out it works quite well. So the first experiment we did to test BioReason is on CCAG, on pathway variant networks from CCAG. And we curated about 1,500 question-answering pairs representing almost 170 variant networks to almost 40 unique diseases. And we use frontier models such as GPT-4o, which back then was the best frontier model, to help us curate such reasoning models. And then we test BioReason on CCAG disease pathway prediction tasks, and we find that as we increase the number of parameters, we get better results. But more importantly, when we apply reinforced learning to teach the language models about the DNA sequence tokens, the language models start to understand better about pathways, about disease, and therefore does a better job in predicting disease pathway associations. So this is a very interesting finding that if we teach the language model reasons through different biological data, the language model eventually start to understand biology. And recently, we published a second BioReason paper. We call it BioReason Pro, which is a reasoning model to understand protein functions. So, this is a collaboration between University Health Network and also Arc Institute and Vector Institute. And the idea, as I mentioned, is can we teach language model to understand protein functions? We all know that AlphaFold and other models kind of solve the problem from protein sequence to understand protein structures. However, there is still a big gap to understand protein functions, and this is exactly what this model is designed to do. And we notice this huge gap because there's over 250 million proteins in UniProt, but less than 0.1% have been experimentally annotated. So the idea is can we build an AI model to start help biologists understand protein functions and therefore better annotate the protein functions? And here, when we talk about protein functions, we kind of take a very narrow perspective. We're specifically focusing on gene ontology or GO terms, and specifically such as biological processes, molecular functions, cellular components, et cetera. So it's a very narrow perspective to tackle these protein functions, but this is the first step. So the dataset we curated to train language model to understand protein functions coming from UniProt, and specifically we curated about 130,000 proteins from different organism in UniProt, and then we generate reasoning data using GPT-5 to help us train the language models to understand the reasoning steps of protein functions.However, one of the first challenge we found to build this reasoning model for protein functions is there's no good tokenizers for GO terms. We all know that GO terms have hierarchical structures. Most of the time it's multi-label. It's extremely messy. So what is the best way to really teach the language model what's a token of GO terms? So in order to build this tokenizer for GO terms, we first train a foundation model, we call it GoGPT, and essentially, it's a supervised model that teaches the model what the GO labels look like. And we use very standard way to build this supervised machine learning model, GoGPT. And we find that just by supervised learning, the GoGPT already kind of achieve very impressive accuracy in understanding the GO terms. But bear in mind, this is a pure supervised learning method that trying to build a reliable tokenization method for incorporate the GO terms. So once we have everything, we start to build this very complicated reasoning language models to help us understand and annotate protein functions. And specifically, the input to this BioReason Pro has three sets of tokens. First, the ESM3 tokens, basically is the sequence information of proteins, and the second one is a GO encoder, which we just trained from GoGPT, and then we have also language tokens from Qwen 3. And once we have these tokens, we merge everything together, we apply reinforced learning on the reasoning datasets from GPT-5, and then we teach Qwen 3 about the reasoning steps to understand protein functions. Again, we use very standard reinforced learning to teach the model how to reason through different steps, and this is very standard reinforced learning invented from DeepSeek, which is very early reasoning models in language side. And we observe very stable training losses, and we kind of give very high confidence that the model is relatively well-trained with reinforced learning. So once the model is trained, the first way to validate this model is we use the approach called LLM as Judge. It basically is just ask more of advanced frontier model here, especially GPT 5.1, to say, "Do you like the predictions in comparison to the ground truth?" And we find that with proper reinforced learning training, language models start to understand the protein sequences and predict different stepwise reasoning datasets for understanding protein functions. And overall, even using language model as a judge, GPT 5.1 think the predictions from our BioReason Pro most of the time is better than many other baseline methods. And we also perform out-of-context test. We look at the proteins that have extremely low structure similarities to the proteins in the training dataset in the hope to test the generalization capacities of BioReason Pro, and we find that our BioReason Pro generalize very well to proteins that have little or less similarities to the proteins in the training datasets. And overall, we find that by teaching language model to understand the protein sequences and therefore reasons through protein functions, BioReason Pro does extremely well across different domains of GO terms, and then across different structure similarity spectrums of different proteins. So this represents the state-of-arts method to understand protein functions. And this really introduce a fundamental shift in building foundation models. The idea is that the language models, the frontier language model is trained so well already, they have very impressive reasoning capacities. And the next step is really to teach these language models to understand biological sequences. Although they never was trained with biological sequences, but with proper alignment, with proper reinforced learning, this frontier language model can do actually really well job to understand biological sequences. We also did human evaluations, where we invited 27 protein experts or biologists, and then to kind of evaluate which one is better in terms of the humans' annotation versus the models' predictions across almost 200 different novel proteins that were not included in the training dataset. And surprisingly, we find that in one of the models, BioReason SFT, almost 80% of the predictions from the BioReason model are better than human annotators. So this is very impressive because for one, BioReason Pro generate reasoning predictions much faster than human annotators. But two, 80% of the time, we actually find that the prediction from BioReason Pro is actually higher quality than human annotators. And this is true across different domains, such as molecular functions, biological processes, cellular components, and also this is true across different spectrum of similarities of different proteins.We open source the model in this link, so all the model data sets, and you can even also put your favorite proteins into this link. And in fact, we released this model about less than two weeks ago, and there's already tens of thousands of users try our model with their favorite proteins, and we receive quite positive comments about the capacity of BioReason Pro in understand the protein functions by just input the protein sequences. So please try our model using this link and send us any feedbacks you may have. All right, so what's next? I talk about virtual cell models such as scGPT and ExCell. I talk about bioreason models, particularly bioreasons through DNA sequences, bioreasons through protein sequences. And a natural next step is bioreason to understand cells. So we are working on a virtual cell model that can reasons through natural languages. And really the idea to build such complicated reasoning model is to start to mine existing literatures and using known knowledge graphs and then using known biological processes to train language models to help us understand lots of cellular data sets such as gene expressions, taxic, and et cetera. And overall, what we really want to understand is how cell responds to different interventions and to reasons using languages. I want to conclude my talk using really kind of a quote from Jensen Huang, who is the CEO of NVIDIA, is that we are at a era where we can start to engineering biology or engineering cells using the context of virtual cell. As we transform biology into engineering, the effect start to compound. We see exponentially improving results using these more advanced languages or agents from previous speakers, and we really see huge improvements at a much faster pace to understand different biological modalities. All right. With that, I want to give credits to all the students. They did most of the work. I'm just the one who talk here. Haotian and Chloe are the main driver behind scGPT, which is the first foundation model in single cell. Chloe, Neil, and Mehran are the main drivers behind ExCell. And Adeeb and Hani are collaborators, the leaders in the two bioreason papers. And all the paper data weights are open sourced, and feel free to try our models and send any comments you may have. Thank you very much. We have seen the power of foundation models and how they can learn structure from unstructured biological data. Before we've seen agents, how they can help us automate procedures. I think a key question now is: How can we take these technical advancements and actually deliver them to our patients in the clinic? And that is the topic that our next speaker will speak to, and it's my great pleasure to introduce Dr. Suchi Saria from Johns Hopkins. She's an AI researcher and has done a lot of seminal work in AI. But then she went one step further, and she actually delivered some of these results to the clinic, and she did it through prospective, multicentric evaluation with mortality endpoints. And I think that's the highest bar for evidence in oncology, and that's what we ultimately need to see as we translate these technologies towards the clinic. So without further ado, please, Dr. Saria, join me here on the stage. Thank you for having me. Can you raise your hand if you're a practicing clinician? Okay, great. And then can you raise your hand if you frequently use real-time clinical AI as part of your decision-making process today? That's perfect. You may not want to answer this, but I'm curious. Can you raise your hand if you feel like it's really not obvious if any of the tools that are available to you are good, decent, verified, validated, and as a result, you're very fearful of where care is headed if you're being forced to use AI at the point of care? Cool. Thanks for responding. So, as Jakob described, I started my research career focusing on model building and developing new technical architectures for making sense of large-scale, multimodal time series data sets broadly, and in particular, when we think about electronic health record and the kind of massively multimodal data sets that are present in electronic health record. Challenges also that exist as a result because these are data sets that are a byproduct of care, which means there are biases embedded in the data. There could be variation site to site in which the way the data are collected or even across clinicians. And so the kinds of methods we need are different from what we might need in other domains.Further, the stakes are much higher here, and I believe we need methods that can really augment frontline clinicians. So in other words, methods that can build trust. And so that's been the career arc on the research front of the work we've led. And then in end of 2018, early 2019, as we started to see more and more evidence for how these models were becoming more and more powerful. And in fact, a lot of the models we talk about today, like transformer-based models, first started to show up in the research literature in 2017. So as we started to apply these technologies and started to see how the models and the architecture were becoming more and more powerful, the next question was, how are we going to actually make use of these within care? And when I first started this work, I thought, "Well, that's easy. I just have to engineer a system that can deliver it. And obviously, if the research shows that these models are really high quality, and if I can find a way to deliver it, then clinicians will adopt it. Boom. And we'll get some fabulous results." But that's not how it turned out. What I started realizing was that there were many barriers to translating these models, including things like how do we translate it in a way that it's easy to use, easy to adopt, builds trust with frontline clinicians, and also from a regulatory standpoint, how do we think about deploying it in a way that's safe, reliable, and responsible? And what that means is it's not just that the model was performant in some offline datasets. But as we implement this model in one clinic, the second clinic, and third clinic, as we implement these models across pre-COVID, COVID, post-COVID, and in just everyday care as practice changes, the models stay performant, and we understand performance as we implement and deploy them, and we're able to track and measure and understand impact. And so that's really the journey that I will describe a little bit today in terms of the work we're doing. Disclosure, I'm founder and CEO of Bayesian Health. Like I mentioned, the company we spun out of Hopkins with almost a decade of work. I'm also an endowed chair at Johns Hopkins doing research in this space, and a conflict of interest policy oversees all of the work produced on both sides and manages work on both sides. Okay. So what's the problem statement? Care is reactive by default today. What that means is, in a way, when a patient is entering the room, you're doing the best you can, given the little bit of time you have to gather information about this patient's history and to do what you need to do. And as a result, you see a lot of variability across maybe it's a busy day, you had less time, maybe there's a wide difference or gap between clinicians with different kinds of experience. There's also depending on what tools happen to be implemented and how easily the EMR is set up, you may have access to data from past settings or not. And as a result, there's a lot of variability in the quality of care that we give. And is there an opportunity to flip it? Which is instead of being reactive, is it possible to have more proactive evidence-based care? It's the kind of thing we've been talking about in research settings for a long time, but the question is, what does it take to deliver that? And ideally, there are real-time clinical intelligence or clinical AI layer that augments the care team at the frontline, which is tirelessly reading through the record, not just from this encounter, but past encounters, pulling all the relevant information, synthesizing this in real time to call your attention to critical bits that you might have missed, making it easy to learn from the expertise of clinicians and high-quality clinical decisions that have come to date so that you're bringing high-quality, consistent care to every visit. And that's what we want to enable and what I think is possible. And so I'll give examples of the work we've been doing to work towards that. Now, the problem with current tools, if you're a practicing clinician, you see tools that may be implemented through the electronic health record. Commonly, the challenge is that the ability to synthesize massively multimodal datasets is quite hard. And so there are really two breeds of tools that you see today. One type is effectively old school, simple, rules-based or analytic alerting systems. Like for instance, most of you who practice have probably seen some alerting system for sepsis, which is a leading cause of hospital deaths. And in that example, what most clinicians would describe is that basically, these alerts, there are a lot of alerts. It's mostly nuisance. It's a lot of noise. They're mostly disliked because they're not offering a lot of true, helpful guidance. And when you look at data in these cases, basically very often they miss a lot of cases. They're often flagging or alerting once a treatment has already begun. The true alert to false positive alert ratio is extremely low. So, 50 alerts to find a single case, which means a lot of noise. And then as a result, they get very little adoption. And so that's the hurdle we have to cross in trying to build tools that can drive much higher adoption. And so what are the challenges in building these tools? So the first is that in many of these condition areas, I'll give you some examples of areas we've been working in, like sepsis.In many patient safety quality areas, these are what I call needle in a haystack problem, which means the prevalence of these conditions is typically in the hospital population or ED population, something close to 2%, 3%, 4%. It is not 70%. Second, simple rules-based alerts create a lot of alarm fatigue because there's a lot of heterogeneity in the way these symptoms present across patient populations. So rather than carrying one average, what you really need is the ability to learn variations and subtypes from this large amount of data. Next, when you're trying to do these learning-based methods from multimodal data, it's very easy to overfit to large-scale multimodal data, and in the process of learning the targets, essentially what you use as your-- When you're building any learning method, you need a notion of ground truth. What do you consider as a case versus a control or what do you use as in this concrete example, say, sepsis versus not sepsis, or when did sepsis occur? And historically, to get this kind of data at very large scale is very hard. So what people do is basically use simple labels that are extracted from the electronic health record, like billing codes, which are very noisy labels, and as a result, you get a lot of garbage in, garbage out. And finally, for these methods to be useful, you need to train them so that not only are they finding a needle in a haystack, but they're also finding them in a timely fashion, right? So for instance, in many of these early detection problems, what you really want is not to just detect it and detect it after the fact, after treatment's begun. You want to detect it early. So how do you solve this super hard signal-to-noise problem? And that's sort of the model-building problem that needs to be solved. So the work that we've done over the years has been on first-- And here's sort of the four-part architecture that has allowed us to make progress, and I'll unpack pieces of this as I go. So the first is the ability to integrate and interact with the EMR so you have real time, not just text data or not just labs data, or not just vitals data, but really real-time longitudinal data for every patient that's walking in the door. And that means integrating with the electronic health record so that you have the ability to ingest and process that data in real time, put AI models on these data, and then be able to put results back into workflow, in clinicians' workflow, so that they can see the outputs and act on it. So that's sort of the first part. And building that is hard because there's a lot of data, and the ability to integrate and synthesize with the electronic health record in real time requires a lot of technical work. Now, it turns out in the beginning when we did this kind of work, we didn't think this was as important. We just thought we had amazing models. So because the model outputs were so good, it would just work. But what we found was that unless the model outputs were very much embedded in real time within clinician workflows, the adoption was going to be very, very difficult. So as a result, we had to build this kind of real-time integration layer. The next piece is the machine learning AI methods that make sense of this data in real time. And so here, mixture of expert-type models that allow us to model subtypes and variation, and I'll, in the next two slides, go a little bit more into it. Once you have these models producing real-time outputs, then moving it back into clinical workflow, and one of our realization was that if you're moving it back into workflow just to drop a flag or an alert, that isn't as helpful to clinicians as it is to be able to have full workflows. And so as examples, workflows where effectively we're catching, flagging, summarizing why the model flagged this case in the first place so that it's transparent, it's not a black box. Facilitating assessments from the clinical team and the care team. Next, being able to then proffer up treatment coordination across the care team so that if they agree this person has this complication, they can rapidly link to order sets or link to orders in order to get done what they need to get done, as well as proffering up documentation so that you're able to then document and wrap up the work that's needed to close that piece of episode. So essentially, the workflows that make it possible to act on that signal in order to realize value from start to end is just as critical. And then finally, some level of a monitoring and governance layer so that you're able to measure and monitor in real time if the system is performing, and also detect if, for instance, there are drifts and shifts that are causing the system to deviate. In many of these areas, my team's written over 50 papers. So I don't cite all of the papers in this particular presentation, but if you just go to my Google Scholar, you should be able to find papers on these topics if you're curious.So let's talk a little bit about the multimodal AI I spoke about. And so here today, when most people think of large language models, which are basically transformer-based models or transformer-based architectures for text data, what we needed to do was to broaden these transformer-based models, not to just incorporate text data, which is in the notes, but to incorporate all the thousands of other kinds of modalities we get in the EMR, like your labs, your vitals, your medications, your treatments. And some of these are event-based data, some of these are continuous time series data. They're occurring over time, and the idea was, how do we integrate these diverse modalities into a single signal? And in doing so, for being able to make high-quality clinical models, we really needed the ability to understand and learn about subpopulations. And so here, essentially, we use these mixture of expert-type architectures that we've designed with these kinds of data sets in mind. And what these do is allow you to integrate lots of different kinds of signal, learn these experts that are pointed towards subpopulations as well as modalities and integrate this to produce signal. And when we do so, what we find is over existing alternatives, basically the ability to reach much higher sensitivity compared to existing models. So, above 80% range, which is how the FDA likes to think about it as a good diagnostic. So 80%, 85%, 90%, but doing so not by heavily compromising on positive predictive value. So being able to do so while still reducing alert volume of existing models where they have significantly lower sensitivity, but 20 to one, 30 to one, and here we're taking it down to three to one. So in other words, far higher positive predictive value and simultaneously improving lead time means these models are able to catch the condition against standard of care much sooner, and where every hour matters in terms of impact on care outcomes, this is critical. And so here's, for example, a paper below that details-- This is the most recent paper. It's actually getting presented this week in Brazil as I speak, but it's the most recent paper of a sequence of papers we've written on this topic of architectures for integrating diverse kinds of data to be able to build high-quality clinical AI models. So we have the models, and then I said I'll talk a little bit about the workflows. So here, for instance, what the workflows are doing is these are modular agents embedded into the EHR to support end-to-end workflows. And what these modular agents are doing is individualizing risk, flagging and explaining why, reducing friction by teeing up order sets and documentation automatically, and then enabling reporting so we understand which patient was flagged, what happened in response. Did the clinician ignore it, or did they respond? How long did it take them to respond? Did they confirm or ignore or disagree? If they confirmed, did they give the relevant treatment? And what that long-term reporting does is not only allows us to understand system behavior but also understand human machine teaming performance. In other words, is the system performing the way we expect to? Are the clinicians who are over-relying on the system, are the clinicians who are under-relying on the system? And that allows us to debug the deployment of these kinds of models. And by doing so, we've now implemented the system in a variety of health systems around the country and have been able to see consistently 80% to 95% clinical adoption, which means every time a patient is flagged, in 80% to 90% of these cases, clinicians go in, they seek it out, they look at the review report, they review the report, they agree, disagree, and they submit an analysis. And if they agree, they do the follow-up treatment as appropriate. And then, like I'd mentioned, the last piece of this four-part workflow was the governance piece. And here, essentially, starting in 2019, I started working through a center of excellence funded by the FDA at Hopkins on research on deploying these models safely. And essentially the questions we wanted to ask was, as we deploy these models, maybe we are doing an initial evaluation pre-deployment. But as time progresses, how do we know that the models are continuing to work? So that's one question. Another question was that before we go and we measure performance, how do we know, from an approval perspective, how do you know what are the populations it's working well on versus it's not working well on? And in real time, as we monitor and observe that system performance is going down for any number of reasons, how do we do root cause analysis to identify what are the subpopulations where things may be going wrong so that we have the ability to mitigate risk and close the loop? And so those are the kinds of methods that we are implementing through this oversight piece into all the models we're deploying. And so now I'll show you an example case study. So this is like I alluded to in the beginning, sepsis is the number one cause of death, excess days and readmissions. This particular case study is data presenting from implementing this at a single system at first called Cleveland Clinic. Cleveland Clinic is a 20-hospital system spread across multiple states in the US.They already have a very high standard of care, and the question was, could AI or the deployment of this kind of an AI augmentation layer dramatically improve the quality of care at their system? They had spent quite a bit of time optimizing the existing EMR-based alerts to improve early detection. They found that most of these alerts were missing a lot of cases. In most scenarios, when the patients were getting alerted, 86% of the alerts were happening after treatment had already initiated. In other words, there wasn't much of an early detection result that they were getting. And there was a lot of alerts, and so there was an opportunity to improve physician experience by improving the quality of the tooling there. And so what we did was, so the partnership here was around an end-to-end infrastructure to enable early identification to treatment to documentation to abstraction. And before deployment, one of the things we did is we implemented it at their site, and then we measured performance both across all patient populations, but also across sub-populations. This happens to be sample data, so reviewing all of that data would take a long time in the context of this keynote. So I just wrote down one slide with some example data, and this is actually a multi-center-- This is evaluation that was done across four different health systems on populations related to oncology. Now, this is sepsis detection in oncologic patients. Now, it turns out in this case, because many patients look like they already have neutropenic fever, it is actually extremely easy to miss sepsis, so detection is especially hard in these cases. And what I'm showing you here is the ability to get very high sensitivity and specificity in these populations. And so step one, we're showing that we're able to achieve very high performance. And so this is what it's showing here is aggregated across Cleveland Clinic, the ability to identify many more cases, so that's almost 60% improvement over existing tool. But more importantly, not just identification, but early identification. So almost 400% improvement over the existing tooling in being able to identify cases earlier and meaningfully earlier, so three and a half hours earlier on average. So this is median lead time. And then simultaneously in this, what we're showing is almost 10X reduction in alert volume. So at a 450-bed hospital, they saw 7,000 fewer alerts per month, which is amazing from a frontline care experience. So then the next thing we did is once we go live in the reporting, we measure what is happening in response. And what we're able to show is consistently over a long period of time, 80% to 90% adoption, which is very exciting. And so if you're able to do earlier detection and you're able to drive adoption, you should see meaningful reduction improvements in outcome. And so here what we're showing is in a pragmatic, simultaneous case control study, basically a very meaningful reduction in delays to treatment. So patients are getting treatment earlier. And simultaneously what we're observing is shorter length of stays and reduced mortality. And so this is extremely exciting and the ability to be able to deploy these tools to not just have an insight, but to drive action is ultimately what impacts patient outcomes. And now, like I described, this is now deployed at the bedside for every single patient that visits the Cleveland Clinic. I spoke a little bit about this case study. I want to talk a little bit about the adoption component. And in 2022, we wrote these trio of papers that came out on the cover of "Nature Medicine." Here are two of them, where one of them focused on this pragmatic case control study across five sites and how we achieve those outcomes. But two of the three papers also focused on driving adoption and studying adoption. And when we studied adoption, there was a quantitative paper that focused on studying quantitative adoption. But what was also really interesting is a qualitative paper that focused on human factors methods for understanding how clinicians build trust and how clinicians build adoption. And this was really a starting point for designing these kinds of systems, for being able to not just make systems that are highly accurate, but to build systems that get adopted and used. Because if we can't do that, we're not going to drive outcomes. And so at a super high level, I'll describe some of our learnings, and I call this at a high level as an activate framework. And so the first is the high-quality AI that monitors and tunes based on your population. So your ability to monitor and really learn about sub-populations, understand clinical practice, understand clinical behavior to surface clinical signals within workflow that is deeply contextualized. So it's not like one of those, you have those assistants or interns that don't understand your context and they are... like interrupting you out of context, and that doesn't build trust. Instead, what you want, signals that really understand the patient longitudinal context, but also the caregiver context to be able to surface signals at the right moment so that they can get acted on. The second is when you're implementing these models, how do you implement a collaborative change management model? And what that means is you want to think about what is the existing care process and how you're augmenting it, and then how are you building a common understanding of what are the goals we're trying to achieve, what are the mechanisms by which we're trying to achieve the goals, and align on workflows to drive change and then measure and hold people accountable in driving them. So this isn't just a technology problem, there's also a people problem, right? You need to change the organization's culture to be able to adapt and integrate these models within their care, and how you do that in its own right deserves a lot of thought process and thinking, and that's basically what encompasses this. The third is tech-enabled process improvement, because today, people are already within health systems used to doing things to improve patient outcomes, right? We have quality improvement or performance improvement they're called. And the way people do that is very often through checklists and education protocols. But those are very expensive and hard to maintain, and they often don't survive the test of time. In other words, you're very focused on one initiative today, and something else comes along, and people get distracted, and they forget. So what we're doing here through infrastructure is to hardwire high-quality ideas within workflow. And what that means is high-quality signals combined with high-quality workflows and embedding it in a way so that they're easy to use and adopt. And finally, incentivized engagement, which is: Is there a way to build workflows that make the right thing the easy thing to do? In other words, if you're taking away some nuisance, terrible workflow, and you're implementing workflows that save the frontline clinicians time, that make their job easier, they're more likely to adopt it. So you want to think about how do you also remove barriers and ease pain, and drive value in implementing these kinds of systems. And so I'll close with the second case studies. This is an example of work we did with Mayo Clinic, with our collaborator, Dr. Jacob Strand, who's the chair of palliative care there. And this is a new use case. This is not in sepsis, just to show you a completely different area. In this example, what I'm showing you is a case you'll recognize. In this particular case, not a patient with cancer, but instead advanced heart failure, though this is a story you'll frequently see in oncology. Basically, it's a case where a patient, 68-year-old woman, she kept coming in, multiple hospitalizations. And each course, she was only deteriorating further and further, but in none of these hospitalizations, her need for palliative care was flagged, and no one had those goals of care conversation. Fast-forward, as we implemented the system in palliative care, for tackling unmet palliative care at Mayo Clinic, what we saw was effectively these patients are now getting reliably identified earlier. By getting identified earlier, clinicians on the palliative care team now have visibility across the entire system to know who are the patients who need it. Then based upon the amount of resources available, they can triage and rank patients very quickly and make sure the patients who need it are now getting it, and then there is a way, again, the workflows make it really easy to be able to do the follow-through that's needed, and we're starting to see the impact on outcomes. So first of all, we're seeing more patients are getting palliative care consults. When a patient's getting flagged, they had a previous alerting system, but the challenge was very few of those alerts were getting attention. Or if they were getting attention, very few of them were converting to real consults and addressing real patient need. So first thing we saw was many more of the right patients getting flagged and flagged earlier and at converting into palliative care consults and patients receiving follow-through care that's appropriate. The second benefit we saw is dramatic reductions in readmissions and 60-day readmissions in this patient population. And then from an administrative perspective, reduction in total cost of care. So I spoke about, I'm right at 9:30, I spoke about a case study in sepsis, I spoke about a case study in palliative care. But the thing that to me is so exciting is that this opportunity is not just limited to these two use cases, but are possible in so many different clinical condition areas. Below, what I'm showing you that this work was done not just through the researchers and students and collaborators in my lab and the team at Bayesian, but also now in many of these areas, we are partnered with a variety of systems, include Cleveland Clinic, Memorial Care, Mayo Clinic, Theta Care, MedStar, University of Rochester, Innova, and what you'll see is a very diverse set of systems. Systems with a large community system, systems that are premier academic medical centers, systems that are regional community hospitals, which means we're not just thinking of AI as the kind of thing that is implemented for the wealthiest hospitals or the most pristine hospitals, but we really want to bring it in a way that we can bring this value to very diverse populations and very diverse settings. And if anyone is interested in learning more or talking to me more about it or collaborating, here's my email address if you'd like me to send an email. And I'll be around between the opening of G and H outside if you'd like to talk more for the next couple of hours. Thank you so much. That was very impressive, I think, and we've seen how these technologies can not just be developed, but also actually brought to clinical practice. And now it's my great pleasure to introduce the final speaker of this session, Dr. Faisal Mahmood from Harvard. He's very well-known. He's maybe the godfather of pathology foundation models. His team has published several of the leading pathology foundation models that are being used by teams worldwide, have been used in thousands of publications, and now I'm very excited that he, I think, is also going to present some new foundation models from his team that are not just for pathology, but go beyond it and actually are foundation models of everything. So I would like to introduce Faisal here. Please join me here on the stage, Faisal. Yep. Thank you so much for the introduction and for inviting me to speak today. So today I'll be talking about some of the work that my group has done in developing foundation models for oncology. We'll talk about some of the pathology foundation models that we have developed, and also some of the new work that is coming out of our group. Here are my disclosures. So my group predominantly works on computational pathology, so we analyze images that look something like this. So these are very large images that come from digitized glass slides. Pathology is often regarded as the last modality in cancer research to be digitized, and that's because the standard of care still remains looking at a glass slide under a microscope. And the goal of our team is really to go from these images to everything in the red box here, from early diagnosis, prognosis, looking at prediction of response to treatment, and so forth. But we also integrate other kinds of data types to answer some of these questions. So this is how we started to approach this problem of trying to represent data in a more meaningful way so we can do all of these different computational tasks downstream. So I always like to talk a little bit about the history of the field, and in particular about Judith Prevette, because she was a forgotten hero of this area. She did some very interesting early work in the 1960s and '70s in trying to analyze these microscopy images. And what a lot of people don't know is that her work became the basis of the very first FDA-approved medical device that used some form of machine learning in the late '80s and early '90s, and that was what's today is known as a Genius Hologic device and is used for Pap smear analysis and other kinds of cytology-based analysis. But at the time, these images were very large, so they were being patched and stored into magnetic tape and using simple line profiles. So it's very interesting to see how her work has had an influence on everything that is being done in computational pathology today. These images are very large, so in more recent times we've used all kinds of weakly supervised methods. So these methods use whole slide images and labels that reside in pathology reports or electronic medical records to train these models. Two of the most common approaches for these include the utility of graph convolutional neural networks or multiple instance learning. So in 2019, this real seminal article was published from the Memorial Sloan Kettering Cancer Center, where they showed that you needed about 10,000 or so whole slide images to train a model that would get you to useful clinical performance. And when we started working in the area, we wanted to see if there's a data-efficient way for us to do this. And that's what led to a number of studies, but we came up with methods and tools that we used to train machine learning models using whole slide images and labels that did not need pixel-level annotations. And we were able to do this in a data-efficient manner, which means we could apply it to much more interesting problems. We also made these tools publicly available, so there's an entire community of people that have used these tools over the past six or seven years. We applied this to cancer of unknown primary in 2021, trying to see if we can predict origins for metastatic and unknown primary tumors using a histology image alone. We looked at endomyocardial biopsy assessment and also trying to integrate histology and molecular data to see if we'll be able to build better prognostic models. But for all of these tools, we used multiple instance learning-based framework, which uses whole slide image, which is patched into smaller patches. We extract meaningful representations from these patches and then learn on top of those representations. Our earlier studies, the studies I just showed, used representations that were just based on a ResNet that's trained on real-world images. But they still performed reasonably, but over time, we realized that the feature extraction component here is very important, and how we represent these data is key to improving these predictive models. So we started experimenting with self-supervised learning early on, and then in 2024, we published these two foundation models. And these foundation models focus on extracting meaningful representations from pathology images, but also trying to contrast with text to improve image representations. This is just in line with what has happened in the machine learning community, where they've tried to contrast images with text representation to improve image representation. So these models were made publicly available and have been downloaded over four million times and have been used at this point in over 2,000 different studies. So this really accelerated the utility of these models show that how important self-supervision is and how important representing data into meaningful representations can be. So these studies were focusing on representing... regions of pathology slides, but we also wanted to scale this to the entire slide. So last year we published Titan, which is a foundation model that focuses on representing the entirety of a whole slide image into a singular vector or representation. And this used a number of different bells and whistles, including iBOT-based training and using what's commonly used for training large language models, where we train short and test long, and train on a limited region of the slide and testing on a large region of the slide. But we also used additional layers of training, which includes using text, both from the report and a generative model that had been developed in our group. And we tried to show that we can improve slide level representations in contrast with some of the other foundation models that were out there. Also, we assessed what the scaling laws looked like. And as our models are getting better, our benchmarks are also getting more complex and difficult because all of the problems that we were trying to target back in 2024, these models already perform almost perfectly on those tasks. So we tried to represent the whole slide, and we made these models publicly available as well. But text is a very subjective modality, so we also wanted to explore that can we contrast with additional modalities such that it will improve image representation? And there's a lot of more objective information about these cases that we have available to us. So we started contrasting with molecular information. We started by looking at IHCs, contrasting with IHCs to see if the representation would improve, and then doing the same with molecular data or transcriptomics in a really toy-based setting. So this is just by using the TCGA. And we showed that you could really improve few-shot performance quite a lot. That means using very, very few labels or very, very few samples for training these models, and still converging to a clinically useful result. So this was early indication that this is an approach that we should scale, and that's where we started to develop Threads, which is a model that uses all of the molecular data that was available to us at MGB as well as at Dana-Farber, where we tried to contrast between H&E images and corresponding molecular data. To improve representation, we showed that you could focus on or target more difficult problems like treatment response prediction directly from the histology slide. So this is a study that will come out in the coming months, and we do have plans to make this model publicly available as well. We also try to expand this to spatial proteomic data. So this is a model for spatial proteomics that we developed in the group using large amounts of data and self-supervision. The goal is the same. Can we represent proteomic data into meaningful representations that can then be used for a host of different downstream tasks? This includes tissue phenotyping, looking at artifact classification and so forth, but also using it as a search engine to retrieve similar regions within spatial proteomic data, as well as for spatial biomarker discovery. So we have an earlier version of this model that was already made available, and if people would like to get access to the newer version of the model, there's a QR code that you can use to sign up for the new version that will be coming out. And my colleague, Shizun, will be giving a talk about this, and we also have a poster on this foundation model, and hopefully the study will also come out later this year. But as we have continued to develop these models, we started by looking at regions within a pathology slide, which we tried to contrast with other modalities to improve image representation. We scaled it to the entirety of the slide, and then we can represent the whole slide. But something that we've been trying to do for the past three years or so is trying to represent the entire patient. So now it's commonly referred to as virtual patient representation, but at the time, we just called it representing the entire patient's record. The goal was that can we use everything digital that we have about the patient and represent the patient into a singular vector that can then be used for a lot of different downstream applications? And these applications can be predictive. They can be related to diagnosis, prognosis, treatment response prediction, next event prediction, hospital operations. But just having something that represents the patient into a meaningful representation would be incredibly valuable. And we wanted this to be dynamic and continuously updating because we have a lot of information about the patient at any given point in time, but that information also continues to evolve as we get more and more data into the system. So we used all of the data that was available to us at Mass General Brigham. This includes about 12 hospitals, and this is data from about 7.2 million patients that were used to develop this model, and it has about 18 different modalities. And the goal is that as more and more data will become available about the patient, we will represent it into a singular vector or singular representation for that patient. So the representation would evolve over time, and we'll also have a patient trajectory as to how the patient and its representation evolves. So once we have this representation, we can look at risks, predictions, similarity searching, statistical analysis, other kinds of treatment response predictions. So the single vector would be incredibly useful for us, right? So these are all downstream tasks or applications that we can potentially look at. But once we have these representations, we can look at how the different concepts in medicine cluster together. This is a very nice plot where we have an entire atlas of medicine being learned on its own from these representations. And you can look at the fact that particularontologies that we know from literature all clustered together, even though they were not given to the model during training. The model is unaware of all the literature that we have about these different ontologies, and yet it manages to cluster these ontologies all together. And then you can also go in and study different relationships that exist across modalities. These are all ways to see how rich these representations are and how meaningful representations does the model learn? We can also look at individual embeddings. So in this case, each dot represents the embedding of the entire patient, or everything that we know about the patient, including structured and unstructured data, and it's colored by age. And then once you have these embeddings and these representations, you're also able to look for similar patients. So in an entire record, in a large record, are you able to find similar patients? Because similarity search or patient matching is a problem that can solve a lot of issues that we have, both in oncology and in medicine in general. This includes looking at patients who might respond to a clinical trial, looking at rare disease diagnosis, and so forth. So here we have examples for how well the model does in finding similar patients, and here's some examples for oncology, and we have a baseline that's for the last progress note. So just embedding the last progress note versus embedding the entire patient's record. We can do similarity search based on text. So we can use text to find if there's a similar patient in the entire record, or we can also use other modalities. Because it's a multimodal model, you can start with an image or text or any other modality to see whether you'll be able to find any similar patients. And there's also a score for how similar the corresponding patient might be. So obviously, this model is very new, and we're still exploring what the capabilities of the model are, but image search is definitely, or patient search is definitely one of those applications that we are very interested in and we continue to devote a lot of effort towards. We can also look at preclinical evidence. So I think one of the basic promises of machine learning and medicine is early diagnosis. And we've seen a host of different studies over the past five years showing that you can use all these different auxiliary modalities to predict disease risk. For example, looking at retinal images, you can predict cardiovascular and cancer risk and so forth. And so in this case, what we have is that we have 18 different modalities all merged into a singular vector or a representation for the patient, and then we use that representation in a more supervised problem downstream to try to see if we were able to predict those diseases early on. And for many of these diseases, we do see that we can predict them earlier. We can also look at disease progression, and trying to see what the progression of the disease would be. The baseline here is just an age, sex baseline, so a very simplistic baseline. But we show that using this representation that includes both structured and unstructured data, you can improve both early diagnosis of disease as well as disease progression. And then you can also look at treatment outcome predictions, because I think this is important. We want to know which drugs the patient would respond to. And you could do that in a retrieval setting where you're finding similar cases, but here it's being posed as a supervised problem. And then you can also, because you have representations of patients, you can also look at clinical operation management as a whole as well. And then here are just some results for cancer survival and progression, and also some baselines for what happens when you use structured and unstructured data, as well as some very fundamental age, sex baselines as well. You can also stratify patients into distinct risk groups. This is just by using the representation itself, and we also studied that these stratifications are also well calibrated. So once we have these representations and we've used them in some kind of supervised downstream tasks, then we can also go back and assess the interpretability of these supervised models as well as those representations. But we want to see that what the model uses in making some of these predictions and how the risk of the patient evolves over time, and this is an example here that we can study how the risk of the patient is evolving over time and what is responsible for that risk evolution. So you can study this at the level of the patient, but you can also study this at the level of the large cohorts, so that can lead to discovery of additional information, additional knowledge. This is something that we are still exploring and will take more time for us to see whether it has any discovery potential. So this is just a flavor of what's coming. This is a very early stage study, and we've been working in this direction for a while, but we will continue to work to validate this model and see what the capabilities of the models are. So you can download a preprint using this QR code, and the preprint would also be available on arXiv in the coming days. So I'll stop here in the interest of time, and I'd like to thank all the PhD students and postdocs who have contributed to this work over the years, as well as all the funding that we've received to make this work possible. Thank you so much. Thank you very much. It's truly an AI revolution in cancer research. Thank you very much for joining today's plenary session. Please come back next year. There's much more to come, and I wish you a nice conference. Thank you.