Radiology as a pioneer of digitalisation

Every day, a university hospital produces thousands of digital radiological images – CT scans, MRI scans and X-rays – which are brimming with information. Even the most experienced radiologist is usually unable to fully analyse all of this within the time available. This is precisely where Artificial Intelligence (AI) can help: it extracts clinically relevant information from enormous volumes of image data – information that remains hidden from the human eye or would be disproportionately time-consuming to analyse manually. At the Institute of Diagnostic and Interventional Radiology at the Technical University of Munich, Lisa Adams and Keno Bressem have established an interdisciplinary team – the ‘AI-Assisted Healthcare’ (AIAH) research group – which develops AI applications ranging from basic research right through to everyday clinical practice. Their work is structured around two central pillars – image-based and language-based AI – which complement one another and work together towards the same goal: a form of medicine that is more precise, faster and more patient-centred.

Image-based AI. Algorithms that learn to see

In radiology, certain trade-offs were long considered unavoidable: lower radiation exposure meant poorer CT image quality, whilst shorter MRI scan times resulted in lower resolution – to the detriment of both patients and medical staff. If patients were to be spared, this resulted in poorer image quality for diagnosis and treatment. If doctors wanted more informative images, they had to expose their patients to higher levels of radiation. AI-based image reconstruction has made these trade-offs a thing of the past and now enables better images to be produced at lower doses and with shorter scan times. For Adams and Bressem, this was the defining moment: AI is changing not only processes, but also what is actually possible in medicine.
Since then, we have consistently refined this approach. Deep-learning algorithms detect tumour-suspicious lesions in the prostate and inflammatory changes in axial spondyloarthritis in MRI scans with an accuracy approaching that of experienced radiologists. A key project, funded by the Wilhelm Sander Foundation, is now applying this principle to the diagnosis of kidney tumours. Around 15 per cent of all kidney tumours that undergo surgery are subsequently found to be benign, as imaging alone does not allow for a reliable distinction. A multimodal AI system that combines image data with laboratory results and clinical information aims to change this and prevent unnecessary operations. The technical basis for this is the MRSegmentator: a model developed by us that automatically recognises over 40 anatomical structures in CT and MRI scans. As an open-source tool, it is freely available to the international research community.

Language-based AI. When machines read and write medical reports

Whilst image-based AI is transforming diagnostics, language-based AI opens up a new dimension: communication. For most patients, a radiology report is a document that is difficult to understand: dense, full of technical jargon and often unsettling. In a prospective study involving 200 cancer patients, we used a locally operated language model to translate CT reports into understandable language. The impact was noticeable and scientifically measurable: cognitive strain decreased significantly and patients’ understanding of their own condition demonstrably improved.

“Several patients said that, for the first time, they felt they were able to fully comprehend their radiological report.”

This result, published in the journal *Radiology*, exemplifies the potential of large language models (LLMs) in medicine. These are AI systems capable of understanding and generating human language. Radiological reports are a particularly suitable application for this: they are text-based, structured and clinically dense. With ‘medAlpaca’, our group has developed one of the first freely available medical language models. We have demonstrated that LLMs can automatically convert free-text findings into structured reports – in multiple languages and with a high degree of accuracy. This approach is complemented by RadioRAG: a system that links language models with real-time knowledge retrieval from specialist radiology databases. This ensures that AI-generated responses are based on current guidelines and specialist literature. The fact that all of this is possible within the hospital’s secure infrastructure, without patient data being passed on to external cloud services, is not merely a technical footnote, but a fundamental ethical requirement.

Outlook. From tool to assistant

Bringing together images and language is the next step. With funding from the Wilhelm Sander Foundation, we are developing a multimodal foundation model for oncological imaging. This is an AI system that is being trained on over 100,000 segmented CT and MRI scans to learn to distinguish tumour regions from healthy tissue. Foundation models can then be adapted to specific clinical questions with minimal effort. They thus form the basis for a new generation of AI assistants that independently support multi-stage workflows: from protocol selection and image analysis to the preparation of reports. Our aim is to identify new image-based biomarkers that enable more personalised treatment planning in oncology. Medical expertise remains indispensable in this process. AI does not replace radiologists, but frees up their time for what really matters: clinical interpretation, discussions with referring doctors and the management of complex cases. Funding from the Wilhelm Sander Foundation makes these translational projects possible – from AI-assisted renal tumour diagnostics and the MRSegmentator to the multimodal Foundation Model. It supports Adams and Bressem in testing innovative technologies where they can deliver the greatest benefit to cancer patients.