Invited Talk, Tuesday 18 June, 16:00-18:00 (Greek time)

Speaker: Panagiotis Kaliosis, Athens University of Economics and Business and KTH Royal Institute of Technology (https://pkaliosis.github.io)

Title: "Data-Driven Guided Decoding for Diagnostic Captioning"

Room: Amphitheater, Archimedes Unit (1 Artemidos str., Marousi, ground floor)

and virtually via MS Teams (meeting information shown below)

Abstract:

Diagnostic Captioning (DC) automatically generates a diagnostic text from one or more medical images (e.g., X-rays, MRIs) of a patient. Treated as a draft, the generated text may assist clinicians, by providing an initial estimation of the patient's condition, speeding up and helping safeguard the diagnostic process. The accuracy of a diagnostic text, however, strongly depends on how well the key medical conditions depicted in the images are expressed. We propose a new data-driven guided decoding method that incorporates medical information, in the form of existing tags capturing  key conditions of the image(s),  into the beam search of the diagnostic text generation process. We evaluate the proposed method on two medical datasets using four DC systems that range from generic image-to-text systems with CNN encoders and RNN decoders to pre-trained Large Language Models. The latter can also be used in few- and zero-shot learning scenarios. In most cases, the proposed mechanism improves performance with respect to all evaluation measures.

For ways to check the latest information about the meetings of Archimedes NLP Theme and AUEB NLP Group, check this Google Calendar. For ways to receive news about the Archimedes Unit and its meetings, check https://archimedesai.gr/en/. To subscribe to the mailing list of Archimedes, send a message with title "subscribe archimedes-news  Firstname LastName” (where Firstname   and LastName  are your First and Last Name respectively) to This email address is being protected from spambots. You need JavaScript enabled to view it.. The body of the message may be blank. If you are an AI researcher or practitioner, please consider becoming a member of the Hellenic Artificial Intelligence Society (EETN, http://www.eetn.gr/en/).