archimedes-Artificial Intelligence, Data Science, Algorithms-greece

 
Artificial Intelligence
 
Data Science
 
Algorithms

[Archimedes NLP Group Invited Talk]Understanding the Trade-Offs Between Hallucinations and Mode Collapse in Language Generation

Dates
2025-01-15 14:30 - 16:00
Venue
Artemidos 1 - Amphitheater
TITLE: Understanding the Trade-Offs Between Hallucinations and Mode Collapse in Language Generation

SPEAKER: Grigoris Velegkas(Yale University, USA)

ABSTRACT: Specifying all desirable properties of a language model is challenging, but certain requirements seem essential. Given samples from an unknown language, the trained model should produce valid strings not seen in the training set, and be expressive enough to capture the language's full breadth. Otherwise, outputting invalid strings constitutes "hallucination," and failing to capture the full breadth leads to "mode collapse." Recent work by Kleinberg and Mullainathan [KM24], building on classical work on the closely related problem of language identification by Gold [Gol67] and Angluin [Ang79, 80], provides a concrete mathematical framework to study the problem of language generation. Kleinberg and Mullainathan showed that for all countable collections of languages, it is possible to create a language model that does not hallucinate but suffers from mode collapse. They asked whether this tension between validity and breadth is inherent for language generation.

In this talk, we define various notions of breadth for language generation, and completely characterize when generation with validity and breadth is possible under each of these notions. Our results answer the question of [KM24] and show that this tension between validity and breadth is indeed inherent for language generation. Moreover, we formalize the notion of stable generation, a natural requirement derived from Gold’s work [Gold67], and discuss when this type of generation is achievable. Finally, we discuss the implications of our results in the universal rates setting of Bousquet, Hanneke, Moran, van Handel, and Yehudayoff [BGMvY21]. The talk is based on joint works with Alkis Kalavasis and Anay Mehrotra.

References:  https://arxiv.org/abs/2411.09642 , https://arxiv.org/abs/2412.18530 

SHORT BIO: Grigoris Velegkas is a final-year PhD student in Computer Science at Yale University, working with Prof. Amin Karbasi. Before that, he studied Electrical and Computer Engineering at the National Technical University of Athens, where he worked with Prof. Dimitris Fotakis. His research lies at the intersection of machine learning and theoretical computer science, and focuses on three main directions: i) understanding generalization properties of ML algorithms, ii) exploring responsible use of ML systems and designing algorithms with provable replicability guarantees, and iii) understanding the interaction between ML algorithms and mechanisms. He was a research intern at Google Research in summer 2023 and summer 2024, and a student researcher from October 2023 to May 2024.


________________________________________________________________________________
Microsoft Teams Need help?
Meeting ID: 364 928 182 762
Passcode: w24fHy
________________________________________________________________________________

 
 

Vision

To position Greece as a leading player in AI and Data Science

image
image

Mission

To build an AI Excellence Hub in Greece where the international research community can connect, groundbreaking ideas can thrive, and the next generation of scientists emerges, shaping a brighter future for Greece and the world

 

ARCHIMEDES IMPACT

Research

Education

Figures as of October 2026

HIGHLIGHTS

Academy of Athens
Greek National AI Team selected through PDTN, the Panhellenic AI Competition, co-organized by Archimedes, Athena Research Center
ERC Grant: Christos Papadimitriou will lead the NEAGT project (starting 2026) on strategic interactions and computational decision-making, opening new directions for algorithmic game theory
Archimedes Talks: Michael I. Jordan (Inria Paris; UC Berkeley) on “Nonnegative Supermartingales, Sequential Testing, and Statistical Contract Theory”, 10 February 2026
Archimedes Talks: Paul Goldberg (University of Oxford) on “Complexity of Unambiguous Problems in Σ₂ᴾ”, 4 February 2026
ERC Grant: Christos Tzamos leads the COMFORMAL project (launched 2025) on rigorous foundations for robust and efficient machine learning algorithms
Archimedes Talks: Mark Girolami (The Alan Turing Institute; University of Cambridge) on “Statistical Finite Element Methods”, 7 November 2025
Archimedes Talks: Mihalis Yannakakis (Columbia University) on “Computing a Fixed Point of Contraction Maps in Polynomial Queries”, 25 June 2025

Highlights will be added soon.

 

NEWS

 
Θερινό Σχολείο Μηχανικής Μάθησης και Τεχνητής Νοημοσύνης 2026 για Μαθητές (GR/EN)

Θερινό Σχολείο Μηχανικής Μάθησης και Τεχνητής Νοημοσύνης 2026 για Μαθητές (GR/EN)

13 October 2026

Με μεγάλη χαρά ανακοινώνουμε ότι από τις 15 έως τις 19 Ιουνίου 2026, στη Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών του Εθνικού Μετσόβιου Πολυτεχνείου, θα πραγματοποιηθεί Θερινό Σχολείο Μηχανικής Μάθησης και Τεχνητής Νοημοσύνης για μαθητές Δευτεροβάθμιας Εκπαίδευσης (Γ΄ Γυμνασίου – Γ΄ Λυκείου), με διδάσκοντα τον Κωνσταντίνο Καραμανή. Το κείμενο είναι διαθέσιμο στα ελληνικά και στα αγγλικά (English version below).

Archimedes Leads EU-Funded AIRIS AI Project

Archimedes Leads EU-Funded AIRIS AI Project

13 October 2026

Archimedes leads a global partnership for trustworthy AI in biomedical research through the EU-funded project AIRIS AI, led by Christos Diou, affiliated member of Archimedes and Associate Professor at Harokopio University of Athens.

Η Εκπαίδευση στην Εποχή της Τεχνητής Νοημοσύνης: 1η Επιμορφωτική Συνάντηση στο 2ο Πρότυπο Γυμνάσιο Αθηνών (GR/EN)

Η Εκπαίδευση στην Εποχή της Τεχνητής Νοημοσύνης: 1η Επιμορφωτική Συνάντηση στο 2ο Πρότυπο Γυμνάσιο Αθηνών (GR/EN)

10 October 2026

Η Ερευνητική Μονάδα «Αρχιμήδης» του ΕΚ «Αθηνά» συμμετείχε στην πρώτη επιμορφωτική συνάντηση για εκπαιδευτικούς με θέμα «Η εκπαίδευση στην εποχή της Τεχνητής Νοημοσύνης», στο 2ο Πρότυπο Γυμνάσιο Αθηνών. Το κείμενο είναι διαθέσιμο στα ελληνικά και στα αγγλικά (English version below).

More news items here
 
 

The project “ARCHIMEDES Unit: Research in Artificial Intelligence, Data Science and Algorithms” with code OPS 5154714 is implemented by the National Recovery and Resilience Plan “Greece 2.0” and is funded by the European Union – NextGenerationEU.

Greece 2.0 – Funded by the European Union – NextGenerationEU