archimedes-Artificial Intelligence, Data Science, Algorithms-greece

 
Artificial Intelligence
 
Data Science
 
Algorithms

[Archimedes Talks Series]A proposal for the mathematical structure computed by large language models

Dates
2024-07-09 16:00 - 18:00
Venue
Artemidos 1 - Amphitheater

Title: A proposal for the mathematical structure computed by large language models

Presenter: Dr.Yiannis Vlassopoulos (Institute for Language and Speech Processing at "Athena" Research Center)

Screen Shot 2024 07 09 at 4pm

Abstract: Large Language Models are transformer neural networks which are trained to produce a probability distribution on the possible next words to given texts in a corpus, in such a way that the most likely word predicted, is the actual word in the training text.

We will explain what is the mathematical structure defined by such conditional probability distributions of text extensions. Changing the viewpoint from probabilities to log probabilities we observe that the data of text extensions are encoded in a directed (non-symmetric) metric structure defined on the space of texts . We then construct a directed metric polyhedron P(), in which  is isometrically embedded as generators of certain special extremal rays. Each such generator encodesextensions of a text along with the corresponding probabilities.

Moreover P() is (min; +) (i.e. tropically) generated by the text extremal rays. This leads to a duality theorem relating the polyhedron P() defined by text ex- tensions to one defined by text restrictions. We also explain that the generator of the extremal ray corresponding to a text is approximated by a Boltzmann weighted linear combination of generators of extremal rays corresponding to the words making up that text. We note that these constructions generalise the familiar view of language as a monoid or as a poset with the subtext order.

This is joint work with Stephane Gaubert.

Bio: 
Yiannis Vlassopoulos earned a degree in Mathematics from the University of Athens in1992 and a Ph.D from Duke University in 1998.
His thesis was on Algebraic Geometry related to String Theory (specifically so called Mirror Symmetry, a duality between Symplectic and Algebraic Geometry). He obtained a Marie Curie Individual Fellowship with Pr. Maxim Kontsevich at the Institut des Hautes Etudes Scientifiques (IHES) in Paris, in 2002.
He subsequently worked as a researcher in IHES for an extended period of time until 2019, on non-Commutative Derived Algebraic Geometry and Topological Quantum Field Theories. In particular, in collaboration with Maxim Kontsevich, they introduced the notion of Pre-Calabi-Yau algebra which is a non-commutative analogue of a Poisson structure.
He also worked as a visiting Professor at the University of Vienna (Austria) and has been a visiting fellow at the University of Miami, Aarhus University (Denmark), the Max Planck Institute for Mathematics in Bonn and the Simons Center for Geometry and Physics at the Stoney Brook University in NY.
He obtained an ENTER fellowship at the University of Athens for the period 2006-2008.

Since 2015 he has focused on Natural Language modelling. Initially, using Tensor Networks and applying algebra and physics technics. He cofounded a company in NY in 2017 in order to develop this technology. Currently he is using Category Theory and Tropical geometry in order to model the structure that Transformer Neural Networks (like GPT) learn when they are trained to guess the next word in a text. One of the main goals is to understand how semantics is encoded and could potentially be controlled as far as logical implications are concerned.

________________________________________________________________________________

Microsoft Teams Need help?

Join the meeting now

Meeting ID: 347 633 128 071

Passcode: Z76hKX

________________________________________________________________________________

 
 

Vision

To position Greece as a leading player in AI and Data Science

image
image

Mission

To build an AI Excellence Hub in Greece where the international research community can connect, groundbreaking ideas can thrive, and the next generation of scientists emerges, shaping a brighter future for Greece and the world

 

Welcome to ARCHIMEDES, a vibrant research hub connecting the global AI and Data Science research community fostering groundbreaking research in Greece and beyond. Its dedicated core team, comprising lead researchers, affiliated researchers, Post-Docs, PhDs and interns, is committed to advancing basic and applied research in Artificial Intelligence and its supporting disciplines, including Algorithms, Statistics, Learning Theory, and Game Theory organized around 8 core research areas. By collaborating with Greek and Foreign Universities and Research Institutes, ARCHIMEDES disseminates its research findings fostering knowledge exchange and providing enriching opportunities for students. Leveraging AI to address real-world challenges, ARCHIMEDES promotes innovation within the Greek ecosystem and extends its societal impact. Established in January 2022, as a research unit of the Athena Research Center with support from the Committee Greece 2021, ARCHIMEDES is funded for its first four years by the EU Recovery and Resilience Facility (RRF).

 
 

NEWS

 
Archimedes Talk by Alexandra Meliou on

Archimedes Talk by Alexandra Meliou on "Data Analysis and Manipulation through a Constrained Optimization Lens"

On Tuesday 4 November, 2025, from 1:00 pm to 2:00 pm, at the Archimedes Amphitheatre (1 Artemidos Street, 15125, Marousi, Archimedes, Athena Research Center, Greece), Professor Alexandra Meliou, Professor at the College of Information and Computer Sciences at the University of Massachusetts Amherst, USA, and a visiting researcher at the Archimedes Research Center, Athena Research Center, Greece, will deliver an Archimedes talk on "Data Analysis and Manipulation through a Constrained Optimization Lens."

Exploring the Frontiers of AI and Biomechanics at ICIST 2025

Exploring the Frontiers of AI and Biomechanics at ICIST 2025

Archimedes, Athena Research Center, Greece is happy to see its work being featured at the 15th International Conference on Information Society and Technology (ICIST 2025), at Kaunas University of Technology, Lithuania, through an invited keynote speech by Dr. Serafeim Moustakidis, titled: “Bringing Biomechanics into the AI Era: A New Frontier for Gait and Joint Health.”

Archimedes Talk by Manos Athanassoulis on

Archimedes Talk by Manos Athanassoulis on "Harnessing the Untapped Benefits of Near-Sortedness for Data Systems"

On Tuesday 21 October, 2025, from 1:00 pm to 2:00 pm, at the Archimedes Amphitheatre (1 Artemidos Street, 15125, Marousi, Archimedes, Athena Research Center, Greece), Professor Manos Athanassoulis, Associate Professor of Computer Science at Boston University (BU), USA, Director and Founder of the BU Data-intensive Systems and Computing Laboratory, BU, USA, and Co-Director of the BU Massive Data Algorithms and Systems Group, BU, USA, will deliver an Archimedes talk on "Harnessing the Untapped Benefits of Near-Sortedness for Data Systems."

GameSec 2025 just Around the Corner!

GameSec 2025 just Around the Corner!

The 16th Conference on Game Theory and AI for Security (GameSec25), an event strongly supported by Archimedes, Athena Research Center, Greece will take place at the National Technical University of Athens, Greece from October 13-15 2025. "With the rapid advancement of artificial intelligence, game theory, and security technologies, the resilience and trustworthiness of modern systems is more critical than ever. The 2025 Conference on Game Theory and AI for Security focuses on leveraging strategic decision-making, adversarial reasoning, and computational intelligence to address security challenges in complex and dynamic environments.

11 Papers Accepted at NeurIPS 2025!

11 Papers Accepted at NeurIPS 2025!

We are happy to announce that 11 papers from Archimedes, Athena Research Center, Greece, have been accepted at the Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS 2025)!NeurIPS is a major annual event in the artificial intelligence and machine learning community and this year it will take place from December 2 to December 7 at the San Diego Convention Center, California, USA.

 
 

The project “ARCHIMEDES Unit: Research in Artificial Intelligence, Data Science and Algorithms” with code OPS 5154714 is implemented by the National Recovery and Resilience Plan “Greece 2.0” and is funded by the European Union – NextGenerationEU.

greece2.0 eu_arch_logo_en

 

Stay connected! Subscribe to our mailing list by emailing sympa@lists.athenarc.gr
with the subject "subscribe archimedes-news Firstname LastName"
(replace with your details)