ISIT 2024 will take place at Intercontinental Athens. On July 7th, attendees will have the opportunity to participate in a series of engaging workshops designed to enhance their skills and knowledge. The following workshops will be offered, each led by experts
in their respective fields, promising an enriching experience for all participants.
Presenters: Yao Xie, Taiji Suzuki, and
Xiuyuan Cheng
As a paradigm for sequential decision making in unknown environments,
reinforcement learning (RL) has received a flurry of attention in recent years. However, the explosion of model complexity in emerging applications and the presence of nonconvexity exacerbate the challenge of achieving efficient RL in sample-starved situations,
where data collection is expensive, time-consuming, or even high-stakes (e.g., in clinical trials, autonomous systems, and online advertising). How to understand and enhance the sample and computational efficiencies of RL algorithms is thus of great interest
and in imminent need. In this tutorial, we aim to present a coherent framework that covers important algorithmic and information-theoretic developments in RL, highlighting the connections between new ideas and classical topics. Employing Markov Decision Processes
as the central mathematical model, we start by introducing classical dynamic programming algorithms when precise descriptions of the environments are available. Equipped with this preliminary background, we introduce four distinctive RL scenarios (i.e., RL
with a generative model, online RL, offline RL, and multi-agent RL), and present three mainstream RL paradigms (i.e., model-based approach, model-free approach, and policy optimization). Our discussions gravitate around two central questions: what are the
information-theoretic limits in each RL scenario, and can we achieve these fundamental limits using computationally efficient algorithms? We will systematically introduce several effective algorithmic ideas (e.g., the optimism principle for online RL, the
pessimism principle for offline RL, variance reduction) that permeate the design of efficient RL algorithms.
Presenters: Ahmad Beirami and Ananda Theertha
Suresh
In recent years, large language models have been used to solve a multitude of natural language tasks. In this tutorial, we give a brief overview of the history of language modeling and the fundamental techniques that led to the development of the modern language
models behind Claude, Gemini, GPT, and Llama. We then present an overview of inference techniques for language models, when they are treated as black-box probabilistic models.
The tutorial is structured as four segments: In the first segment, we provide a historical perspective on language models and present a black-box probabilistic model that serves as a foundation for understanding the inference from these models. In the second
segment, we examine black-box techniques aimed at aligning models towards various goals (e.g., safety), such as controlled decoding and the best-of-N algorithm. In the third segment, our focus then shifts to efficiency, where we examine information-theoretic
techniques designed to improve inference latency, such as model compression or speculative decoding. If time permits, in the final segment, we discuss in-context learning, which has become one of the standard ways to solve new tasks using black-box language
models.
Throughout the tutorial, we highlight open problems that might be of interest to the audience. The tutorial does not assume any prior knowledge on language models.