Archimedes Talk Series on: "Towards Trustworthy AI: Understanding Memorization, Privacy, and Security in Deep Learning" by Dr. Deepak Ravikumar (Amazon, Purdue University, USA)

Abstract

As deep learning systems are increasingly deployed in safety-critical domains such as healthcare, finance, and autonomous navigation, ensuring that these systems are not only accurate but also trustworthy has become essential. Trustworthy AI is grounded in six foundational pillars: human agency and oversight, fairness, explainability, robustness, privacy, and accountability. This talk advances trustworthy AI by addressing three deeply interconnected challenges: memorization, privacy, and robustness. Although these represent a subset of the broader framework, the pillars are interdependent, progress in one often reinforces others. We begin by studying memorization, where models overfit specific training samples, including noisy, rare, or mislabeled data. To quantify memorization efficiently, we introduce two novel metrics: Cumulative Sample Loss (CSL) and Cumulative Sample Gradient (CSG). These proxies track training dynamics, correlate with traditional stability-based memorization scores, and are orders of magnitude more efficient. We show that CSL and CSG theoretically bound both memorization and learning time, enabling scalable detection of mislabeled data, dataset bias, and duplicates. Additionally, CSG facilitates early stopping without a validation set. We then connect memorization to privacy, showing that memorized samples are more vulnerable to membership inference attacks. We derive theoretical bounds linking memorization, input loss curvature, and differential privacy. Leveraging these insights, we develop a black-box membership inference attack based on input loss curvature, achieving state-of the-art performance. Finally, we address robustness in the face of adversarial perturbations and out-of-distribution (OoD) examples. We propose Intra-Class Mixup and Norm-Scaling, which enhance OoD detection. To improve ensemble robustness, we introduce TREND (Transferability-based Robust Ensemble Design), which leverages adversarial transferability for principled ensemble construction. We also present In-Distribution Knowledge Distillation (IDKD), which supports robust decentralized learning under non-IID data distribution. Collectively, this talk offers a theoretically grounded and practically relevant framework for enhancing memorization, privacy, and robustness in deep learning, contributing key tools and insights for building more trustworthy AI systems.

Short Biography

Deepak is currently an Applied Scientist II at Amazon, where he researches building better ML models to represent sellers on the platform. He conducted his Ph.D. research under Prof. Kaushik Roy, where his research gained significant recognition. His work has been spotlighted at leading conferences, earning the 2024 NeurIPS Spotlight Paper Award (Top 2%), the 2024 ICML Spotlight Paper Award (Top 3.5%) and Estus H. and Vashti L. Magoon Research Excellence Award 2025. At Purdue, he has been awarded the College of Engineering Scholarship and the ECE Summer Research Grant. His research focuses on deep learning algorithms, with a particular emphasis on deep learning memorization and trustworthy machine learning. He previously worked as an ML Research intern at Microsoft, where he focused on predictive time-series machine learning models. Prior to that, at National Instruments R&D, he developed innovative signal acquisition and processing frameworks, with his work being recognized as one of the Top 3 Best Papers at the National Instruments Tech Conference in 2017. Deepak earned his M.S. in Electrical and Computer Engineering from Purdue University in 2019, where he received the prestigious Magoon Teaching Excellence Award for his outstanding contributions as a teaching assistant. He completed his B.E. in Electronics Engineering from M. S. Ramaiah Institute of Technology, India, in 2016, graduating as a bronze medallist for academic excellence.