Research Interests

My general research interests are in AI safety and responsible AI, aiming to build systems that are maximally useful to society with provable guarantees. In my current research, I'm working at the intersection of uncertainty quantification and decision making. I'm studying the calibration properties induced by minimizing generalized risk measures as an aggregation mechanism. I'm also interested in safe statistics, imprecise probabilities, and possibility theory.

Previously, I worked on studying the calibration properties of learning to defer (L2D) systems [ICML'22], extending L2D systems to allow for multiple experts [AISTATS'23], and studying the out-of-distribution behavior of L2D systems (in preparation). I also collaborated on a project on the test-time adaption of L2D to new experts [AISTATS'24].

Selected Publications

 (* Denotes equal contribution)

• Rajeev Verma, Volker Fischer, Eric Nalisnick.
  On the Calibration of Conditional-Value-at-Risk.
  In ICML 2024 Workshop on Next Generation of AI Safety.
  
  
• Dharmesh Tailor, Aditya Patra, Rajeev Verma, Putra Manggala, Eric Nalisnick.
  Learning to Defer to a Population: A Meta-Learning Approach.
  In Twenty-seventh Conference on Artificial Intelligence and Statistics (AISTATS 2024).
  [Oral, Student paper award (top 1%)].
  
  
• Rajeev Verma^*, Daniel Barrejón^*, Eric Nalisnick.
  Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles.
  In Twenty-sixth Conference on Artificial Intelligence and Statistics (AISTATS 2023).
  Note: Also appeared at the ICML 2022 Workshop on Human-Machine Collaboration and Teaming as On the Calibration of Learning to Defer to Multiple Experts.
  
  
• Rajeev Verma, Eric Nalisnick.
  Calibrated Learning to Defer with One-vs-All Classifiers.
  In International Conference on Machine Learning (ICML 2022).
  
  
• Rajeev Verma.
  On the Calibration of Learning to Defer Systems.
  Master's Thesis 2022 (UvA).
  
  

Service

 Reviewer:

 ICML: 2023, 2024
 NeurIPS: 2023 (Top reviewer)
 UAI: 2024
 ICLR: 2023
 ACL ARR: 2024

 Teaching:

 Human-in-the-Loop Machine Learning (Teaching Assistant)
 Deep Learning 2 (Teaching Assistant)