Riccardo Fogliato

I am a senior research scientist on the Responsible AI team at Microsoft Core AI, working on LLM safety. Previously, I was an applied scientist on the Responsible AI team at AWS, where I worked on staistical methods for evaluating language and vision models.

I received a PhD in Statistics from CMU under the supervision of Alexandra Chouldechova and Zachary Lipton. During the PhD, I interned at MSR with Besmira Nushi, Kori Inkpen, and Eric Horvitz. I also was a research fellow at the Partnership on AI where I worked with Alice Xiang. Before going to CMU, I studied at Collegio Carlo Alberto, at the Universities of Torino and Padova, and spent some time at ENS Cachan.

You can contact me at riccardofogliato [at] gmail [dot] com.

Resume / GitHub / LinkedIn / Google Scholar

Papers

Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge
Evangelia Spiliopoulou*, Riccardo Fogliato*, et al
arXiv arXiv
Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles
Kimberly Truong, Riccardo Fogliato, Hoda Heidari, Steven Wu
EMNLP 2025 arXiv
Stronger Neyman Regret Guarantees for Adaptive Experimental Design
Georgy Noarov, Riccardo Fogliato, Martin Bertran, Aaron Roth
ICML 2025 arXiv code
Improving LLM Group Fairness on Tabular Data via In-Context Learning
V. Cherepanova, CJ Lee, N. Akpinar, R. Fogliato, M. Bertran, M. Kearns, J. Zou
AIES 2025 arXiv
Precise Model Benchmarking with Only a Few Observations
Riccardo Fogliato, Pratik Patil, Nil-Jana Akpinar, Mathew Monfort
EMNLP 2024 arXiv
A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimation
Riccardo Fogliato, Pratik Patil, Mathew Monfort, Pietro Perona
ECCV 2024 arXiv ECCV code
Multicalibration for Confidence Scoring in LLMs
Gianluca Detommaso, Martin Bertran*, Riccardo Fogliato*, Aaron Roth
ICML 2024 arXiv code
Confidence Intervals for Error Rates in 1:1 Matching Tasks: Critical Statistical Analysis and Recommendations
Riccardo Fogliato, Pratik Patil, Pietro Perona
International Journal of Computer Vision arXiv (w/ power analysis in C.5) IJCV code PyPI
Estimating the Likelihood of Arrest from Police Records in Presence of Unreported Crimes
R. Fogliato, AK Kuchibhotla, Z. Lipton, D. Nagin, A. Xiang, and A. Chouldechova
Annals of Applied Statistics arXiv AOAS code
The Progression of Disparities within the Criminal Justice System: Differential Enforcement and Risk Assessment Instruments
M. Zilka, R. Fogliato, J. Hron, B. Butcher, C. Ashurst, A. Weller
FAccT 2023 arXiv
Homophily and Incentive Effects in Use of Algorithms
Riccardo Fogliato, Sina Fazelpour, Shantanu Gupta, Zachary Lipton, David Danks
CogSci 2022 arXiv
Human Discernment of Algorithmic Errors: A Case Study in Child Welfare
Maria De-Arteaga*, Riccardo Fogliato*, and Alexandra Chouldechova SSRN
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
R. Fogliato, S. Chappidi, M. Fitzke, M. Parkinson, D. Wilson, P. Fisher, M. Lungren, E. Horvitz, K. Inkpen, B. Nushi
FAccT 2022 arXiv platform
Racial Disparities in the Enforcement of Marijuana Violations in the US
Bradley Butcher, Chris Robinson, Miri Zilka, Riccardo Fogliato, Carolyn Ashurst, Adrian Weller
AIES 2022 arXiv code
On the Validity of Arrest as a Proxy for Offense: Race and the Likelihood of Arrest for Violent Crimes
Riccardo Fogliato, Alice Xiang, Zachary Lipton, Daniel Nagin, Alexandra Chouldechova
AIES 2021 (oral) arXiv ACM code
The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies
Riccardo Fogliato, Alexandra Chouldechova, Zachary Lipton
CSCW 2021 arXiv ACM data+code
maars: an R implementation of Models As Approximations
Riccardo Fogliato*, Shamindra Shrotriya*, and Arun Kumar Kuchibhotla code arXiv talk
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
U. Bhatt, Y. Zhang, J. Antorán, Q.V. Liao, P. Sattigeri, R. Fogliato, G.G. Melançon, R. Krishnan, J. Stanley, O. Tickoo, L. Nachman, R. Chunara, A. Weller, A. Xiang
AIES 2021 arXiv ACM
Lessons from the Deployment of an Algorithmic Tool in Child Welfare
Riccardo Fogliato*, Maria De-Arteaga*, Alexandra Chouldechova
Fair & Responsible AI Workshop, CHI 2020 workshop
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
Maria De-Arteaga*, Riccardo Fogliato*, Alexandra Chouldechova (* co-first)
CHI 2020 arXiv ACM Medium post
Fairness Evaluation in the Presence of Biased Noisy Labels
Riccardo Fogliato, Max G'Sell, Alexandra Chouldechova
AISTATS 2020 arXiv PMLR
TRAP: A Predictive Framework for Trail Running Assessment of Performance
Riccardo Fogliato, Natalia Oliveira, Ronald Yurko
Journal of Quantitative Analysis in Sports arXiv JQAS Talk @ MIT SSAC
^† Best poster award at NESSIS 2019 and at CMSAC 2019 (1 of 4) poster
Trajectories of Prescription Opioids Filled Over Time
J. Elmer, R. Fogliato, N. Setia, W. Mui, M. Lynch, E. Hulsey, D. Nagin
PLOS one 2019 PLOS

Why PATTERN Should Not Be Used: The Perils of Using Algorithmic Risk Assessment Tools During COVID-19
Riccardo Fogliato, Alice Xiang, Alexandra Chouldechova
Issue brief of the Partnership on AI issue brief

Riccardo Fogliato

Papers

Misc