Publications
* Equal contribution- ASSERT: A Measurement Pipeline for GenAI Audits EMNLP 2026 Industry Track paper arXiv code website
- Many AI Analysts, One Dataset: Navigating the Agentic Data Science Multiverse PNAS 2026 paper arXiv
- SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents NeurIPS 2026 Workshop on Foundations of Language Model Security arXiv
- Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality Preprint 2026 arXiv
- Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge Preprint 2025 arXiv
- Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles EMNLP 2025 paper arXiv
- Stronger Neyman Regret Guarantees for Adaptive Experimental Design ICML 2025 paper arXiv code
- Improving LLM Group Fairness on Tabular Data via In-Context Learning AIES 2025 paper arXiv
- Precise Model Benchmarking with Only a Few Observations EMNLP 2024 paper arXiv
- A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimation ECCV 2024 paper arXiv code
- Multicalibration for Confidence Scoring in LLMs ICML 2024 paper arXiv code
- Confidence Intervals for Error Rates in 1:1 Matching Tasks: Critical Statistical Analysis and Recommendations IJCV 2024 paper arXiv code PyPI The arXiv version includes a power analysis (Appendix C.5).
- Estimating the Likelihood of Arrest from Police Records in Presence of Unreported Crimes Annals of Applied Statistics 2024 paper arXiv code
Earlier publications (2019–2023)
- The Progression of Disparities within the Criminal Justice System: Differential Enforcement and Risk Assessment Instruments FAccT 2023 paper arXiv
- Homophily and Incentive Effects in Use of Algorithms CogSci 2022 paper arXiv
- Human Discernment of Algorithmic Errors: A Case Study in Child Welfare Working paper 2022 SSRN
- Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging FAccT 2022 paper arXiv code
- Racial Disparities in the Enforcement of Marijuana Violations in the US AIES 2022 paper arXiv code
- On the Validity of Arrest as a Proxy for Offense: Race and the Likelihood of Arrest for Violent Crimes AIES 2021 (oral) paper arXiv code
- The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies CSCW 2021 paper arXiv code & data
-
maars: Tidy Inference under the ‘Models as Approximations’ Framework in R Preprint 2021 arXiv code talk - Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty AIES 2021 paper arXiv
- Why PATTERN Should Not Be Used: The Perils of Using Algorithmic Risk Assessment Tools During COVID-19 Partnership on AI Issue Brief 2020 paper
- Lessons from the Deployment of an Algorithmic Tool in Child Welfare CHI 2020 Workshop on Fair & Responsible AI paper (archived)
- A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores CHI 2020 paper arXiv blog post
- Fairness Evaluation in Presence of Biased Noisy Labels AISTATS 2020 paper arXiv
- TRAP: A Predictive Framework for Trail Running Assessment of Performance Journal of Quantitative Analysis in Sports 2020 paper arXiv talk Best poster award at NESSIS 2019 and at CMSAC 2019 (1 of 4); talk at MIT SSAC.
- Trajectories of Prescription Opioids Filled Over Time PLOS ONE 2019 paper