Josiah Aklilu

PhD student at Stanford

Currently, I’m a Biomedical Data Science PhD student at Stanford University where I’m fortunately advised by Prof. Serena Yeung in the MARVL lab. My research interests are at the intersection of machine learning and healthcare, centered in computer vision and vision-language models for surgical skill assessment and training. I’m also very interested in developing zero-shot methods for video understanding in biomedical domains.

Previously, I received my Bachelor’s in Mathematics, Computer Science, and Physics at the University of Denver with a Distinction in Computer Science.

selected publications

2024

TMLR

Revisiting Active Learning in the Era of Vision Foundation Models

Sanket Rajan Gupte*, Josiah Aklilu*, Jeffrey J Nirschl, and Serena Yeung-Levy

Transactions on Machine Learning Research, 2024

Abstract PDF Code

Foundation vision or vision-language models are trained on large unlabeled or noisy data and learn robust representations that can achieve impressive zero- or few-shot performance on diverse tasks. Given these properties, they are a natural fit for active learning (AL), which aims to maximize labeling efficiency, but the full potential of foundation models has not been explored in the context of AL, specifically in the low-budget regime. In this work, we evaluate how foundation models influence three critical components of effective AL, namely, 1) initial labeled pool selection, 2) ensuring diverse sampling, and 3) the trade-off between representative and uncertainty sampling. We systematically study how the robust representations of foundation models (DINOv2, OpenCLIP) challenge existing findings in active learning. Our observations inform the principled construction of a new simple and elegant AL strategy that balances uncertainty estimated via dropout with sample diversity. We extensively test our strategy on many challenging image classification benchmarks, including natural images as well as out-of-domain biomedical images that are relatively understudied in the AL literature. Source code will be made available.
NEJM AI

Artificial Intelligence Identifies Factors Associated with Blood Loss and Surgical Experience in Cholecystectomy

Josiah Aklilu, Min Woo Sun, Shelly Goel, Sebastiano Bartoletti, and 12 more authors

NEJM AI, 2024

Abstract PDF Code

A computer vision model developed to recognize fine-grained surgical activity in laparoscopic cholecystectomy videos reveals associations among surgical behaviors, adverse outcomes, and surgical skills.

2022

MLHC

ALGES: Active Learning with Gradient Embeddings for Semantic Segmentation of Laparoscopic Surgical Images

Josiah Aklilu, and Serena Yeung-Levy

In Proceedings of the 7th Machine Learning for Healthcare Conference, Aug 2022

Abstract PDF Code

Annotating medical images for the purposes of training computer vision models is an extremely laborious task that takes time and resources away from expert clinicians. Active learning (AL) is a machine learning paradigm that mitigates this problem by deliberately proposing data points that should be labeled in order to maximize model performance. We propose a novel AL algorithm for segmentation, ALGES, that utilizes gradient embeddings to effectively select laparoscopic images to be labeled by some external oracle while reducing annotation effort. Given any unlabeled image, our algorithm treats predicted segmentations as truth and computes gradients with respect to the model parameters of the last layer in a segmentation network. The norms of these per-pixel gradient vectors correspond to the magnitude of the induced change in model parameters and contain rich information about the model’s predictive uncertainty. Our algorithm then computes gradients embeddings in two ways, and we employ a center-finding algorithm with these embeddings to procure representative and diverse batches in each round of AL. An advantage of our approach is extensibility to any model architecture and differentiable loss scheme for semantic segmentation. We apply our approach to a public data set of laparoscopic cholecystectomy images and show that it outperforms current AL algorithms in selecting the most informative data points for improving the segmentation model. Our code is available at https://github.com/josaklil-ai/surg-active-learning.