- Uncertainty quantification using token-level log-probabilities over generated sequences.
- SHAP-based explanation as an independent signal for calibrating LLM confidence.
- Rule-based, logically grounded actions derived from confidence scores and explanations.
- Reflective, memory-augmented agents for open-world clinical case reasoning.
Hi! This is Aref Mahjoubfar. 👋
I am a physician and researcher working at the intersection of medicine and artificial intelligence. My research asks a single practical question: when should a clinical AI system be trusted, and how do we measure that? I work on uncertainty quantification and calibration for large language models, explainable and rule-grounded decision policies, medical computer vision, and the architecture of agentic AI systems for clinical environments.
About
My interest in computing came long before my medical training. I started designing custom websites from scratch as a teenager, moved into application development with C#, and gradually shifted toward machine learning — well before it became the obvious thing to study.
During my medical education I teamed up with a group of friends to explore computer vision in clinical practice. That work became my thesis: diagnosing acute lymphoblastic and myeloblastic leukemia (ALL and AML) from blood and bone marrow slide images, using a dataset we collected ourselves over the span of a year — end to end, from clinical data collection through preprocessing, model development, and evaluation.
Since then my work has moved from images to language. Working with collaborators at the Trustworthy AI Lab at Ontario Tech University, Mount Sinai, and Iran University of Medical Sciences, I have studied how large language models express confidence in medical tasks — whether that confidence means anything, how badly it is calibrated, and what a clinical system should actually do with it.
That last part is what holds my attention now. A probability is not a decision. My current research is about closing the gap between the two: translating token-level uncertainty signals into explicit, auditable policies that decide when a model may answer, when it must defer, and when it must escalate to a clinician.
I combine a clinical background with hands-on engineering — Python 🐍, PyTorch, LLM evaluation pipelines, containerized reproducible experiments — because I think the algorithmic and the infrastructural questions in medical AI are the same question seen from two sides.
My VisionLife moves on, and I am genuinely excited about the future and the next wave of technological innovation (this is what immortality is for 😉). I firmly believe in open science and in combining our efforts, step by step, to lay the foundations for tomorrow's breakthroughs. This is how I get to experience more in less time :)
Research InterestsCurrent research
Uncertainty quantification in medical LLMs
I study how to measure — and how to use — the uncertainty in large language model outputs on medical tasks. This line of work connects:
- token-level log-probabilities and sequence-level confidence measures
- semantic entropy, self-consistency, and agreement across paraphrases
- calibration metrics (AUROC, Brier score, Expected Calibration Error) and the decision thresholds derived from them
- explanation methods such as SHAP as a second, independent signal
- selective prediction: knowing when the right answer is "I don't know"
Our review of the field, The challenge of uncertainty quantification of large language models in medicine, frames uncertainty not as a defect to be removed but as information to be communicated. A follow-up study evaluated five LLMs across 156 experimental configurations on a Persian medical exam dataset, and found that chain-of-thought prompting improves accuracy while worsening overconfidence — and that emotional prompting makes it worse still. Accuracy and calibration have to be optimized together; improving one can silently damage the other.
From confidence to care: rule-grounded clinical escalation
Reporting a confidence score is not the same as acting on it. In From Confidence to Care (IEEE CASCON 2025), we map confidence and robustness signals onto a transparent, logic-based policy that outputs discrete care levels aligned with recognizable triage standards (ESI, CTAS, NEWS2), together with a human-readable trace explaining why each decision was made.
The goal is an AI system whose uncertainty is not a number in a log file but a governing component of its behavior — auditable by a clinician, and answerable for.
Reflective and agentic AI for clinical reasoning
My current work concerns AI agents that reason over clinical cases rather than single model calls: an LLM diagnostic core paired with an explicit critic, a reflector, a deterministic interpreter, and multi-tier memory that accumulates reasoning strategies across cases.
The questions I find most interesting here are about behavior over time. Does reflection produce durable improvement, or drift? What happens when an agent learns from incomplete, unlabeled, open-world cases instead of clean labeled ones? And what does that cost in latency, throughput, and reliability when the system has to run in a real clinical setting?
Selected projects
Publications
My published work spans artificial intelligence in medicine, medical imaging, machine learning, and clinical evidence synthesis. A complete and current list is on my Google Scholar profile.
- Atf Z, Safavi-Naini SAA, Lewis PR, Mahjoubfar A, Naderi N, Savage TR, Soroush A. The challenge of uncertainty quantification of large language models in medicine. arXiv:2504.05278. [arXiv]
- Naderi N, Atf Z, Lewis PR, Mahjoubfar A, Safavi-Naini SAA, Soroush A. Evaluating prompt engineering techniques for accuracy and confidence elicitation in medical LLMs. arXiv:2506.00072. [arXiv]
- Atf Z, Mahjoubfar A, Lewis PR. From confidence to care: rule-based escalation for trustworthy clinical AI. IEEE International Conference on Collaborative Advances in Software and Computing (CASCON), 2025.
- Azizi S, Tabrizi T, Vatankhah M, Naghiloo F, Ganji Z, Mahjoubfar A. Recent machine learning approaches in diagnosing leukemia using bone marrow slides: a systematic review. Authorea, 2025.
- Noori M, Azizi S, Abbasi Varaki F, Fayyaz F, Mahjoubfar A, et al. Efficacy and safety of immune checkpoint inhibitors for patients with prostate cancer: a systematic review and meta-analysis. Frontiers in Immunology, 14:1181051. [journal]
- Abbasi Varaki F, Azizi S, Mahjoubfar A. The accuracy of convolutional neural network–long short-term memory (CNN-LSTM) in diagnosing different types of acute lymphocytic leukemia based on peripheral blood smear. First International Congress of Artificial Intelligence in Medical Sciences.
- Noori M, Azizi S, Fayyaz F, Mahjoubfar A, Rezaei N. Immune checkpoint inhibitors plus chemotherapy versus chemotherapy alone as first-line therapy for advanced gastric and esophageal cancers: a systematic review and meta-analysis. International Immunopharmacology, 113:109317. [journal]
Technical expertise
Open science & direction
Progress in medical AI depends on more than more capable models. It depends on transparent methods, reproducible experiments, honest evaluation, and collaboration across medicine, computer science, and engineering. I try to make my work reproducible and available — through code, documentation, and openly accessible research outputs.
I am building a research career in AI for medicine, in environments where clinical knowledge, machine learning, and systems engineering meet. The long-term goal is medical AI that is:
- Capable enough to be useful.
- Transparent enough to be understood.
- Calibrated enough to be trusted — and to be doubted — at the right times.
- Robust enough to survive contact with a real clinic.
Collaboration
I am open to collaboration on medical AI, machine learning for healthcare, medical imaging, large language models, trustworthy and explainable AI, uncertainty quantification, clinical decision support, and AI agents in healthcare systems. If your work overlaps with any of these, I would be glad to hear from you.