Erfan Esmaeili

Ph.D. student, Electrical & Computer Engineering, Purdue University
Advised by Prof. Qiang Qiu · Ph.D. in Theoretical Physics

I work on post-training generative models with reinforcement learning: designing rewards that capture what we actually want from a model, and optimizing toward them. My recent work fine-tunes diffusion models with RL using gradient-based influence rewards (ECCV 2026), so that the synthetic data they generate measurably helps downstream classifiers. I am now carrying the same questions (reward design, credit assignment, reward hacking) over to RL post-training of LLMs.

I also study how attention works inside generative models, including training-free attention control for text-to-image alignment (CVPR 2025, co-first author). Before machine learning, I earned a Ph.D. in theoretical physics.

Seeking Research Scientist / Research Intern positions for Summer 2027. U.S. Permanent Resident. No sponsorship required.

CV (PDF) Email Google Scholar GitHub LinkedIn

News

Oct 2026Looking for research scientist internships for Summer 2027. Feel free to reach out.
Jul 2026Steering Diffusion Models via Class-Contrastive Influence for Few-Shot Medical Classification, our RL fine-tuning paper, was accepted at ECCV 2026.
Dec 2025Wrapped up the LLM Model-Identification Platform at Purdue RCAC. It is now used in Purdue's cybersecurity course (CNIT 27000).
Jun 2025T-SAM, our attention-control method for text-to-image alignment, appeared at CVPR 2025.
Aug 2022Started my Ph.D. in ECE at Purdue with Prof. Qiang Qiu.

Research

RL post-training · Diffusion

Learned rewards for useful synthetic data

Class-Contrastive Influence scores how much a generated sample helps a downstream classifier. Used as an RL reward, it fine-tunes diffusion models to produce data that improves few-shot medical classification.

RL post-training · LLMs (in progress)

Reward design for LLM agents

GRPO post-training of LLMs on multi-turn tool use. The question carried over from diffusion: which rewards produce the behavior we want, and which ones get hacked?

Attention in generative models

Attention control for text-to-image alignment

Cross-attention in diffusion models ignores word relations that the text encoder's self-attention already captures. Steering cross-attention maps toward that structure, with no training, fixes attribute binding and missing objects.

Publications

* equal contribution · Full list on Google Scholar

Diffusion model → synthetic samples → influence reward → RL update

Steering Diffusion Models via Class-Contrastive Influence for Few-Shot Medical Classification

Jeeyung Kim, Erfan Esmaeili, Qiang Qiu

ECCV 2026European Conference on Computer Vision

A gradient-based influence score used as an RL reward to fine-tune diffusion models, improving accuracy and robustness over diffusion-augmentation baselines on few-shot medical imaging benchmarks.

Paper

BibTeX

@inproceedings{kim2026steering,
  title     = {Steering Diffusion Models via Class-Contrastive Influence
               for Few-Shot Medical Classification},
  author    = {Kim, Jeeyung and Esmaeili, Erfan and Qiu, Qiang},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

T-SAM method overview: cross-attention similarity matrix is steered toward the text self-attention matrix

Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps

Jeeyung Kim*, Erfan Esmaeili*, Qiang Qiu

CVPR 2025IEEE/CVF Conference on Computer Vision and Pattern Recognition

A training-free method that aligns cross-attention maps with the text encoder's self-attention, improving attribute binding and object fidelity in Stable Diffusion.

Paper Project page

BibTeX

@inproceedings{kim2025tsam,
  title     = {Text Embedding is Not All You Need: Attention Control for
               Text-to-Image Semantic Alignment with Text Self-Attention Maps},
  author    = {Kim, Jeeyung and Esmaeili, Erfan and Qiu, Qiang},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision
               and Pattern Recognition (CVPR)},
  year      = {2025}
}

Earlier work: 7 papers in theoretical physics, listed on Google Scholar.

Background

Experience

Graduate Research Assistant, Purdue ECE2022 – present

RL post-training of generative models · Advisor: Qiang Qiu

Rosen Center for Advanced Computing, PurdueMay – Dec 2025

Built the LLM Model-Identification Platform for teaching

Postdoctoral Researcher, IPM2020 – 2022

Theory of wide neural networks · Tehran, Iran

Education

Ph.D., Electrical & Computer Engineering2022 – present

Purdue University

Ph.D., Physics2015 – 2020

Institute for Research in Fundamental Sciences (IPM) · asymptotic symmetries

M.Sc. & B.Sc., Physics2009 – 2015

Sharif University of Technology

Gold Medal, International Olympiad on Astronomy and Astrophysics2007

Skills

ML: PyTorch, RL fine-tuning of generative models, Hugging Face (Diffusers, Transformers, PEFT), vLLM, LoRA
Engineering: Python, Bash, SQL, Docker, Kubernetes, GitHub Actions, GCP, Linux / multi-GPU servers