Abu Dhabi, UAEOpen to research collaboration

MSc Computer Vision at MBZUAI

Ritesh Thawkar

AI Researcher · Computer Vision · Multimodal Systems

I build multimodal AI systems that move between research prototypes and production-scale agentic applications.

Currently pursuing an MSc in Computer Vision at MBZUAI, where I work on VLM reasoning, self-evolving LMMs, and retrieval-backed AI agents.

I like building systems that connect research ideas with useful, reliable AI products.

Doodle avatar of Ritesh Thawkar

Research direction

What I investigate

4 active themes

My current interests center on multimodal models that improve from their own interaction traces: asking better questions, selecting reliable evidence, generating useful supervision, and evaluating the loop with grounded metrics.

Research collaboration on multimodal reasoning, self-evolving VLMs/MLLMs, unified understanding and generation models, and evaluation. Applied AI work around RAG systems, LLM agents, retrieval pipelines, and production ML backends.

  1. 01

    Self-evolving multimodal understanding and generation from unlabeled visual data

  2. 02

    Unified understanding and generation models

  3. 03

    LLM and VLM evaluation for reasoning, retrieval, grounding, and generation

  4. 04

    Applied AI systems using RAG, agents, dashboards, and domain assistants

Research output

Publications

Peer-reviewed work and preprints, ordered by release date.

ECCV 2026Conference

Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

Shravan Venkatraman, Ritesh Thawkar, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Salman Khan, Fahad Shahbaz Khan

Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi-role self-play and self-consistency reward schemes in existing self-evolving LMMs optimize answer agreement without ensuring the decoder attends to visual content, relying instead on statistical language priors to produce self consistent outputs. This leads to a persistent failure mode we term visual under-conditioning, where the decoder relies on language priors rather than the image during generation, manifesting as insufficient attention to visual tokens. As a result, current self-evolving LMMs struggle on vision--language understanding tasks such as image captioning and visual question answering. To address this, we propose VISE (Visual Invariance Self-Evolution), a purely unsupervised self-evolving framework that directly regularizes the model's visual conditioning policy through two complementary invariance-based rewards: a geometric invariance reward that enforces spatial consistency under known transformations, and a semantic invariance reward that penalizes evidence-agnostic generation by requiring the model to recognize the absence of evidence when predicted regions are perturbed. VISE operates within a single model without specialist roles, external reward models, or annotations, and is trained on raw unlabeled images. Experiments on 18 benchmarks demonstrate the efficacy of our approach. Using Qwen3-VL-2B as the base model, VISE achieves gains of $+16.85$ CIDEr on COCO and $+19.66$ CIDEr on TextCaps, reduces object hallucination by $5.0$ Chair-I points, and generalizes across four model families and scales. Our code and models are available at https://mbzuai-oryx.github.io/VISE

MICCAI 2026ConferenceSpotlight

How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning?

Wafa Al Ghallabi, Ritesh Thawkar, Sara Ghaboura, Omkar Thawakar, Numan Saeed, Dana Al Nuaimi, Ajnas Alkatheeri, Salman Khan, Fahad Shahbaz Khan

Magnetic Resonance Imaging (MRI) interpretation is fundamental to clinical decision-making, requiring radiologists to integrate multi-view anatomical planes across sequential timepoints while precisely localizing interval changes. However, existing vision-language benchmarks remain confined to single-timepoint, single-view interpretation, failing to capture the temporal-spatial reasoning essential to radiologic practice. We introduce the Time-Aware Multi-View MRI Benchmark, an evaluation framework unifying multi-view anatomical input, temporal reasoning across longitudinal scans, and structured localization guidance. The benchmark comprises 3,920 expert-verified question-answer pairs derived from 890 patients across over 3,200 longitudinal MRI timepoints, drawn from seven clinical cohorts covering glioblastoma, neurodegeneration, vestibular schwannoma, and brain metastases, in open-ended, multiple-choice, and binary formats, requiring models to identify anatomical regions of maximal change, characterize progression across sequences and views, and provide structured guidance specifying boundaries, imaging features, and confounders. Experiments across 16 vision-language models reveal moderate temporal alignment but systematic failure on change direction recognition and volumetric quantification, while multi-view inputs improve spatial localization yet degrade temporal reasoning in compact architectures. Our benchmark provides a systematic framework for evaluating progression tracking, interval change localization, and temporal ordering, which are essential for clinical deployment. Code, evaluation splits, and the dataset are available at: https://github.com/wafaAlghallabi/Time-Aware-MRI.

BMVC 2026ConferenceAccepted

Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar, Abdelrahman Shaker, Rao Muhammad Anwer

Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce \textbf{CVPD} (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of $+3.60$ on OCRBench, $+3.38$ on MMStar Fine-Grained Perception, and $+3.08$ on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.

arXiv 2026Preprint

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar, Abdelrahman Shaker, Fahad Khan, Hisham Cholakkal, Salman Khan, Rao Muhammad Anwer

Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a $+3.5\%$ absolute gain on MMMU and improves GenEval image generation performance from $82\%$ to $85\%$. Code and models are publicly released.

WACV 2026Conference

A Multi-Agent Diffusion Approach for MRI Anomaly Segmentation via Modality-Specific LoRA Specialization

Wafa Al Ghallabi, Muhammad Zaigham Zaheer, Ritesh Thawkar, Omkar Thawakar, Salman Khan, Fahad Shahbaz Khan

Unsupervised anomaly segmentation in multi-sequence MRI is a promising way to scale lesion screening, but existing reconstruction-based methods face three persistent issues: they fail to generalize across modalities, they depend on hand-crafted masking or paired translations, and they often require separate models with high inference cost. In this work, we take a stepwise approach to address these limitations. In the first stage, we fully finetune a diffusion model on healthy brain MRI slices pooled across T1, T2, and FLAIR, which produces anatomically consistent reconstructions. To further improve, we introduce a lightweight second stage where modality-specific LoRA adapters are trained on top of the pretrained diffusion backbone. A simple router automatically selects the right adapter for each input, effectively turning the system into a modality-aware, agentic-like framework. To further stabilize reconstructions, we incorporate a learnable latent-frequency mask that suppresses non-informative spectral components and preserves structural detail. This design allows the model to emphasize healthy anatomy while efficiently capturing modality-dependent contrasts. This two-stage strategy boosts Dice to 88% on BraTS2021 (FLAIR), achieving state-of-the-art performance. Experiments on BraTS2021, ISLES, and ATLAS datasets confirm that the approach consistently improves Dice and SSIM across all modalities, outperforming diffusion, masking, and cycle-based baselines, and offering a practical balance between accuracy and efficiency for clinical MRI anomaly detection. Source code is available at https://github.com/wafaalghallabi/mri-router.

Selected work

Built to be used

8 systems

Research systems, production agents, and open-source tools shaped around measurable use.

Background

Experience

3 roles

Jun 2026 - Jul 2026

Bengaluru, India

Research Intern

Microsoft Research India
  • Conducted a systematic evaluation of multimodal models through MS-VISTA, integrating two visual benchmarks and one reasoning-analysis study across perception, cognition, and spatial intelligence.
  • Developed reproducible VLM/MLLM evaluation pipelines covering model inference, answer extraction, experiment tracking, and comparative performance analysis.
  • Built a Flask and React research leaderboard with authenticated submissions, benchmark scoring, capability-level analysis, and auditable evaluation reports.

Sep 2024 - Present

Abu Dhabi, UAE

Research Assistant

Mohamed Bin Zayed University of Artificial Intelligence
  • Engineered visual-reasoning benchmarks for VLMs/MLLMs, evaluating sequential logic, grounding, and multimodal reasoning across research tasks.
  • Developed composed-video-retrieval methods using dense video descriptions and multimodal embeddings for fine-grained retrieval and temporal understanding.
  • Deployed document-retrieval pipelines and scalable LLM web agents for Lawa.AI using FastAPI, vector databases, and retrieval-augmented generation.
  • Optimized Django and PostgreSQL AI infrastructure for Nutrigenics.care, integrating image-aware LLM recommendations into personalized nutrition workflows.

Feb 2024 - May 2024

Remote

Framework Engineer Intern

Quantiphi Analytics
  • Engineered data-ingestion pipelines for small and medium business operations, supporting reliable multi-source operational tracking.
  • Automated business-insight extraction with ML-driven analytics workflows, converting operational data into decision-ready outputs.
  • Optimized enterprise data-processing workflows, reducing end-to-end processing time by 20%.

Education

2 programs

MSc Computer Vision

MBZUAI

Aug 2024 - Present

Abu Dhabi, UAE

GPA 3.60 / 4.00

BTech Computer Science & Engineering

Dr. Babasaheb Ambedkar Technological University

Feb 2021 - May 2024

Lonere, India

GPA 8.51 / 10.00

Technical practice

Tools I work with

7 areas
01/ Primary

AI Research & Evaluation

Methods for studying multimodal capabilities, limitations, and reasoning behavior.

VLMs / MLLMsMultimodal BenchmarkingBenchmark DesignCapability AnalysisError Analysis
02

LLM Systems

Retrieval-grounded and tool-using language model systems for applied AI workflows.

Agentic RAGTool-Using AgentsRetrievalVector Databases
03

Languages

Core implementation languages for model tooling and production systems.

PythonTypeScriptJavaScriptC
04

ML / Deep Learning

Model development and experimentation across vision and language tasks.

PyTorchTransformersTensorFlowScikit-learn
05

Research Engineering

Tools for repeatable experimentation, evaluation, documentation, and delivery.

Evaluation PipelinesExperiment TrackingLaTeXGitDocker
06

Web & Data

Application frameworks and data stores for research tools and AI products.

FastAPIFlaskDjangoReactPostgreSQLSQLite
07

Cloud

Cloud platforms and automation for deploying research and production systems.

AWSAzureGitHub Actions

Contact

Have something in mind?

For research collaboration, engineering work, or a useful technical conversation, send a note.