🚨Open Position: Visual Compositional Generation Research 🚨

We are excited to announce an open research position for a project under Dr. Rohban at the RIML Lab (Sharif University of Technology). The project focuses on improving text-to-image generation in diffusion-based models by addressing compositional challenges.

🔍 Project Description:

Large-scale diffusion-based models excel at text-to-image (T2I) synthesis, but still face issues like object missing and improper attribute binding. This project aims to study and resolve these compositional failures to improve the quality of T2I models.

Key Papers:
- T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional T2I Generation
- Attend-and-Excite: Attention-Based Semantic Guidance for T2I Diffusion Models
- If at First You Don’t Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection
- ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization

🎯 Requirements:

- Must: PyTorch, Deep Learning,
- Recommended: Transformers and Diffusion Models.
- Able to dedicate significant time to the project.

🗓 Important Dates:

- Application Deadline: 2024/10/12 (23:59 UTC+3:30)

📌 Apply here:
Application Form

For questions:
📧 [email protected]
💬 @amirkasaei

@RIMLLab
#research_application
#open_position

💠 Compositional Learning Journal Club

Join us this week for an in-depth discussion on Compositional Learning in the context of cutting-edge text-to-image generative models. We will explore recent breakthroughs and challenges, focusing on how these models handle compositional tasks and where improvements can be made.

✅ This Week's Presentation:

🔹 Title: A semiotic methodology for assessing the compositional effectiveness of generative text-to-image models

🔸 Presenter: Amir Kasaei

🌀 Abstract:
A new methodology for evaluating text-to-image generation models is being proposed, addressing limitations in current evaluation techniques. Existing methods, which use metrics such as fidelity and CLIPScore, often combine criteria like position, action, and photorealism in their assessments. This new approach adapts model analysis from visual semiotics, establishing distinct visual composition criteria. It highlights three key dimensions: plastic categories, multimodal translation, and enunciation, each with specific sub-criteria. The methodology is tested on Midjourney and DALL·E, providing a structured framework that can be used for future quantitative analyses of generated images.

📄 Paper: A semiotic methodology for assessing the compositional effectiveness of generative text-to-image models

Session Details:
- 📅 Date: Sunday
- 🕒 Time: 5:00 - 6:00 PM
- 🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

💠 Compositional Learning Journal Club

Join us this week for an in-depth discussion on Compositional Learning in the context of cutting-edge text-to-image generative models. We will explore recent breakthroughs and challenges, focusing on how these models handle compositional tasks and where improvements can be made.

✅ This Week's Presentation:

🔹 Title: Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback

🔸 Presenter: Amir Kasaei

🌀 Abstract:
Recent advancements in text-conditioned image generation, particularly through latent diffusion models, have achieved significant progress. However, as text complexity increases, these models often struggle to accurately capture the semantics of prompts, and existing tools like CLIP frequently fail to detect these misalignments.

This presentation introduces a Decompositional-Alignment-Score, which breaks down complex prompts into individual assertions and evaluates their alignment with generated images using a visual question answering (VQA) model. These scores are then combined to produce a final alignment score. Experimental results show this method aligns better with human judgments compared to traditional CLIP and BLIP scores. Moreover, it enables an iterative process that improves text-to-image alignment by 8.7% over previous methods.

This approach not only enhances evaluation but also provides actionable feedback for generating more accurate images from complex textual inputs.

📄 Paper: Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback

Session Details:
- 📅 Date: Sunday
- 🕒 Time: 2:00 - 3:00 PM
- 🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

Project Description:
This project is a collaborative effort between Dr. Rohban, Dr. Soleymani, and Dr. Asgari. Together, we aim to push the boundaries of language model evaluation for the Persian language. In this project, our primary goal is to benchmark and develop innovative methods for evaluating language models on the Persian language both robustly and comprehensively. Our approach will encompass both static and dynamic assessments to ensure thorough analysis. This initiative seeks to advance the field by addressing unique challenges posed by Persian language processing.

For more in-depth insights, please refer to the following papers:

"Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?"

Requirments:
Familiarity with LLM Concepts: Understanding the fundamentals and advancements in large language models.
Deep Learning Expertise: Practical knowledge and experience in deep learning techniques.
PyTorch Proficiency: Hands-on experience with the PyTorch framework is essential.
Commitment: Ability to dedicate significant time and maintain consistency throughout the project.

To apply for this position, please read the suggested papers and send your resume along with a brief summary of your research interests to [email protected]. We are eager to hear from motivated individuals who are passionate about advancing language model evaluation.

For any inquiries, feel free to reach out to us via the above email.

#open_position
#research_application

#اخبار_پژوهشی_آزمایشگاه

مقالات برتر چاپ شده از آغاز سال ۲۰۲۳ تحت نظارت آقای دکتر رهبان

Fake It Until You Make It: Towards Accurate Near-Distribution Novelty Detection
آقای حسین میرزایی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس ICLR

Lagrangian objective function leads to improved unforeseen attack generalization
آقای محمد عزیزملایری دانشجوی دکترا آزمایشگاه RIML - چاپ شده در ژورنال Machine Learning

Compositions and methods for treating proliferative diseases
US Patent App.

Zerograd: Costless conscious remedies for catastrophic overfitting in the fgsm adversarial training
خانم زینب گلگونی دانشجوی دکترا آزمایشگاه RIML - چاپ شده در ژورنال Intelligent Systems with Applications

A deep learning framework to scale linear facial measurements to actual size using horizontal visible iris diameter: a study on an Iranian population
آقای دکتر حسین محمدرحیمی محقق در آزمایشگاه RIML - چاپ شده در ژورنال Scientific Reports

Weakly-Supervised Drug Efficiency Estimation with Confidence Score: Application to COVID-19 Drug Discovery
خانم نهال میرزایی و آقای محمد ولی‌ثانیان دانشجویان کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس MICCAI

Forecasting influenza hemagglutinin mutations through the lens of anomaly detection
آقای محمدرضا صالحی دانشجوی اسبق آزمایشگاه RIML - چاپ شده در ژورنال Scientific Reports

Borderless azerbaijani processing: Linguistic resources and a transformer-based approach for azerbaijani transliteration
خانم ریحانه زهرابی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس ACL - تحت نظر دکتر بیگی، دکتر عسگری و دکتر رهبان

Examination of lemon bruising using different CNN-based classifiers and local spectral-spatial hyperspectral imaging
آقای دکتر سجاد سبزی پسادکترا آزمایشگاه RIML - چاپ شده در ژورنال Algorithms

A Robust Heterogeneous Offloading Setup Using Adversarial Training
آقای مهدی امیری دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در ژورنال IEEE Transactions on Mobile Computing - تحت نظر دکتر رهبان و دکتر حسابی

Universal Novelty Detection Through Adaptive Contrastive Learning
آقای حسین میرزایی و آقای مجتبی نافذ دانشجویان کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس CVPR

Killing It With Zero-Shot: Adversarially Robust Novelty Detection
آقای حسین میرزایی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس IEEE ICASSP

User Voices, Platform Choices: Social Media Policy Puzzle with Decentralization Salt
جمعی از دانشجویان کارشناسی - چاپ شده در کنفرانس CHI

Comparison of 2D and 3D convolutional neural networks in hyperspectral image analysis of fruits applied to orange bruise detection
آقای دکتر سجاد سبزی پسادکترا و خانم ریحانه زهرابی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در Journal of Food Science

RODEO: Robust Outlier Detection via Exposing Adaptive Outliers
آقای حسین میرزایی و آقای مجتبی نافذ دانشجویان کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس ICML

Virtual screening for small-molecule pathway regulators by image-profile matching
آقای دکتر رهبان و خانم دکتر Anne E. Carpenter - چاپ شده در ژورنال Cell systems

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
Coming Soon! :)

و ده‌ها مقاله دیگر که در Google Scholar دکتر رهبان می‌توانید مشاهده کنید.

تبریک خدمت تمامی اعضای آزمایشگاه به دلیل تلاش‌‌، کوشش و پژوهش در جهت رفع مشکلات جامعه و کشور و چاپ مقالات در برترین کنفرانس‌ها و مجلات AI

Adversarial robust learning and its generalization issues

This is a research project in the group of Dr. Rohban (RIML lab) from Sharif University of Technology

Project description:
Despite deep neural networks impressive success in many real-world problems, their instability under test-time adversarial noises is the major issue against their use in safety-critical applications. Therefore, the problem of learning robust deep networks (not only accurate on original samples, but also accurate on adversarially perturbed ones) has become an active area of research.
Training the model based on the adversarial samples in each mini-batch, which is known as “Adversarial training” (AT), has been empirically established as a general and effective approach to remedy this issue. However, real challenges in practice and also theoretical aspects have remained. Especially, we face some critical generalization issues in this new learning paradigm including the larger generalization gap between test and train data in comparison with standard training or the specific phenomenon called catastrophic overfitting. Achieving a better understanding of this topic can be a good help to provide more robust models.

In this project, we aim to analyze generalization in robust learning in a more comprehensive, deep, and detailed way. The project has both theoretical and practical aspects; So having interest, capability, and perseverance in both aspects is needed.

Estimated time for the project is 6 months although it may change depending on the progress and results of the project.

For more information, you can read the following paper:
Zerograd: Costless conscious remedies for catastrophic overfitting in the FGSM adversarial training

Requirements:
- Familiarity with linear algebra fundamentals
- Familiarity with statistics and probability
- Familiarity with ML and deep learning fundamentals
- Hands-on experience in ML and deep learning
- Hands-on experience with PyTorch framework
- Dedicating considerable time and consistency to the project 
- Enthusiasm to learn and tackle research problems

Preferred qualifications:
* Familiarity with Jax framework 
Familiarity with adversarial robustness

To apply for the position, please read the suggested paper and send your resume as well as your research interests to [email protected]

We would be happy to answer any questions you may have through the above email.

#open_position
#research_application

ویدئوهای درس پردازش هوشمند تصاویر زیست-پزشکی، دکتر رهبان، پائیز ۱۴۰۲:
https://www.aparat.com/playlist/7606670

ویدئوهای درس یادگیری تقویتی، دکتر رهبان، بهار ۱۴۰۳:
https://www.aparat.com/playlist/9319081