ECCV 2024 Papers — Page 8
European Conference on Computer Vision · 2387 papers
Energy-induced Explicit quantification for Multi-modality MRI fusion
Xiaoming Qi (Southeast University), Shuo Li (Case Western Reserve University)
ClassificationSegmentationMultimodalityBiomedical DataMagnetic Resonance Imaging
🎯 What it does: Propose a multi-modal MRI fusion method based on energy models, Energy-induced Explicit Propagation and Alignment (EPA), which explicitly quantifies and optimizes aggregation patterns under different diseases.
Enhanced Motion Forecasting with Visual Relation Reasoning
Sungjune Kim (Korea University), Sangpil Kim (Korea University)
Autonomous DrivingGraph Neural NetworkImagePoint Cloud
🎯 What it does: This paper proposes a visual relationship reasoning module called ViRR, aimed at improving motion prediction accuracy in autonomous driving scenarios by leveraging graph-structured visual semantic relationships.
Enhanced Sparsification via Stimulative Training
Shengji Tang (Fudan University), Tao Chen (Fudan University)
Computational EfficiencyKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: Enhance network sparsification through self-distillation training, proposing the STP framework to achieve one-stage multi-dimensional (depth + width) structured pruning. Before pruning, self-distillation is used to maintain the magnitude of pruned weights, enhancing the expressiveness of retained weights.
Enhancing Cross-Subject fMRI-to-Video Decoding with Global-Local Functional Alignment
Chong Li (Fudan University), Jianfeng Feng (Fudan University)
GenerationData SynthesisTransformerDiffusion modelContrastive LearningVideoBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a cross-subject fMRI-to-video decoding pipeline, which projects fMRI data from different subjects into a unified brain space through Global-Local Functional Alignment (GLFA), and combines a pre-trained fMRI-PTE Transformer encoder with a diffusion-based video generation model to achieve high-quality reconstruction from brain signals to dynamic videos.
Enhancing Diffusion Models with Text-Encoder Reinforcement Learning
Chaofeng Chen (Nanyang Technological University), Weisi Lin (Nanyang Technological University)
GenerationSupervised Fine-TuningReinforcement LearningDiffusion modelImageTextMultimodality
🎯 What it does: Fine-tune the text encoder using reinforcement learning combined with LoRA to enhance text-image alignment and visual quality in diffusion models, and seamlessly integrate with existing U-Net fine-tuning models;
Enhancing Optimization Robustness in 1-bit Neural Networks through Stochastic Sign Descent
NianHui Guo, Haojin Yang (Hasso Plattner Institut, University of Potsdam)
ClassificationRecognitionOptimizationImageTextBenchmark
🎯 What it does: This paper proposes the Diode optimizer specifically designed for binary neural networks (BNNs), achieving random sign descent updates without hidden weights by leveraging gradient sign information.
Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models
Claudio Rota (University of Milano-Bicocca), Joost van de Weijer (Universitat Autònoma de Barcelona)
Super ResolutionConvolutional Neural NetworkDiffusion modelOptical FlowVideo
🎯 What it does: This paper proposes the StableVSR method based on diffusion models for video super-resolution, aiming to enhance perceptual quality and ensure temporal consistency.
Enhancing Plausibility Evaluation for Generated Designs with Denoising Autoencoder
Jiajie Fan (BMW Group), Hao Wang (Leiden University)
GenerationAuto EncoderImageMesh
🎯 What it does: Proposed a structural feasibility assessment metric called FDD based on denoising autoencoders to evaluate the structural rationality of generated design images, and validated its superiority on multiple datasets.
Enhancing Recipe Retrieval with Foundation Models: A Data Augmentation Perspective
Fangzhou Song (University of Science and Technology of China), Shuo Wang (University of Science and Technology of China)
SegmentationRetrievalData-Centric LearningTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: Propose to use Llama2 and SAM for data augmentation, and insert a lightweight adapter and multi-level circular loss into CLIP to enhance cross-modal recipe retrieval performance.
Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
Yang Zhang (National University of Singapore), Kenji Kawaguchi (National University of Singapore)
GenerationVision Language ModelDiffusion modelImageText
🎯 What it does: This paper proposes a training-agnostic method that enables real-time modulation of cross-attention during the inference phase of diffusion models, aiming to enhance semantic fidelity in text-to-image synthesis.
Enhancing Source-Free Domain Adaptive Object Detection with Low-confidence Pseudo Label Distillation
Ilhoon Yoon (Yonsei University), Kwanghoon Sohn (Yonsei University)
Object DetectionDomain AdaptationKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: Propose a low-confidence pseudo-label distillation loss to help source-free domain adaptive object detection models identify overlooked hard-to-detect instances.
Enhancing Tampered Text Detection through Frequency Feature Fusion and Decomposition
Zhongxi Chen (Xiamen University), Rongrong Ji (Xiamen University)
SegmentationAnomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: Propose a document image tampering detection network FFDN based on frequency domain information fusion and decomposition.
Enhancing Tracking Robustness with Auxiliary Adversarial Defense Networks
Zhewei Wu (University of Electronic Science and Technology of China), Shijie Zhou (University of Electronic Science and Technology of China)
Object TrackingAdversarial AttackConvolutional Neural NetworkImageVideo
🎯 What it does: Proposes the Auxiliary Adversarial Defense Network (AADN) to preprocess input images during visual tracking, thereby enhancing the tracker's robustness against imperceptible perturbations.
Enhancing Vectorized Map Perception with Historical Rasterized Maps
Xiaoyu Zhang (Chinese University of Hong Kong), Ji Zhao (Huixi Technology)
Object DetectionObject TrackingAutonomous DrivingTransformerImageRetrieval-Augmented Generation
🎯 What it does: Propose the HRMapNet framework, which leverages low-cost historical raster maps to enhance online vectorized map perception.
Enriching Information and Preserving Semantic Consistency in Expanding Curvilinear Object Segmentation Datasets
Qin Lei (Chongqing University), Qizhu Dai (Chongqing University)
SegmentationData SynthesisVision Language ModelDiffusion modelImageTextBiomedical Data
🎯 What it does: This paper proposes a text feature-based semantic map generation and control network (SCP ControlNet) to expand the dataset for curve-shaped object segmentation, enhancing model performance by generating synthetic data with high information content that differs from the original distribution.
EntAugment: Entropy-Driven Adaptive Data Augmentation Framework for Image Classification
Suorong Yang (Nanjing University), Jian Zhao (Nanjing University)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: Proposed an adaptive data augmentation framework called EntAugment based on information entropy, which can dynamically adjust the augmentation intensity during training according to sample difficulty and model status, and introduced Entropy Regularization Loss (EntLoss) to enhance the model's classification confidence.
EpipolarGAN: Omnidirectional Image Synthesis with Explicit Camera Control
Christopher May (Purdue University), Daniel Aliaga (Purdue University)
GenerationGenerative Adversarial NetworkImage
🎯 What it does: Designed an Omnidirectional GAN capable of generating indoor scenes with free camera positioning without requiring depth or camera pose information, while enhancing multi-view consistency through camera reprojection and epipolar loss.
Equi-GSPR: Equivariant SE(3) Graph Network Model for Sparse Point Cloud Registration
Xueyang Kang (KU Leuven), Bing WANG
Pose EstimationAutonomous DrivingGraph Neural NetworkPoint CloudBenchmark
🎯 What it does: Proposes a sparse point cloud registration framework based on SE(3) equivariant graph neural networks, which aggregates point features using equivariant graph layers, computes similarity, and decodes relative transformations;
Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection
Deepti Hegde (Johns Hopkins University), Vishal M. Patel (Johns Hopkins University)
Object DetectionContrastive LearningPoint Cloud
🎯 What it does: The paper proposes a self-supervised pre-training framework for LiDAR point clouds that leverages spatial and temporal equivariance to enhance 3D object detection performance.
EraseDraw : Learning to Insert Objects by Erasing Them from Images
Alper Canberk (Columbia University), Carl Vondrick (Columbia University)
Image HarmonizationGenerationData SynthesisPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderImageTextMultimodality
🎯 What it does: Propose the EraseDraw system, which first automatically removes objects from images and generates corresponding descriptions, thereby constructing a large number of training samples consisting of source images, target images, and text prompts. Subsequently, high-quality object insertion is achieved using text-conditioned diffusion models.
Eta Inversion: Designing an Optimal Eta Function for Diffusion-based Real Image Editing
Wonjun Kang (FuriosaAI), Hyung Il Koo (FuriosaAI)
GenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelImageBenchmark
🎯 What it does: Propose a diffusion model inversion method called Eta Inversion, which injects real Gaussian noise into the reverse sampling process using a time- and region-variant η function, thereby achieving more flexible and high-quality text-to-image editing while preserving the structural integrity of the source image.
Evaluating Text-to-Visual Generation with Image-to-Text Generation
Zhiqiu Lin (Carnegie Mellon University), Deva Ramanan (Carnegie Mellon University)
GenerationTransformerSupervised Fine-TuningVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes a VQAScore metric based on a Visual Question Answering (VQA) model to evaluate the alignment between text and visual generation, and constructs the GenAI-Bench benchmark containing 1,600 synthetic text prompts.
Evaluating the Adversarial Robustness of Semantic Segmentation: Trying Harder Pays Off
Levente Halmosi (Szechenyi Istvan University), Márk Jelasity (Szechenyi Istvan University)
SegmentationAdversarial AttackImage
🎯 What it does: This paper proposes a new adversarial training method that simultaneously minimizes cross-entropy loss and a new prediction consistency loss during training, thereby enhancing model robustness.
Event Camera Data Dense Pre-training
Yan Yang (Australian National University), Liu liu
SegmentationDepth EstimationConvolutional Neural NetworkTransformerContrastive LearningOptical FlowVideo
🎯 What it does: Proposes a self-supervised framework that pretrains neural networks using event camera data, enabling them to perform well on dense prediction tasks such as segmentation, depth estimation, and optical flow estimation.
Event Trojan: Asynchronous Event-based Backdoor Attacks
Ruofei Wang (Hong Kong Baptist University), Renjie Wan (Hong Kong Baptist University)
Adversarial AttackImage
🎯 What it does: This paper proposes Event Trojan, a backdoor attack method targeting asynchronous event data, achieving stealthy control of models by injecting variable and immutable triggers into event streams.
Event-Adapted Video Super-Resolution
Zeyu Xiao (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
Super ResolutionConvolutional Neural NetworkSupervised Fine-TuningVideo
🎯 What it does: Propose EATER, a parameter-efficient fine-tuning based adapter that can migrate existing frame-level VSR models to event camera input VSR tasks without changing their original parameters, while significantly improving performance.
Event-Aided Time-To-Collision Estimation for Autonomous Driving
Jinghang Li (Hunan University), Yi Zhou (Hunan University)
Object DetectionObject TrackingAutonomous DrivingOptical FlowImageMultimodality
🎯 What it does: This paper proposes a Time-to-Collision (TTC) estimation method based on event cameras for collision warning of vehicles ahead in autonomous driving.
Event-based Head Pose Estimation: Benchmark and Method
Jiahui Yuan (University of Science and Technology of China), Xiaoyan Sun (University of Science and Technology of China)
Pose EstimationConvolutional Neural NetworkVideoBenchmark
🎯 What it does: This paper proposes an event camera-based head pose estimation (EV-HPE) framework, constructs two large-scale event head pose datasets (Prophesee-HP and Davis-HP), and implements and evaluates head pose estimation on these datasets.
Event-based Mosaicing Bundle Adjustment
Shuang Guo (TU Berlin and Robotics Institute Germany), Guillermo Gallego (TU Berlin and Robotics Institute Germany)
Pose EstimationOptimization
🎯 What it does: Designed a direct photometric bundle adjustment (EMBA) method based on event cameras, which directly optimizes the camera rotation trajectory and panoramic gradient maps using event streams, ultimately reconstructing high-quality grayscale panoramic images from gradients.
Event-Based Motion Magnification
Yutian Chen (Shanghai AI Laboratory), Tianfan Xue (Chinese University of Hong Kong)
RestorationSuper ResolutionRecurrent Neural NetworkMultimodalityBenchmark
🎯 What it does: Proposes a complete solution that utilizes an event camera and RGB camera dual-camera system to amplify low-amplitude high-frequency motion and perform high-frame-rate interpolation;
EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding
jiazhou zhou, Lin Wang (Hong Kong University of Science and Technology)
RecognitionRetrievalRepresentation LearningTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: This paper proposes the EventBind framework, which leverages the pre-trained CLIP model to achieve a unified multimodal representation of event camera data, completing event recognition and retrieval tasks.
Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization
Ming-Yang Ho (National Taiwan University), Yufeng Jane Tseng
Image TranslationConvolutional Neural NetworkGenerative Adversarial NetworkImageBiomedical Data
🎯 What it does: Propose the Dense Normalization (DN) layer, which achieves pixel-level statistics estimation to address seams and jitter artifacts in ultra-high-resolution unpaired image translation, while preserving local color and contrast.
EvSign: Sign Language Recognition and Translation with Streaming Events
Pengyu Zhang (Dalian University of Technology), Xu Jia (Dalian University of Technology)
RecognitionImage TranslationTransformerTime SeriesSequentialBenchmark
🎯 What it does: Explored the possibility of using event cameras for continuous sign language recognition and translation, collected the EvSign dataset, and proposed a Transformer framework.
Ex2Eg-MAE: A Framework for Adaptation of Exocentric Video Masked Autoencoders for Egocentric Social Role Understanding
Minh Tran (University of Southern California), Mohammad Soleymani (University of Southern California)
Data SynthesisDomain AdaptationKnowledge DistillationTransformerAuto EncoderContrastive LearningVideo
🎯 What it does: Proposes the Ex2Eg-MAE framework, which generates dynamic perspective synthetic first-person videos by performing 3D facial synthesis on outdoor videos, and recovers the original outdoor video using a masked autoencoder to achieve self-supervised learning;
Exact Diffusion Inversion via Bidirectional Integration Approximation
Guoqiang Zhang (University of Exeter), W. Bastiaan Kleijn (Victoria University of Wellington)
RestorationGenerationDiffusion modelImageOrdinary Differential Equation
🎯 What it does: Propose a novel bidirectional integral approximation (BDIA) method to achieve precise forward and reverse diffusion inversion in diffusion models (DDIM) without significantly increasing computational costs, addressing the state inconsistency issues caused by existing methods (e.g., EDICT).
Exemplar-free Continual Representation Learning via Learnable Drift Compensation
Alex Gomez-Villa (University of Science and Technology of China), Joost van de Weijer (Eindhoven University of Technology)
Knowledge DistillationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an adaptive incremental learning framework that combines pseudo labels with cross-entropy loss to achieve rapid learning on new tasks while maintaining knowledge from old tasks.
ExMatch: Self-guided Exploitation for Semi-Supervised Learning with Scarce Labeled Samples
Noo-ri Kim (Sungkyunkwan University), Jee-Hyong Lee (Sungkyunkwan University)
ClassificationDomain AdaptationRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Propose a self-guided semi-supervised learning method called ExMatch that leverages unlabeled samples, aiming to address confirmation bias when only a small number of labeled samples are available.
Expanding Scene Graph Boundaries: Fully Open-vocabulary Scene Graph Generation via Visual-Concept Alignment and Retention
Zuyao Chen (Hong Kong Polytechnic University), Chang Wen Chen (Hong Kong Polytechnic University)
RecognitionObject DetectionGenerationKnowledge DistillationTransformerVision Language ModelImageText
🎯 What it does: Proposed a fully open-vocabulary scene graph generation framework called OvSGTR, which can identify unseen objects and relations under four different open-vocabulary settings (closed-set, open objects, open relations, open objects+relations).
Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Concepts
Andong Tan (Hong Kong University of Science and Technology), Hao Chen (Hong Kong University of Science and Technology)
ClassificationExplainability and InterpretabilityConvolutional Neural NetworkVision Language ModelContrastive LearningImageText
🎯 What it does: Proposes a concept bottleneck model (OpenCBM) that can utilize arbitrary open-vocabulary concepts, enabling the free addition, deletion, or replacement of concepts after model training without requiring retraining.
Explicitly Guided Information Interaction Network for Cross-modal Point Cloud Completion
Xu Hang (Wuhan University), Bisheng Yang (Wuhan University)
GenerationTransformerVision Language ModelMultimodalityPoint Cloud
🎯 What it does: Proposes the EGIInet view-guided point cloud completion network, achieving cross-modal feature fusion through a unified encoder and explicit guidance information interaction.
Exploiting Dual-Correlation for Multi-frame Time-of-Flight Denoising
Guanting Dong (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
RestorationDepth EstimationConvolutional Neural NetworkImage
🎯 What it does: Propose a learning-based denoising framework called MTDNet based on multi-frame time-of-flight (ToF) depth maps, leveraging inter-frame correlations to efficiently suppress multi-path interference (MPI) and scattering noise.
Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models
Minchan Kim (Seoul National University), Buru Chang (Seoul National University)
GenerationReinforcement LearningDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose an unsupervised reinforcement learning framework named ESREAL to suppress hallucinations in vision-language models when generating long paragraphs.
Exploiting Supervised Poison Vulnerability to Strengthen Self-Supervised Defense
Jeremy Styborski (Nanyang Technological University), Adams Kong (Nanyang Technological University)
Representation LearningAdversarial AttackContrastive LearningImage
🎯 What it does: The study generates adversarial perturbations by exploiting the vulnerability of supervised learning models in availability poisoning scenarios, using them as an enhancement method to improve the defense effectiveness of self-supervised learning (SSL), proposing the VESPR method.
Explorative Inbetweening of Time and Space
Haiwen Feng (Max Planck Institute For Intelligent Systems), Xuaner Zhang (Max Planck Institute For Intelligent Systems)
GenerationDiffusion modelImage
🎯 What it does: Propose the 'bounded generation' task, utilizing start and end frames in potential diffusion models to achieve arbitrary motion interpolation between subjects and cameras, and introduce the Time Reversal Fusion sampling strategy to realize untrained boundary control.
Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentation
Tong Shao (Harbin Institute of Technology), Jingyong Su (Harbin Institute of Technology)
SegmentationTransformerVision Language ModelContrastive LearningImage
🎯 What it does: Propose CLIPtrase, a training-free method that restores the local semantic correlations of CLIP's visual features through self-correlation, and achieves open-vocabulary semantic segmentation via DBSCAN clustering and denoising.
Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling
Wonho Bae (University of British Columbia), Danica J. Sutherland (University of British Columbia)
ClassificationMeta LearningImage
🎯 What it does: Explores applying active learning to context set labeling in meta-learning, proposes a Gaussian Mixture Model (GMM)-based active sampling algorithm, and validates it experimentally across multiple meta-learning frameworks.
Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection
Ting Lei (Peking University), Yang Liu (Peking University)
Object DetectionTransformerPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: Proposed a zero-shot human-object interaction (HOI) detection framework called CMMP, which utilizes conditional multimodal prompts to separate visual and language tasks, enhancing the ability to recognize unseen interactions.
Exploring Guided Sampling of Conditional GANs
Yifei Zhang (Shanghai Jiao Tong Univeristy), Fan Cheng (Shanghai Jiao Tong Univeristy)
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: Propose a Guided Sampling method (GANdance) based on GAN latent space vector arithmetic, achieving improved conditional generation with no training or light training.
Exploring Phrase-Level Grounding with Text-to-Image Diffusion Model
Danni Yang (Xiamen University), Rongrong Ji (Xiamen University)
SegmentationGenerationTransformerPrompt EngineeringDiffusion modelImageText
🎯 What it does: Investigated the phrase-level visual understanding capability of text-to-image diffusion models beyond the sentence level, proposing the DiffPNG framework to achieve zero-shot Panoptic Narrative Grounding.
Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation
Xuelu Feng (University at Buffalo), Zixin Zhu (University at Buffalo)
SegmentationConvolutional Neural NetworkTransformerVision Language ModelDiffusion modelVideoTextMultimodality
🎯 What it does: Proposes a Referring Video Object Segmentation (R-VOS) framework named VD-IT based on pre-trained text-video diffusion models, which extracts visual features from a fixed diffusion model using text-guided image projection and video-specific noise prediction, and designs a multi-scale, Transformer-based mask decoder for video object segmentation.
Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation
Yuwen Pan (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
SegmentationTransformerImage
🎯 What it does: Proposed an end-to-end framework called NightFormer for nighttime semantic segmentation, which includes a pixel-level texture enhancement module and an object-level reliable matching module.
Exploring the Feature Extraction and Relation Modeling For Light-Weight Transformer Tracking
Jikai Zheng (Northwest A&F University), Jifeng Ning (Tencent AI-Lab)
Object TrackingConvolutional Neural NetworkTransformerVideo
🎯 What it does: This paper proposes a lightweight Transformer tracker called FERMT, which splits attention into two stages: feature extraction and relation modeling. It incorporates CNN-based dual attention units into a state-of-the-art architecture to enhance speed and accuracy.
Exploring Vulnerabilities in Spiking Neural Networks: Direct Adversarial Attacks on Raw Event Data
Yanmeng Yao (Nanjing University of Information Science and Technology), Bin Gu (Mohamed bin Zayed University of Artificial Intelligence)
Adversarial AttackSpiking Neural NetworkVideo
🎯 What it does: This paper proposes an adversarial attack method directly targeting raw event data (COO format), which can efficiently and controllably attack vision models based on spiking neural networks (SNNs) without relying on grid-based intermediate representations.
Expressive Whole-Body 3D Gaussian Avatar
Gyeongsik Moon (DGIST), Shunsuke Saito (Meta)
GenerationGaussian SplattingVideoMesh
🎯 What it does: Train ExAvatar using short-term monocular videos to generate an animatable full-body 3D Gaussian Avatar that supports free driving of facial expressions, hand movements, and body poses.
External Knowledge Enhanced 3D Scene Generation from Sketch
Zijie Wu (Hunan University), Ajmal Mian (University of Western Australia)
GenerationGraph Neural NetworkTransformerDiffusion modelImagePoint CloudMesh
🎯 What it does: Developed a 3D scene generation architecture SEK based on hand-drawn sketches combined with external relational knowledge, achieving diverse, customizable, and reasonable indoor scene generation and completion through the integration of diffusion models and graph attention.
F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions
Jie Yang (Chinese University of Hong Kong Shenzhen), Siyuan Huang (State Key Laboratory of General Artificial Intelligence)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextMultimodalityPoint CloudMesh
🎯 What it does: Propose a fine-grained semantic alignment 3D human-robot interaction task and dataset Semantic-HOI, and design a unified model F-HOI to accomplish understanding, reasoning, generation, and reconstruction tasks.
Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control
Yue Han (Zhejiang University), Yong Liu (Tencent)
GenerationTransformerVision Language ModelDiffusion modelImage
🎯 What it does: Proposed a lightweight Face-Adapter that can achieve facial reenactment and face swapping tasks on pre-trained diffusion models, supporting both tasks with a single model.
Face Reconstruction Transfer Attack as Out-of-Distribution Generalization
Yoon Gyo Jung (Northeastern University), Octavia Camps (Northeastern University)
Adversarial AttackGenerative Adversarial NetworkImageBenchmark
🎯 What it does: Aiming at 'face reconstruction transfer attack' (FRTA) in facial recognition systems, this paper proposes an attack method based on an implicit generator that can successfully reconstruct identity-matching synthetic faces on unseen encoders.
Faceptor: A Generalist Model for Face Perception
Lixiong Qin (Beijing University of Posts and Telecommunications), Weihong Deng (Beijing University of Posts and Telecommunications)
ClassificationRecognitionSegmentationPose EstimationTransformerSupervised Fine-TuningImage
🎯 What it does: Developed a unified general model called Faceptor that can simultaneously handle multiple facial analysis tasks
Facial Affective Behavior Analysis with Instruction Tuning
Yifan Li (Michigan State University), Yu Kong (Arizona State University)
ClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes a framework for instruction-based facial affect behavior analysis (FABA): ① Construct the FABA-Instruct dataset (approximately 19K real-world facial images, with fine-grained emotion and AU descriptions automatically generated by GPT-4V); ② Design the FABA-Bench benchmark, incorporating new evaluation metrics that combine recognition and text generation (REGE); ③ Develop the EmoLA model based on LLaVA-1.5, integrating a facial prior expert (InsightFace keypoint extraction) into the visual encoder and employing LoRA for efficient parameter fine-tuning, achieving both generation and recognition of FABA instructions.
Factorized Diffusion: Perceptual Illusions by Noise Decomposition
Daniel Geng (University of Michigan), Andrew Owens (University of Michigan)
GenerationPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Leveraging pre-trained text-conditioned diffusion models, by decomposing noise estimates during the reverse diffusion process, enabling different image components (such as frequency subbands, color, motion blur, etc.) to be separately controlled via distinct text prompts, thereby generating various perceptual illusion images (hybrid images, color hybrid images, motion hybrid images, inverse hybrid images)
Factorizing Text-to-Video Generation by Explicit Image Conditioning
Rohit Girdhar (GenAI), Ishan Misra (GenAI)
GenerationData SynthesisVision Language ModelDiffusion modelVideoText
🎯 What it does: Proposed Emu Video, a text-to-video model that first generates images corresponding to the text and then generates videos conditioned on both the image and text;
FAFA: Frequency-Aware Flow-Aided Self-Supervision for Underwater Object Pose Estimation
Jingyi Tang (Tsinghua University), Xiangyang Ji (Tsinghua University)
Pose EstimationOptical FlowImage
🎯 What it does: Proposed the FAFA two-stage self-supervised framework, achieving underwater 6D pose estimation through frequency domain enhancement and flow-assisted consistency.
FairDomain: Achieving Fairness in Cross-Domain Medical Image Segmentation and Classification
Yu Tian (Harvard University), Mengyu Wang (Harvard University)
ClassificationSegmentationDomain AdaptationTransformerBiomedical Data
🎯 What it does: This study systematically explores the issue of algorithmic fairness in the process of cross-domain migration (domain adaptation and domain generalization) in medical imaging, and proposes a pluggable Fair Identity Attention (FIA) module to enhance the fairness and performance of cross-domain models.
Fairness-aware Vision Transformer via Debiased Self-Attention
Yao Qiang (Wayne State University), Dongxiao Zhu (Wayne State University)
Safty and PrivacyAdversarial AttackTransformerImage
🎯 What it does: This paper proposes a Vision Transformer framework called DSA based on debiased attention, which locates and masks pseudo features by leveraging a biased model and adversarial attacks before training, and then aligns attention weights during training to achieve fair prediction.
FairViT: Fair Vision Transformer via Adaptive Masking
Bowei Tian (Wuhan University), Yanning Shen (University of California, Irvine)
ClassificationTransformerImage
🎯 What it does: Propose the FairViT framework, integrating adaptive masks and distance loss to balance accuracy and fairness within Vision Transformers.
Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection
Yuzhen Lin (Shenzhen University), Qiushi Li (Shenzhen University)
ClassificationAnomaly DetectionConvolutional Neural NetworkTransformerVideo
🎯 What it does: Proposed a deepfake detection framework named CDFA, which enhances the generalization capability of detection models on unseen datasets and unseen forgery methods by gradually introducing forged augmented samples (p-fake) during training and dynamically adjusting augmentation strategies.
FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performance
Jiedong Zhuang (Zhejiang University), Haoji Hu (Zhejiang University)
ClassificationRecognitionTransformerPrompt EngineeringVision Language ModelImagePoint Cloud
🎯 What it does: This paper proposes the FALIP method, which inserts retinal attention masks into the multi-head self-attention module of CLIP, achieving improved zero-shot performance without training during inference and without altering the image content.
FAMOUS: High-Fidelity Monocular 3D Human Digitization Using View Synthesis
Vishnu Mani Hema (Carnegie Mellon University), Fernando de la Torre (Carnegie Mellon University)
Image TranslationGenerationData SynthesisPose EstimationDomain AdaptationGenerative Adversarial NetworkImageMesh
🎯 What it does: Generate a complete human back view from a single RGB image using a perspective hallucinator, then predict geometry with an implicit function (PIFuHD) and combine front and back view textures to produce a high-fidelity fully textured 3D mesh.
Fast Context-Based Low-Light Image Enhancement via Neural Implicit Representations
Tomáš Chobola (Technical University of Munich), Tingying Peng (Technical University of Munich)
RestorationNeural Radiance FieldImage
🎯 What it does: Propose the CoLIE method, which utilizes neural implicit representations to map 2D coordinates to the illumination component of low-light images, recovers brightness within the HSV space, and achieves efficient processing through guided filtering.
Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation
Nina Weng (Technical University of Denmark), Siavash Arjomand Bigdeli (Technical University of Denmark)
Data SynthesisExplainability and InterpretabilityConvolutional Neural NetworkDiffusion modelImageBiomedical Data
🎯 What it does: Propose FastDiME, a fast method for generating contrast samples based on diffusion models, used to detect and quantify the model's dependence on 'shortcut' features.
Fast Encoding and Decoding for Implicit Video Representation
Hao Chen (ByteDance), Abhinav Shrivastava (University of Maryland)
CompressionConvolutional Neural NetworkTransformerVideo
🎯 What it does: Propose a fast implicit video encoding and decoding framework, NeRV-Enc and NeRV-Dec
Fast Point Cloud Geometry Compression with Context-based Residual Coding and INR-based Refinement
Hao Xu (McMaster University), Xiaolin Wu (McMaster University)
CompressionGraph Neural NetworkNeural Radiance FieldPoint Cloud
🎯 What it does: This paper proposes a point cloud geometry compression method (CRCIR) based on contextual residual encoding and implicit neural representation.
Fast Registration of Photorealistic Avatars for VR Facial Animation
Chaitanya Patel (Stanford University), Shih-En Wei (Meta Reality Labs)
Image TranslationPose EstimationDomain AdaptationConvolutional Neural NetworkTransformerImageVideoMesh
🎯 What it does: Propose a system combining style transfer and iterative refinement on VR headsets to quickly and accurately align infrared images from headset cameras with personalized light-rendered face models, enabling online estimation of facial expressions and head pose.
Fast Sprite Decomposition from Animated Graphics
Tomoyuki Suzuki (CyberAgent), Kota Yamaguchi (CyberAgent)
OptimizationComputational EfficiencyConvolutional Neural NetworkVideo
🎯 What it does: Propose an optimization-based fast animation graphic sprite decomposition method that can split raster animation videos into static textures and time-varying affine animation parameters, facilitating subsequent editing;
Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation
Shentong Mo (MBZUAI), Zhenguo Li (Huawei Noah's Ark Lab)
GenerationTransformerMixture of ExpertsDiffusion modelAuto EncoderPoint Cloud
🎯 What it does: Propose FastDiT-3D, which rapidly generates high-quality 3D point clouds with multi-class support by leveraging an extremely high proportion (≈99%) of foreground-background aware masking and an encoder-decoder architecture with global and window self-attention.
Fast View Synthesis of Casual Videos with Soup-of-Planes
Yao-Chih Lee (University of Maryland), Feng Liu (Adobe Research)
GenerationData SynthesisDepth EstimationComputational EfficiencyNeural Radiance FieldGaussian SplattingOptical FlowVideoPoint Cloud
🎯 What it does: Designed a hybrid explicit video representation that leverages view-aligned plane soup (soup-of-planes) to capture static scenes while employing per-frame point clouds to represent dynamic content, enabling efficient synthesis of novel time-varying views from monocular videos.
FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos
Florian Maximilian Langer (Qualcomm Technologies, Inc.), Mohsen Ghafoorian (Qualcomm Technologies, Inc.)
Pose EstimationRetrievalKnowledge DistillationConvolutional Neural NetworkTransformerContrastive LearningVideoPoint Cloud
🎯 What it does: FastCAD is a single-stage method capable of real-time simultaneous retrieval and alignment of all CAD models in a scene; it directly predicts the category, orientation, front face, and shape embeddings for each detected object, then retrieves the nearest neighbor CAD models in a pre-learned embedding space and aligns them according to the predicted orientation;
FastPCI: Motion-Structure Guided Fast Point Cloud Frame Interpolation
tianyu zhang, Jian Yang (Nankai University)
Autonomous DrivingComputational EfficiencyConvolutional Neural NetworkTransformerPoint Cloud
🎯 What it does: Proposed FastPCI, a pyramid convolution-Transformer architecture for fast and high-precision point cloud frame interpolation.
Feature Diversification and Adaptation for Federated Domain Generalization
Seunghan Yang (Qualcomm AI Research), Sungrack Yun (Qualcomm AI Research)
Domain AdaptationFederated LearningImage
🎯 What it does: In the federated learning scenario, feature diversity of local data is achieved through global feature statistics, and during the inference phase, instance feature adaptation is used to enhance generalization to unknown domains.
Federated Learning with Local Openset Noisy Labels
Zonglin Di (University of California, Santa Cruz), Yang Liu (University of California, Santa Cruz)
OptimizationFederated LearningSafty and PrivacyContrastive LearningImageBenchmark
🎯 What it does: This paper proposes a new federated learning framework, FedDPCont, to address the problem of local open-set noisy labels on each client.
FedHARM: Harmonizing Model Architectural Diversity in Federated Learning
Anestis Kastellos (Centre for Research and Technology Hellas), Petros Daras (Centre for Research and Technology Hellas)
ClassificationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: Designed the FedHARM framework to enable collaborative training of different CNN architectures (ResNet, EfficientNet, MobileNetV3) under the same federated learning scenario, achieving model-agnostic aggregation through gradient feature extraction and block-level feature alignment.
FedHide: Federated Learning by Hiding in the Neighbors
Hyunsin Park (Qualcomm Technologies Inc), Sungrack Yun (Qualcomm Technologies Inc)
Federated LearningSafty and PrivacyContrastive LearningImageAudio
🎯 What it does: This paper proposes a federated learning framework called FedHide for training embedding networks in scenarios where each client has data from only a single class, achieving privacy protection by generating proxy class prototypes.
FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clients
Shangchao Su (Fudan University), Xiangyang Xue (Fudan University)
Federated LearningTransformerSupervised Fine-TuningImage
🎯 What it does: Proposed FedRA, a random sublayer allocation strategy for heterogeneous client federated tuning, addressing the problem of base models being unable to be fully trained on resource-constrained devices.
FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learning
Boyu Fan, Pan HUI
Data SynthesisFederated LearningDiffusion modelImageBenchmark
🎯 What it does: Propose FedTSA, which utilizes resource-based clustering and two-phase aggregation (intra-cluster weight averaging + server-side deep mutual learning) to achieve model heterogeneity in federated learning.
FedVAD: Enhancing Federated Video Anomaly Detection with GPT-Driven Semantic Distillation
Fan Qi (Tianjin University of Technology), Changsheng Xu (Chinese Academy of Sciences)
Anomaly DetectionFederated LearningKnowledge DistillationTransformerLarge Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: Proposes FedVAD, a privacy-friendly video anomaly detection framework based on federated learning, integrating visual consistency clustering and GPT-driven semantic distillation.
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
Keen You (Apple), Zhe Gan (Apple)
ClassificationObject DetectionTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: Developed Ferret-UI, a multimodal large language model designed for mobile UI screen design, capable of performing fine-grained reference, localization, description, and interactive reasoning tasks;
Few-Shot Anomaly-Driven Generation for Anomaly Classification and Segmentation
Guan Gui (Tencent YouTu Lab), Yunsheng Wu
ClassificationSegmentationAnomaly DetectionDiffusion modelImageBenchmark
🎯 What it does: Propose a generation method driven by a small number of real anomaly samples (AnoGen), which learns embeddings in a pre-trained diffusion model and combines bounding box guidance to generate anomaly images with semantic consistency and spatial controllability. These generated images are then used to train a weakly supervised anomaly detection model, improving performance on anomaly classification and segmentation tasks.
Few-shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt
Chenxi Liu (University of Maryland), Heng Huang (University of Maryland)
ClassificationRepresentation LearningTransformerPrompt EngineeringImageBenchmark
🎯 What it does: Propose the Attention‑Aware Self‑Adaptive Prompt (ASP) framework, which utilizes a fixed pre-trained ViT and inserts task-invariant prompts (TIP) and adaptive task-specific prompts (TSP) between respective attention layers to achieve few-shot class incremental learning.
Few-shot Defect Image Generation based on Consistency Modeling
Qingfeng Shi (Chinese Academy of Sciences), Zhengtao Zhang (Chinese Academy of Sciences)
GenerationAnomaly DetectionTransformerDiffusion modelImage
🎯 What it does: For few-shot generation of industrial defect images, the DefectDiffu model is proposed, which can generate high-quality, diverse defect images under the condition of only a small number of defect samples and can automatically generate corresponding masks.
Few-Shot Image Generation by Conditional Relaxing Diffusion Inversion
Yu Cao (Queen Mary University of London), Shaogang Gong (Queen Mary University of London)
GenerationDomain AdaptationDiffusion modelScore-based ModelImage
🎯 What it does: Propose an untrained conditional relaxation diffusion inversion (CRDI) framework that achieves few-shot image generation by learning sample-specific guidance embeddings (SGE).
Few-shot NeRF by Adaptive Rendering Loss Regularization
Qingshan Xu (Nanyang Technological University), Hanwang Zhang (Nanyang Technological University)
GenerationNeural Radiance FieldImage
🎯 What it does: Improve NeRF under sparse views by proposing an adaptive rendering loss regularization method, achieving frequency relation alignment through two-stage rendering supervision and uncertainty learning to better learn global structures and local details.
Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments
Djamahl Etchegaray (University of Queensland), Yadan Luo (University of Queensland)
Object DetectionAutonomous DrivingVision Language ModelImagePoint Cloud
🎯 What it does: Propose an open-vocabulary 3D object detection method called Find n' Propagate tailored for urban environments, aiming to enhance the recall of novel targets and propagate via self-training to farther regions.
Finding a needle in a haystack: A Black-Box Approach to Invisible Watermark Detection
Minzhou Pan (Northeastern University), Xue Lin (Northeastern University)
Anomaly DetectionConvolutional Neural NetworkTransformerImage
🎯 What it does: In a black-box, unlabeled environment, the WaterMark Detector (WMD) method is proposed to automatically detect invisible watermarks within a given image dataset.
Finding Meaning in Points: Weakly Supervised Semantic Segmentation for Event Cameras
Hoonhee Cho (KAIST), Kuk-Jin Yoon (KAIST)
SegmentationAutonomous DrivingKnowledge DistillationConvolutional Neural NetworkContrastive LearningVideo
🎯 What it does: Proposed a weakly supervised semantic segmentation method for event cameras (EV-WSSE), achieving point-level annotation with only one labeled point per class (1-class-1-click), while enhancing segmentation performance under sparse annotations through bidirectional student learning, prototype contrastive learning, and cross-student prototype distillation.
Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentation
Seongsu Ha (Seoul National University), Joonseok Lee (Seoul National University)
SegmentationRetrievalVision Language ModelImageMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposed a mosaic data augmentation method called NeMo based on negative sample mining to improve the training of referential image segmentation models.
Finding Visual Task Vectors
Alberto Hojel (University Of California Berkeley), Amir Bar (University Of California Berkeley)
Image TranslationRestorationSegmentationComputational EfficiencyRepresentation LearningTransformerReinforcement LearningImage
🎯 What it does: Studied and discovered visual task vectors by replacing the activation of specific attention heads in the MAE-VQGAN model to guide the model to complete different visual tasks without requiring input-output examples.
Fine-grained Dynamic Network for Generic Event Boundary Detection
Ziwei Zheng (Xi'an Jiaotong University), Fan Li (Xi'an Jiaotong University)
SegmentationConvolutional Neural NetworkTransformerContrastive LearningVideo
🎯 What it does: Propose DyBDet dynamic network for generalized event boundary detection, which can adaptively allocate subnetworks and achieve precise boundary localization.
Fine-Grained Scene Graph Generation via Sample-Level Bias Prediction
Yansheng Li, Wenbin Wang
GenerationConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: Propose a fine-grained scene graph generation method based on sample-level bias prediction (SBP), which predicts sample-specific bias using joint features and global priors through a generative adversarial network (BGAN) to correct coarse-grained relationship predictions into fine-grained relationships.
FineMatch: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction
Hang Hua (University of Rochester), Jiebo Luo (Adobe Research)
Anomaly DetectionRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the FineMatch benchmark, focusing on fine-grained aspect mismatch detection and correction between images and text, constructing 49,906 human-annotated image-text pairs and defining a new ITM-IoU evaluation metric.