ECCV 2024 Papers — Page 22
European Conference on Computer Vision · 2387 papers
Text Motion Translator: A Bi-Directional Model for Enhanced 3D Human Motion Generation from Open-Vocabulary Descriptions
Yijun Qian (Carnegie Melon University), Jungdam Won (Seoul National University)
GenerationPose EstimationTransformerLarge Language ModelTextMultimodality
🎯 What it does: Construct a large-scale 3D action dataset LaViMo and propose the TMT bidirectional text-motion translation model, achieving generation of 3D actions from natural language and text from actions without relying on action duration.
Text to Layer-wise 3D Clothed Human Generation
Junting Dong (Shanghai AI Laboratory), Bo Dai (Shanghai AI Laboratory)
GenerationVision Language ModelDiffusion modelScore-based ModelNeural Radiance FieldTextMesh
🎯 What it does: TELA achieves layer-by-layer generation of a 3D clothed human model through text descriptions, generating a nude body first and then sequentially generating each piece of clothing, enabling independent control of the human body and clothing.
Text-Anchored Score Composition: Tackling Condition Misalignment in Text-to-Image Diffusion Models
Luozhou Wang (Hong Kong University of Science and Technology), Yingcong Chen (Adobe Research)
GenerationDiffusion modelScore-based ModelImageTextMultimodality
🎯 What it does: Studied the control generation problem when text and multiple control conditions are not fully aligned in diffusion models, proposing a training-free text-anchored score combination (TASC) method.
Text-Conditioned Resampler For Long Form Video Understanding
Bruno Korbar (University of Oxford), Federico Tombari (Google)
RecognitionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningVideoText
🎯 What it does: This paper proposes the Text-Conditioned Resampler (TCR), a lightweight visual-language adaptation module that can select and compress the most relevant visual features from long videos based on given text task instructions, then input them as fixed-length queries into large language models (LLMs) for text generation.
Text-Guided Video Masked Autoencoder
David Fan (Amazon Prime Video), Xinyu Li (Amazon Prime Video)
RecognitionRetrievalTransformerVision Language ModelAuto EncoderContrastive LearningVideoText
🎯 What it does: Proposes a text-guided masking strategy (Text-Guided Masking, TGM) for video autoencoders (MAE), and jointly trains MAE with video-text contrastive learning to enhance downstream recognition performance after unsupervised pre-training.
Text-to-Sticker: Style Tailoring Latent Diffusion Models for Human Expression
Animesh Sinha (GenAI), Dhruv Mahajan (GenAI)
GenerationData SynthesisSupervised Fine-TuningPrompt EngineeringVision Language ModelDiffusion modelImageText
🎯 What it does: For the sticker generation task, multi-stage fine-tuning is applied to a pre-trained Latent Diffusion model, ultimately proposing the Style Tailoring method.
Text2LiDAR: Text-guided LiDAR Point Clouds Generation via Equirectangular Transformer
Yang Wu, Jian Yang (Nanjing University of Science and Technology)
GenerationData SynthesisAutonomous DrivingTransformerDiffusion modelTextPoint Cloud
🎯 What it does: Proposed Text2LiDAR, a text-controlled LiDAR point cloud generation framework capable of converting natural language descriptions into high-quality, controllable 360° LiDAR data.
Text2Place: Affordance-aware Text Guided Human Placement
Rishubh Parihar (Indian Institute of Science), Venkatesh Babu RADHAKRISHNAN (Indian Institute of Science)
GenerationVision Language ModelDiffusion modelScore-based ModelImageText
🎯 What it does: This paper proposes a two-stage method to achieve realistic human placement through text-guided semantic masks and subject-oriented inpainting.
TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering
Jingye Chen (HKUST), Furu Wei (Microsoft Research)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringDiffusion modelImageTextBenchmark
🎯 What it does: Achieve the automatic generation of high-quality multi-line text images from natural language prompts through two-phase training, utilizing large language models (LLM) for text layout planning and text layout encoding.
Textual Grounding for Open-vocabulary Visual Information Extraction in Layout-diversified Documents
Mengjun Cheng (Peking University), Jie Chen (Baidu Inc.)
RecognitionTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose an open-vocabulary visual information extraction (VIE) method based on text localization, employing an end-to-end OCR-free text-region alignment framework combined with layout-aware cross-modal encoding and two-stage pre-training.
Textual Knowledge Matters: Cross-Modality Co-Teaching for Generalized Visual Class Discovery
Haiyang Zheng (University of Trento), Zhun Zhong (University of Trento)
ClassificationRecognitionLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose a two-stage framework called TextGCD, which first constructs image captions through retrieval-based text generation, and then improves the accuracy of general category discovery by utilizing cross-modal co-teaching and alignment.
Textual Query-Driven Mask Transformer for Domain Generalized Segmentation
Byeonghyun Pak (Agency for Defense Development), Hoseong Kim (Agency for Defense Development)
SegmentationDomain AdaptationAutonomous DrivingTransformerVision Language ModelImageText
🎯 What it does: This paper proposes a domain-generalized semantic segmentation method called Textual Query-Driven Mask Transformer (tqdm), which utilizes visual language model (VLM) text embeddings as object queries.
Textual-Visual Logic Challenge: Understanding and Reasoning in Text-to-Image Generation
Peixi Xiong (Intel Labs), Nilesh Jain (Intel Labs)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelVision Language ModelGenerative Adversarial NetworkImageTextMultimodality
🎯 What it does: Proposed a logic-rich text-to-image generation task and constructed the TV-Logic dataset along with the baseline UnR-GAN model.
Texture-GS: Disentangle the Geometry and Texture for 3D Gaussian Splatting Editing
Tianxing Xu, Song-Hai Zhang (Tsinghua University)
GenerationGaussian SplattingImagePoint Cloud
🎯 What it does: Developed a Texture-GS method that decouples 3D Gaussian geometry from 2D texture using UV-MLP, enabling editable textures (e.g., texture replacement) while maintaining real-time rendering performance of 3D Gaussian Splatting.
TF-FAS: Twofold-Element Fine-Grained Semantic Guidance for Generalizable Face Anti-Spoofing
Xudong Wang (Xiamen University), Rongrong Ji (Xiamen University)
Anomaly DetectionTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a multimodal framework named TF-FAS for generalized face anti-spoofing, leveraging two fine-grained semantic guidance mechanisms (content and category) to enhance the model's cross-domain generalization capability.
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World
Weiyun Wang (Fudan University), Jifeng Dai (OpenGVLab, Shanghai AI Laboratory)
RecognitionObject DetectionGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the Relation Conversation (ReC) task, constructs a high-quality dataset AS-V2, and designs an evaluation benchmark CRPE. Based on these resources, the All-Seeing Model v2 (ASMv2) was trained, achieving significant improvements in image relation understanding, scene graph generation, and general vision-language tasks.
The Devil is in the Statistics: Mitigating and Exploiting Statistics Difference for Generalizable Semi-supervised Medical Image Segmentation
Muyang Qiu (Nanjing University), Yang Gao (Nanjing University)
SegmentationDomain AdaptationConvolutional Neural NetworkBiomedical DataMagnetic Resonance Imaging
🎯 What it does: Propose a semi-supervised domain generalization framework for medical image segmentation, which utilizes statistical individual branches (SIBs) to obtain reliable pseudo-labels, learns domain-invariant features through statistical aggregation branches (SAB), and simulates unknown domains by introducing multi-level perturbations at both image and feature levels.
The Fabrication of Reality and Fantasy: Scene Generation with LLM-Assisted Prompt Interpretation
Yi Yao (National Yang Ming Chiao Tung University), Wen-Huang Cheng (National Taiwan University)
GenerationData SynthesisLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextBenchmark
🎯 What it does: Proposed the RFBench benchmark and developed a training-free RFNet, combining LLM-guided diffusion models to achieve generation of real and fantasy scenes.
The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?
Qinyu Zhao (Australian National University), Stephen Gould (Australian National University)
Safty and PrivacyTransformerSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: By performing linear probing on the logit distribution of the first token from large vision-language models (LVLMs), the method determines whether the model should answer a question, thereby identifying unanswerable visual questions, jailbreak attacks, and deceptive questions. During the generation process, a decoding strategy based on the probing results is adopted to enhance the safety and reliability of the generated content.
The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-Explainable Model with Counterfactual Explanations
Anselm Haselhoff (Ruhr West University of Applied Sciences), Jonas Schneider (e:fs TechHub GmbH)
GenerationExplainability and InterpretabilityAuto EncoderImage
🎯 What it does: Propose a Gaussian Discriminant Variational Autoencoder (GdVAE) based on Conditional Variational Autoencoder (CVAE), which combines self-explainable and counterfactual explanation capabilities, enabling the generation of realistic, approximate, and consistent adversarial examples directly in the latent space.
The Hard Positive Truth about Vision-Language Compositionality
Amita Kamath (University of Washington), Ranjay Krishna (University of Washington)
RetrievalRepresentation LearningTransformerSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper constructs an evaluation dataset and training set containing hard positive samples to investigate the compositional reasoning ability of vision-language models for image description.
The Lottery Ticket Hypothesis in Denoising: Towards Semantic-Driven Initialization
Jiafeng Mao (CyberAgent), Kiyoharu Aizawa (University of Tokyo)
GenerationData SynthesisConvolutional Neural NetworkPrompt EngineeringDiffusion modelImageText
🎯 What it does: Proposes the concept of 'lucky tickets' in the denoising process of diffusion models, referring to pixel blocks in random noise images that can naturally denoise into specific concepts, and constructs a semantics-driven initial image based on this.
The Nerfect Match: Exploring NeRF Features for Visual Localization
Qunjie Zhou (NVIDIA), Laura Leal-Taixé (NVIDIA)
Pose EstimationRetrievalConvolutional Neural NetworkTransformerNeural Radiance FieldImage
🎯 What it does: Utilize the internal features of pre-trained NeRF for 2D-3D matching, propose the NeRFMatch matching network, and incorporate pose refinement into the structured localization pipeline to accomplish the visual localization task.
The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
Seungwoo Son (Pohang University of Science and Technology), Jaeho Lee (Pohang University of Science and Technology)
ClassificationKnowledge DistillationTransformerContrastive LearningImageAudio
🎯 What it does: This paper proposes a knowledge distillation framework (MaskedKD) that masks the teacher ViT input based on the student's attention weights, significantly reducing the teacher's supervision cost.
The Sky's the Limit: Relightable Outdoor Scenes via a Sky-pixel Constrained Illumination Prior and Outside-In Visibility
James A D Gardner, William Smith
Image TranslationGenerationNeural Radiance FieldImage
🎯 What it does: This paper proposes the NeuSky method, which utilizes sky pixel constraints and an outward-inward visibility network to achieve inverse rendering for outdoor scenes, decoupling geometry, albedo, distant lighting, and sky visibility.
Thermal3D-GS: Physics-induced 3D Gaussians for Thermal Infrared Novel-view Synthesis
Qian Chen (Beihang University), Xiangzhi Bai (Beihang University)
GenerationData SynthesisGaussian SplattingImageBenchmarkPhysics Related
🎯 What it does: Proposed a physics-driven 3D Gaussian jitter method called Thermal3D-GS for novel view synthesis in thermal infrared images.
Think before Placement: Common Sense Enhanced Transformer for Object Placement
Yaxuan Qin (Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences), Xilin Chen (Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences)
GenerationTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodalityChain-of-Thought
🎯 What it does: Propose a 'think before placing' framework named CSENet, which generates location descriptions using a large multimodal model, then predicts the scale and coordinates of foreground objects to achieve object placement that is more semantically and visually consistent.
Think2Drive: Efficient Reinforcement Learning by Thinking with Latent World Model for Autonomous Driving (in CARLA-v2)
Qifeng Li (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Autonomous DrivingReinforcement LearningWorld Model
🎯 What it does: This paper proposes a model-based reinforcement learning method called Think2Drive, which realizes autonomous driving planning using a neural planner and a latent world model;
Thinking Outside the BBox: Unconstrained Generative Object Compositing
Gemma Canet Tarrés (University of Surrey), Soo Ye Kim (Adobe Research)
GenerationDiffusion modelImage
🎯 What it does: Propose a mask-free generative object synthesis model that can automatically place and synthesize objects in images.
This Probably Looks Exactly Like That: An Invertible Prototypical Network
Zachariah Carmichael (University of Notre Dame), Walter Scheirer
ClassificationGenerationFlow-based ModelImage
🎯 What it does: Propose ProtoFlow, a reversible prototype network that combines conceptual neural networks with flow models to achieve joint generation and prediction, where prototypes represent latent space distributions and can be directly mapped back to the data space for visualization.
Three Things We Need to Know About Transferring Stable Diffusion to Visual Dense Prediciton Tasks
Manyuan Zhang (MMLAB Chinese University of Hong Kong), Hongsheng Li (SenseTime Research)
SegmentationDepth EstimationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Directly using the UNet of Stable Diffusion as a feature encoder for visual dense prediction tasks such as semantic segmentation and depth estimation, and verifying its transferability through multiple experiments.
TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models
Aditya Chinchure (University of British Columbia), Matthew Turk (Toyota Technological Institute at Chicago)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Proposes the TIBET framework to automatically identify and quantify social and incidental biases in text-to-image generation models (e.g., Stable Diffusion) under arbitrary input prompts;
Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers
Ekaterina Grishina (HSE University), Maxim Rakhuba
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: Studied the upper bound of the spectral norm of the Jacobian in convolutional layers, proposed a new upper bound, and proved that this upper bound can be efficiently and differentiably computed during training.
Time-Efficient and Identity-Consistent Virtual Try-On Using A Variant of Altered Diffusion Models
Phuong Hoang Dam (Korea Advanced Institute of Science and Technology), Daeyoung Kim (Korea Advanced Institute of Science and Technology)
Image TranslationImage HarmonizationVision Language ModelDiffusion modelFlow-based ModelImage
🎯 What it does: Propose a time-efficient and identity-consistent virtual try-on system, FIP-VITON, which achieves fast generation using a single-step improved diffusion model while preserving garment textures and user identity.
TimeCraft: Navigate Weakly-Supervised Temporal Grounded Video Question Answering via Bi-directional Reasoning
Huabin Liu (Shanghai Jiao Tong University), Weiyao Lin (Lenovo Research)
TransformerLarge Language ModelGaussian SplattingVideoMultimodality
🎯 What it does: Propose the TimeCraft framework, which accomplishes video question answering and corresponding temporal localization under weakly supervised settings through bidirectional reasoning.
TimeLens-XL: Real-time Event-based Video Frame Interpolation with Large Motion
Shi Guo, Yongrui Ma (Chinese University of Hong Kong)
RestorationFlow-based ModelOptical FlowVideoMultimodality
🎯 What it does: Propose a real-time video frame interpolation method based on event cameras, TimeLens-XL, which achieves 27FPS frame interpolation at 1280×720 resolution in large motion scenarios.
Timestep-Aware Correction for Quantized Diffusion Models
Yuzhe Yao (Xi'an Jiaotong University), Jingdong Wang (Baidu Inc)
GenerationComputational EfficiencyDiffusion modelImage
🎯 What it does: Investigated a time-step aware correction method for dynamically correcting quantization errors during low-precision quantization diffusion model inference, reducing error accumulation and improving image quality.
Tiny Models are the Computational Saver for Large Models
Qingyuan Wang (University College Dublin), Deepu John (University College Dublin)
Computational EfficiencyKnowledge DistillationMixture of ExpertsImage
🎯 What it does: Propose TinySaver, a framework that dynamically compresses large models by using a pre-trained mini model for early exit before inference;
TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data
Siyi Du (Imperial College London), Chen Qin (Imperial College London)
ClassificationData-Centric LearningTransformerAuto EncoderContrastive LearningImageMultimodalityTabularBiomedical Data
🎯 What it does: Proposed the TIP framework, achieving multi-modal pre-training and downstream classification on tabular and image data with missing values;
TLControl: Trajectory and Language Control for Human Motion Synthesis
Weilin Wan (University of Hong Kong), Lingjie Liu (University of Pennsylvania)
GenerationData SynthesisTransformerAuto EncoderTextMultimodalitySequential
🎯 What it does: This paper proposes a human motion synthesis framework called TLControl that simultaneously supports language descriptions and trajectory constraints, capable of generating complete and constraint-compliant high-quality motion sequences based on user-provided partial trajectories and text prompts.
To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now
Yimeng Zhang (Michigan State University), Sijia Liu (Michigan State University)
GenerationAdversarial AttackPrompt EngineeringDiffusion modelImageText
🎯 What it does: Proposed a method called UnlearnDiffAtk for generating adversarial prompts based on the internal 'free' classifier of diffusion models, aimed at evaluating the robustness of safety-driven unlearned diffusion models (unlearned DMs).
To Supervise or Not to Supervise: Understanding and Addressing the Key Challenges of Point Cloud Transfer Learning
Souhail Hadgi, Maks Ovsjanikov (Lix Ecole Polytechnique)
ClassificationSegmentationDomain AdaptationConvolutional Neural NetworkSupervised Fine-TuningContrastive LearningPoint Cloud
🎯 What it does: This paper systematically evaluates two mainstream strategies in point cloud transfer learning: supervised pre-training and contrastive learning (PointContrast). It compares the performance of different 3D network architectures (PointNet, PointMLP, DGCNN, MinkowskiNet, PCT) in linear probing and full fine-tuning across multiple downstream tasks (ModelNet40, ScanObjectNN, S3DIS), and proposes a simple regularization method based on early-layer geometric features, significantly enhancing the transferability of supervised pre-training.
TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenes
Bu Jin (Chinese Academy of Sciences), Hao Zhao (Li Auto)
Object DetectionAutonomous DrivingTransformerLarge Language ModelVision Language ModelImageTextMultimodalityPoint Cloud
🎯 What it does: Proposed the 3D dense description task for outdoor scenes and designed an end-to-end TOD Cap 3 network to output 3D object boxes and natural language descriptions under LiDAR point cloud and panoramic RGB image inputs.
Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning
Shibo Jie (Peking University), Yunhe Wang (Huawei)
ClassificationComputational EfficiencyKnowledge DistillationTransformerImage
🎯 What it does: Propose the Token Compensator (ToCom) module, which dynamically compensates for the token compression ratio differences between training and inference stages of Vision Transformer (ViT) using a lightweight LoRA plugin during inference, achieving acceleration without training or reparameterization.
Tokenize Anything via Prompting
Ting Pan, Shiguang Shan
RecognitionSegmentationGenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A unified, promptable vision foundation model named TAP was constructed, capable of simultaneously performing segmentation, identification, and generating descriptions for any region.
Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture
Xuanchen Li (Shanghai Jiao Tong University), Yichao Yan (Xueshen AI)
GenerationGaussian SplattingSimultaneous Localization and MappingVideoMesh
🎯 What it does: Proposes the Topo4D framework to automatically generate 4D facial meshes with fixed topology and 8K textures from multi-view videos.
Topology-Preserving Downsampling of Binary Images
Chia-Chia Chen (National Yang Ming Chiao Tung University), Chi-Han Peng (National Yang Ming Chiao Tung University)
OptimizationImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Proposed a discrete optimization-based method for binary image downsampling, which ensures the downsampled image retains the same topological features as the original (zeroth and first Betti numbers) through integer programming, while maximizing similarity with the original image (IoU/Dice).
Toward INT4 Fixed-Point Training via Exploring Quantization Error for Gradients
Dohyung Kim, Bumsub Ham (Yonsei University)
ClassificationObject DetectionSuper ResolutionImage
🎯 What it does: This paper analyzes the gradient quantization error in low-bit fixed-point training and proposes an adaptive method to update the quantization range to reduce the quantization error of large gradients.
Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learning
Yan Li, Wenxian Yu (Tongji University)
Object DetectionKnowledge DistillationConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningImage
🎯 What it does: Propose CastDet, a student-teacher self-learning framework based on the pre-trained RemoteCLIP model, aiming to achieve open-vocabulary object detection (OVD) from a drone perspective, capable of identifying target categories outside the training set without additional annotated data.
Toward Tiny and High-quality Facial Makeup with Data Amplify Learning
Qiaoqiao Jin (Shanghai Jiao Tong University), Bingbing Ni (Shanghai Jiao Tong University)
Image TranslationConvolutional Neural NetworkSupervised Fine-TuningDiffusion modelImage
🎯 What it does: Using a small number of annotated makeup images, generate a large amount of pseudo-paired data through diffusion models, and train the extremely compact TinyBeauty model on this data to achieve high-quality facial makeup.
Towards a Density Preserving Objective Function for Learning on Point Sets
Haritha Jayasinghe (University of Cambridge), Ioannis Brilakis (University of Cambridge)
OptimizationRepresentation LearningPoint Cloud
🎯 What it does: Propose a new point cloud distance metric called UniformChamferDistance (UniformCD), which incorporates the local density ratio into the correspondence search of Chamfer Distance to suppress clustering phenomena during point cloud training.
Towards Adaptive Pseudo-label Learning for Semi-Supervised Temporal Action Localization
Feixiang Zhou (Lancaster University), Hossein Rahmani (Lancaster University)
Object DetectionTransformerContrastive LearningVideo
🎯 What it does: This paper proposes a semi-supervised temporal action localization framework named APL, aiming to improve model performance by adaptively selecting high-quality pseudo labels.
Towards Architecture-Agnostic Untrained Networks Priors for Image Reconstruction with Frequency Regularization
Yilin Liu (University of North Carolina at Chapel Hill), Pew-Thian Yap (University of North Carolina at Chapel Hill)
RestorationConvolutional Neural NetworkBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes three frequency regularization methods (bandwidth-limited input, controllable upsampler, and learnable Lipschitz regularization), which enhance the performance of untrained networks (e.g., DIP) in medical image reconstruction by directly adjusting the network's frequency bias, eliminating dependency on specific architectures.
Towards Certifiably Robust Face Recognition
Seunghun Paik (Hanyang University), Jae Hong Seo (Hanyang University)
RecognitionContrastive LearningImage
🎯 What it does: This paper proposes a provably robust face recognition system, providing robustness criteria under the 1-Lipschitz condition in open-set recognition scenarios, and constructs a provably robust model based on this.
Towards compact reversible image representations for neural style transfer
Xiyao Liu (Central South University), Hui Fang (Loughborough University)
Image TranslationFlow-based ModelContrastive LearningImage
🎯 What it does: Proposed an InfoMinRev module based on reversible flow networks, achieving a balance between feature compression and expressive power in neural style transfer, thereby generating more natural stylized images with fewer artifacts.
Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation
Rong Wang (Australian National University), HONGDONG LI
GenerationGraph Neural NetworkMesh
🎯 What it does: Propose a data-driven 3D motion transfer method that can animate target stylized characters while generating realistic clothing animations.
Towards Image Ambient Lighting Normalization
Florin-Alexandru Vasluianu (University of Würzburg), Radu Timofte (University of Würzburg)
RestorationConvolutional Neural NetworkImageBenchmark
🎯 What it does: This paper proposes and studies the Ambient Lighting Normalization (ALN) task, aiming to recover image details in complex multi-light source and self-shadowed scenes.
Towards Latent Masked Image Modeling for Self-Supervised Visual Representation Learning
Yibing Wei (University of Wisconsin Madison), Pedro Morgado (Carnegie Mellon University)
Representation LearningTransformerAuto EncoderContrastive LearningImage
🎯 What it does: Propose and systematically evaluate a framework for Latent Masked Image Modeling (Latent MIM) in the latent space, leveraging reconstruction of latent features to learn unsupervised visual representations.
Towards Model-Agnostic Dataset Condensation by Heterogeneous Models
Jun-Yeong Moon (Kyung Hee University), Gyeong-Moon Park (Kyung Hee University)
Data SynthesisKnowledge DistillationData-Centric LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose a model-agnostic dataset compression method called HMDC, which generates general synthetic images by using heterogeneous models collaboratively.
Towards More Practical Group Activity Detection: A New Benchmark and Model
Dongkeun Kim (Pohang University of Science and Technology), Suha Kwak (Pohang University of Science and Technology)
Object DetectionConvolutional Neural NetworkTransformerContrastive LearningVideoBenchmark
🎯 What it does: This paper proposes a new large-scale multi-perspective group activity detection dataset (Café) and an end-to-end group activity detection model based on Transformer.
Towards Multi-modal Transformers in Federated Learning
Guangyu Sun (University of Central Florida), Chen Chen (University of Central Florida)
Federated LearningRepresentation LearningTransformerMixture of ExpertsImageTextMultimodalityBiomedical Data
🎯 What it does: Propose the FedCola framework, addressing cross-modal and modality gap issues between single-modal and multi-modal clients in federated learning through a multi-modal Transformer.
Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision
Hao Dong (ETH Zürich), Olga Fink (EPFL)
Domain AdaptationAnomaly DetectionAuto EncoderContrastive LearningVideoMultimodalityAudio
🎯 What it does: Proposed the MOOSA method for Multimodal Open-Set Domain Generalization and Adaptation (MMOSDG/MM-OSDA), which learns cross-modal representations through self-supervised tasks and achieves unknown class detection.
Towards Multimodal Sentiment Analysis Debiasing via Bias Purification
Dingkang Yang (Fudan University), Lihua Zhang (Tencent Youtu Lab)
ClassificationHyperparameter SearchTransformerContrastive LearningMultimodality
🎯 What it does: Construct a multi-modal sentiment analysis framework MCIS based on causal inference, purifying label bias and context bias through two adversarial pseudo-experiments (without multi-modal input and retaining only contextual words) during the inference phase of existing trained models, and obtaining unbiased predictions by subtracting with a tunable coefficient;
Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching
Meng Chu (National University of Singapore), Tat-Seng Chua (National University of Singapore)
RetrievalAutonomous DrivingTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: Proposed the GeoText-1652 dataset and a natural language guided UAV localization framework based on spatial relationship matching, integrating multi-platform images with fine-grained text-box annotations;
Towards Neuro-Symbolic Video Understanding
Minkyu Choi (University of Texas at Austin), Sandeep Chinchali (University of Texas at Austin)
RetrievalExplainability and InterpretabilityTransformerVision Language ModelVideoText
🎯 What it does: Propose a neuro-symbolic video retrieval framework that integrates visual language models with temporal logic and probabilistic automata to real-time locate complex event scenarios in long videos.
Towards Open Domain Text-Driven Synthesis of Multi-Person Motions
Mengyi Shan (University of Washington), Mitchell K Hill
GenerationData SynthesisPose EstimationTransformerVision Language ModelDiffusion modelImageVideoTextMultimodality
🎯 What it does: Proposes a framework for generating multi-person human motion based on text descriptions, supporting any number of people and generating natural and diverse group actions from open-domain prompts.
Towards Open-ended Visual Quality Comparison
Haoning Wu (Nanyang Technological University), Weisi Lin (Sensetime Research)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityBenchmark
🎯 What it does: Developed the LMM Co-Instruct for open-ended visual quality comparison along with its corresponding training set and benchmark.
Towards Open-Ended Visual Recognition with Large Language Models
Qihang Yu (ByteDance), Liang-Chieh Chen (ByteDance)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImage
🎯 What it does: Proposes the OmniScient Model (OSM), a mask classifier based on large language models, achieving open-vision recognition;
Towards Open-World Object-based Anomaly Detection via Self-Supervised Outlier Synthesis
Brian Kostadinov Shalon Isaac-Medina, Toby P Breckon
Data SynthesisAnomaly DetectionFlow-based ModelMultimodality
🎯 What it does: This paper proposes an open-world object detection framework based on self-supervised virtual anomaly synthesis (SSOS) to achieve object-level anomaly detection without class labels.
Towards Physical World Backdoor Attacks against Skeleton Action Recognition
Qichen Zheng (Nanyang Technological University), Alex Kot (Nanyang Technological University)
RecognitionAdversarial AttackGraph Neural NetworkTransformerVideoPoint CloudGraph
🎯 What it does: Proposed the first physical-world backdoor attack method for skeletal action recognition, named PSBA, which can inject a small number of trigger actions into the training set, causing the model to misclassify as the target category when detecting the trigger action, without affecting the accuracy of normal samples.
Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models
Jiaqi Xu (Chinese University of Hong Kong), Pheng-Ann Heng (Chinese University of Hong Kong)
RestorationPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: Propose a semi-supervised learning framework named WResVLM based on vision-language models (VLM) to enhance the clarity and semantic restoration of images under various adverse weather conditions in real-world environments.
Towards Real-world Event-guided Low-light Video Enhancement and Deblurring
Taewoo Kim (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
RestorationConvolutional Neural NetworkTransformerMultimodality
🎯 What it does: This paper proposes an end-to-end framework that utilizes an event camera to simultaneously enhance low-light videos and remove motion blur.
Towards Reliable Advertising Image Generation Using Human Feedback
Zhenbang Du (Huazhong University of Science and Technology), Jingping Shao (JD)
GenerationData SynthesisConvolutional Neural NetworkTransformerReinforcement LearningVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Proposed a reliable system for generating advertising images, utilizing a multi-modal RFNet to simulate human review and combining cyclic generation with RFFT-refined diffusion models, significantly improving the proportion of usable images.
Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models
Francesco Croce (EPFL), Matthias Hein (University of Tübingen)
SegmentationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a reliable robustness evaluation and fast training method for semantic segmentation models, including a novel attack loss function, attack ensemble, and adversarial training using a robust ImageNet backbone.
Towards Robust Event-based Networks for Nighttime via Unpaired Day-to-Night Event Translation
Yuhwan Jeong (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
Image TranslationData SynthesisDomain AdaptationConvolutional Neural NetworkDiffusion modelGenerative Adversarial NetworkContrastive LearningTime Series
🎯 What it does: Through unpaired event-to-event translation, daytime event data is converted into nighttime events, helping networks that perform well on daytime data maintain performance at night.
Towards Robust Full Low-bit Quantization of Super Resolution Networks
Denis S. Makhov (Noah's Ark Lab Huawei Technologies), Kirill Solodskikh (Noah's Ark Lab Huawei Technologies)
Super ResolutionConvolutional Neural NetworkImageOrdinary Differential Equation
🎯 What it does: This paper proposes a full low-bit quantization method for super-resolution networks. The core idea is to first map images to the differential operator (edge, texture) domain, enhance the image using a quantized CNN in this domain, and then revert the result back to the original domain through a regularized partial differential equation (PDE) solver, achieving high-quality super-resolution under full 4-bit quantization.
Towards Scene Graph Anticipation
Rohith Peddi (University of Texas at Dallas), Vibhav Gogate (University of Texas at Dallas)
GenerationConvolutional Neural NetworkTransformerVideoStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes the Scene Graph Anticipation (SGA) task, which predicts future relationships between observed objects in videos, and introduces an end-to-end model called SceneSayer. The model generates future relationships by leveraging object-centric representations, spatial context encoding, and continuous-time latent dynamics (NeuralODE/NeuralSDE).
Towards Stable 3D Object Detection
Jiabao Wang (Nankai University), Qibin Hou (Nankai University)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: This paper proposes a Stability Index (SI) metric to measure the temporal stability of 3D object detectors, and develops a Prediction Consistency Learning (PCL) training strategy based on this, significantly improving the consistency of confidence, position, size, and orientation of detectors across consecutive frames.
Towards Unified Representation of Invariant-Specific Features in Missing Modality Face Anti-Spoofing
Guanghao Zheng (Shanghai Jiao Tong University), Hongkai Xiong (Shanghai Jiao Tong University)
Anomaly DetectionConvolutional Neural NetworkTransformerContrastive LearningMultimodality
🎯 What it does: Propose the MMA-FAS framework to address the missing modality problem in multimodal face anti-spoofing (FAS), jointly extracting modality-invariant and modality-specific features, and enhancing robustness through LBP-guided contrastive learning and adaptive modality sampling.
TP2O: Creative Text Pair-to-Object Generation using Balance Swap-Sampling
Jun Li (Nanjing University of Science and Technology), Jian Yang (Nanjing University of Science and Technology)
GenerationVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: This paper proposes a sampling method (BASS) based on text embedding column vector exchange and CLIP distance balance, which can generate composite object images with high novelty, surprise, and value using only two object description texts without additional training.
TPA3D: Triplane Attention for Fast Text-to-3D Generation
Bin-Shih Wu (National Taiwan University), Yu-Chiang Frank Wang (NVIDIA)
GenerationData SynthesisTransformerVision Language ModelGenerative Adversarial NetworkTextMesh
🎯 What it does: Propose a GAN-based tri-plane attention network (TPA3D) that refines tri-plane features using sentence and word-level text features, enabling the rapid generation of high-quality textured 3D meshes from text.
Track Everything Everywhere Fast and Robustly
Yunzhou Song (University of Pennsylvania), Kostas Daniilidis (University of Pennsylvania)
Object TrackingDepth EstimationOptimizationVision Language ModelOptical FlowVideo
🎯 What it does: This paper proposes a long-term pixel tracking method based on test-time optimization called CaDeX++, achieving fast and robust pixel tracking through a reversible local deformation network, monocular depth estimation, and DINOv2 semantic matching.
Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation
Homanga Bharadhwaj (Carnegie Mellon University), Shubham Tulsiani (Carnegie Mellon University)
Robotic IntelligenceTransformerDiffusion modelVideo
🎯 What it does: Proposed the Track2Act framework, which generates robot action sequences by predicting point trajectories from internet videos and learns residual policies through a few robot demonstrations, achieving general-purpose robotic manipulation without requiring training during testing.
Trackastra: Transformer-based cell tracking for live-cell microscopy
Benjamin Gallusser (EPFL), Martin Weigert (EPFL)
Object TrackingTransformerBiomedical Data
🎯 What it does: Propose Trackastra, a Transformer-based cell tracking method that directly learns associations between detections and achieves biologically consistent tracking.
TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Tracks
Jinjie Mai (King Abdullah University of Science and Technology), Bernard Ghanem (King Abdullah University of Science and Technology)
Pose EstimationDepth EstimationNeural Radiance FieldImage
🎯 What it does: Propose TrackNeRF, achieving globally consistent bundle adjustment-based NeRF training through feature trajectories, addressing sparse and noisy camera poses to obtain high-quality novel view synthesis.
TrafficNight : An Aerial Multimodal Benchmark For Nighttime Vehicle Surveillance
Guoxing Zhang (Hong Kong Polytechnic University), HUANG Hailong
Object DetectionSegmentationConvolutional Neural NetworkImageVideoMultimodalityPoint CloudBenchmark
🎯 What it does: This paper introduces the TrafficNight dataset, providing synchronized thermal infrared and sRGB aerial images, vehicle annotations with oriented bounding boxes (OBB), HD maps, and raw videos, with a focus on large vehicles and semi-trailers.
Train Till You Drop: Towards Stable and Robust Source-free Unsupervised 3D Domain Adaptation
Björn Michele (valeo.ai), Nicolas Courty (CNRS, IRISA, Univ. Bretagne Sud)
SegmentationDomain AdaptationPoint Cloud
🎯 What it does: Propose a source-agnostic unsupervised domain adaptation framework (TTYD) for 3D LiDAR semantic segmentation, which performs fine-tuning on the target domain through two regularization terms: confidence entropy and class distribution similarity, and provides unsupervised stopping and validation criteria.
Trainable Highly-expressive Activation Functions
Irit Chelly (Ben-Gurion University of the Negev), Oren Freifeld (Ben-Gurion University of the Negev)
ClassificationSegmentationGenerationImage
🎯 What it does: This paper proposes a trainable, high-expressive activation function called DiTAC, which achieves significant performance improvements across various tasks.
Training A Secure Model against Data-Free Model Extraction
Zhenyi Wang (University of Maryland), Mingchen Gao (University of Sydney)
OptimizationSafty and PrivacyAdversarial AttackData-Centric LearningMeta LearningImage
🎯 What it does: Proposed and implemented a security model training framework AAUG based on random weight perturbation, aiming to significantly reduce the success rate of data-free model extraction (DFME) during deployment.
Training A Small Emotional Vision Language Model for Visual Art Comprehension
Jing Zhang (Hefei University of Technology), Dan Guo (Hefei University of Technology)
ClassificationGenerationTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Trained and evaluated a small-scale emotion vision-language model SEVLM for emotion classification and emotion explanation generation in visual artworks.
Training-free Composite Scene Generation for Layout-to-Image Synthesis
Jiaqi Liu (University of Sydney), Chang Xu (University of Sydney)
Image TranslationGenerationData SynthesisDiffusion modelImage
🎯 What it does: Propose an untrained composite scene generation method that utilizes layout information to guide Stable Diffusion for multi-object image synthesis.
Training-Free Model Merging for Multi-target Domain Adaptation
Wenyi Li (Tsinghua University), Hao Zhao (Tsinghua University)
SegmentationDomain AdaptationImage
🎯 What it does: Proposes a training-data-free model merging method to combine models trained for different target domains into a single robust model in multi-target domain adaptation.
Training-free Video Temporal Grounding using Large-scale Pre-trained Models
Minghang Zheng (Peking University), Yang Liu (Peking University)
RetrievalTransformerLarge Language ModelVision Language ModelVideoTextChain-of-Thought
🎯 What it does: Propose a training-free video temporal localization method, which first splits queries into sub-events and infers their order and relationships using a large language model, then performs dynamic and static matching for each sub-event with a vision-language model, ultimately obtaining video segments.
Trajectory-aligned Space-time Tokens for Few-shot Action Recognition
Pulkit Kumar (University of Maryland), Abhinav Shrivastava (University of Maryland)
RecognitionMeta LearningTransformerContrastive LearningVideo
🎯 What it does: Proposed Trajectory-Aligned Spatiotemporal Tokens (TAT) and Masked Spatiotemporal Transformer for few-shot action recognition, focusing on decomposing motion and appearance representations.
TrajPrompt: Aligning Color Trajectory with Vision-Language Representations
Li-Wu Tsao (National Yang Ming Chiao Tung University), Wen-Huang Cheng (National Taiwan University)
Autonomous DrivingRepresentation LearningPrompt EngineeringVision Language ModelContrastive LearningImage
🎯 What it does: Propose TrajPrompt, a prompt-based method that draws colored trajectories on bird's-eye view (BEV) scenes and aligns with a pre-trained CLIP vision-language model for future trajectory prediction.
TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos
Yufu Wang (University of Pennsylvania), Kostas Daniilidis (University of Pennsylvania)
Pose EstimationDepth EstimationTransformerSupervised Fine-TuningSimultaneous Localization and MappingVideo
🎯 What it does: Proposed a two-stage method TRAM for recovering the global trajectory and motion of the human body from natural videos, separating and combining camera trajectory with body motion;
TransCAD: A Hierarchical Transformer for CAD Sequence Inference from Point Clouds
Elona Dupont, Djamila Aouada (University of Luxembourg)
GenerationConvolutional Neural NetworkTransformerPoint CloudSequential
🎯 What it does: Propose TransCAD, an end-to-end hierarchical Transformer that directly predicts CAD generation sequences from point clouds, achieving feature-based reverse engineering.
Transferable 3D Adversarial Shape Completion using Diffusion Models
Xuelong Dai (Hong Kong Polytechnic University), Bin Xiao (Hong Kong Polytechnic University)
Adversarial AttackDiffusion modelPoint Cloud
🎯 What it does: Achieve adversarial shape completion for 3D point clouds via diffusion models, generating high-quality adversarial point clouds.
TransFusion -- A Transparency-Based Diffusion Model for Anomaly Detection
Matic Fučka (University of Ljubljana), Danijel Skočaj (University of Ljubljana)
Anomaly DetectionConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Proposed a single-stage surface anomaly detection method called TransFusion based on transparency diffusion, which simultaneously reconstructs normal appearance and localizes anomalies in one iteration process;
Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priors
Jae Joong Lee (Purdue University), Bedrich Benes (Purdue University)
SegmentationData SynthesisDiffusion modelScore-based ModelNeural Radiance FieldImageMeshBenchmarkAgriculture Related
🎯 What it does: Based on a single tree RGB image, a complete 3D tree shell is first generated using species-specific diffusion models, then combined with species-based spatial colonization algorithms to produce high-quality 3D tree models suitable for growth simulation, constructing a simulation-ready dataset of 600,000 real trees.