ECCV 2024 Papers — Page 24
European Conference on Computer Vision · 2387 papers
View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields
Haodi He (Stanford University), Leonidas Guibas
SegmentationDepth EstimationNeural Radiance FieldContrastive LearningMesh
🎯 What it does: Propose a perspective-consistent hierarchical 3D segmentation method based on Ultrametric Feature Field, which can automatically generate coarse-to-fine segmentation results at different resolution thresholds.
ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformers
Jinke Li (Uisee Foundation Research & Development), Dan Zhang (Uisee Foundation Research & Development)
SegmentationAutonomous DrivingConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: Propose the ViewFormer framework, which fuses learned perspective attention and streaming spatiotemporal attention to achieve spatial feature aggregation and spatiotemporal modeling of multi-view images for 3D occupancy awareness and occupancy flow prediction.
Viewpoint textual inversion: discovering scene representations and 3D view control in 2D diffusion models
James Burgess (Stanford University), Serena Yeung-Levy (Stanford University)
GenerationRepresentation LearningPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelImageText
🎯 What it does: By learning a '3D perspective token' in the embedding space of Stable Diffusion, the camera perspective of generated images can be controlled without modifying the model weights, and this method is used to examine whether the model internally contains an implicit 3D scene representation.
ViG-Bias: Visually Grounded Bias Discovery and Mitigation
Badr-Eddine Marani, Enzo Ferrante (CentraleSupelec, Universite Paris Saclay)
ClassificationRecognitionExplainability and InterpretabilityVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Propose the ViG-Bias method, which leverages visual explanations (e.g., GradCAM heatmaps) to assist models in discovering and mitigating biases in visual recognition.
ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling
Siming Yan (University of Texas at Austin), Li Erran Li (AWS AI)
Object DetectionReinforcement Learning from Human FeedbackTransformerVision Language ModelImageTextMultimodality
🎯 What it does: Proposes the ViGoR framework, which efficiently fine-tunes large vision-language models (e.g., LLaVA) for visual grounding through fine-grained reward modeling and automated evaluation methods, significantly reducing hallucinations and information omissions in text.
ViLA: Efficient Video-Language Alignment for Video Question Answering
Xijun Wang (University of Maryland College Park), Shan Yang (Amazon)
Knowledge DistillationTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Designed and implemented the ViLA network, which includes a text-guided Frame-Prompter and a cross-modal QFormer-Distiller, to efficiently sample key frames from videos and align them with pre-trained LLMs, thereby completing video question answering tasks.
ViPer: Visual Personalization of Generative Models via Individual Preference Learning
Sogand Salehi (Swiss Federal Institute of Technology Lausanne), Amir Zamir (Swiss Federal Institute of Technology Lausanne)
GenerationTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: ViPer leverages large language models to extract structured visual attributes by allowing users to freely comment on a few images, then embeds these attributes into Stable Diffusion to accomplish one-time personalized image generation.
VISA: Reasoning Video Object Segmentation via Large Language Model
Cilin Yan (Beihang University University Of Amsterdam), Efstratios Gavves (Shanghai Jiao Tong University)
SegmentationTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodality
🎯 What it does: The study proposes the ReasonVOS task and designs the VISA model, achieving video object segmentation and reasoning based on large language models.
VISAGE: Video Instance Segmentation with Appearance-Guided Enhancement
Hanjung Kim (Yonsei University), Seon Joo Kim (Yonsei University)
Object TrackingSegmentationTransformerContrastive LearningVideo
🎯 What it does: Propose the VISAGE method, which generates appearance queries through mask pooling in video instance segmentation, improves query matching via contrastive learning, reduces dependence on positional information, and implements a simplified tracker and memory pool to enhance instance tracking accuracy.
VisFocus: Prompt-Guided Vision Encoders for OCR-Free Dense Document Understanding
Ofir Abramovich (Reichman University), R. Manmatha (AWS AI Labs)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningMultimodality
🎯 What it does: Proposed an OCR-free visual document understanding framework called VisFocus, which can directly introduce prompt information into the visual encoder and guide the model to focus on document regions related to the prompts through a new pre-training task.
Visible and Clear: Finding Tiny Objects in Difference Map
Bing Cao (Tianjin University), Qinghua Hu (Tianjin University)
Object DetectionConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: Propose a self-reconstruction based tiny object detection framework SR-TOD, which enhances the visibility of tiny objects by adding a reconstruction head in the detector's neck layer to generate difference maps, thereby guiding feature enhancement.
Vision-Language Action Knowledge Learning for Semantic-Aware Action Quality Assessment
Huangbiao Xu (Fuzhou University), Wenzhong Guo (Fuzhou University)
RecognitionConvolutional Neural NetworkTransformerVision-Language-Action ModelContrastive LearningVideoText
🎯 What it does: This paper proposes a multi-grained visual-language alignment framework and a semantic-aware collaborative attention module, which embeds professional terminology text knowledge into video's frame-level, stage-level, and video-level representations to improve action quality assessment performance.
Vision-Language Dual-Pattern Matching for Out-of-Distribution Detection
Zihan Zhang (Huazhong University of Science and Technology), Xiang Xiang (Huazhong University of Science and Technology)
Anomaly DetectionTransformerPrompt EngineeringVision Language ModelContrastive LearningImageText
🎯 What it does: This paper proposes a dual-modal matching framework based on CLIP (Dual-Pattern Matching, DPM), which constructs visual patterns and text patterns by simultaneously utilizing visual features and text features of ID (in-distribution) samples to enhance the accuracy of OOD (out-of-distribution) detection. Two implementations are provided: no-training (DPM-F) and lightweight training (DPM-T).
VisionLLaMA: A Unified LLaMA Backbone for Vision Tasks
Xiangxiang Chu (Meituan Inc), Chunhua Shen (Zhejiang University)
ClassificationSegmentationGenerationTransformerLarge Language ModelDiffusion modelAuto EncoderContrastive LearningImage
🎯 What it does: Propose VisionLLaMA—a unified vision transformer architecture based on LLaMA, applicable to multiple visual tasks such as image generation, classification, segmentation, and detection.
VisionTrap: Vision-Augmented Trajectory Prediction Guided by Textual Descriptions
Seokha Moon (Korea University), Jinkyu Kim (Korea University)
Autonomous DrivingComputational EfficiencyTransformerVision Language ModelContrastive LearningImageVideoTextMultimodality
🎯 What it does: Propose VisionTrap, a vision-enhanced trajectory prediction model that utilizes visual semantics from surround-view cameras and text descriptions generated by VLM+LLM to guide the model in learning, thereby better capturing the behaviors and environmental interactions of road agents such as pedestrians and vehicles, ultimately achieving real-time (53 ms) trajectory prediction.
Vista3D: unravel the 3d darkside of a single image
Qiuhong Shen (National University of Singapore), Xinchao Wang (National University of Singapore)
GenerationDiffusion modelScore-based ModelGaussian SplattingImageMesh
🎯 What it does: Propose the VISTA3D framework, which employs a two-stage coarse-to-fine grid generation approach (first rapidly constructing a rough geometry using Gaussian Splatting, then converting it into an SDF and refining it via FlexiCubes), while achieving texture decoupling and angular composition from multi-source diffusion models, generating diverse and consistent high-quality 3D objects from a single image within 5–20 minutes.
Visual Alignment Pre-training for Sign Language Translation
Peiqi Jiao (Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences), Xilin Chen (Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences)
RecognitionTransformerVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: Propose Visual Alignment Pre-training (VAP), which generates gloss-like alignments in sign language translation without gloss annotations by greedily matching visual and textual tokens, thereby enhancing the semantic representations of the visual encoder.
Visual Grounding for Object-Level Generalization in Reinforcement Learning
Haobin Jiang (Peking University), Zongqing Lu (Peking University)
Reinforcement LearningVision Language ModelVideo
🎯 What it does: This study proposes the COPL method, which leverages the vision-language knowledge of MineCLIP for visual localization and reward design, thereby achieving zero-shot generalization for unseen targets in Minecraft tasks.
Visual Prompting via Partial Optimal Transport
Mengyu Zheng (University of Sydney), Chang Xu (Huawei Noah's Ark Lab)
ClassificationDomain AdaptationPrompt EngineeringImage
🎯 What it does: Propose a label mapping method called OTLM based on partial optimal transport (Partial Optimal Transport), which significantly reduces the feature transfer burden required for visual prompting while keeping the pre-trained model parameters unchanged, thereby improving the model's performance on downstream tasks.
Visual Relationship Transformation
Xiaoyu Xu (University of Waterloo), Zhou Wang (University of Waterloo)
RecognitionObject DetectionDepth EstimationGraph Neural NetworkImageVideo
🎯 What it does: This paper proposes the Visual Relationship Transformation (VRT) task, aiming to predict object relationships in unseen perspectives and determine their visibility using only a single observation view and perspective transformation matrix.
Visual Text Generation in the Wild
Yuanzhi Zhu (Alibaba Group), Zhibo Yang (Alibaba Group)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Proposed a two-stage visual text generation method called SceneVTG, which first recommends suitable text regions and content using a multi-modal large language model, and then generates high-quality text images on real backgrounds using a local conditional diffusion model.
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models
Shicheng Li (Peking University), Lu Hou (Huawei)
RetrievalLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: Create the VITATECS dataset and use it to assess the temporal concept understanding ability of video-language models.
VividDreamer: Invariant Score Distillation for Hyper-Realistic Text-to-3D Generation
Wenjie Zhuo (State Key Laboratory of Brain-machine Intelligence, Zhejiang University), Yi Yang (State Key Laboratory of Brain-machine Intelligence, Zhejiang University)
GenerationTransformerDiffusion modelScore-based ModelNeural Radiance FieldGaussian SplattingText
🎯 What it does: Proposes the Invariant Score Distillation (ISD) method to address the issues of over-saturation and over-smoothing in Score Distillation Sampling (SDS) for text-to-3D generation.
VLAD-BuFF: Burst-aware Fast Feature Aggregation for Visual Place Recognition
Ahmad Khaliq (Queensland University of Technology), Sourav Garg (University of Adelaide)
RetrievalRepresentation LearningTransformerContrastive LearningImageBenchmark
🎯 What it does: Propose VLAD-BuFF, improving traditional VLAD aggregation by suppressing feature burstiness through self-similarity soft counting weights, and accelerating aggregation using PCA-initialized low-dimensional projection.
Volumetric Rendering with Baked Quadrature Fields
Gopal Sharma, Andrea Tagliasacchi (Google DeepMind)
GenerationComputational EfficiencySupervised Fine-TuningNeural Radiance FieldImageMesh
🎯 What it does: Propose a NeRF representation utilizing textured polygons, using meshes extracted from the quadrature field to obtain ray-field intersection points as quadrature samples required for volume rendering, achieving real-time rendering of transparent and non-transparent scenes without distortion.
VP-SAM: Taming Segment Anything Model for Video Polyp Segmentation via Disentanglement and Spatio-temporal Side Network
Zhixue Fang (Shenzhen University), Jing Qin
SegmentationTransformerVideoBiomedical Data
🎯 What it does: This paper addresses the video polyp segmentation task by proposing a VP-SAM model based on the Segment Anything Model (SAM).
VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space
Guénolé Fiche (CentraleSupélec), Francesc Moreno
Pose EstimationConvolutional Neural NetworkTransformerAuto EncoderImageMesh
🎯 What it does: Propose a framework called VQ-HPS that transforms human pose and shape estimation into a classification task in a vector-quantized latent space, leveraging Mesh-VQ-VAE to learn discrete mesh encodings and using Transformer to predict images.
VQA-Diff: Exploiting VQA and Diffusion for Zero-Shot Image-to-3D Vehicle Asset Generation in Autonomous Driving
YIBO LIU (Huawei Noah Ark Lab), JINJUN SHAN (Huawei Noah Ark Lab)
GenerationAutonomous DrivingVision Language ModelDiffusion modelImageTextMesh
🎯 What it does: Leverage VQA and diffusion models for zero-shot generation of vehicle 3D assets
VSViG: Real-time Video-based Seizure Detection via Skeleton-based Spatiotemporal ViG
Yankun Xu (Westlake University), Mohamad Sawan (Zhejiang University)
Anomaly DetectionComputational EfficiencyConvolutional Neural NetworkGraph Neural NetworkSupervised Fine-TuningVideoBiomedical Data
🎯 What it does: Designed and implemented a real-time epileptic seizure detection model called VSViG, based on skeleton patch embeddings and spatiotemporal visual graph neural networks, achieving early seizure identification by outputting probabilities through a regression task.
Walker: Self-supervised Multiple Object Tracking by Walking on Temporal Object Appearance Graphs
Mattia Segù, Bernt Schiele (ETH Zurich)
Object TrackingGraph Neural NetworkContrastive LearningVideo
🎯 What it does: Proposes a self-supervised multi-object tracking model called Walker, which can perform detection and tracking using only sparse bounding box annotations without requiring instance ID labels.
WAS: Dataset and Methods for Artistic Text Segmentation
Xudong Xie (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
SegmentationTransformerDiffusion modelImage
🎯 What it does: Construct an artistic text segmentation dataset (real WAS-R and synthetic WAS-S), and propose the WASNet model tailored for this task.
WaSt-3D: Wasserstein-2 Distance for Scene-to-Scene Stylization on 3D Gaussians
Dmytro Kotovenko (Lmu Munich), Bjorn Ommer (Meta Reality Labs)
Image TranslationGenerationGaussian SplattingPoint Cloud
🎯 What it does: Propose the WaSt-3D method, which achieves untrained scene-to-scene style transfer using 3D Gaussian splatting and Sinkhorn divergence.
Watch Your Steps: Local Image and Scene Editing by Text Instructions
Ashkan Mirzaei (Samsung AI Centre Toronto), Igor Gilitschenski (University of Toronto)
Image TranslationGenerationPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderImage
🎯 What it does: Proposes a method that automatically locates edit regions in images and 3D NeRF scenes by computing a 'correlation map' through the difference in noise prediction from InstructPix2Pix (IP2P), and elevates this correlation information into a 3D correlation field to achieve localized, controllable text-driven editing.
Watching it in Dark: A Target-aware Representation Learning Framework for High-Level Vision Tasks in Low Illumination
Yunan Li (Xidian University), Qiguang Miao (Xidian University)
Object DetectionDomain AdaptationRepresentation LearningContrastive LearningImageVideo
🎯 What it does: This paper proposes a target-aware representation learning framework for low-light environments, leveraging bidirectional domain alignment and target saliency mechanisms to enhance the performance of high-level visual tasks.
WAVE: Warping DDIM Inversion Features for Zero-shot Text-to-Video Editing
Yutang Feng (Beihang University), Angela Yao (National University of Singapore)
GenerationData SynthesisDiffusion modelOptical FlowVideoText
🎯 What it does: Designed a zero-shot method that utilizes flow-guided feature warping during the DDIM inversion and denoising stages to achieve text-driven video editing.
Wavelength-Embedding-guided Filter-Array Transformer for Spectral Demosaicing
Haijin Zeng (IMEC-Ghent University), Wilfried Philips (IMEC-Ghent University)
RestorationConvolutional Neural NetworkTransformerImage
🎯 What it does: A transformer-based de-mosaicking network called WeFAT is proposed for spectral de-mosaicking tasks in multi-spectral filter array (MSFA) cameras, which can adapt to different wavelength distributions and MSFA structures of cameras with only a single dataset for training.
Wavelet Convolutions for Large Receptive Fields
Shahaf E Finder, Oren Freifeld (Ben-Gurion University of the Negev)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkImage
🎯 What it does: Proposed a wavelet transform-based convolutional layer (WTConv) that achieves an extremely large receptive field without increasing the number of parameters, and can replace traditional depth convolutions in convolutional neural networks.
WBP: Training-time Backdoor Attacks through Hardware-based Weight Bit Poisoning
Kunbei Cai (University of Central Florida), Fan Yao (University of Central Florida)
Adversarial AttackConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose a task-agnostic backdoor attack during training, named WBP, which induces binary bit flips in pre-trained model weights using hardware row-hammer, enabling backdoor implantation during fine-tuning.
Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detection
Kwanyong Park (ETRI), Donghyun Kim (Korea University)
Object DetectionData SynthesisTransformerLarge Language ModelDiffusion modelContrastive LearningMultimodality
🎯 What it does: Generate text-image triplets with free-text descriptions and bounding boxes using large language models and text-to-image diffusion models, and use this synthetic data to enhance compositional understanding in language-based object detection.
Weakly Supervised 3D Object Detection via Multi-Level Visual Guidance
Kuan-Chih Huang (University of California, Merced), Ming-Hsuan Yang (University of California, Merced)
Object DetectionContrastive LearningImagePoint Cloud
🎯 What it does: Proposed a 3D object detection framework (VG-W3D) that utilizes only 2D bounding boxes as weak supervision signals, achieving learning from point clouds to 3D bounding boxes through multi-layer visual guidance.
Weakly Supervised Co-training with Swapping Assignments for Semantic Segmentation
Xinyu Yang (Lancaster University), Bryan M Williams
SegmentationTransformerContrastive LearningImage
🎯 What it does: Propose an end-to-end weakly supervised semantic segmentation framework CoSA, utilizing a dual-stream network to achieve collaborative training between CAM and the segmentation model, eliminating traditional CAM post-processing.
Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance
Yufei Zhang (Rensselaer Polytechnic Institute), Qiang Ji (Rensselaer Polytechnic Institute)
Pose EstimationConvolutional Neural NetworkImageMesh
🎯 What it does: Propose a weakly supervised monocular 3D hand reconstruction method that constructs a differentiable prior using biomechanical, functional anatomical, and physical knowledge of hands, and improves training effectiveness through a negative log-likelihood loss that models uncertainty in image observations.
Weakly-supervised Camera Localization by Ground-to-satellite Image Registration
Yujiao Shi (ShanghaiTech University), Ankit Vora (Ford Motor Company)
Pose EstimationAutonomous DrivingConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: Propose a weakly supervised ground-to-satellite image registration framework. First, self-supervised regression estimates relative rotation, then contrastive learning maximizes positive sample similarity to estimate translation, thereby improving camera pose accuracy.
Weakly-Supervised Spatio-Temporal Video Grounding with Variational Cross-Modal Alignment
Yang Jin (Peking University), Yadong Mu (Peking University)
Object DetectionTransformerVision Language ModelContrastive LearningVideoText
🎯 What it does: This paper proposes a weakly supervised spatiotemporal video localization framework that achieves spatial-temporal localization of target objects by leveraging the correspondence between video sentences.
Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment
Mengting Chen (Alibaba Group), Shuai Xiao (Alibaba Group)
Image TranslationPose EstimationConvolutional Neural NetworkVision Language ModelDiffusion modelImage
🎯 What it does: Proposes a virtual try-on framework named Wear-Any-Way, supporting high-quality virtual try-on for multiple garments, model-to-model, and complex scenarios, with freely adjustable wearing styles achieved through point control.
WebRPG: Automatic Web Rendering Parameters Generation for Visual Presentation
Zirui Shao (Zhejiang Provincial Key Laboratory of Service Robot Zhejiang University), Cong Yao (Alibaba Group)
GenerationTransformerDiffusion modelAuto EncoderText
🎯 What it does: This paper proposes the WebRPG task, which involves automatically generating rendering parameters for each element based on HTML code to achieve web page visualization without CSS code.
WeConvene: Learned Image Compression with Wavelet-Domain Convolution and Entropy Model
Haisheng Fu (Simon Fraser University), Guohe Zhang (Xi'an Jiaotong University)
CompressionConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: Proposes the WeConvene framework, integrating discrete wavelet transform into convolutional layers and entropy models to achieve more efficient learned image compression.
WeCromCL: Weakly Supervised Cross-Modality Contrastive Learning for Transcription-only Supervised Text Spotting
Jingjing Wu (Harbin Institute of Technology), Wenjie Pei (Harbin Institute of Technology)
RecognitionObject DetectionTransformerContrastive LearningMultimodality
🎯 What it does: Propose a weakly supervised cross-modal contrastive learning framework named WeCromCL, which utilizes text transcription information to locate scene text anchors under unbounded annotation conditions. Subsequently, the anchors are used as pseudo labels to train a single-point supervised text detector, achieving text detection and recognition relying solely on transcribed text.
Weight Conditioning for Smooth Optimization of Neural Networks
Hemanth Saratchandran (University of Adelaide), Simon Lucey (Sorbonne Université)
ClassificationOptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose a new weight conditioning method by applying row balancing preprocessing to the neural network weight matrix, reducing the matrix condition number to smooth the loss surface and accelerate gradient descent.
Weighted Ensemble Models Are Strong Continual Learners
Imad Eddine MAROUF, Stéphane Lathuilière (Télécom-Paris Institut Polytechnique de Paris)
OptimizationMeta LearningTransformerSupervised Fine-TuningImageBenchmark
🎯 What it does: This paper proposes a continuous learning method that utilizes model averaging of pre-trained weights to balance the adaptability of new tasks and the memorization of old tasks, introducing CoMA and its improved version CoFiMA.
Weighting Pseudo-Labels via High-Activation Feature Index Similarity and Object Detection for Semi-Supervised Segmentation
Prantik Howlader (Stony Brook University), Dimitris Samaras (Stony Brook University)
Object DetectionSegmentationConvolutional Neural NetworkImageBenchmark
🎯 What it does: This paper proposes a method for weighting pseudo-labels in semi-supervised semantic segmentation. By jointly training a semantic segmentation model and an object detection model, reliable pseudo-label pixels are identified. Learning weights are assigned to each pseudo-label pixel using the ranking similarity of high-activation feature dimensions, thereby mitigating the negative impact of pseudo-label noise on training.
WHAC: World-grounded Humans and Cameras
Wanqi Yin (SenseTime Research), Lei Yang (University of Tokyo)
Pose EstimationDepth EstimationSimultaneous Localization and MappingVideo
🎯 What it does: Propose the WHAC framework, jointly estimating the camera and human trajectories in the world coordinate system, achieving scalable human pose and shape recovery under monocular video.
When and How do negative prompts take effect?
Yuanhao Ban (University of California, Los Angeles), Cho-Jui Hsieh (University of California, Los Angeles)
RestorationGenerationPrompt EngineeringDiffusion modelMultimodality
🎯 What it does: A systematic study on negative prompts in text-to-image diffusion models such as Stable Diffusion, analyzing when and how they take effect, and proposing a controlled image inpainting method based on this analysis.
When Do We Not Need Larger Vision Models?
Baifeng Shi (UC Berkeley Microsoft Research), Trevor Darrell (UC Berkeley Microsoft Research)
ClassificationSegmentationDepth EstimationRetrievalRobotic IntelligenceImageMultimodality
🎯 What it does: Propose and verify the 'Scaling on Scales' (S²) method, which enhances visual model performance by extracting features from multi-scale images while keeping the model size unchanged.
When Fast Fourier Transform Meets Transformer for Image Restoration
Xingyu Jiang (Beihang University), Yue Deng (Beihang University)
RestorationSuper ResolutionTransformerImageBenchmark
🎯 What it does: Proposes an efficient image restoration framework named SFHformer, which integrates Fast Fourier Transform (FFT) with Transformer, achieving the fusion of local and global features in a dual perception structure across spatial and frequency domains, capable of uniformly handling various image degradation problems.
When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset
Yi Zhang (SenseTime Research and Tetras.AI), Wentao Liu (SenseTime Research and Tetras.AI)
Object DetectionTransformerMultimodalityPoint CloudBenchmark
🎯 What it does: Developed a general multi-modal pedestrian detection model called MMPedestron, and constructed a large-scale benchmark MMPD containing multiple modalities such as RGB, IR, Depth, LiDAR, and Event.
Where am I? Scene Retrieval with Language
Jiaqi Chen (ETH Zürich), Hermann Blum (ETH Zürich)
RetrievalGraph Neural NetworkLarge Language ModelContrastive LearningTextGraph
🎯 What it does: Proposed a natural language-based 3D scene retrieval method called Text2SceneGraphMatcher, which can convert open-text descriptions into text graphs and locate the scenes described by users through joint embedding and semantic scene graph matching.
Which Model Generated This Image? A Model-Agnostic Approach for Origin Attribution
Fengyuan Liu (University of Oxford), Jindong Gu (University of Oxford)
ClassificationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: Propose a few-shot single-class classification framework called OCC-CLIP, which determines whether an image originates from the same generative model, thereby achieving source attribution of generated images.
WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language
Zhenxiang Lin (ShanghaiTech University), Yuexin Ma (Shanghai AI Laboratory)
Object DetectionConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningImageTextMultimodalityPoint Cloud
🎯 What it does: This paper proposes the 3D Visual Grounding in the Wild (3DVGW) task, which utilizes synchronized LiDAR point clouds and industrial camera images along with natural language descriptions to locate target objects in large-scale dynamic scenes, and constructs two large-scale datasets, STRefer and LifeRefer, based on this task;
WildVidFit: Video Virtual Try-On in the Wild via Image-Based Controlled Diffusion Models
Zijian He (Sun Yat-sen University), Liang Lin (Sun Yat-sen University)
Image TranslationGenerationConvolutional Neural NetworkTransformerDiffusion modelContrastive LearningImageVideo
🎯 What it does: A real-time video try-on framework named WildVidFit is proposed, which does not require video training and achieves high-quality video try-on for complex poses and occlusions by leveraging image-based control diffusion models.
WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensing
Shuokang Huang (Imperial College London), Julie A. McCann (Imperial College London)
ClassificationRecognitionConvolutional Neural NetworkRecurrent Neural NetworkTransformerVideoMultimodalityBenchmark
🎯 What it does: This paper proposes WiMANS, a novel dual-band WiFi CSI and synchronized video combined multi-user activity sensing benchmark dataset, and conducts benchmark experiments on multi-user identification, localization, and activity recognition tasks using this dataset.
WindPoly: Polygonal Mesh Reconstruction via Winding Numbers
Xin He (Shenzhen University), Hui Huang (Shenzhen University)
GenerationPoint CloudMesh
🎯 What it does: Proposed the WindPoly method, which achieves low-polygon mesh reconstruction of building point clouds by utilizing polygon plane detection, adaptive spatial partitioning, and polygon-based winding number optimization.
Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequence
Yutong Chen (Shanghai Artificial Intelligence Laboratory), Yinqiang Zheng (Shanghai Artificial Intelligence Laboratory)
GenerationNeural Radiance FieldOptical FlowVideo
🎯 What it does: Propose the Dyco method, which utilizes pose sequences and a local dynamic context encoder to achieve inertia-aware 3D human modeling
WordRobe: Text-Guided Generation of Textured 3D Garments
Astitva Srivastava (IIIT Hyderabad), Avinash Sharma (IIIT Hyderabad)
GenerationGraph Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelTextMesh
🎯 What it does: Designed and implemented the WordRobe framework, capable of generating high-quality pose-free, textured 3D garment meshes through user-friendly text prompts.
WorldPose: A World Cup Dataset for Global 3D Human Pose Estimation
Tianjian Jiang (ETH Zürich), Jie Song (ETH Zürich)
Pose EstimationOptical FlowVideoBenchmark
🎯 What it does: This paper constructs the WorldPose dataset based on multi-view cameras from the 2022 World Cup live broadcasts, providing 2.5 million global 3D human poses and trajectories, and achieving high-precision calibration of mobile cameras;
WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generation
Jiachen Lu (Fudan University), Li Zhang (Fudan University)
Autonomous DrivingTransformerVision Language ModelDiffusion modelAuto EncoderVideoTextMultimodality
🎯 What it does: Propose the WoVoGen framework, which first predicts future world states using a 4D world volume (time + space + height), and then generates multi-camera street-level videos from this volume, supporting controllable edits for weather, lighting, urban style, etc.
WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Models
Xin-Jian Wu (State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences), Cheng-Lin Liu (State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences)
SegmentationKnowledge DistillationTransformerPrompt EngineeringImage
🎯 What it does: Propose a weakly supervised part segmentation framework called WPS-SAM, which leverages the pre-trained Segment Anything Model (SAM) to automatically generate prompts and complete pixel-level part segmentation during training using only bounding box or point-level annotations.
WRIM-Net: Wide-Ranging Information Mining Network for Visible-Infrared Person Re-Identification
Yonggan Wu (University of Science and Technology of China), Hong-Qiang Wang (Institute of Intelligent Machines)
RecognitionRetrievalConvolutional Neural NetworkContrastive LearningMultimodality
🎯 What it does: Proposed a Wide-Ranging Information Mining Network (WRIM-Net) for person re-identification tasks in visible and infrared images.
WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering
Pingyi Chen (Zhejiang University), Lin Yang (Westlake University)
Data SynthesisExplainability and InterpretabilityTransformerLarge Language ModelVision Language ModelMultimodalityBiomedical Data
🎯 What it does: Proposed the WSI-VQA framework, transforming various whole slide image (WSI)-level tasks into generative visual question answering (VQA) tasks, and constructed the Wsi2Text Transformer (W2T) model; simultaneously automatically generated and made publicly available the WSI-VQA dataset containing 977 WSI and 8,672 question-answer pairs.
WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding
Quan Kong (Woven by Toyota), Norimasa Kobori (Woven by Toyota)
Object TrackingAutonomous DrivingLarge Language ModelVision Language ModelVideoMultimodalityBenchmark
🎯 What it does: Constructed and made public a traffic video dataset named WTS, which focuses on pedestrians and includes multi-perspective, synchronized 2D/3D views, fine-grained video descriptions, and 3D gaze information; developed the LLMScorer evaluation metric and the Instance-VideoLLM baseline model based on this dataset;
X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs
Sirnam Swetha (University of Central Florida), Mubarak Shah (Amazon)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelAuto EncoderContrastive LearningMultimodality
🎯 What it does: Proposed and implemented X-Former, a lightweight Transformer module for unifying contrastive learning (CLIP) and reconstruction learning (MAE) in multi-modal large language models (MLLM), to simultaneously capture global semantics and local detail information.
X-Pose: Detecting Any Keypoints
Jie Yang (IDEA), Lei Zhang (IDEA)
Pose EstimationTransformerPrompt EngineeringContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes the X-Pose framework, which can simultaneously detect arbitrary keypoints of multiple objects in complex scenes.
XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution
Qu Yunpeng, Chao Zhou (Kuaishou Technology)
Super ResolutionVision Language ModelDiffusion modelImageMultimodality
🎯 What it does: This paper proposes the XPSR framework, which utilizes cross-modal priors (high-level semantics and low-level degradation information) to drive diffusion models for image super-resolution, generating high-fidelity and realistic high-resolution images.
YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information
Chien-Yao Wang (Academia Sinica), Hong-Yuan Mark Liao (National Taipei University Of Technology)
Object DetectionComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper proposes Programmable Gradient Information (PGI) and Generalized Efficient Layer Aggregation Network (GELAN), integrating them into YOLOv9, aiming to significantly enhance the training effectiveness and inference performance of lightweight and large-scale object detection models by reprogramming gradient transmission through reversible branches and multi-level auxiliary information;
You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception
Sheng Jin (University of Hong Kong), Ping Luo (University of Hong Kong)
Object DetectionSegmentationPose EstimationTransformerContrastive LearningImageMesh
🎯 What it does: This paper proposes the HQNet framework, which achieves single-stage multi-task human-centric perception (detection, segmentation, pose, attribute recognition, and 3D mesh restoration) by learning a unified human query (Human Query) to share cross-task features;
You Only Need One Step: Fast Super-Resolution with Stable Diffusion via Scale Distillation
Mehdi Noroozi (Samsung AI Cambridge), Georgios Tzimiropoulos
Super ResolutionKnowledge DistillationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Propose YONOS-SR, a super-resolution model based on Stable Diffusion, achieving one-step inference through scale distillation.
Zero-Shot Adaptation for Approximate Posterior Sampling of Diffusion Models in Inverse Problems
Yasar U Alcalar, Mehmet Akcakaya
Diffusion modelScore-based ModelImagePhysics Related
🎯 What it does: Proposes the Zero-shot Approximate Posterior Sampling (ZAPS) method for efficient posterior sampling in inverse problems using diffusion models.
Zero-Shot Detection of AI-Generated Images
Davide Cozzolino (Università di Napoli Federico II), Luisa Verdoliva (Università di Napoli Federico II)
CompressionImage
🎯 What it does: This paper proposes an image compression framework based on a pixel-level hybrid density network.
Zero-Shot Image Feature Consensus with Deep Functional Maps
Xinle Cheng, Leonidas Guibas (Stanford University)
Graph Neural NetworkOptical FlowImage
🎯 What it does: This paper proposes a zero-shot method that utilizes functional mapping to fuse features from two major visual models to achieve dense image correspondence.
Zero-Shot Multi-Object Scene Completion
Shun Iwase (Carnegie Mellon University), Sergey Zakharov (Toyota Research Institute)
RestorationGenerationConvolutional Neural NetworkTransformerAuto EncoderImagePoint CloudMeshBenchmark
🎯 What it does: Propose a hybrid network called OctMAE, combining Octree U-Net with implicit 3D MAE, which can rapidly and accurately reconstruct 3D shapes of multi-object scenes, including occluded regions, using only RGB-D images and foreground masks as single-frame inputs.
Zero-shot Object Counting with Good Exemplars
Huilin Zhu (Wuhan University of Technology), Shengfeng He (Singapore Management University)
Object DetectionTransformerVision Language ModelContrastive LearningImage
🎯 What it does: Proposed the VA-Count framework, combining an example enhancement module (EEM) and a noise suppression module (NSM) to achieve zero-shot object counting.
Zero-shot Text-guided Infinite Image Synthesis with LLM guidance
Soyeong Kwon (UNIST), Taehwan Kim (UNIST)
GenerationData SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Propose a framework that utilizes large language models (LLM) for zero-shot text-guided infinite image outpainting, enabling the generation of images of arbitrary sizes without requiring high-resolution text-image paired datasets.
ZeroI2V: Zero-Cost Adaptation of Pre-Trained Transformers from Image to Video
Xinhao Li (State Key Laboratory for Novel Software Technology, Nanjing University), Limin Wang (State Key Laboratory for Novel Software Technology, Nanjing University)
RecognitionDomain AdaptationTransformerVideo
🎯 What it does: Propose a zero-cost method for migrating a pre-trained image Transformer to video tasks, named ZeroI2V, which utilizes spatiotemporal dual-head attention (STDHA) to achieve spatiotemporal modeling, employs a linear adapter for parameter-efficient transfer, and integrates it into the original model during inference through structural reparameterization, achieving no additional parameters or computations during inference.
ZeST: Zero-Shot Material Transfer from a Single Image
Ta-Ying Cheng (University of Oxford), Varun Jampani (Stability AI)
Image TranslationGenerationPrompt EngineeringDiffusion modelImageMesh
🎯 What it does: Implemented a zero-shot, single-image material transfer method called ZeST, which can transfer the appearance of any material sample to objects in the target image.
ZigMa: A DiT-style Zigzag Mamba Diffusion Model
Vincent Tao Hu (CompVis LMU Munich), Bjorn Ommer
GenerationData SynthesisTransformerDiffusion modelImageVideo
🎯 What it does: This paper proposes a Zigzag Mamba network based on Mamba to enhance the scalability and efficiency of diffusion models in high-resolution visual generation.
ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
Viraj Shah (Google Research), Varun Jampani (Google Research)
GenerationSupervised Fine-TuningDiffusion modelImage
🎯 What it does: Proposed a ZipLoRA method that effectively merges independently trained style and content LoRAs by optimizing the mixing coefficient, enabling the generation of any subject in any style.
ZoLA: Zero-Shot Creative Long Animation Generation with Short Video Model
Fu-Yun Wang (MMLab CUHK), Hongsheng Li (Shanghai AI Lab)
GenerationDiffusion modelScore-based ModelVideoStochastic Differential Equation
🎯 What it does: Propose a zero-shot method called ZoLA, which generates long animations (tens of seconds) using existing short video diffusion models, supporting diverse generation scenarios such as control guidance, editing, action customization, and multi-prompt switching.