ECCV 2024 Papers — Page 15
European Conference on Computer Vision · 2387 papers
Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling
Jaehyeok Kim (HKUST), Dan Xu (HKUST)
GenerationNeural Radiance FieldVideo
🎯 What it does: This paper proposes a motion-oriented composable neural radiance field (MoCo-NeRF) for free-viewpoint rendering of dynamically clothed humans from monocular videos, modeling non-rigid motion by learning radiance residual fields.
Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
Friedhelm Hamann, Kostas Daniilidis
SegmentationConvolutional Neural NetworkBiomedical Data
🎯 What it does: This paper proposes a medical image segmentation method based on multi-scale deep supervision and regularization loss.
MotionChain: Conversational Motion Controllers via Multimodal Prompts
Biao Jiang (Fudan University), Jiayuan Fan (Tencent)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelAuto EncoderImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposed the MotionChain framework, which utilizes multimodal prompts to achieve conversational control over continuous human motion, supporting tasks such as generation, editing, and translation.
MotionDirector: Motion Customization of Text-to-Video Diffusion Models
Rui Zhao (Show Lab), Mike Zheng Shou (Show Lab)
GenerationConvolutional Neural NetworkDiffusion modelVideoTextBenchmark
🎯 What it does: This paper proposes MotionDirector, which enables motion customization for text-to-video diffusion models, learning and generating specific motions while maintaining appearance diversity.
MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model
Wenxun Dai (Tsinghua University), Yansong Tang (Tsinghua University)
GenerationTransformerDiffusion modelAuto EncoderTextTime SeriesOrdinary Differential Equation
🎯 What it does: Propose MotionLCM, a real-time controllable motion generation framework based on latent consistency models; by distilling diffusion models in the motion latent space and incorporating ControlNet, achieve real-time (first or few-order sampling) generation of high-quality human motions under text and initial joint trajectory controls;
MoVideo: Motion-Aware Video Generation with Diffusion Models
Jingyun Liang (ETH Zurich), Rakesh Ranjan (Meta Inc)
GenerationDepth EstimationConvolutional Neural NetworkDiffusion modelGenerative Adversarial NetworkOptical FlowVideoTextMultimodality
🎯 What it does: Proposes the MoVideo framework, achieving text or image-to-video generation through depth and optical flow guided diffusion models.
MRSP: Learn Multi-Representations of Single Primitive for Compositional Zero-Shot Learning
Dongyao Jiang (Xi'an Jiaotong University), Nanning Zheng (Xi'an Jiaotong University)
ClassificationRepresentation LearningGraph Neural NetworkTransformerVision Language ModelContrastive LearningImage
🎯 What it does: Propose the MRSP framework, which enhances compositional zero-shot learning (CZSL) performance by constructing multiple representations of a single primitive, and employs a three-branch cross-attention decoder for concept decomposition in images.
MSD: A Benchmark Dataset for Floor Plan Generation of Building Complexes
Casper van Engelenburg (Delft University of Technology), Seyran Khademi (Delft University of Technology)
GenerationConvolutional Neural NetworkGraph Neural NetworkDiffusion modelImageGraphBenchmark
🎯 What it does: Constructed a large-scale multi-family residential floor plan dataset named Modified Swiss Dwellings (MSD), and conducted experimental evaluations on two baseline generative models (Modified HouseDiffusion and Graph-informed U-Net) based on this dataset;
MTA-CLIP: Language-Guided Semantic Segmentation with Mask-Text Alignment
Anurag Das (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)
SegmentationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageText
🎯 What it does: Propose the MTA-CLIP framework, transferring the semantic segmentation task from pixel-text alignment to mask-text alignment, and achieving vision-language representation learning based on masks.
MTaDCS: Moving Trace and Feature Density-based Confidence Sample Selection under Label Noise
Qingzheng Huang (Shenzhen University), Zitong Yu (Great Bay University)
ClassificationAnomaly DetectionConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Propose the MTaDCS method, which selects confident samples through movement trajectory and local feature density to address the confusion between noisy labels and difficult samples.
MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution
Yuxuan Jiang (University of Bristol), David Bull (University of Bristol)
Super ResolutionKnowledge DistillationTransformerImage
🎯 What it does: Propose a multi-teacher knowledge distillation framework called MTKD, which uses a knowledge aggregation network to fuse outputs from multiple teachers and guide a lightweight student network to learn image super-resolution.
MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders
Baijiong Lin (Hong Kong University of Science and Technology), Yingcong Chen (Hong Kong University of Science and Technology)
SegmentationDepth EstimationTransformerImage
🎯 What it does: Propose a Mamba-based multi-task dense scene understanding framework named MTMamba, which includes two core modules: Self-Task Mamba (STM) and Cross-Task Mamba (CTM), aiming to simultaneously accomplish dense prediction tasks such as semantic segmentation, depth estimation, and human parsing.
Multi-branch Collaborative Learning Network for 3D Visual Grounding
Zhipeng Qian (Xiamen University), Rongrong Ji (Xiamen University)
Object DetectionSegmentationConvolutional Neural NetworkTransformerPoint Cloud
🎯 What it does: Propose a multi-branch collaborative learning network (MCLN) that simultaneously handles the 3DREC and 3DRES tasks in 3D visual localization;
Multi-Granularity Sparse Relationship Matrix Prediction Network for End-to-End Scene Graph Generation
lei wang, Badong Chen (Xi'an Jiaotong University)
Object DetectionGenerationTransformerImageMultimodality
🎯 What it does: Designed an end-to-end scene graph generation method based on a multi-granularity sparse relationship matrix prediction network, eliminating the relation matching step and enhancing relation prediction by leveraging entity semantics and positional information.
Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot
Fabien Baradel (NAVER LABS Europe), Gregory Rogez
Pose EstimationDepth EstimationTransformerImageMesh
🎯 What it does: Propose Multi-HMR, a single-frame multi-human full-body 3D mesh recovery model based on Vision Transformer, which can simultaneously detect multiple people, regress SMPL-X parameters, predict 3D positions, and optionally utilize camera intrinsics from a single RGB image.
Multi-Label Cluster Discrimination for Visual Representation Learning
Xiang An (DeepGlint), Jiankang Deng (Huawei Noah's Ark Lab)
RetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: This paper proposes a multi-label clustering discriminative (MLCD) framework to enhance the semantic structural representation of visual representation learning.
Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification
Jiangming Shi (Xiamen University), Yanyun Qu (Xiamen University)
RecognitionRetrievalDomain AdaptationContrastive LearningImageMultimodality
🎯 What it does: Proposes a Multi-Memory Matching (MMM) framework for unsupervised visible-infrared person re-identification, generating high-quality pseudo labels and establishing reliable cross-modal correspondences through cross-modal clustering, multi-memory learning and matching, and soft clustering-level alignment.
Multi-modal Crowd Counting via a Broker Modality
Haoliang Meng (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
Object DetectionKnowledge DistillationConvolutional Neural NetworkTransformerImageMultimodality
🎯 What it does: Propose an intermediate 'broker' modality inserted between visual and thermal imaging, reformulating the dual-modal crowd counting task into a tri-modal learning framework. A lightweight non-diffusion fusion module generates this intermediate modality, followed by joint density estimation using tri-modal features.
Multi-modal Relation Distillation for Unified 3D Representation Learning
Huiqun Wang (Beihang University), Di Huang (Beihang University)
RetrievalKnowledge DistillationRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextPoint Cloud
🎯 What it does: Propose a multimodal relational distillation framework MRD to unify 3D point cloud representation learning, and distill internal and cross-modal relationships among images, text, and point clouds within 3D models.
Multi-Modal Video Dialog State Tracking in the Wild
Adnen Abdessaied (University of Stuttgart), Andreas Bulling (University of Stuttgart)
Convolutional Neural NetworkGraph Neural NetworkTransformerVideoTextMultimodalityAudio
🎯 What it does: Proposed the MST MIXER model for achieving multi-modal dialogue state tracking in real-world video dialogue tasks.
Multi-Person Pose Forecasting with Individual Interaction Perceptron and Prior Learning
Peng Xiao (South China University of Technology), Huaidong Zhang (South China University of Technology)
Pose EstimationGraph Neural NetworkTransformerSequential
🎯 What it does: This paper proposes a multi-person human pose prediction framework called IAFormer, which integrates individual and interaction information.
Multi-RoI Human Mesh Recovery with Camera Consistency and Contrastive Losses
Yongwei Nie (South China University of Technology), Hongmin Cai (South China University of Technology)
Pose EstimationConvolutional Neural NetworkContrastive LearningImageMesh
🎯 What it does: This paper proposes a multi-RoI human mesh recovery method that simultaneously estimates the global 3D mesh and local cameras using multiple local views, and improves accuracy through camera consistency loss and contrastive loss.
Multi-scale Cross Distillation for Object Detection in Aerial Images
Kun Wang (National University of Defense Technology), Yang Li (National University of Defense Technology)
Object DetectionConvolutional Neural NetworkImageBenchmark
🎯 What it does: Propose a multi-scale cross distillation (MSCD) framework that utilizes a shared-parameter multi-branch network and an adaptive cross-scale distillation module to achieve target detection performance comparable to or better than multi-scale inference under single-scale inference.
Multi-Sentence Grounding for Long-term Instructional Video
Zeqian Li (Coop Medianet Innovation Center Shanghai Jiao Tong University), Weidi Xie (Coop Medianet Innovation Center Shanghai Jiao Tong University)
Data SynthesisRepresentation LearningTransformerLarge Language ModelContrastive LearningVideoText
🎯 What it does: This paper constructs an automated, scalable workflow for cleaning large-scale educational video data and generating a high-quality, multi-step aligned video-text dataset named HowToStep.
Multi-Task Domain Adaptation for Language Grounding with 3D Objects
Penglei Sun (Hong Kong University of Science and Technology), Xiaowen Chu (Hong Kong University of Science and Technology)
Domain AdaptationTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: This paper proposes a multi-task domain adaptation framework named DA4LG, aiming to enhance 3D object language localization performance through visual domain adapters and multi-task learning.
MultiDelete for Multimodal Machine Unlearning
Jiali Cheng (University of Massachusetts Lowell), Hadi Amiri (University of Massachusetts Lowell)
Safty and PrivacyTransformerMultimodality
🎯 What it does: To address the forgetting task in multimodal machine learning models, this paper proposes the MultiDelete method, which can forget the multimodal associations of specified samples without retraining the entire model, while maintaining the original functionality;
MultiGen: Zero-shot Image Generation from Multi-modal Prompts
Zhi-Fan Wu (Alibaba Group), Yu Liu (Alibaba Group)
GenerationTransformerPrompt EngineeringDiffusion modelMultimodality
🎯 What it does: Designed and implemented the MultiGen model, which integrates text, coordinates, and image multimodal information into a latent diffusion model through object-level enhanced tokens, achieving zero-shot multi-object multimodal image generation.
Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognition
Masashi Hatano (Keio University), Hideo Saito (Keio University)
RecognitionDomain AdaptationKnowledge DistillationMeta LearningTransformerContrastive LearningVideoMultimodality
🎯 What it does: Propose a multi-modal cross-domain few-shot learning framework, MM-CDFSL, for first-person perspective action recognition.
Multimodal Label Relevance Ranking via Reinforcement Learning
Taian Guo (Tencent Youtu Lab), Xing Sun (Tencent Youtu Lab)
Domain AdaptationReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: Propose a reinforcement learning-based multimodal tag relevance ranking method called LR PPO, and construct a new dataset specifically for this task named LRMovieNet; this method can train a base model on the source domain and quickly transfer to the target domain with only a small amount of preference annotations.
Multiscale Graph Texture Network
Ravishankar Evani (Nanyang Technological University), Shangbo Mao (Nanyang Technological University)
RecognitionConvolutional Neural NetworkGraph Neural NetworkImageGraph
🎯 What it does: Propose the Graph Texture Network (GTN), which maps multi-layer convolutional features into a graph structure and achieves adaptive fusion of latent texture attributes through a learnable undirected masked adjacency matrix and residual message passing;
Multiscale Sliced Wasserstein Distances as Perceptual Color Difference Measures
Jiaqi He (City University of Hong Kong), Kede Ma
Image
🎯 What it does: Propose an image color difference evaluation method based on multi-scale sliced Wasserstein distance (MS-SWD), which uses non-local patch matching for potentially mismatched photographic images to assess color differences.
Multistain Pretraining for Slide Representation Learning in Pathology
Guillaume Jaume (Harvard Medical School), Faisal Mahmood (Mass General Brigham)
ClassificationRepresentation LearningTransformerContrastive LearningMultimodalityBiomedical Data
🎯 What it does: This paper proposes Madeleine, a multi-modal self-supervised pre-training framework based on multiple staining (e.g., H&E and IHC), designed to learn general representations for whole slide images (WSI).
MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty
Tim Broedermann, Luc Van Gool (ETH Zürich)
SegmentationAutonomous DrivingConvolutional Neural NetworkImageMultimodalityPoint CloudBenchmark
🎯 What it does: This paper presents and makes public the MUSES dataset, a multi-sensor (RGB camera, radar, LiDAR, event camera, IMU/GNSS) driving scenario dataset, along with 2500 panoptic images annotated with 2D semantics, instances, and difficulty labels. The authors designed a two-stage annotation process, leveraging auxiliary sensors to enhance annotation coverage and accuracy, and introduced a multi-dimensional uncertainty annotation based on pixel-level difficulty, thereby defining a new task called uncertainty-aware panoptic segmentation. Finally, the paper proposes the UPQ (Uncertainty-Aware Panoptic Quality) evaluation metric and conducts benchmark experiments on three tasks (semantic segmentation, panoptic segmentation, UPQ segmentation) across both monomodal and multimodal tracks.
MutDet: Mutually Optimizing Pre-training for Remote Sensing Object Detection
Ziyue Huang (Beihang University), Yunhong Wang (Beihang University)
Object DetectionTransformerContrastive LearningImage
🎯 What it does: This paper proposes MutDet, a mutual optimization pre-training framework for orientation object detection in remote sensing images, aiming to address the feature mismatch problem caused by differences in feature extraction methods in traditional DETR pre-training.
Mutual Learning for Acoustic Matching and Dereverberation via Visual Scene-driven Diffusion
Jian Ma (Southern University of Science and Technology), Feng Zheng (Zhejiang University)
RestorationRetrievalConvolutional Neural NetworkDiffusion modelMultimodality
🎯 What it does: Propose a mutual learning framework named MVSD based on diffusion models, achieving bidirectional collaborative training for visual-audio matching (VAM) and dereverberation.
MVDD: Multi-View Depth Diffusion Models
Zhen Wang (Google), Yinda Zhang (Google)
GenerationDiffusion modelPoint CloudMesh
🎯 What it does: Propose a multi-view depth map-based diffusion model called MVDD to generate high-quality 3D shapes with more than 20K points and further recover them into meshes.
MVDiffHD: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction
Shitao Tang (Simon Fraser University), Rakesh Ranjan (Meta Reality Labs)
GenerationDiffusion modelAuto EncoderImageMesh
🎯 What it does: Propose MVDiffusion++, a diffusion model that can generate dense high-resolution multi-view images under single-view or sparse-view conditions without camera poses, and use them for 3D reconstruction.
MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views
Wangze Xu (Peking University), Ronggang Wang (Peking University)
GenerationTransformerGaussian SplattingImage
🎯 What it does: This paper proposes a few-view novel view synthesis method called MVPGS based on 3D Gaussian splatting, achieving high-quality real-time rendering through MVS priors, forward projection appearance constraints, and geometric regularization.
MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo
Tianqi Liu (Huazhong University of Science and Technology), Ziwei Liu (Nanyang Technological University)
GenerationDepth EstimationComputational EfficiencyConvolutional Neural NetworkGaussian SplattingImagePoint Cloud
🎯 What it does: Propose a general Gaussian splatting method called MVSGaussian based on multi-view stereo vision (MVS), which can rapidly construct 3D Gaussian point clouds and achieve real-time view synthesis with only a few input images, while supporting fast fine-tuning for individual scenes through geometric consistency aggregation.
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
Yuedong Chen (Monash University), Jianfei Cai (Monash University)
GenerationComputational EfficiencyTransformerGaussian SplattingImageBenchmark
🎯 What it does: Propose an efficient multi-view sparse image to 3D Gaussian distribution inference model, MVSplat, which achieves 3D Gaussian representation and novel view rendering with a single forward pass.
MyVLM: Personalizing VLMs for User-Specific Queries
Yuval Alaluf, Danny Cohen-Or
SegmentationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an attention-based semantic segmentation framework that can automatically learn category embeddings and perform precise pixel segmentation during training.
N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fields
Yash Bhalgat (University of Oxford), Andrea Vedaldi (University of Oxford)
SegmentationVision Language ModelGaussian SplattingTextPoint Cloud
🎯 What it does: Proposes the N2F2 (Nested Neural Feature Fields) technique, which learns a single high-dimensional feature field using hierarchical supervision, enabling different dimensions to correspond to different scales and semantic granularities, thereby achieving multi-scale, open-vocabulary 3D scene segmentation and localization;
NAMER: Non-Autoregressive Modeling for Handwritten Mathematical Expression Recognition
Chenyu Liu (University of Science and Technology of China), Qingfeng Liu (University of Science and Technology of China)
RecognitionConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a non-autoregressive (Non-Autoregressive) model-based framework for Handwritten Mathematical Expression Recognition (HMER) called NAMER. It first uses a Visual Word Tokenizer (VAT) to parallelly recognize visible symbols and local relationships, then employs a Parallel Graph Decoder (PGD) to correct symbols and predict their connectivity, ultimately constructing a directed acyclic graph (DAG) to obtain a LaTeX expression.
NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models
Gengze Zhou (University of Adelaide), Qi Wu (University of Adelaide)
Autonomous DrivingExplainability and InterpretabilityGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: Develop NavGPT-2 by integrating a frozen large vision-language model with topological graph navigation strategies, enabling interpretable navigation reasoning and interaction in visual environments.
Navigating Text-to-Image Generative Bias across Indic Languages
Surbhi Mittal (IIT Jodhpur), Tal Hassner (Weir P.B.C.)
GenerationDiffusion modelImageTextBenchmark
🎯 What it does: This study constructs the IndicTTI benchmark to evaluate bias and cultural adaptability of 30 Indian languages in text-to-image generation models.
Navigation Instruction Generation with BEV Perception and Large Language Models
Sheng Fan (Zhejiang University), Yi Yang (Zhejiang University)
Autonomous DrivingTransformerLarge Language ModelPrompt EngineeringVision-Language-Action ModelImageTextMultimodality
🎯 What it does: Developed a navigation instruction generation framework called BEVInstructor, which integrates BEV perception and large language models (LLM) to generate natural language navigation instructions by fusing multi-view images with bird's-eye-view (BEV) features.
NePhi: Neural Deformation Fields for Approximately Diffeomorphic Medical Image Registration
Lin Tian (UNC Chapel Hill), Marc Niethammer (UNC Chapel Hill)
OptimizationComputational EfficiencyConvolutional Neural NetworkBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Propose a MLP-based implicit neural deformation field called NePhi, which can achieve approximate diffeomorphic medical image registration with low memory consumption, and balance speed and accuracy through a hybrid multi-resolution strategy.
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
Muhammad Zubair Irshad (Toyota Research Institute), Rares Ambrus (Toyota Research Institute)
Object DetectionSegmentationRepresentation LearningConvolutional Neural NetworkTransformerNeural Radiance FieldAuto EncoderImage
🎯 What it does: Using an occlusion autoencoder on NeRF voxel grids for self-supervised pre-training to learn high-quality 3D representations and improve downstream task performance.
NeRF-XL: NeRF at Any Scale with Multi-GPU
Ruilong Li (NVIDIA), Francis Williams (UC Berkeley)
GenerationOptimizationComputational EfficiencyNeural Radiance FieldImage
🎯 What it does: Proposes NeRF-XL, capable of distributed training and rendering arbitrarily large-scale NeRF models on multi-GPU setups.
NeRMo: Learning Implicit Neural Representations for 3D Human Motion Prediction
Dong Wei (Nanjing University of Science and Technology), Shengxiang Hu (Nanjing University of Science and Technology)
Pose EstimationNeural Radiance FieldTime SeriesSequential
🎯 What it does: Propose NeRMo, a continuous-time 3D human motion prediction framework based on implicit neural representations;
Neural graphics texture compression supporting random access
Farzad Farhadzadeh (Qualcomm AI Research), Fatih Porikli (Qualcomm AI Research)
CompressionTransformerAuto EncoderImage
🎯 What it does: Proposes a neural texture compression framework that supports random access, enabling real-time decoding of multi-channel textures at any MIP level and texture coordinates.
Neural Metamorphosis
Xingyi Yang (National University of Singapore), Xinchao Wang (National University of Singapore)
ClassificationSegmentationGenerationImage
🎯 What it does: This paper proposes Neural Metamorphosis (NeuMeta), which generates a continuous weight manifold by training a neural implicit function (INR), enabling a single training process to generate weights for networks of any size.
Neural Poisson Solver: A Universal and Continuous Framework for Natural Signal Blending
Delong Wu (Nanjing University), Zhan Ma (Nanjing University)
Image HarmonizationRepresentation LearningNeural Radiance FieldImagePhysics Related
🎯 What it does: This paper proposes a neural Poisson solver without boundary conditions, which can directly solve the continuous Poisson equation on implicit neural representations (INR), achieving multi-dimensional natural signal fusion.
Neural Spectral Decomposition for Dataset Distillation
Shaolei Yang (Xi'an Jiaotong University), Shuaicheng Liu (Megvii Technology)
ClassificationKnowledge DistillationImage
🎯 What it does: Propose a dataset distillation framework based on neural spectral decomposition, decomposing the entire dataset into spectral tensors and transformation matrices, achieving efficient distillation through low-rank representation and information sharing.
Neural Surface Detection for Unsigned Distance Fields
Federico Stella (École Polytechnique Fédérale de Lausanne), Pascal Fua (École Polytechnique Fédérale de Lausanne)
SegmentationComputational EfficiencyAuto EncoderMesh
🎯 What it does: This paper proposes a local discriminative method based on neural networks, converting unsigned distance field (UDF) into pseudo signed distance field (pseudo-SDF), followed by using traditional triangulation algorithms (such as Marching Cubes or Dual Contouring) to obtain high-quality open surface meshes.
Neural Volumetric World Models for Autonomous Driving
Zanming Huang (Boston University), Eshed Ohn-Bar (Boston University)
Autonomous DrivingTransformerWorld ModelImage
🎯 What it does: Propose the NeMo framework based on voxels, achieving a self-supervised learning 3D world model for end-to-end decision making in autonomous driving.
NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Driving
William Ljungbergh (Zenseact), Christoffer Petersson (Delft University of Technology)
Autonomous DrivingNeural Radiance FieldVideoPoint Cloud
🎯 What it does: Propose a NeRF-based closed-loop simulation framework called NeuroNCAP to test the performance of end-to-end autonomous driving models on photorealistic sensor data in safety-critical scenarios; by rendering nuScenes logs and dynamically editing actors, construct Euro NCAP-inspired three categories of collision scenarios (stationary, forward, lateral) and conduct multiple closed-loop evaluations.
NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation
Jingyang Huo (Fudan University), Jianfeng Feng
RestorationTransformerVision Language ModelDiffusion modelImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: Propose NeuroPictor, a framework that reconstructs images from fMRI through multi-agent pre-training and multi-level modulation, enabling precise control over low-level structures and high-level semantics.
NGP-RT: Fusing Multi-Level Hash Features with Lightweight Attention for Real-Time Novel View Synthesis
Yubin Hu (Tsinghua University), Yong-Jin Liu (Tencent Games)
GenerationComputational EfficiencyNeural Radiance FieldImage
🎯 What it does: Leverage the multi-level hash features of Instant-NGP combined with a lightweight attention mechanism to achieve real-time novel view rendering; simultaneously introduce an occupancy distance grid to optimize ray stepping.
Nickel and Diming Your GAN: A Dual-Method Approach to Enhancing GAN Efficiency via Knowledge Distillation
Sangyeop Yeo (Ulsan National Institute of Science and Technology), Jaejun Yoo (Ulsan National Institute of Science and Technology)
GenerationComputational EfficiencyKnowledge DistillationGenerative Adversarial NetworkImage
🎯 What it does: This study proposes two novel GAN compression methods: DiME (utilizing a base model as a kernel for distribution matching) and NICKEL (achieving network interaction compression through knowledge exchange between the generator and discriminator), significantly reducing the parameters and FLOPs of StyleGAN2 on datasets such as FFHQ.
NICP: Neural ICP for 3D Human Registration at Scale
Riccardo Marin (University of Tübingen), Gerard Pons-Moll (University of Tübingen)
Pose EstimationNeural Radiance FieldPoint Cloud
🎯 What it does: Propose a neural scalable registration (NSR) workflow that combines local neural fields (LoVD) and self-supervised neural ICP (NICP) to achieve high-precision, global registration of large-scale 3D human point clouds.
NL2Contact: Natural Language Guided 3D Hand-Object Contact Modeling with Diffusion Model
Zhongqun Zhang (National University of Singapore), Hyung Jin Chang (University of Birmingham)
GenerationConvolutional Neural NetworkTransformerVision Language ModelDiffusion modelTextPoint Cloud
🎯 What it does: Studies how to control 3D hand-object contact models through natural language descriptions to generate corresponding hand poses and contact maps.
Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
Qinyu Yang, Ying Shan (Tencent AI Lab)
RestorationDiffusion modelScore-based ModelVideoStochastic Differential Equation
🎯 What it does: By performing 1–3 iterations of noise calibration on random noise, combined with the pre-trained diffusion model SDEdit, an enhancement method is achieved that improves video quality while maintaining video content consistency.
Noise-assisted Prompt Learning for Image Forgery Detection and Localization
Dong Li (University of Science and Technology of China), Zheng-Jun Zha (University of Science and Technology of China)
Anomaly DetectionTransformerPrompt EngineeringContrastive LearningImageMultimodality
🎯 What it does: Propose a CLIP-based image forgery detection and localization framework called CLIP-IFDL, combining noise-assisted prompt learning with forgery-enhanced noise adapters to achieve high-precision detection and localization.
Non-Exemplar Domain Incremental Learning via Cross-Domain Concept Integration
Qiang Wang (Xi'an Jiaotong University), Yihong Gong (Xi'an Jiaotong University)
ClassificationDomain AdaptationTransformerPrompt EngineeringImageBenchmark
🎯 What it does: Propose a non-sample domain incremental learning method called PINA, achieving cross-domain concept integration through a unified classifier, domain-specific alignment module, and patch shuffling selector.
Non-Line-of-Sight Estimation of Fast Human Motion with Slow Scanning Imagers
Javier Grau Chopite, Matthias B Hullin
RestorationBiomedical DataUltrasound
🎯 What it does: Proposed a novel ultrasound image reconstruction method based on three-dimensional Bézier interpolation and weighted linear combination
Non-parametric Sensor Noise Modeling and Synthesis
Ali Mosleh (Samsung AI Center), Michael S Brown
RestorationData SynthesisImage
🎯 What it does: Proposes a non-parametric camera sensor noise modeling and synthesis method that directly constructs the probability mass function (PMF) for each pixel intensity from images and performs noise synthesis.
Non-transferable Pruning
Ruyi Ding (Northeastern University), Yunsi Fei (Northeastern University)
Safty and PrivacyContrastive LearningImage
🎯 What it does: By performing target-domain-specific sparse pruning on pre-trained models to reduce their transferability performance on unauthorized domains, thereby achieving controllable intellectual property authorization.
Nonverbal Interaction Detection
Jianan Wei (Zhejiang University), Wenguan Wang (Zhejiang University)
Object DetectionGraph Neural NetworkTransformerImageMultimodalityBenchmark
🎯 What it does: Constructed a large-scale non-verbal interaction dataset called NVI, proposed the NVI-DET task, and designed a detection model named NVI-DEHR based on a dual-scale hypergraph.
Norface: Improving Facial Expression Analysis by Identity Normalization
Hanwei Liu (Tongji University), Yu Ding (Hebei Agricultural University)
ClassificationRecognitionMixture of ExpertsAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: Proposes the Norface framework, which utilizes identity normalization and Mixture of Experts to unify AU detection, AU intensity estimation, and FER tasks.
Norma: A Noise Robust Memory-Augmented Framework for Whole Slide Image Classification
Yu Bai (Beijing University of Posts and Telecommunications), Wendong Wang (Beijing University of Posts and Telecommunications)
ClassificationTransformerContrastive LearningImageBiomedical Data
🎯 What it does: Proposes a Whole Slide Image classification framework called Norma, based on serialized and memory-enhanced Vision Transformer, and enhances robustness through cyclic training and noise label detection.
NOVUM: Neural Object Volumes for Robust Object Classification
Artur Jesslen (University of Freiburg), Adam Kortylewski (University of Freiburg)
ClassificationPose EstimationConvolutional Neural NetworkContrastive LearningGaussian SplattingImage
🎯 What it does: Proposes the NOVUM framework that embeds 3D composable object volumes into deep networks for image classification.
nuCraft: Crafting High Resolution 3D Semantic Occupancy for Unified 3D Scene Understanding
Benjin Zhu (MMLab, Chinese University of Hong Kong), Hongsheng Li (SenseTime Research)
Object DetectionSegmentationData SynthesisPose EstimationAutonomous DrivingAuto EncoderPoint Cloud
🎯 What it does: Constructed a high-resolution, accurate 3D semantic occupancy dataset called nuCraft, and proposed the VQ-Occ method to achieve high-resolution occupancy prediction.
Nuvo: Neural UV Mapping for Unruly 3D Representations
Pratul Srinivasan (Google), Ben Mildenhall (Google)
GenerationNeural Radiance FieldGaussian SplattingMesh
🎯 What it does: Propose a UV mapping method based on neural fields (Nuvo), which can generate editable, low-fragmentation texture atlases for irregular geometries such as NeRF and text-to-3D models.
NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Image
Yoonwoo Jeong (POSTECH), Doyup Lee (Runway)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelScore-based ModelContrastive LearningImageMesh
🎯 What it does: Proposed a pluggable NVS-Adapter module that generates multi-view images from a single image without fine-tuning pre-trained text-image models, leveraging perspective consistency cross-attention and global semantic modulation.
Nymeria: A Massive Collection of Egocentric Multi-modal Human Motion in the Wild
Lingni Ma (Meta Reality Labs Research), Richard Newcombe (Meta Reality Labs Research)
Pose EstimationSimultaneous Localization and MappingImageMultimodalityTime SeriesBenchmark
🎯 What it does: In the paper, the authors constructed a large-scale, multimodal self-perspective daily action dataset named Nymeria, integrating data from full-body motion capture, AR glasses, wristbands, and other devices, along with hierarchical natural language descriptions.
O2V-Mapping: Online Open-Vocabulary Mapping with Neural Implicit Representation
Muer Tie (Fudan University), Wenchao Ding (Fudan University)
SegmentationComputational EfficiencyRepresentation LearningLarge Language ModelVision Language ModelNeural Radiance FieldSimultaneous Localization and MappingImage
🎯 What it does: Developed an online open vocabulary mapping framework called O2V-mapping, which utilizes voxel neural implicit representations to real-time construct open vocabulary semantic scenes from RGBD streams.
OAPT: Offset-Aware Partition Transformer for Double JPEG Artifacts Removal
Qiao Mo (Kuaishou Technology), Shuyuan Zhu (University of Electronic Science and Technology of China)
RestorationConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose an Offset-Aware Partition Transformer (OAPT) for removing artifacts in double JPEG compressed images.
OAT: Object-Level Attention Transformer for Gaze Scanpath Prediction
Yini Fang (Hong Kong University of Science and Technology), Bertram E Shi
TransformerImage
🎯 What it does: This paper proposes an Object-Level Attention Network (OAT) based on Transformer for predicting human eye movement scan paths in visual search tasks, modeling scan paths as a sequence of object fixations rather than pixel-level fixations.
Object-Aware NIR-to-Visible Translation
Yunyi Gao, Ying Fu (Beijing Institute Of Technology)
Image TranslationTransformerGenerative Adversarial NetworkImage
🎯 What it does: Propose an object-aware NIR-to-visible image conversion framework and collect a large-scale fully aligned FANVID dataset.
Object-Aware Query Perturbation for Cross-Modal Image-Text Retrieval
Naoya Sogi (NEC Corporation), Makoto Terao (NEC Corporation)
Object DetectionRetrievalVision Language ModelMultimodality
🎯 What it does: Designed an object-aware perturbation for queries in pre-trained vision-language models to enhance the retrieval capability of small objects in images.
Object-Centric Diffusion for Efficient Video Editing
Kumara Kahatapitiya (Qualcomm AI Research), Amirhossein Habibian (Qualcomm AI Research)
GenerationComputational EfficiencyDiffusion modelVideo
🎯 What it does: Proposes an Object-Centric Diffusion (OCD) framework specifically for video editing tasks, significantly improving efficiency through foreground-object-driven sampling and token merging.
Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Models
Yasi Zhang (University of California, Los Angeles), Ying Nian Wu (University of California, Los Angeles)
GenerationTransformerDiffusion modelImageTextMultimodality
🎯 What it does: Propose an energy-based alignment (EBAMA) method for optimizing attention maps in text-guided diffusion models during inference, along with corresponding object-level attribute binding loss and strength regularization terms.
Object-Oriented Anchoring and Modal Alignment in Multimodal Learning
Shibin Mei, BiLian Ke
RetrievalRepresentation LearningTransformerContrastive LearningMultimodality
🎯 What it does: This paper proposes a method for image-text alignment in dual-stream multimodal pre-training using object-oriented anchors, achieving fine-grained alignment through optimal transport and multi-perspective text mining.
ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion
Daniel Winter (Google Research), Yedid Hoshen (Google Research)
Image TranslationImage HarmonizationGenerationData SynthesisSupervised Fine-TuningDiffusion modelImage
🎯 What it does: Propose a training method for object removal and insertion in images by utilizing physically collected counterfactual image pairs.
OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving
Guoqing Wang (Shanghai Jiao Tong University), Chao Ma (Shanghai Jiao Tong University)
GenerationAutonomous DrivingConvolutional Neural NetworkTransformerDiffusion modelMultimodalityPoint Cloud
🎯 What it does: Proposes OccGen, a multimodal 3D semantic occupancy prediction framework based on diffusion models, which can gradually generate detailed 3D occupancy maps from random noise.
Occluded Gait Recognition with Mixture of Experts: An Action Detection Perspective
Panjian Huang (Beijing Normal University), Yongzhen Huang (Beijing Normal University)
RecognitionConvolutional Neural NetworkMixture of ExpertsContrastive LearningVideo
🎯 What it does: To address gait recognition in occluded environments, we propose a hybrid expert model called GaitMoE based on action detection, and create the Occluded Gait (OccGait) database.
Occlusion Handling in 3D Human Pose Estimation with Perturbed Positional Encoding
Niloofar Azizi (Graz University of Technology), Horst Bischof (Microsoft)
Pose EstimationGraph Neural NetworkVideo
🎯 What it does: Studied a pose encoding method called PerturbPE for addressing occlusion in 3D human pose estimation, which uses Rayleigh-Schrodinger series theory to repeatedly perturb and average the graph Laplacian matrix to extract stable spectral features and inject them into a GCN network;
Occlusion-Aware Seamless Segmentation
Yihong Cao (Hunan University), Kailun Yang (Hunan University)
SegmentationDomain AdaptationAutonomous DrivingTransformerImageBenchmark
🎯 What it does: Propose the Occlusion-Aware Seamless Segmentation (OASS) task, construct the UnmaskFormer model to achieve multi-task segmentation (semantic, instance, amodal) on panoramic images, and address three major challenges—field-of-view extension, occlusion, and domain difference—via Unmasking Attention (UA) and Amodal-oriented Mix (AoMix).
Occupancy as Set of Points
Yiang Shi (Huazhong University of Science & Technology), Xinggang Wang (Huazhong University of Science & Technology)
Autonomous DrivingConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose a 3D occupancy prediction framework called Occupancy as Set of Points (OSP) based on point interest points (PoI), which uses sparse points instead of traditional voxel grids for sampling and prediction.
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
Wenzhao Zheng (Tsinghua University), Jiwen Lu (Tsinghua University)
Autonomous DrivingTransformerAuto EncoderWorld ModelImagePoint Cloud
🎯 What it does: This paper constructs a world model called OccWorld based on 3D occupied space, used for simultaneously predicting future 4D occupancy information and the trajectory of autonomous vehicles.
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Jingkang Yang (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
Robotic IntelligenceAI Code AssistantTransformerLarge Language ModelReinforcement LearningVision Language ModelVision-Language-Action ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes an embodied vision-language programmer called Octopus, which can convert natural language task descriptions into executable code and perform dynamic planning and execution through visual perception; meanwhile, it designs three simulation environments (OctoGibson, OctoMC, OctoGTA) named OctoVerse for training and evaluation; it generates training data automatically using GPT-4 and achieves adaptive reinforcement learning by combining it with RLEF (Reinforcement Learning with Environmental Feedback); finally, it demonstrates excellent task completion rates and code executability in the three environments.
OGNI-DC: Robust Depth Completion with Optimization-Guided Neural Iterations
Yiming Zuo (Princeton University), Jia Deng (Princeton University)
Depth EstimationOptimizationConvolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: Propose the OGNI-DC framework, which achieves sparse depth completion through differentiable optimization-guided neural iteration, using depth gradients as the learning objective and leveraging DDI integration to obtain complete depth maps.
OLAF: A Plug-and-Play Framework for Enhanced Multi-object Multi-part Scene Parsing
Pranav Gupta (IIIT Hyderabad), Ravi Kiran Sarvadevabhatla (Google Research)
SegmentationConvolutional Neural NetworkTransformerPrompt EngineeringImage
🎯 What it does: Proposed a plug-and-play framework called OLAF, which enhances multi-object, multi-part scene segmentation performance by incorporating foreground/background and edge masks into RGB input, along with a low-level dense feature guidance module.
OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models
Zhe Kong, Wenhan Luo (Shenzhen University)
GenerationData SynthesisDiffusion modelScore-based ModelImage
🎯 What it does: Propose a two-phase sampling framework named OMG, specifically designed to address identity preservation, occlusion conflicts, and foreground-background lighting inconsistency issues in multi-concept personalized generation;
Omni-Recon: Harnessing Image-based Rendering for General-Purpose Neural Radiance Fields
Yonggan Fu (Georgia Institute of Technology), Yingyan (Celine) Lin (Georgia Institute of Technology)
GenerationData SynthesisTransformerDiffusion modelNeural Radiance FieldImage
🎯 What it does: Propose the Omni-Recon framework, constructing a general NeRF model to achieve real-time 3D reconstruction, zero-shot field multi-task scene understanding, and rapid adaptation to downstream applications such as real-time rendering and text-guided editing.
Omni6D: Large-Vocabulary 3D Object Dataset for Category-Level 6D Object Pose Estimation
Mengchen Zhang (Zhejiang University), Dahua Lin (Shanghai Artificial Intelligence Laboratory)
Pose EstimationSupervised Fine-TuningImageBenchmark
🎯 What it does: This study proposes Omni6D—a large vocabulary 3D object dataset for category-level 6D pose estimation—and conducts systematic evaluations of multiple existing methods on this dataset; it also introduces rotation symmetry adaptive evaluation metrics and a Fine-tune strategy.
Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Tracking
Jiyao Zhang (Peking University), Hao Dong (Peking University)
Data SynthesisPose EstimationConvolutional Neural NetworkTransformerDiffusion modelImageMeshBenchmark
🎯 What it does: Constructed a large-scale, category-rich 6D pose estimation dataset called Omni6DPose, and proposed an improved GenPose++ model for category-level pose estimation and tracking.
OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web
Raghav Kapoor (Carnegie Mellon University), Ruslan Salakhutdinov (Writer.com)
TransformerLarge Language ModelSupervised Fine-TuningAgentic AIVision Language ModelVision-Language-Action ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the OmniACT dataset and benchmark for evaluating the execution capabilities of multi-modal general autonomous agents on desktop and web applications. The dataset contains 9,800 pairs of screenshots and natural language tasks, along with corresponding executable PyAutoGUI scripts.
OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objects
Akshay Krishnan (Google Research), Matthew Brown (Google Research)
Pose EstimationTransformerImagePoint Cloud
🎯 What it does: Propose the OmniNOCS dataset and the NOCSformer model, unifying the prediction of 3D position, pose, and shape from 2D detection boxes.