ECCV 2024 Papers — Page 13
European Conference on Computer Vision · 2387 papers
Learning to Build by Building Your Own Instructions
Aaron T Walsman (University of Washington), Dieter Fox (University of Washington)
Robotic IntelligenceTransformerReinforcement LearningVision-Language-Action ModelImageTextSequential
🎯 What it does: Designed and implemented InstructioNet, which utilizes self-generated instruction stacks to solve the LEGO structure Break-and-Make task.
Learning to Complement and to Defer to Multiple Users
Zheng Zhang (University of Surrey), Gustavo Carneiro (University of Surrey)
ClassificationImageBiomedical Data
🎯 What it does: Proposes a unified human-machine collaborative classification framework named LECODU, integrating learning compensation, learning delay, and optimal user number decision-making, and achieving dual optimization of accuracy and collaboration cost through multi-noise label training.
Learning to Detect Multi-class Anomalies with Just One Normal Image Prompt
Bin-Bin Gao (Tencent YouTu Lab)
Anomaly DetectionTransformerPrompt EngineeringAuto EncoderImage
🎯 What it does: Designed a unified anomaly detection framework called OneNIP based on a single normal image prompt, utilizing self-attention reconstruction and cross-attention recovery, combined with supervised refinement for pixel-level anomaly segmentation.
Learning to Distinguish Samples for Generalized Category Discovery
Fengxiang Yang (Xiamen University), Zhun Zhong (University of Trento)
ClassificationRepresentation LearningGraph Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes the NGCN model based on neighbor information and the cross-view consistency strategy CVCS, improving pseudo-label generation and noise filtering in the general category discovery task.
Learning to Drive via Asymmetric Self-Play
Chris Zhang (Waabi), Raquel Urtasun (Waabi)
Autonomous DrivingTransformerReinforcement LearningTime Series
🎯 What it does: This paper proposes an asymmetric self-play framework, where a teacher first demonstrates safe paths to all vehicles in a multi-agent traffic simulation, then interacts with a student, forcing the student to fail in challenging scenarios generated by the teacher and learn to resolve them, thereby enhancing the safety and realism of driving strategies.
Learning to Enhance Aperture Phasor Field for Non-Line-of-Sight Imaging
In Cho (Yonsei University), Seon Joo Kim (Yonsei University)
RestorationData SynthesisSuper ResolutionConvolutional Neural NetworkAuto EncoderImagePhysics Related
🎯 What it does: Propose a light phase field enhanced network named LEAP, which recovers complete and clean measurement data from noisy partial observations in sparse sampling and small aperture non-line-of-sight (NLOS) imaging, and further performs scene reconstruction.
Learning to Generate Conditional Tri-plane for 3D-aware Expression Controllable Portrait Animation
Taekyung Ki (DeepBrainAI Inc.), Gyeongsu Chae (DeepBrainAI Inc.)
GenerationSuper ResolutionConvolutional Neural NetworkTransformerNeural Radiance FieldContrastive LearningImageVideo
🎯 What it does: This paper proposes Export3D, a one-shot 3D view and expression controllable human portrait animation method. It directly maps the source image and driven expression parameters to 3D priors via a triplet-plane generator, and generates multi-view images using differentiable volume rendering and super-resolution networks, achieving cross-identity expression transfer without appearance swapping.
Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment
Yuxiao Chen (Rutgers University), Dimitris N. Metaxas (Michigan State University)
RecognitionTransformerLarge Language ModelVision-Language-Action ModelContrastive LearningVideoTextBenchmark
🎯 What it does: This paper proposes a method that utilizes a large language model (LLM) to extract task-related operational steps and generates reliable pseudo-labels through multi-path text-video alignment, enabling temporal localization of action steps in educational videos under unsupervised or weakly supervised conditions.
Learning to Make Keypoints Sub-Pixel Accurate
Shinjeong Kim (ETH Zürich), Daniel Barath (ETH Zürich)
Pose EstimationConvolutional Neural NetworkImage
🎯 What it does: Add a learnable offset module to any learning-based keypoint detector to achieve sub-pixel level precise localization.
Learning to Obstruct Few-Shot Image Classification over Restricted Classes
Amber Yijia Zheng (Purdue University), Raymond A. Yeh (Purdue University)
ClassificationSafty and PrivacyConvolutional Neural NetworkSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningImageText
🎯 What it does: To address the safety hazards in few-shot classification (FSC), we propose a novel pre-training model obstruction method: Learning to Obstruct (LTO). This method learns parameters with a 'poor initialization' for specified restricted classes (R) through meta-learning on the pre-trained backbone network, causing subsequent FSC classifiers to perform poorly on restricted classes while maintaining normal or better performance on non-restricted classes (R').
Learning to Robustly Reconstruct Dynamic Scenes from Low-light Spike Streams
Liwen Hu (Peking University), Tiejun Huang (Peking University)
RestorationData SynthesisConvolutional Neural NetworkRecurrent Neural NetworkSequential
🎯 What it does: This paper proposes a bidirectional recursive framework for reconstructing high-speed dynamic scenes from spike streams under low-light conditions, while simultaneously constructing synthetic and real datasets tailored for low-light high-frame-rate scenarios.
Learning to Unlearn for Robust Machine Unlearning
Mark He Huang (Singapore University of Technology and Design), Jun Liu (Singapore University of Technology and Design)
Safty and PrivacyComputational EfficiencyMeta LearningImageBenchmark
🎯 What it does: Proposes the Learning-to-Unlearn (LTU) framework, which utilizes meta-learning methods to forget specified forgetting sets after model training, while maintaining the model's overall performance under the condition of using only a small number of remaining samples.
Learning Trimodal Relation for Audio-Visual Question Answering with Missing Modality
Kyu Ri Park (Kyung Hee University), Jung Uk Kim (Kyung Hee University)
RestorationTransformerDiffusion modelVideoMultimodalityAudio
🎯 What it does: Propose a robust framework for audio-visual question answering (AVQA) that recovers missing modal features through an RMM generator when input modalities are missing, and further enhances audio-visual features using an AVR diffusion model to achieve accurate answers.
Learning Unified Reference Representation for Unsupervised Multi-class Anomaly Detection
Liren He (Fudan University), Chengjie Wang (Tencent Youtu Lab)
Anomaly DetectionTransformerImage
🎯 What it does: Propose a unified multi-class anomaly detection framework RLR based on learnable reference representations, achieving feature reconstruction from learnable references and avoiding learning shortcuts.
Learning Unsigned Distance Functions from Multi-view Images with Volume Rendering Priors
Wenyuan Zhang (Tsinghua University), Zhizhong Han (Wayne State University)
Neural Radiance FieldImage
🎯 What it does: This paper infers an unsigned distance function (UDF) from multi-view images by learning a neural network-based differentiable renderer (volume rendering prior), achieving high-precision reconstruction of open surfaces.
Learning Video Context as Interleaved Multimodal Sequences
Kevin Qinghong Lin (National University of Singapore), Mike Zheng Shou (National University of Singapore)
ClassificationRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelVideoMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Introduces MovieSeq, a general video understanding framework that embeds narrative videos into interleaved multimodal sequences and uses large language models for instruction tuning.
Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information
Luca Di Giammarino (Sapienza University of Rome), Daniel Barath (ETH Zürich)
Pose EstimationOptimizationData-Centric LearningRobotic IntelligenceSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: Based on the known SfM model, a discretized spatial-pose map is constructed. Visibility checks are used to filter the most beneficial camera perspectives for visual localization, and a lightweight MLP is trained to score the localization quality of perspectives, achieving self-supervised active viewpoint selection for localization.
Learning with Counterfactual Explanations for Radiology Report Generation
Mingjie Li (Stanford University), Xiaojun Chang (MBZUAI)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringContrastive LearningImageTextBiomedical Data
🎯 What it does: This paper proposes a pulmonary radiology report generation framework based on contrastive learning and counterfactual explanations (CoFE), which leverages counterfactual images and learnable prompts to enhance the model's focus on abnormal features, generating more accurate and complete medical reports.
Learning with Unmasked Tokens Drives Stronger Vision Learners
Taekyung Kim (NAVER AI Lab), Dongyoon Han (NAVER AI Lab)
ClassificationRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: Proposes a LUT method that incorporates supervision for unmasked tokens in Masked Image Modeling (MIM) training to enhance the representation learning of vision Transformers.
Learning-based Axial Video Motion Magnification
Kwon Byung-Ki (POSTECH), Tae-Hyun Oh (POSTECH)
RestorationData SynthesisConvolutional Neural NetworkOptical FlowVideo
🎯 What it does: Proposed a learning-based axial video motion magnification method that can amplify subtle motions in user-specified directions (e.g., axial) to a visible level while keeping motions in other directions unaffected.
LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning
Bolin Lai (GenAI, Meta), Miao Liu (University of Illinois Urbana-Champaign)
GenerationLarge Language ModelSupervised Fine-TuningVision Language ModelVision-Language-Action ModelDiffusion modelImageVideoTextMultimodality
🎯 What it does: This paper proposes a model called LEGO, which generates images displaying specific action execution states from a first-person perspective based on user text prompts and perspective images.
Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Models
Saman Motamed (Sofia University), Luc Van Gool (ETH Zurich)
GenerationDiffusion modelContrastive LearningImageText
🎯 What it does: Propose a text inversion method called Lego, which can learn from only four example images and reverse entangled adjective and verb concepts with the subject, migrating them to new target subjects.
LEIA: Latent View-invariant Embeddings for Implicit 3D Articulation
Archana Swaminathan (University of Maryland), Abhinav Shrivastava (University of Maryland)
GenerationPose EstimationNeural Radiance FieldImage
🎯 What it does: Learn viewpoint-invariant latent embeddings from multi-view static images and generate NeRF weights via a hypernetwork, achieving high-quality 3D reconstruction and interpolation for objects with variable poses.
Length-Aware Motion Synthesis via Latent Diffusion
Alessio Sampieri (Sapienza University of Rome), Fabio Galasso (Sapienza University of Rome)
GenerationData SynthesisTransformerVision-Language-Action ModelDiffusion modelAuto EncoderTextMultimodalitySequential
🎯 What it does: Propose a length-aware 3D human motion synthesis framework called LADiff, which can generate corresponding motion sequences based on text descriptions while controlling the target duration.
LEROjD: Lidar Extended Radar-Only Object Detection
Patrick Palmer (TU Dortmund University), Torsten Bertram (TU Dortmund University)
Object DetectionKnowledge DistillationConvolutional Neural NetworkPoint Cloud
🎯 What it does: This paper studies how to leverage LiDAR (LiDAR) point clouds during the training phase to enhance the 3D object detection performance of systems that use only imaging radar (3+1D Radar), proposing two methods: Multi-Stage Sparse Training (MSTM) and Cross-Modal Knowledge Distillation (KD).
Let the Avatar Talk using Texts without Paired Training Data
Xiuzhe Wu (University of Hong Kong), Xiaojuan Qi (Chinese University of Hong Kong)
GenerationTransformerVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageVideoText
🎯 What it does: Proposes a zero-shot, text-driven 3D avatar dialogue generation method, leveraging the pre-trained 3D avatar generation model EG3D along with CLIP and 3DMM for avatar generation and animation, and designing a self-supervised InpaintNet to repair warping artifacts, ultimately generating high-quality 3D avatar videos that match textual descriptions and include speech.
LetsMap: Unsupervised Representation Learning for Label-Efficient Semantic BEV Mapping
Nikhil Gosala (University of Freiburg), Abhinav Valada (Federal University of Rio Grande)
Autonomous DrivingConvolutional Neural NetworkTransformerSupervised Fine-TuningAuto EncoderContrastive LearningVideo
🎯 What it does: Proposes a label-efficient semantic BEV mapping framework called LetsMap, which leverages unsupervised representation learning. It first independently learns scene geometry and scene representations from monocular front-view sequences, then fine-tunes with minimal BEV labels to generate semantic BEV maps;
Leveraging Enhanced Queries of Point Sets for Vectorized Map Construction
Zihao Liu (Shanghai Jiao Tong University), Ningyi Xu (Shanghai Jiao Tong University)
Autonomous DrivingTransformerSimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper proposes a novel online vectorization method for high-definition maps called MapQR, based on the DETR structure. It employs scatter-and-gather queries and position embeddings to simultaneously model content and location information, and improves the BEV encoder (GKT-h) to enhance performance.
Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation
Haizhong Zheng (University of Michigan), Atul Prakash (University of Michigan)
Data SynthesisComputational EfficiencyData-Centric LearningImage
🎯 What it does: Design and implement Hierarchical Memory Network (HMN) as a data parameterization container to efficiently compress large datasets into compact synthetic datasets.
Leveraging Imperfect Restoration for Data Availability Attack
YI HUANG (Nanyang Technological University), Wai-Kin Adams Kong (Nanyang Technological University)
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Proposed a novel data availability attack method called IRP to prevent models from learning unauthorized data in supervised and self-supervised learning scenarios.
Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos
Akshay Paruchuri (University of North Carolina at Chapel Hill), Roni Sengupta (University of North Carolina at Chapel Hill)
Depth EstimationConvolutional Neural NetworkTransformerVideoBiomedical Data
🎯 What it does: The study utilizes endoscopic near-field illumination information to generate high-quality depth maps through monocular depth estimation methods.
Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detection
Christos Koutlis (Information Technologies Institute), Symeon Papadopoulos (Information Technologies Institute)
Anomaly DetectionTransformerVision Language ModelContrastive LearningImage
🎯 What it does: This paper proposes a method for detecting synthetic images by leveraging representations from intermediate Transformer blocks of the CLIP image encoder, combined with a lightweight network and a trainable importance estimator.
Leveraging scale- and orientation-covariant features for planar motion estimation
Marcus Valtonen Örnhag (Ericsson Research), Alberto Jaenal (Ericsson Research)
Pose EstimationAutonomous DrivingImage
🎯 What it does: This paper derives linear constraints for planar motion from scale and orientation covariant features (e.g., SIFT), proposes a minimization solver that requires only a single SIFT correspondence point, and integrates it into a robust estimation framework;
Leveraging temporal contextualization for video action recognition
Minji Kim (Seoul National University), Bohyung Han (Seoul National University)
RecognitionTransformerPrompt EngineeringVision-Language-Action ModelVideo
🎯 What it does: Propose the TC-CLIP framework, utilizing temporal contextualization techniques to enhance video action recognition.
Leveraging Text Localization for Scene Text Removal via Text-aware Masked Image Modeling
Zixiao Wang (University of Science and Technology of China), Pengwei Liu (IntSig Information Co Ltd)
RestorationSupervised Fine-TuningPrompt EngineeringImage
🎯 What it does: This paper proposes a text-aware masked image modeling (TMIM) pre-training framework based on weakly supervised text detection labels to enhance the performance of scene text removal (STR) models.
Leveraging Thermal Modality to Enhance Reconstruction in Low-Light Conditions
Jiacong Xu (Johns Hopkins University), Vishal Patel (Johns Hopkins University)
RestorationNeural Radiance FieldImageMultimodality
🎯 What it does: Integrate thermal imaging and visible-light raw images to propose Thermal-NeRF, achieving high-quality 3D scene reconstruction and view synthesis under low-light conditions.
LG-Gaze: Learning Geometry-aware Continuous Prompts for Language-Guided Gaze Estimation
Pengwei Yin (Hikvision Research Institute), Jiang Zhu (Hikvision Research Institute)
Pose EstimationConvolutional Neural NetworkTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposed a language-guided gaze estimation framework called LG-Gaze, which aligns visual features with continuous language embeddings using the CLIP language encoder, thereby enhancing the robustness of cross-domain gaze prediction.
LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation
Jiaxiang Tang (Peking University), Ziwei Liu (Nanyang Technological University)
GenerationConvolutional Neural NetworkDiffusion modelNeural Radiance FieldGaussian SplattingImageTextMultimodalityMesh
🎯 What it does: Proposed a large-scale multi-view Gaussian model (LGM) that can quickly generate high-resolution (512) 3D models from text or single-view images.
LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model
Dilxat Muhtar (Nanjing University), Pengfeng Xiao (Nanjing University)
ClassificationRecognitionData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: Proposed a multi-modal large language model specifically for remote sensing image understanding, named LHRS-Bot, and constructed a large-scale remote sensing image-text pair dataset LHRS-Align, an instruction-based dataset LHRS-Instruct, and an evaluation benchmark LHRS-Bench.
LiDAR-based All-weather 3D Object Detection via Prompting and Distilling 4D Radar
Yujeong Chae (KAIST), Kuk-Jin Yoon (KAIST)
Object DetectionKnowledge DistillationPrompt EngineeringMultimodalityPoint Cloud
🎯 What it does: This paper proposes a 3D detection framework using only LiDAR, which maintains high-precision detection under both normal and adverse weather conditions by incorporating 4D radar prompt learning and four-level cross/inner-modal knowledge distillation during the training phase.
LiDAR-Event Stereo Fusion with Hallucinations
Luca Bartolomei (University of Bologna), Stefano Mattoccia (University of Bologna)
Depth EstimationMultimodalityPoint Cloud
🎯 What it does: Integrate LiDAR sparse depth information with event stereo cameras, and synthesize virtual events in event data through two hallucination strategies (VSH and BTH) to enhance matching accuracy, thereby improving the performance of event stereo depth estimation.
LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors
Saksham Suri (University of Maryland), Abhinav Shrivastava (University of Maryland)
Object DetectionSegmentationPose EstimationSuper ResolutionRepresentation LearningConvolutional Neural NetworkTransformerAuto EncoderImage
🎯 What it does: This paper proposes a lightweight self-supervised feature densification module called LiFT, which enhances low-resolution features from a frozen ViT to high-resolution, thereby improving the performance of dense visual tasks.
Light-in-Flight for a World-in-Motion
Jongho Lee (University of Wisconsin-Madison), Mohit Gupta (University of Wisconsin-Madison)
Depth EstimationOptical FlowImage
🎯 What it does: This paper proposes a new method that simultaneously estimates 3D geometry, intensity, and 3D motion in dynamic scenes using a single indirect ToF camera.
LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models
Hai Jiang (Sichuan University), Shuaicheng Liu (University of Electronic Science and Technology of China)
RestorationDiffusion modelImage
🎯 What it does: Propose an unsupervised low-light image enhancement framework called LightenDiffusion, combining Retinex theory with diffusion models, utilizing a content transfer decomposition network in the latent space to achieve purer reflection and illumination separation, and improving reconstruction quality through self-constrained consistency loss.
Linearly Controllable GAN: Unsupervised Feature Categorization and Decomposition for Image Generation and Manipulation
sehyung lee (Rakuten Institute of Technology, Rakuten Group, Inc.), Bjorn Stenger (Rakuten Institute of Technology, Rakuten Group, Inc.)
GenerationGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: Propose an unsupervised linear controllable GAN (LC-GAN), achieving fine-grained semantic control over image generation by decomposing noise vectors into geometric and appearance codes, along with spectral regularization, without relying on pre-trained classifiers or supervisory signals.
LineFit: A Geometric Approach for Fitting Line Segments in Images
Marion Boyer, Florent Lafarge
ClassificationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a novel convolutional neural network based on a multi-scale attention mechanism for image classification tasks.
LingoQA: Video Question Answering for Autonomous Driving
Ana-Maria Marcu (Wayve Technologies), Oleg Sinavski (Wayve Technologies)
Autonomous DrivingTransformerLarge Language ModelVision Language ModelVideoTextBenchmark
🎯 What it does: Developed the LingoQA dataset and benchmark for autonomous driving visual question answering.
Linking in Style: Understanding learned features in deep learning models
Maren Wehrheim, Matthias Kaschube (Frankfurt Institute for Advanced Studies)
GenerationExplainability and InterpretabilityRepresentation LearningConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This paper trains a 'link network' to map the second-to-last layer representation space of a pre-trained CNN classifier to the latent space of StyleGAN-XL, enabling visualization and systematic analysis of the abstract features learned by the classifier; subsequently, unsupervised feature matching and few-shot image segmentation methods are used to automatically quantify these features, further exploring the concept representation of individual units and distributed decision boundaries;
LISO: Lidar-only Self-Supervised 3D Object Detection
Stefan Andreas Baur (Mercedes-Benz), Andreas Geiger (Mercedes-Benz)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: Proposes LISO, a self-supervised 3D object detection framework that uses only LiDAR point cloud sequences, leveraging self-supervised scene flow to generate high-precision pseudo labels, and enhancing detection performance through trajectory-regularized self-training iterations;
Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation
Bolin Lai (Georgia Institute of Technology), James M Rehg
RecognitionRepresentation LearningTransformerContrastive LearningVideoMultimodalityAudio
🎯 What it does: Proposed and implemented the first egocentric gaze prediction model that simultaneously leverages video and audio information.
LITA: Language Instructed Temporal-Localization Assistant
De-An Huang (Nvidia), Jan Kautz (Nvidia)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelVideoText
🎯 What it does: Proposed a specialized temporal localization assistant LITA for video large language models, capable of answering 'when'-type questions and achieving precise time interval localization.
LiteSAM is Actually what you Need for segment Everything
Jianhai Fu (National Engineering Research Center for Information Security), Zhiyu Xiang (National Engineering Research Center for Information Security)
SegmentationConvolutional Neural NetworkImage
🎯 What it does: Propose an attention-based multi-branch network that leverages PPN and Mask branches in parallel to learn small object segmentation;
LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environment
Yiming Ren (ShanghaiTech University), Yuexin Ma (ShanghaiTech University)
Pose EstimationConvolutional Neural NetworkRecurrent Neural NetworkTransformerPoint Cloud
🎯 What it does: The paper proposes LiveHPS++, a robust and coherent human motion capture framework based on a single LiDAR, which accurately estimates human motion in dynamic noise environments using three modules.
LivePhoto: Real Image Animation with Text-guided Motion Control
Xi Chen (University of Hong Kong), Hengshuang Zhao (University of Hong Kong)
GenerationPrompt EngineeringDiffusion modelImageVideoText
🎯 What it does: Proposed a system called LivePhoto that allows users to animate images through text descriptions, addressing the lack of temporal motion control in existing text-to-video generation methods.
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Yanwei Li (CUHK), Jiaya Jia (CUHK)
Computational EfficiencyTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Propose LLaMA-VID, which effectively represents video frames through dual tokens (context token and content token), addressing the issue of excessive tokens in Vision-Language Models (VLMs) for long videos.
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
Hao Zhang (Hong Kong University of Science and Technology), Jianwei Yang (Microsoft Research)
Object DetectionSegmentationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Constructed a grounded visual chat (GVC) dataset with 150K scale, proposed LLaVA-Grounding which connects large multimodal models with segmentation/detection models, and created the Grounding-Bench evaluation benchmark based on this.
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
Shilong Liu (Tsinghua University), Chunyuan Li (Microsoft Research)
TransformerSupervised Fine-TuningAgentic AIVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: Developed a multi-modal assistant named LLaVA-Plus, which learns and instantly invokes various visual and vision-language tools during training through a pluggable tool library to accomplish complex tasks;
LLaVA-UHD: an LMM Perceiving any Aspect Ratio and High-Resolution Images
Zonghao Guo (University of Chinese Academy of Sciences), Gao Huang (Tsinghua University)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageMultimodality
🎯 What it does: This paper first reveals systematic defects in high-resolution image processing by experimenting with the visual encoding strategies of GPT-4V and LLaVA-1.5; subsequently, it proposes LLaVA-UHD, a multimodal model capable of efficiently perceiving images with arbitrary aspect ratios and high resolutions.
LLM as Copilot for Coarse-grained Vision-and-Language Navigation
Yanyuan Qiao (Australian Institute for Machine Learning, University of Adelaide), Qi Wu (Australian Institute for Machine Learning, University of Adelaide)
TransformerLarge Language ModelAgentic AIPrompt EngineeringVision-Language-Action ModelImageTextMultimodality
🎯 What it does: Proposed the VLN-Copilot framework, enabling vision-language navigation agents to proactively seek real-time navigation guidance from large language models (LLMs) when encountering uncertainty;
LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Model
Yulin Luo (Peking University), Shanghang Zhang (Peking University)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: This paper proposes a subpopulation structure discovery framework based on large language models (SSD-LLM), which leverages multimodal LLMs (e.g., LLaVA1.5) to generate text descriptions rich in image information. It then uses GPT-4 to extract dimensions and attributes, structure hierarchies, and recursively refine through self-consistency, ultimately partitioning data into interpretable subpopulations. Subsequently, this structure is applied to downstream tasks such as data organization, subpopulation shift compensation, and slice discovery.
LLMCO4MR: LLMs-aided Neural Combinatorial Optimization for Ancient Manuscript Restoration from Fragments with Case Studies on Dunhuang
Yuqing Zhang (Zhejiang University), Fei Wu (Zhejiang University)
RestorationOptimizationGraph Neural NetworkTransformerLarge Language ModelContrastive LearningImageMultimodality
🎯 What it does: Propose a two-stage framework for ancient manuscript fragment reassembly, first using a neural combinatorial optimization (CO) solver to select TopK potentially relevant fragments from a large collection containing foreign fragments, then utilizing a multi-modal large language model (MLLM) for zero-shot mutual matching and orientation determination of fragments, thereby achieving automatic拼接 of ancient manuscript fragments.
LLMGA: Multimodal Large Language Model based Generation Assistant
bin xia, Jiaya Jia (ByteDance Inc)
RestorationGenerationData SynthesisLarge Language ModelPrompt EngineeringMixture of ExpertsVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Developed a multimodal large language model generation assistant LLMGA, utilizing language-generated detailed prompts to guide Stable Diffusion, supporting image generation, similar image generation, inpainting and outpainting, as well as instruction-based editing.
LMT-GP: Combined Latent Mean-Teacher and Gaussian Process for Semi-supervised Low-light Image Enhancement
Ye Yu (Hefei University of Technology), Zhen Kan (University of Science and Technology of China)
RestorationConvolutional Neural NetworkImage
🎯 What it does: Propose a semi-supervised low-light image enhancement method named LMT-GP, combining the latent mean-teacher framework with Gaussian Process, leveraging unlabelled low-light images to enhance performance and improve generalization.
LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation
Yushi Lan (Nanyang Technological University), Chen Change Loy (Shanghai Artificial Intelligence Laboratory)
GenerationTransformerVision Language ModelDiffusion modelAuto EncoderImageTextMultimodalityMesh
🎯 What it does: In this work, a framework named LN3Diff is proposed, which combines VAE-based 3D semantic compression with diffusion learning in the compressed space, for high-quality monocular 3D reconstruction and text-to-3D generation.
LNL+K: Enhancing Learning with Noisy Labels Through Noise Source Knowledge Integration
Siqi Wang (Boston University), Bryan Plummer (Boston University)
ClassificationImageBiomedical Data
🎯 What it does: This paper proposes incorporating noise source knowledge (LNL+K) into the learning with noisy labels (LNL) task, enhancing the model's ability to identify and train on clean samples through cross-category comparisons of prior information about noise source categories.
LoA-Trans: Enhancing Visual Grounding by Location-Aware Transformers
Ziling Huang (National Institute of Informatics), Shin'ichi Satoh (National Institute of Informatics)
RecognitionSegmentationTransformerPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: This paper proposes the LoA-Trans model, which jointly learns Referring Expression Comprehension (REC) and Referring Expression Segmentation (RES) within a unified framework. It achieves precise inter-task information exchange through center prompts, query selection mechanisms, and the TaskSyn network, enhancing visual alignment and target localization accuracy.
Loc3Diff: Local Diffusion for 3D Human Head Synthesis and Editing
Yushi Lan (Google S-Lab, Nanyang Technological University), Yinda Zhang (Google S-Lab, Nanyang Technological University)
GenerationData SynthesisSuper ResolutionDiffusion modelAuto EncoderGaussian SplattingImage
🎯 What it does: Proposed a Diffusion model based on local tri-plane and 3D Gaussian for generating and editing realistic 3D heads.
Local Action-Guided Motion Diffusion Model for Text-to-Motion Generation
Peng Jin (Peking University), Jie Chen (Peking University)
GenerationGraph Neural NetworkTransformerVision Language ModelDiffusion modelAuto EncoderTextMultimodalitySequential
🎯 What it does: Proposed a motion diffusion model called GuidedMotion based on local action guidance, achieving text-driven human motion generation and supporting fine-grained control through local actions.
Local All-Pair Correspondence for Point Tracking
Seokju Cho, Joon-Young Lee
Object TrackingConvolutional Neural NetworkTransformerVideo
🎯 What it does: Propose LocoTrack, a model that achieves efficient and accurate arbitrary point tracking using local all-to-all 4D correlation.
Local and Global Flatness for Federated Domain Generalization
Hao Yan (Carleton University), Yuhong Guo (Carleton University)
Domain AdaptationFederated LearningImageBenchmark
🎯 What it does: Train models that can generalize across domains under a federated learning framework using local and global flatness regularization.
Local Occupancy-Enhanced Object Grasping with Multiple Triplanar Projection
Kangqi Ma (Peking University), Yadong Mu (Peking University)
Pose EstimationRobotic IntelligenceConvolutional Neural NetworkPoint CloudBenchmark
🎯 What it does: This paper proposes a method to enhance object grasping under single-view point cloud input by leveraging local occupancy prediction; it first samples candidate grasp points and defines local reachable regions, extracts global context using multiple sets of tri-plane projections, queries voxels in the local region, fuses occupancy information, and finally refines the grasp orientation on the completed local shape to decode the full 6-DoF grasp pose.
Localization and Expansion: A Decoupled Framework for Point Cloud Few-shot Semantic Segmentation
Zhaoyang Li (University of Science and Technology of China), Tianzhu Zhang (University of Science and Technology of China)
SegmentationMeta LearningGraph Neural NetworkPoint Cloud
🎯 What it does: This paper proposes a Decoupled Localization and Expansion (DLE) framework for point cloud few-shot semantic segmentation.
LogoSticker: Inserting Logos into Diffusion Models for Customized Generation
Mingkang Zhu (CUHK), Jiaya Jia (CUHK)
Image TranslationGenerationSupervised Fine-TuningReinforcement LearningDiffusion modelImage
🎯 What it does: Study the logo insertion task, proposing the LogoSticker two-stage pipeline to achieve accurate learning and contextual generation of user-provided logos.
Long-CLIP: Unlocking the Long-Text Capability of CLIP
Beichen Zhang (Shanghai AI Laboratory), Jiaqi Wang (Shanghai AI Laboratory)
ClassificationRetrievalComputational EfficiencyRepresentation LearningData-Centric LearningTransformerSupervised Fine-TuningVision Language ModelContrastive LearningMultimodality
🎯 What it does: This paper proposes Long-CLIP, a multimodal model that can directly replace CLIP, capable of processing long texts while maintaining or even enhancing zero-shot capabilities.
Long-range Turbulence Mitigation: A Large-scale Dataset and A Coarse-to-fine Framework
Shengqi Xu (Huazhong University of Science and Technology), Luxin Yan (Huazhong University of Science and Technology)
RestorationOptical FlowImageBenchmark
🎯 What it does: Constructed the first large-scale long-range atmospheric turbulence dataset (RLR-AT) and proposed a coarse-to-fine hierarchical framework (CDSP) for long-range turbulence removal, leveraging dynamic turbulence statistics and static background low-rank tensor priors.
Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustment
Zhanzhong Pang (National University of Singapore), Angela Yao (National University of Singapore)
SegmentationData-Centric LearningConvolutional Neural NetworkTransformerVideo
🎯 What it does: To address the long-tail action distribution in procedural videos, this paper proposes the Group-based Temporal Log Adjustment (G-TLA) framework to improve the segmentation accuracy of tail actions.
Long-term Temporal Context Gathering for Neural Video Compression
Linfeng Qi (University of Science and Technology of China), Yan Lu (Microsoft Research Asia)
CompressionOptical FlowVideo
🎯 What it does: This paper proposes a neural video compression framework that leverages the collaboration between long-term and short-term context, called DCVC-LCG.
LongVLM: Efficient Long Video Understanding via Large Language Models
Yuetian Weng (Monash University), Bohan Zhuang (Monash University)
RecognitionComputational EfficiencyTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Propose LongVLM, a VideoLLM that achieves efficient fine-grained long video understanding by segmenting long videos, performing hierarchical token merging on each segment, and incorporating global semantics.
Look Around and Learn: Self-Training Object Detection by Exploration
Gianluca Scarpellini (Fondazione Istituto Italiano di Tecnologia), Alessio Del Bue (Fondazione Istituto Italiano di Tecnologia)
Object DetectionKnowledge DistillationRobotic IntelligenceConvolutional Neural NetworkReinforcement LearningContrastive LearningImagePoint Cloud
🎯 What it does: Designed a fully self-supervised robotic visual exploration framework called 'Look Around and Learn,' which collects multi-view images through the exploration strategy 'Look Around' and generates consistent pseudo-labels using a 'Disagreement Reconciliation' mechanism to fine-tune pre-trained object detectors.
Look Hear: Gaze Prediction for Speech-directed Human Attention
Sounak Mondal (Stony Brook University), Minh Hoai (Stony Brook University)
RecognitionObject DetectionConvolutional Neural NetworkTransformerImageTextMultimodality
🎯 What it does: Proposed and implemented an eye movement scan path prediction model called ART for incremental object reference tasks, and collected a large-scale human eye movement dataset named RefCOCO-Gaze;
LookupViT: Compressing visual information to a limited number of tokens
Rajat Koner (Google DeepMind Ludwig Maximilian University of Munich), Prateek Jain
ClassificationCompressionComputational EfficiencyTransformerImageVideo
🎯 What it does: Propose LookupViT, which uses multi-head bidirectional cross-attention to compress visual information into a fixed number of tokens, significantly reducing ViT inference costs.
Lossy Image Compression with Foundation Diffusion Models
Lucas Relic (ETH Zürich), Christopher Schroers (Disney Research Studios)
CompressionTransformerDiffusion modelImage
🎯 What it does: Proposed a lossless image compression method utilizing a basic latent diffusion model
Lost and Found: Overcoming Detector Failures in Online Multi-Object Tracking
Lorenzo Vaquero (Fondazione Bruno Kessler), Manuel Mucientes (University of Santiago de Compostela)
Object TrackingData SynthesisTransformerVideoBenchmark
🎯 What it does: Propose a fully online framework named BUSCA, which can detect and track targets missed by detectors in any tracking-by-detection (TbD) system, especially maintaining trajectories when targets are occluded.
Lost in Translation: Latent Concept Misalignment in Text-to-Image Diffusion Models
Juntu Zhao (Shanghai Jiao Tong University), Dequan Wang (Shanghai Jiao Tong University)
GenerationLarge Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Investigate and address the 'latent concept mismatch' (LC-Mis) problem in text-to-image diffusion models, construct an automated data collection pipeline based on large language models (LLMs), and propose the MoCE method to alleviate mismatch by staged input of concepts.
Lost in Translation: Modern Neural Networks Still Struggle With Small Realistic Image Transformations
Ofir Shifman (Hebrew University of Jerusalem), Yair Weiss (Hebrew University of Jerusalem)
ClassificationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper investigates the robustness deficiency of modern deep networks when facing small real image translations, and proposes a new method for achieving robust inference through cropping selection (RICS).
LPViT: Low-Power Semi-structured Pruning for Vision Transformers
Kaixin Xu (Agency for Science, Technology and Research), Weisi Lin (Nanyang Technological University)
Computational EfficiencyTransformerImage
🎯 What it does: For Vision Transformer, we propose low-power semi-structured pruning (LPViT), achieving block-level sparsity and significantly accelerating inference while maintaining model accuracy.
LRSLAM: Low-rank Representation of Signed Distance Fields in Dense Visual SLAM System
Hongbeen Park (Korea University), Jinkyu Kim (Korea University)
Simultaneous Localization and MappingImage
🎯 What it does: Propose a low-rank decomposition visual SLAM method called LRSLAM, combining six-axis decomposition and CP decomposition to efficiently represent Signed Distance Field (SDF) and achieve RGB-D map construction and localization.
m&m’s: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks
Zixian Ma (University of Washington), Ranjay Krishna (Allen Institute of Artificial Intelligence)
Large Language ModelAgentic AIPrompt EngineeringMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposes m&m's, a multimodal multi-step tool usage benchmark containing 4K+ real-world tasks and 33 tools, providing a human-validated executable subset.
M^2Depth: Self-supervised Two-Frame Multi-camera Metric Depth Estimation
Yingshuang Zou (Tsinghua University), Haotian Zhang (MachDrive)
Depth EstimationAutonomous DrivingSimultaneous Localization and MappingImage
🎯 What it does: Propose a self-supervised two-frame multi-camera scale-aware depth estimation network, M Depth, capable of recovering a scale-consistent 3D structure of the 360° surrounding environment in autonomous driving scenarios.
M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models
Seunggeun Chi (Honda Research Institute USA), Kwonjoon Lee (Honda Research Institute USA)
GenerationTransformerDiffusion modelTextMultimodality
🎯 What it does: Propose the M2D2M model, which utilizes a discrete diffusion framework combined with two-stage sampling to generate coherent human motion sequences from multi-action text.
M3DBench: Towards Omni 3D Assistant with Interleaved Multi-modal Instructions
Mingsheng Li (Fudan University), Tao Chen (Fudan University)
Object DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityPoint CloudBenchmark
🎯 What it does: Introduces the M3DBench multimodal instruction dataset and baseline model, supporting interactive multimodal instruction understanding and execution in 3D scenes.
MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion
Lehong Wu (Peking University), Jiaying Liu (Peking University)
GenerationPose EstimationRepresentation LearningTransformerDiffusion modelContrastive LearningGraph
🎯 What it does: Proposes Masked Conditional Diffusion (MacDiff), a self-supervised skeletal model framework that integrates a semantic encoder with a diffusion decoder.
MAD-DR: Map Compression for Visual Localization with Matchness Aware Descriptor Dimension Reduction
Qiang Wang (EasyAR Mega)
Pose EstimationRetrievalCompressionGraph Neural NetworkTransformerImage
🎯 What it does: Designed and implemented a jointly trained local feature compression and matching framework, which can compress descriptors such as SuperPoint to 1/256 of their original size while maintaining high matching and localization performance.
Made to Order: Discovering monotonic temporal changes via self-supervised video ordering
Charig Yang (University of Oxford), Andrew Zisserman (University of Oxford)
TransformerVideoBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a self-supervised video sorting method that learns and locates time-monotonic change regions by sorting shuffled image sequences, enabling the identification and localization of time-monotonic visual features without manual annotations.
MagDiff: Multi-Alignment Diffusion for High-Fidelity Video Generation and Editing
Haoyu Zhao (Fudan University), Yu-Gang Jiang (Huawei)
GenerationVision Language ModelDiffusion modelAuto EncoderVideoText
🎯 What it does: Proposed a unified multi-alignment diffusion model called MagDiff, which can simultaneously accomplish high-fidelity video generation and editing tasks.
MagicEraser: Erasing Any Objects via Semantics-Aware Control
Fan Li (Huawei Noah's Ark Lab), Songcen Xu (Huawei Noah's Ark Lab)
GenerationPrompt EngineeringDiffusion modelImage
🎯 What it does: Propose the MagicEraser framework, which completes the object erasure task based on diffusion models, divided into two stages: content initialization (using traditional inpainting pre-filling) and controlled generation (prompt tuning + semantic-aware attention reassembly).
MagicMirror: Fast and High-Quality Avatar Generation with Constrained Search Space
Armand Comas (Google), Thabo Beeler (Google)
GenerationData SynthesisSupervised Fine-TuningPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldOptical FlowImageText
🎯 What it does: Propose the MagicMirror framework, which rapidly generates high-quality 3D head avatars using text prompts and supports personalized editing
MagMax: Leveraging Model Merging for Seamless Continual Learning
Daniel Marczak (IDEAS NCBR), Sebastian Cygert (IDEAS NCBR)
TransformerSupervised Fine-TuningVision Language ModelImage
🎯 What it does: Proposed the MagMax method, achieving continuous learning in large pre-trained models through sequential fine-tuning and maximum amplitude parameter selection;
MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment
Kanglei Zhou (Beihang University), Xiaohui Liang (Durham University)
RecognitionRepresentation LearningGraph Neural NetworkVideo
🎯 What it does: Propose the Continual Action Quality Assessment (CAQA) framework, achieving continual learning without forgetting for new and old data through Manifold-Aligned Graph Regularization (MAGR);
Mahalanobis Distance-based Multi-view Optimal Transport for Multi-view Crowd Localization
Qi Zhang (Shenzhen University), Hui Huang (Shenzhen University)
Object DetectionConvolutional Neural NetworkImage
🎯 What it does: Studied the multi-view crowd localization problem, proposing to use a Mahalanobis distance improved multi-view optimal transport loss to replace traditional Gaussian density map supervision.