IEEE/CVF Conference on Computer Vision and Pattern Recognition · 892 papers
SEED-Bench: Benchmarking Multimodal Large Language Models
Bohao Li (Tencent AI Lab), Ying Shan (Tencent AI Lab)
CodeGenerationData SynthesisTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: A multi-modal large language model (MLLM) evaluation benchmark named SEED-Bench has been constructed, covering multi-level capabilities from text understanding to image generation, and evaluated across 27 dimensions using 24,000 multiple-choice questions.
🎯 What it does: This study proposes a nighttime dynamic scene imaging method based on event cameras and constructs the first real low-light event and high-quality image alignment dataset, RLED.
🎯 What it does: This paper proposes a method called SeeSR that utilizes semantic prompts (hard labels and soft features) to control pre-trained text-image diffusion models for real-world image super-resolution.
🎯 What it does: Maps anomaly scores to box prompts, using a promptable segmentation model (such as SAM) to directly generate high-quality OoD object masks, achieving OoD detection without the need for thresholds.
🎯 What it does: This paper proposes a selective 'hourglass' mapping strategy based on diffusion models, utilizing shared distribution mapping and strong conditional guidance to achieve unified image restoration.
🎯 What it does: A Selective-Stereo framework is proposed, which incorporates Selective Recurrent Unit (SRU) and Contextual Spatial Attention (CSA) modules into traditional iterative stereo matching networks to adaptively fuse information of different frequencies (high-frequency details and low-frequency smoothness), thereby improving the quality of dense disparity estimation.
🎯 What it does: A lightweight self-distillation masked autoencoder is proposed for video anomaly detection, combining motion gradient weighting, teacher-student decoding, and synthetic anomaly enhancement.
🎯 What it does: This paper proposes a self-supervised Dual Contouring method (SDC) that directly predicts mesh vertices from SDF grids through two types of geometric consistency losses, eliminating the reliance on QEF solving and manually trained data. This self-supervised framework is applied for the regularization of Deep Implicit Networks (DIN) and for end-to-end reconstruction from single-view images to meshes.
SelfPose3d: Self-Supervised Multi-Person Multi-View 3d Pose Estimation
Vinkle Srivastav (University of Strasbourg), Nicolas Padoy (University of Strasbourg)
CodePose EstimationImage
🎯 What it does: A fully self-supervised multi-camera multi-person 3D pose estimation method called SelfPose3d is proposed, which utilizes 2D pseudo-poses and multi-view geometric constraints to achieve 3D pose reconstruction without the need for 2D or 3D real annotations.
🎯 What it does: This paper proposes a semantic-aware multi-label adversarial attack framework (GMLA) that can generate effective attacks while maintaining label semantic consistency.
🎯 What it does: Research on class-incremental learning based on pre-trained Vision Transformers is conducted, proposing a parameter-free shared Adapter incremental fine-tuning combined with semantic drift estimation to retrain a unified classifier.
Yimeng Fan (Tianjin University), Wenrui Lu (Tianjin University)
CodeObject DetectionSpiking Neural NetworkImage
🎯 What it does: For target detection with event cameras, we propose the Spiking Fusion Object Detector (SFOD), which implements multi-scale feature fusion and completes the target detection task within the SNN framework.
Shadow Generation for Composite Image Using Diffusion Model
Qingyang Liu (Shanghai Jiao Tong University), Li Niu
CodeGenerationData SynthesisDiffusion modelImage
🎯 What it does: A shadow generation method based on diffusion models (SGDiffusion) is proposed, which can generate natural shadows for inserted foregrounds in synthetic images.
🎯 What it does: A shallow-deep collaborative learning framework (SDCL) based on Transformer is proposed for unsupervised visible-infrared portrait recognition.
🎯 What it does: This paper proposes a Transformer-based multi-person eye tracking architecture called Sharingan, which can predict the gaze points of all individuals in an image at once.
SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design
Seokju Yun (Machine Intelligence Laboratory, University of Seoul), Youngmin Ro (Machine Intelligence Laboratory, University of Seoul)
CodeObject DetectionTransformerImage
🎯 What it does: A single-head visual Transformer SHViT is proposed, achieving low latency and high accuracy through a large stride patchify stem and a single-head attention module.
🎯 What it does: A continuous sign language recognition model called SignGraph based on graph convolutional networks has been constructed, utilizing local graphs (LSG) and temporal graphs (TSG) modules to capture cross-region and cross-frame features at the graph level, and learning sign language representations of different granularities at multiple scales.
SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models
Feifei Wang (University of Science and Technology of China), Qidong Huang (University of Science and Technology of China)
CodeGenerationSafty and PrivacyAdversarial AttackDiffusion modelImage
🎯 What it does: The SimAC method is proposed to suppress the model's personalized reproduction of user facial images by adding adversarial noise to the images in text-to-image diffusion models, thereby protecting user privacy.
🎯 What it does: This study investigates the MPCount model for single-domain generalization in crowd counting, addressing domain shift and label ambiguity issues.
🎯 What it does: This paper proposes an end-to-end framework named S2HGrasp for generating physically constrained human grasp poses from single-view scene point clouds.
🎯 What it does: A single-step diffusion-based image super-resolution method SinSR is proposed, compressing the inference steps of the diffusion model into one.
🎯 What it does: Proposes the Skeleton-in-Context framework, which utilizes in-context learning to unify the handling of various skeletal sequence tasks.
🎯 What it does: An implicit neural representation called SketchINR is proposed for high-fidelity compression and reconstruction of sequential vector sketches.
🎯 What it does: A universal adversarial patch (SlowFormer) is designed to significantly increase the model's FLOPs and power consumption by forcing adaptive efficient visual Transformers to revert to full computation by pasting a fixed patch in the input image.
🎯 What it does: A small-scale data-free knowledge distillation method SSD-KD is proposed, which utilizes a teacher network to inversely synthesize a minimal amount of high-quality synthetic samples, and employs prioritized sampling to accelerate training during the distillation process.
🎯 What it does: Designed and implemented the Smart Help challenge, constructing a multi-agent home task environment based on AI2-THOR, and proposed a help robot model that can actively adapt to user capabilities and goals.
SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models
Yuzhou Huang (Chinese University of Hong Kong), Ying Shan (Tencent)
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageMultimodality
🎯 What it does: A directive image editing framework named SmartEdit is proposed, which can understand and execute complex instructions that include attributes such as position, size, color, mirror relationships, and require world knowledge reasoning.
Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models
Jiayi Guo (Tsinghua University), Humphrey Shi (Georgia Tech)
CodeGenerationDiffusion modelImage
🎯 What it does: This paper proposes the Smooth Diffusion model, which enhances the performance of downstream tasks such as image interpolation, inversion, and editing by incorporating Step-wise Variation Regularization during training to make the latent space of the diffusion model smoother.
🎯 What it does: A spatiotemporal video grounding framework called SnAG is proposed for multi-query long videos, aiming to address the scalability issues of traditional methods in scenarios involving long videos and numerous queries.
CodeNeural Radiance FieldSimultaneous Localization and MappingPoint Cloud
🎯 What it does: SNI-SLAM is a real-time semantic SLAM system based on NeRF that can simultaneously perform semantic mapping, surface reconstruction, and camera tracking.
🎯 What it does: This paper proposes SOAC, a target-independent, self-supervised spatiotemporal multi-sensor calibration method using NeRF corresponding to multiple cameras.
🎯 What it does: This paper proposes an angle-based social circle (SocialCircle) representation to capture social interactions in pedestrian trajectory prediction and integrates it into various baseline models.
🎯 What it does: A JPDVT method based on diffusion visual Transformer is proposed, which can simultaneously solve the image and video jigsaw puzzle (including missing fragments) problem.
🎯 What it does: The LegoGCD framework is proposed, combining SimGCD with local entropy regularization (LER) and dual-view KL constraints (DKL) to alleviate the catastrophic forgetting problem of known categories in Generalized Category Discovery.
🎯 What it does: Proposes Spatial-Aware Regression (SAR), which incorporates spatial location information into regression-based keypoint localization to achieve efficient and robust keypoint detection.
SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction
Zhiyang Yao (Tsinghua University), Lu Fang (Tsinghua University)
CodeRestorationTransformerImage
🎯 What it does: A Transformer model specifically designed for high-resolution hyperspectral image reconstruction, called SPECAT, has been developed and implemented.
🎯 What it does: A coarse-to-fine 3D instance segmentation framework called Spherical Mask is proposed, which utilizes spherical polygons for coarse detection and achieves refinement through spherical point migration.
SpiderMatch: 3D Shape Matching with Global Optimality and Geometric Consistency
Paul Roetzer (University of Bonn), Florian Bernard (University of Bonn)
CodeOptimizationMesh
🎯 What it does: A 3D shape matching method based on SpiderCurve is proposed, using integer linear programming to solve for globally optimal and geometrically consistent correspondences.
🎯 What it does: Designed and implemented a three-stage spike camera guided motion deblurring network UaSDN, achieving high-quality deblurring under the condition of unaligned RGB and spike data.
🎯 What it does: A NeRF model based on continuous pulse streams from Spike cameras, called SpikeNeRF, is proposed, which can learn dense 3D scene representations and generate high-quality novel view images using only pulse data.
SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks
Xinyu Shi (Peking University), Zhaofei Yu (Peking University)
CodeSpiking Neural NetworkTransformerImage
🎯 What it does: A dual-pulse self-attention mechanism (DSSA) is proposed, and based on it, a multi-stage ResNet-Transformer structure called SpikingResformer is designed, constructing a complete and directly trainable spiking neural network.
🎯 What it does: An unsupervised natural light non-calibrated photogrammetry method called Spin-UP is proposed, which utilizes a rotating platform to achieve uniform ambient light and recovers surface normals, ambient light, and isotropic reflectance through inverse rendering.
🎯 What it does: This paper proposes the UniMoS framework, which separates the CLIP visual features into language-associated (LAC) and vision-associated (VAC) parts through a modality separation network, and achieves unsupervised domain adaptation using modality fusion training and a modality discriminator.
🎯 What it does: This paper enhances the segmentation performance of unsupervised object center learning by incorporating self-training and sequence permutation techniques into the slot-based autoencoder.
🎯 What it does: A point cloud upsampling method based on self-supervised learning utilizes a mesh interpolation and recursive feature aggregation deformation module to achieve the transformation from sparse point clouds to high-resolution uniformly distributed point clouds.
🎯 What it does: This study investigates the use of a fixed d-Simplex classifier to learn static feature representations for achieving model compatibility, and proposes a High-Order Compatibility (HOC) loss.
🎯 What it does: A StegoGAN model based on CycleGAN is proposed, which utilizes steganography techniques to explicitly separate matching and non-matching information in the feature space, thereby suppressing the generation of pseudo-features in non-injective image translation.
🎯 What it does: This paper proposes BlindNet, a method for domain generalization semantic segmentation achieved through covariance alignment and semantic consistency contrastive learning.
Teng Hu (Shanghai Jiao Tong University), Yu-Kun Lai (Cardiff University)
CodeGenerationData SynthesisTransformerImage
🎯 What it does: SuperSVG is proposed, a two-stage self-supervised framework based on superpixels, which first captures the main structure using a coarse model and then refines the details with a fine model, ultimately generating high-quality SVG vector graphics.
Supervised Anomaly Detection for Complex Industrial Images
Aimira Baitieva (Valeo), Olivier Bernard (Valeo)
CodeAnomaly DetectionImage
🎯 What it does: This paper presents a new industrial defect detection dataset VAD and designs a segmentation-based supervised anomaly detection method SegAD for efficient anomaly detection in complex industrial images.
🎯 What it does: A multi-modal domain generalization framework MMDG is proposed, which utilizes the U-Adapter to suppress unreliable information across modalities and adaptively adjusts gradients through ReGrad to address modality imbalance, significantly improving cross-domain facial deception detection performance.
SVDTree: Semantic Voxel Diffusion for Single Image Tree Reconstruction
Yuan Li (Institute of Automation, Chinese Academy of Sciences), Jianwei Guo (Institute of Automation, Chinese Academy of Sciences)
CodeSegmentationGenerationDiffusion modelImage
🎯 What it does: This paper proposes a single-image tree model reconstruction framework based on semantic voxel diffusion, called SVDTree, which can generate high-fidelity three-dimensional tree geometry from a single tree photograph.
SynSP: Synergy of Smoothness and Precision in Pose Sequences Refinement
Tao Wang (Beijing University of Posts and Telecommunications), Jian Zhao (Northwestern Polytechnical University)
CodePose EstimationOptimizationTransformerVideo
🎯 What it does: This paper proposes a posture sequence optimization network named SynSP, which aims to improve the accuracy and smoothness of posture estimation while maintaining low latency.
CodeClassificationSegmentationGenerationRetrievalOptimizationTransformerVision Language ModelDiffusion modelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes a step-by-step generation of a candidate set of images that differ only in a specific attribute, and based on this, constructs the SPEC fine-grained visual-language understanding benchmark. Subsequently, this benchmark is used to diagnose the performance of mainstream VLMs, and CLIP is fine-tuned by adding hard negative samples to enhance fine-grained understanding capabilities.
🎯 What it does: The paper conducts a theoretical analysis of the singularities at the time endpoints of diffusion models and proposes the SingDiffusion plugin for sampling at the initial singularity moment, thereby eliminating average brightness imbalance and enhancing image quality.
Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting
Zijie Chen (Zhejiang University), Zhenzhong Lan (Westlake University)
CodeGenerationRetrievalTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a personalized prompt rewriting method based on user historical interactions and constructs a PIP dataset containing 300,237 prompts to enhance the personalization effect of text-to-image generation.
Taming Self-Training for Open-Vocabulary Object Detection
Shiyu Zhao (Rutgers University), Dimitris N. Metaxas (Rutgers University)
CodeObject DetectionVision Language ModelImage
🎯 What it does: A self-supervised training-based open vocabulary object detection framework SAS-Det is designed, utilizing CLIP to generate pseudo-labels and enhancing detection performance through a branch detection head and periodic teacher updates.
🎯 What it does: The TASeg framework is proposed, combining Temporal LiDAR Aggregation and Distillation (TLAD), Temporal Image Aggregation and Fusion (TIAF), and Static-Moving Switch Augmentation (SMSA) to achieve semantic segmentation of multi-frame LiDAR and multi-temporal images.
🎯 What it does: For sample-free incremental learning, the Task-Adaptive Saliency Supervision (TASS) method combines boundary guidance, low-level task assistance, and saliency noise injection to suppress saliency drift, enhancing the model's adaptability to new tasks and memory retention.
Task-Customized Mixture of Adapters for General Image Fusion
Pengfei Zhu (Tianjin University), Qinghua Hu (Tianjin University)
CodeImage TranslationRestorationPrompt EngineeringMixture of ExpertsImageMultimodalityMagnetic Resonance Imaging
🎯 What it does: A task-customized mixed adapter (TC-MoA) is designed to adaptively handle multi-modal, multi-exposure, and multi-focal image fusion tasks within the same base model.
🎯 What it does: A framework called CERM is proposed for training deep networks with strict constraints (such as wavelet filters) by performing gradient descent on constrained subspaces;
TE-TAD: Towards Full End-to-End Temporal Action Detection via Time-Aligned Coordinate Expression
Ho-Joong Kim (Korea University), Seong-Whan Lee (Korea University)
CodeRecognitionObject DetectionTransformerVideo
🎯 What it does: This paper proposes an end-to-end temporal action detection Transformer (TE-TAD) that achieves detection without relying on sliding windows and NMS through time-aligned coordinate representation.
🎯 What it does: Enhancing the model's generalization ability under distribution shift during the testing phase through the Energy Adaptation (TEA) method.
🎯 What it does: A text-driven stylization method for multi-object 3D meshes, TeMO, is proposed, achieving precise parsing and style transfer for multi-object scenes.
🎯 What it does: This study investigates the linear relationship between the OOD scores generated by current OOD detection methods and network features, and based on this, proposes a robust testing-time linear correction method (RTL, RTL++) and its online version.
Benedetta Liberatori (University of Trento), Elisa Ricci (University of Trento)
CodeRecognitionObject DetectionTransformerVision Language ModelContrastive LearningVideoText
🎯 What it does: In the absence of labeled training data, a test-time adaptive zero-shot temporal action localization method called T3AL is proposed, which utilizes a pre-trained vision-language model to achieve video-level pseudo-label generation, prediction refinement based on self-supervised learning, and subtitle-guided region suppression.
🎯 What it does: A point cloud descriptor TetraSphere based on a tunable TetraTransform layer and vector neural networks is proposed, which remains invariant under arbitrary rotations and reflections.
🎯 What it does: This paper proposes a zero-shot text-guided extreme super-resolution method that can explore multiple high-resolution reconstruction results consistent with low-resolution input and semantically coherent through natural language prompts.
🎯 What it does: A semantic text-guided image fusion framework (Text-IF) is proposed, achieving adaptive processing of degraded images and supporting user interactive generation of fusion results.
🎯 What it does: The GSGEN method is proposed, utilizing 3D Gaussian splatting for text-to-3D generation, and achieving high-quality, geometrically consistent 3D assets through a two-stage process (geometric optimization + appearance refinement).
🎯 What it does: This paper proposes a complete framework for generating 3D hand-object interaction actions from text prompts and object meshes, capable of producing diverse and physically feasible interaction sequences.
🎯 What it does: Proposes the Text2QR method, which uses Stable Diffusion to generate QR codes that align with user aesthetics, and ensures scannability through subsequent optimization.
🎯 What it does: Fine-tuning the text encoder of Stable Diffusion improves the quality of generated images and text-image alignment, and can be combined with UNet fine-tuning.
🎯 What it does: Utilize NeRF for geometric modeling of real scenes, and insert and edit text in three-dimensional space to achieve controllable scene text image synthesis;
🎯 What it does: Proposes the Texture-Preserving Diffusion (TPD) model, which implements texture transfer in diffusion models using self-attention without the need for additional image encoders, and achieves high-fidelity virtual try-on by combining decoupled mask prediction.
🎯 What it does: A post-training quantization framework specifically designed for diffusion models, TFMQ-DM, is proposed to maintain temporal features to reduce the impact of quantization on generation quality.
The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding
Lorenzo Bianchi (Italian National Research Council), Fabrizio Falchi (Italian National Research Council)
CodeObject DetectionTransformerLarge Language ModelImageBenchmark
🎯 What it does: This paper proposes the fine-grained open vocabulary detection (FG-OVD) task and its evaluation protocol, and constructs a fine-grained evaluation benchmark based on a dynamic vocabulary.
The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement
Gabriele Trivigno (Politecnico di Torino), Torsten Sattler (Czech Technical University in Prague)
CodePose EstimationOptimizationConvolutional Neural NetworkContrastive LearningGaussian SplattingSimultaneous Localization and MappingImageMesh
🎯 What it does: A simple pose refinement method utilizing pre-trained features and particle filters is proposed, which can iteratively improve the initial pose without the need for any scene-specific training.
Theoretically Achieving Continuous Representation of Oriented Bounding Boxes
Zikai Xiao, Shimin Hu
CodeObject DetectionImage
🎯 What it does: A continuous oriented bounding box (COBB) representation is proposed to address the discontinuity issues of rotation and aspect ratio in traditional OOB representations.
🎯 What it does: In the multi-center medical imaging federated learning scenario, this paper proposes a new federated evidence active learning framework (FEAL) to select the most informative unlabeled samples for labeling under domain transfer.
🎯 What it does: In the context of autonomous driving, the ScaLR method achieves high-quality 3D feature distillation through three main pillars: 3D backbone expansion, 2D backbone pre-training, and cross-dataset pre-training.
TIM: A Time Interval Machine for Audio-Visual Action Recognition
Jacob Chalk (University of Bristol), Dima Damen (Czech Technical University in Prague)
CodeRecognitionTransformerVideoMultimodalityAudio
🎯 What it does: This paper proposes a multi-modal Transformer based on time interval queries (Time Interval Machine, TIM), which can simultaneously recognize audio and visual actions in long videos and achieve cross-modal context aggregation through unified time encoding.
🎯 What it does: A method for training a lightweight parallel side network (LoSA) on a frozen large-scale visual pre-trained model is proposed to reduce training time, memory usage, and the number of learnable parameters.
🎯 What it does: Using Stable Diffusion for image editing, the TiNO-Edit method is proposed to achieve various controllable editing tasks by automatically optimizing noise and diffusion step size.
🎯 What it does: By combining the Segment Anything Model (SAM) with a hybrid implicit-explicit surface representation and a mesh region growing method, 3D reconstruction and decomposition of indoor scenes from sparse pose multi-view images is achieved, requiring only a minimal number of human clicks (approximately 1.4) for object-level separation.
Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts
Jiawen Zhu (Singapore Management University), Guansong Pang (Singapore Management University)
CodeAnomaly DetectionVision Language ModelImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: A general anomaly detection model, InCTRL, is designed to learn residual features for cross-domain unsupervised anomaly detection using a small number of normal images as contextual prompts.
🎯 What it does: A multi-modal personalized face generation framework is proposed, capable of simultaneously controlling identity, expression, and background, and achieving fine-grained expression synthesis.
🎯 What it does: A post-training quantization framework for diffusion models (APQ-DM) is proposed, which significantly improves the generation quality and inference efficiency of low-bit-width models through group quantization and active calibration set generation.
Towards CLIP-driven Language-free 3D Visual Grounding via 2D-3D Relational Enhancement and Consistency
Yuqi Zhang (Sichuan University), Yinjie Lei (Sichuan University)
CodeObject DetectionSegmentationTransformerVision Language ModelContrastive LearningImagePoint Cloud
🎯 What it does: A language-free unsupervised 3D visual localization framework based on CLIP is proposed, which utilizes multi-view images to generate pseudo-language features for aligning 3D vision with text.
Towards Co-Evaluation of Cameras HDR and Algorithms for Industrial-Grade 6DoF Pose Estimation
Agastya Kalra (Intrinsic Innovation LLC), Michael Stark (Intrinsic Innovation LLC)
CodePose EstimationMultimodality
🎯 What it does: Proposes an industrial-grade 6DoF pose estimation co-evaluation dataset IPD and provides a high-precision evaluation method based on robot consistency.
🎯 What it does: A Fairness-Aware Adversarial Learning (FAAL) framework is proposed, utilizing distributed robust optimization to enhance class fairness in adversarial training.