European Conference on Computer Vision Β· 980 papers
Enriching Information and Preserving Semantic Consistency in Expanding Curvilinear Object Segmentation Datasets
Qin Lei (Chongqing University), Qizhu Dai (Chongqing University)
CodeSegmentationData SynthesisVision Language ModelDiffusion modelImageTextBiomedical Data
π― What it does: This paper proposes a text feature-based semantic map generation and control network (SCP ControlNet) to expand the dataset for curve-shaped object segmentation, enhancing model performance by generating synthetic data with high information content that differs from the original distribution.
π― What it does: Proposed an adaptive data augmentation framework called EntAugment based on information entropy, which can dynamically adjust the augmentation intensity during training according to sample difficulty and model status, and introduced Entropy Regularization Loss (EntLoss) to enhance the model's classification confidence.
Eta Inversion: Designing an Optimal Eta Function for Diffusion-based Real Image Editing
Wonjun Kang (FuriosaAI), Hyung Il Koo (FuriosaAI)
CodeGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelImageBenchmark
π― What it does: Propose a diffusion model inversion method called Eta Inversion, which injects real Gaussian noise into the reverse sampling process using a time- and region-variant Ξ· function, thereby achieving more flexible and high-quality text-to-image editing while preserving the structural integrity of the source image.
π― What it does: This paper proposes an event camera-based head pose estimation (EV-HPE) framework, constructs two large-scale event head pose datasets (Prophesee-HP and Davis-HP), and implements and evaluates head pose estimation on these datasets.
Shuang Guo (TU Berlin and Robotics Institute Germany), Guillermo Gallego (TU Berlin and Robotics Institute Germany)
CodePose EstimationOptimization
π― What it does: Designed a direct photometric bundle adjustment (EMBA) method based on event cameras, which directly optimizes the camera rotation trajectory and panoramic gradient maps using event streams, ultimately reconstructing high-quality grayscale panoramic images from gradients.
Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization
Ming-Yang Ho (National Taiwan University), βͺYufeng Jane Tseng
CodeImage TranslationConvolutional Neural NetworkGenerative Adversarial NetworkImageBiomedical Data
π― What it does: Propose the Dense Normalization (DN) layer, which achieves pixel-level statistics estimation to address seams and jitter artifacts in ultra-high-resolution unpaired image translation, while preserving local color and contrast.
π― What it does: Explored the possibility of using event cameras for continuous sign language recognition and translation, collected the EvSign dataset, and proposed a Transformer framework.
Expanding Scene Graph Boundaries: Fully Open-vocabulary Scene Graph Generation via Visual-Concept Alignment and Retention
Zuyao Chen (Hong Kong Polytechnic University), Chang Wen Chen (Hong Kong Polytechnic University)
CodeRecognitionObject DetectionGenerationKnowledge DistillationTransformerVision Language ModelImageText
π― What it does: Proposed a fully open-vocabulary scene graph generation framework called OvSGTR, which can identify unseen objects and relations under four different open-vocabulary settings (closed-set, open objects, open relations, open objects+relations).
Explicitly Guided Information Interaction Network for Cross-modal Point Cloud Completion
Xu Hang (Wuhan University), Bisheng Yang (Wuhan University)
CodeGenerationTransformerVision Language ModelMultimodalityPoint Cloud
π― What it does: Proposes the EGIInet view-guided point cloud completion network, achieving cross-modal feature fusion through a unified encoder and explicit guidance information interaction.
π― What it does: Propose a learning-based denoising framework called MTDNet based on multi-frame time-of-flight (ToF) depth maps, leveraging inter-frame correlations to efficiently suppress multi-path interference (MPI) and scattering noise.
π― What it does: The study generates adversarial perturbations by exploiting the vulnerability of supervised learning models in availability poisoning scenarios, using them as an enhancement method to improve the defense effectiveness of self-supervised learning (SSL), proposing the VESPR method.
Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentation
Tong Shao (Harbin Institute of Technology), Jingyong Su (Harbin Institute of Technology)
CodeSegmentationTransformerVision Language ModelContrastive LearningImage
π― What it does: Propose CLIPtrase, a training-free method that restores the local semantic correlations of CLIP's visual features through self-correlation, and achieves open-vocabulary semantic segmentation via DBSCAN clustering and denoising.
Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection
Ting Lei (Peking University), Yang Liu (Peking University)
CodeObject DetectionTransformerPrompt EngineeringVision Language ModelMultimodality
π― What it does: Proposed a zero-shot human-object interaction (HOI) detection framework called CMMP, which utilizes conditional multimodal prompts to separate visual and language tasks, enhancing the ability to recognize unseen interactions.
π― What it does: Propose a Guided Sampling method (GANdance) based on GAN latent space vector arithmetic, achieving improved conditional generation with no training or light training.
π― What it does: Investigated the phrase-level visual understanding capability of text-to-image diffusion models beyond the sentence level, proposing the DiffPNG framework to achieve zero-shot Panoptic Narrative Grounding.
Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation
Xuelu Feng (University at Buffalo), Zixin Zhu (University at Buffalo)
CodeSegmentationConvolutional Neural NetworkTransformerVision Language ModelDiffusion modelVideoTextMultimodality
π― What it does: Proposes a Referring Video Object Segmentation (R-VOS) framework named VD-IT based on pre-trained text-video diffusion models, which extracts visual features from a fixed diffusion model using text-guided image projection and video-specific noise prediction, and designs a multi-scale, Transformer-based mask decoder for video object segmentation.
π― What it does: This paper proposes a lightweight Transformer tracker called FERMT, which splits attention into two stages: feature extraction and relation modeling. It incorporates CNN-based dual attention units into a state-of-the-art architecture to enhance speed and accuracy.
Exploring Vulnerabilities in Spiking Neural Networks: Direct Adversarial Attacks on Raw Event Data
Yanmeng Yao (Nanjing University of Information Science and Technology), Bin Gu (Mohamed bin Zayed University of Artificial Intelligence)
CodeAdversarial AttackSpiking Neural NetworkVideo
π― What it does: This paper proposes an adversarial attack method directly targeting raw event data (COO format), which can efficiently and controllably attack vision models based on spiking neural networks (SNNs) without relying on grid-based intermediate representations.
π― What it does: Aiming at 'face reconstruction transfer attack' (FRTA) in facial recognition systems, this paper proposes an attack method based on an implicit generator that can successfully reconstruct identity-matching synthetic faces on unseen encoders.
Fairness-aware Vision Transformer via Debiased Self-Attention
Yao Qiang (Wayne State University), Dongxiao Zhu (Wayne State University)
CodeSafty and PrivacyAdversarial AttackTransformerImage
π― What it does: This paper proposes a Vision Transformer framework called DSA based on debiased attention, which locates and masks pseudo features by leveraging a biased model and adversarial attacks before training, and then aligns attention weights during training to achieve fair prediction.
FairViT: Fair Vision Transformer via Adaptive Masking
Bowei Tian (Wuhan University), Yanning Shen (University of California, Irvine)
CodeClassificationTransformerImage
π― What it does: Propose the FairViT framework, integrating adaptive masks and distance loss to balance accuracy and fairness within Vision Transformers.
π― What it does: Generate a complete human back view from a single RGB image using a perspective hallucinator, then predict geometry with an implicit function (PIFuHD) and combine front and back view textures to produce a high-fidelity fully textured 3D mesh.
Fast Context-Based Low-Light Image Enhancement via Neural Implicit Representations
TomΓ‘Ε‘ Chobola (Technical University of Munich), Tingying Peng (Technical University of Munich)
CodeRestorationNeural Radiance FieldImage
π― What it does: Propose the CoLIE method, which utilizes neural implicit representations to map 2D coordinates to the illumination component of low-light images, recovers brightness within the HSV space, and achieves efficient processing through guided filtering.
π― What it does: Propose an optimization-based fast animation graphic sprite decomposition method that can split raster animation videos into static textures and time-varying affine animation parameters, facilitating subsequent editing;
Federated Learning with Local Openset Noisy Labels
Zonglin Di (University of California, Santa Cruz), Yang Liu (University of California, Santa Cruz)
CodeOptimizationFederated LearningSafty and PrivacyContrastive LearningImageBenchmark
π― What it does: This paper proposes a new federated learning framework, FedDPCont, to address the problem of local open-set noisy labels on each client.
π― What it does: Designed the FedHARM framework to enable collaborative training of different CNN architectures (ResNet, EfficientNet, MobileNetV3) under the same federated learning scenario, achieving model-agnostic aggregation through gradient feature extraction and block-level feature alignment.
FedVAD: Enhancing Federated Video Anomaly Detection with GPT-Driven Semantic Distillation
Fan Qi (Tianjin University of Technology), Changsheng Xu (Chinese Academy of Sciences)
CodeAnomaly DetectionFederated LearningKnowledge DistillationTransformerLarge Language ModelVideoTextMultimodalityBenchmark
π― What it does: Proposes FedVAD, a privacy-friendly video anomaly detection framework based on federated learning, integrating visual consistency clustering and GPT-driven semantic distillation.
π― What it does: Propose a generation method driven by a small number of real anomaly samples (AnoGen), which learns embeddings in a pre-trained diffusion model and combines bounding box guidance to generate anomaly images with semantic consistency and spatial controllability. These generated images are then used to train a weakly supervised anomaly detection model, improving performance on anomaly classification and segmentation tasks.
π― What it does: Propose the AttentionβAware SelfβAdaptive Prompt (ASP) framework, which utilizes a fixed pre-trained ViT and inserts task-invariant prompts (TIP) and adaptive task-specific prompts (TSP) between respective attention layers to achieve few-shot class incremental learning.
π― What it does: For few-shot generation of industrial defect images, the DefectDiffu model is proposed, which can generate high-quality, diverse defect images under the condition of only a small number of defect samples and can automatically generate corresponding masks.
π― What it does: Improve NeRF under sparse views by proposing an adaptive rendering loss regularization method, achieving frequency relation alignment through two-stage rendering supervision and uncertainty learning to better learn global structures and local details.
π― What it does: Propose DyBDet dynamic network for generalized event boundary detection, which can adaptively allocate subnetworks and achieve precise boundary localization.
π― What it does: Propose a fine-grained scene graph generation method based on sample-level bias prediction (SBP), which predicts sample-specific bias using joint features and global priors through a generative adversarial network (BGAN) to correct coarse-grained relationship predictions into fine-grained relationships.
FipTR: A Simple yet Effective Transformer Framework for Future Instance Prediction in Autonomous Driving
Xingtai Gui, Chi Zhang
CodeAutonomous DrivingTransformerImage
π― What it does: Proposed a fully end-to-end Transformer framework called FipTR, which directly predicts BEV instance segmentation and motion states in future perspectives using instance queries, eliminating auxiliary outputs and post-processing;
Fisher Calibration for Backdoor-Robust Heterogeneous Federated Learning
Wenke Huang (Wuhan University), Dacheng Tao (Nanyang Technological University)
CodeFederated LearningAdversarial AttackImage
π― What it does: Propose the Self-Driven Fisher Calibration (SDFC) method, which in heterogeneous federated learning measures the difference in parameter importance between local and validation distributions through the Fisher information matrix, then introduces FDReg regularization during local training and assigns weighted aggregation during model aggregation based on the differences, thereby defending against backdoor attacks.
π― What it does: This paper proposes a method to efficiently map 2D segmentation masks to 3D Gaussian splatting rendering, completing 3D segmentation with a single global optimal solution.
π― What it does: This paper proposes a Sequential Backdoor Learning (SBL) framework that utilizes continual learning techniques to actively resist fine-tuning defenses during training, maintaining high attack success rates for backdoor models.
π― What it does: Propose a framework named ADELLO for long-tailed semi-supervised learning, with the core being Flexible Distribution Alignment (FlexDA) and complementary consistency regularization, dynamically aligning the distribution of unlabeled data and achieving model calibration.
π― What it does: Proposed a flow-based contrastive learning method called FlowCon for out-of-distribution (OOD) detection without modifying the pre-trained classifier.
Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models
Yuchen Yang (Johns Hopkins University), Shao-Yuan Lo (Honda Research Institute USA)
CodeAnomaly DetectionTransformerLarge Language ModelVision Language ModelVideoText
π― What it does: Developed a rule-based reasoning framework for anomaly video detection called AnomalyRuler, which induces LLMs to generate detection rules using a small number of normal samples and identifies anomalies based on rules during inference.
π― What it does: This paper proposes a training-free method called FouriScale, which utilizes dilated convolutions and low-pass filtering in the frequency domain to achieve structural and scale consistency with pre-trained diffusion models, thereby generating high-resolution images of arbitrary sizes without retraining.
π― What it does: Proposes FreeCompose, which achieves general zero-shot image synthesis by leveraging the generative prior of pre-trained diffusion models.
π― What it does: Propose a text-driven image editing method called FreeDiff, which does not require fine-tuning and is based on frequency domain truncation. It uses advanced frequency truncation to progressively refine the guidance of diffusion models, enabling unified processing for various editing tasks.
π― What it does: Proposed a frequency-spectrum and spatial entanglement learning framework (FSEL) that combines frequency domain and spatial domain features for concealed object detection.
π― What it does: Propose a two-stage training pipeline FFR, pre-training on balanced synthetic images first, then fine-tuning on real images to reduce spurious correlation in visual recognition models;
π― What it does: Proposed Frontier-Enhanced Topological Memory (FTM), which integrates ghost nodes with perceptual features into topological maps, predicts visual information for them using an online-trained NeRF, and achieves end-to-end image goal navigation with a multi-stage memory extraction module.
π― What it does: Propose a Foreground Self-Distillation (FSD) framework to enhance the performance of Birdβs-Eye-View (BEV) 3D object detection based on multi-view cameras.
Haisong Liu (Nanjing University), Limin Wang (Nanjing University)
CodeAutonomous DrivingTransformerImagePoint Cloud
π― What it does: Designed a fully sparse 3D occupancy prediction network called SparseOcc, which first recovers a sparse 3D structure from multi-view images using a sparse voxel decoder, and then predicts semantic/instance occupancy information within the sparse space through a mask transformer.
Fundamental Matrix Estimation Using Relative Depths
Yaqing Ding (Czech Technical University in Prague), Zuzana Kukelova (Czech Technical University in Prague)
CodePose EstimationDepth EstimationImage
π― What it does: Proposed a minimal solver for estimating the fundamental matrix of dual cameras using relative depth information, and presented three solving methods: 4p4d, 4p3d, and 3p3d;
Binzhu Xie (Beijing University of Posts and Telecommunications), Ziwei Liu (Nanyang Technological University)
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityBenchmarkRetrieval-Augmented Generation
π― What it does: Propose the FunQA video question-answering dataset, focusing on humorous, creative, and magic surprise videos, and design multi-task evaluation.
FuseTeacher: Modality-fused Encoders are Strong Vision Supervisors
Chen-Wei Xie (Alibaba Group), Yun Zheng (University Of Science And Technology Of China)
CodeClassificationRetrievalKnowledge DistillationRepresentation LearningTransformerVision Language ModelContrastive LearningMultimodality
π― What it does: By introducing a lightweight fusion encoder to perform multi-modal fusion of images and text, and using the fused representation for classification distillation and retrieval distillation in the visual encoder, the FuseTeacher method achieves stronger visual representation learning.
π― What it does: This paper proposes the GAReT method, which migrates a Vision Transformer-based image geolocation model to video input through a GeoAdapter, and generates temporally consistent GPS trajectories using TransRetriever.
π― What it does: Proposes the Gated Temporal Diffusion (GTD) model, which jointly models the uncertainty of observations and future actions using diffusion models to generate diverse long-term action predictions;
GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting
Xinjie Zhang (Hong Kong University of Science and Technology), Jun Zhang (Hong Kong University of Science and Technology)
CodeCompressionGaussian SplattingImage
π― What it does: Proposed a GaussianImage method based on 2D Gaussian distribution for image representation and compression, achieving an image encoder with low memory usage, fast training, and extremely high rendering speed.
π― What it does: Designed a unified task-oriented video segmentation framework called GvSeg, capable of simultaneously handling four types of tasks: example-guided, instance, semantic, and panoptic video segmentation.
General Geometry-aware Weakly Supervised 3D Object Detection
Guowen Zhang (The Hong Kong Polytechnic University), Lei Zhang (The Hong Kong Polytechnic University)
CodeObject DetectionAutonomous DrivingConvolutional Neural NetworkLarge Language ModelImagePoint Cloud
π― What it does: Proposes a generic geometry-aware weakly supervised 3D object detection framework GGA, which learns 3D bounding boxes by leveraging 2D boxes and shape ratio priors generated by LLMs.
GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features
Luc P.J. StrΓ€ter (University of Amsterdam), Yuki M. Asano (University of Amsterdam)
CodeAnomaly DetectionTransformerImage
π― What it does: Designed a general-purpose anomaly detection framework called GeneralAD, which leverages pre-trained Vision Transformer features to generate pseudo-anomalous features and achieves image-level and pixel-level anomaly detection and localization through a cross-patch attention discriminator.
Yuhang Zhang (Beijing University of Posts and Telecommunications), Weihong Deng (Beijing University of Posts and Telecommunications)
CodeRecognitionDomain AdaptationVision Language ModelContrastive LearningImage
π― What it does: This paper proposes a general expression recognition method that utilizes fixed CLIP facial features and a learnable Sigmoid mask, enabling training only on the training set and zero-shot inference on multi-domain test sets.
π― What it does: The paper proposes Symbolic Optimizer Learner (SOL), a method that directly learns and outputs interpretable symbolic optimizers on target tasks.
GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes
Ibrahim Ethem Hamamci (University Of Zurich), Bjoern Menze (Istanbul Medipol University)
CodeGenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelDiffusion modelTextBiomedical DataComputed Tomography
π― What it does: This study proposes the GenerateCT framework, which can generate high-resolution, three-dimensional chest CT volumes based on free-text prompts;
π― What it does: This paper proposes the GenAD framework, treating autonomous driving as a generative problem, and simultaneously performs motion prediction and planning in the structural latent space using an instance-centric scene tokenizer and a VAE+GRU structure.
GENIXER: Empowering Multimodal Large Language Models as a Powerful Data Generator
Henry Hengyuan Zhao, Mike Zheng Shou (National University of Singapore)
CodeData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningImageTextBenchmark
π― What it does: Propose the Genixer data generation pipeline, train LLaVA1.5 and Shikra to generate vision instruction tuning data, construct two synthetic datasets (Genixer-915K and Genixer-350K), and verify their effectiveness in improving the performance of multimodal large language models.
π― What it does: Propose GenQ, which uses Stable Diffusion to generate high-quality synthetic images and employs energy, BN/Patch filtering for quantization training with low data volume;
π― What it does: Studied a self-supervised learning view generation framework called GenView based on pre-trained generative models to enhance the quality and diversity of positive sample views.
π― What it does: Propose GeoCalib, a method that combines a deep network to predict the Perspective Field with differentiable geometric optimization (LevenbergβMarquardt) to achieve single-image camera calibration (focal length, gravity direction, distortion).
Michael Tschannen (Google DeepMind), Fabian Mentzer (Google DeepMind)
CodeSegmentationGenerationDepth EstimationTransformerMixture of ExpertsAuto EncoderImage
π― What it does: Propose Generative Infinite-Vocabulary Transformer (GIVT), which directly generates continuous vector sequences instead of discrete tokens, using the continuous latent space of Ξ²-VAE as input; two modifications are made to the Transformer: input uses linear projection instead of vocabulary embeddings, and output uses a multivariate Gaussian mixture model instead of classification distribution.
GKGNet: Group K-Nearest Neighbor based Graph Convolutional Network for Multi-Label Image Recognition
Ruijie Yao (Tsinghua University), Ji Wu (Tsinghua University)
CodeRecognitionGraph Neural NetworkImage
π― What it does: Proposes a fully graph convolution-based multi-label image recognition model called GKGNet, which utilizes a unified graph structure to combine image patches and label embeddings for interaction, enabling more accurate label predictions on targets of different scales and shapes.
π― What it does: Proposed a global and local adaptive diffusion model GLAD for unsupervised anomaly detection, achieving more accurate reconstruction and anomaly localization by dynamically determining denoising steps per image and fusing local features.
GLARE: Low Light Image Enhancement via Generative Latent Feature based Codebook Retrieval
Han Zhou (McMaster University), Jun Chen (McMaster University)
CodeRestorationFlow-based ModelAuto EncoderImage
π― What it does: Proposes the GLARE method, which utilizes three modules: a VQ codebook learned from normally illuminated images, invertible latent normalization flows, and adaptive feature transformations to achieve natural enhancement of low-light images to normal illumination.
BartΕomiej Sobieski (Warsaw University of Technology), Przemyslaw Biecek (Warsaw University of Technology)
CodeExplainability and InterpretabilityDiffusion modelAuto EncoderImageBiomedical Data
π― What it does: Proposes a black-box visual counterfactual explanation method that generates cross-image set counterfactual images by leveraging global directions (GCDs) in the semantic latent space of a diffusion autoencoder (DiffAE), and implements black-box latent integral gradient (BB-LIG) explanations using finite differences.
CodePose EstimationRetrievalOptimizationSimultaneous Localization and MappingImageBenchmark
π― What it does: Proposed a Global Structure Optical Flow (GLOMAP) system that achieves joint localization of camera pose and 3D points, bypassing traditional translation averaging plus global triangulation steps;
Global-Local Collaborative Inference with LLM for Lidar-Based Open-Vocabulary Detection
Xingyu Peng (Beihang University), Si Liu (Meituan)
CodeObject DetectionTransformerLarge Language ModelPoint CloudChain-of-Thought
π― What it does: This paper proposes a global-local collaborative reasoning framework called GLIS, which combines global scene information and local object information using LiDAR point clouds, and employs a Large Language Model (LLM) for chain reasoning to achieve open-vocabulary detection in point clouds.
π― What it does: Proposed a Geometry-Driven Multi-Reference Texture Transfer (GMT) module to enhance the quality of novel view synthesis images based on Generalizable NeRF (G-NeRF) through geometry-driven feature alignment and multi-reference texture transfer.
π― What it does: Proposed a Transformer-based point cloud understanding model called GPSFormer, with core components including the Global Perception Module (GPM) (using ADGConv, residual cross-attention, and multi-head attention) and the Local Structure Fitting Convolution (LSFConv) (leveraging low-order and high-order convolutions derived from Taylor series decomposition).
π― What it does: Designed and implemented a lightweight GRA module to replace traditional convolutions, enhancing orientation perception capability for tilted object detection.
π― What it does: This paper addresses the class imbalance problem in semi-supervised medical image segmentation by proposing a gradient-aware (GA) loss, improving the gradient behavior of Dice and cross-entropy to enhance training stability and performance on small classes.
Taha Entesari (Johns Hopkins University), Mahyar Fazlyab (Johns Hopkins University)
CodeAnomaly DetectionImage
π― What it does: Propose a gradient-regularized OOD detection method GReg (along with its accompanying energy sampling algorithm GReg+), which allows the network to learn both OOD scores and their local gradients during training, thereby achieving a smoother score manifold;
Graph Neural Network Causal Explanation via Neural Causal Models
Arman Behnam (Illinois Institute of Technology), Binghui Wang (Illinois Institute of Technology)
CodeExplainability and InterpretabilityDrug DiscoveryGraph Neural NetworkGraphBiomedical Data
π― What it does: This paper proposes a graph neural network interpreter CXGNN based on causal inference, which can automatically identify the causal subgraph leading to model predictions.
π― What it does: This paper proposes the GraphBEV framework, addressing the issue of local and global misalignment in LiDAR and camera Birdβs-Eye-View (BEV) feature fusion caused by projection errors, and constructs a robust multi-modal 3D object detection model.
Gravity-aligned Rotation Averaging with Circular Regression
Linfei Pan (ETH Zurich), Daniel Barath
CodePose EstimationAutonomous DrivingOptimizationSimultaneous Localization and MappingImageVideo
π― What it does: Studied a rotational averaging method that utilizes gravity direction information to reduce degrees of freedom and improve the accuracy of camera poses in global SfM.
π― What it does: Propose the Grid-Attention (GridAttn) module, which can directly replace the multi-head attention layers in large visual Transformers (e.g., SAM, Stable Diffusion, SegFormer) without training or fine-tuning, to improve inference efficiency while maintaining or slightly enhancing performance.
GRiT: A Generative Region-to-text Transformer for Object Understanding
Jialian Wu (State University of New York at Buffalo), Lijuan Wang (State University of New York at Buffalo)
CodeRecognitionObject DetectionTransformerLarge Language ModelVision Language ModelImageText
π― What it does: A generative region-to-text Transformer (GRiT) was studied, which can simultaneously locate objects in images and freely describe their attributes with natural language, achieving a unified framework for object detection and dense description.
π― What it does: Propose the GroCo framework, which leverages ground prior in self-supervised monocular depth estimation to recover scale and enhance model generalization.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Shilong Liu (Tsinghua University), Lei Zhang (International Digital Economy Academy)
CodeObject DetectionTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodality
π― What it does: Developed an open-set object detection model called Grounding DINO based on DINO, which can detect any object through text or pointing expressions.
CodeRecognitionTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodalityBenchmarkRetrieval-Augmented Generation
π― What it does: Propose AutoVER, a retrieval-enhanced constraint generation-based autoregressive multimodal large language model for visual entity recognition.
Yuming Jiang (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
CodeGenerationPose EstimationDiffusion modelImage
π― What it does: Propose GroupDiff, a unified framework based on diffusion models for group portrait editing, enabling insertion, deletion, and interactive editing of individuals.
GTMS: A Gradient-driven Tree-guided Mask-free Referring Image Segmentation Method
Haoxin Lv (Beijing Institute of Technology), Sanyuan Zhao (Beijing Institute of Technology)
CodeSegmentationTransformerVision Language ModelImageTextMultimodality
π― What it does: Proposed a gradient-driven, tree-guided, mask-free reference image segmentation method called GTMS, which uses only bounding box supervision. It combines structural trees with GradCAM to generate high-quality pseudo-labels, thereby improving the performance of weakly supervised RIS.
GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation
Haonan Wang (Nanjing University), Yong Wang (Cainiao Network)
CodePose EstimationTransformerImage
π― What it does: Proposed a Transformer-based efficient human pose estimation framework called GTPT, which significantly reduces computational cost while maintaining high accuracy, especially suitable for full-body pose estimation with a large number of keypoints.
π― What it does: Propose a self-guided 'Guide-and-Rescale' framework that enables various edits (e.g., local modifications, style transfer) on real images without fine-tuning or reconstructing diffusion models
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning
Zhecan Wang (Columbia University), Golnaz Ghiasi (Google DeepMind)
CodeData SynthesisLarge Language ModelSupervised Fine-TuningDiffusion modelImageTextMultimodalityBenchmark
π― What it does: Propose the HaloQuest visual question answering dataset, combining real and synthetic images to specifically evaluate multimodal hallucinations and provide fine-tuning resources.
π― What it does: Designed a denoising adaptive graph transformer, HandDAGT, which accurately estimates 3D hand pose using multi-modal inputs of depth maps and point clouds.
Handling The Non-Smooth Challenge in Tensor SVD: A Multi-Objective Tensor Recovery Framework
Jingjing Zheng (University of British Columbia), Xianta Jiang (Memorial University of Newfoundland)
CodeRestorationOptimizationImageVideo
π― What it does: This paper proposes a multi-objective tensor recovery framework based on a learnable tensor nuclear norm to address the non-smooth challenges in tensor SVD and achieve high-order tensor completion.
π― What it does: Propose an animation-driven head avatar generation framework called HeadStudio, which combines 3D Gaussian Splatting with FLAME head prior to generate high-quality avatars that can be rendered in real-time.
π― What it does: Propose a hierarchical geometric learning framework (HGL) that extracts geometric information from three levelsβpoint-level, object-level, and frame-levelβduring test-time adaptation for point cloud segmentation to generate pseudo labels and update the model.