π― What it does: A novel unsupervised self-supervised framework is proposed, which can decouple pose and expression in the latent space of videos, enabling independent editing of pose and expression in portrait videos.
π― What it does: A dense prediction field (DPF) model based on implicit neural functions is proposed, utilizing point-level weakly supervised learning for dense prediction tasks such as semantic segmentation and intrinsic image decomposition.
π― What it does: A full-process pipeline based on physical self-supervision is proposed, capable of generating various garments through a single network and fitting fabrics on different body shapes and poses.
π― What it does: This paper proposes self-supervised pre-training of Masked Autoencoder (MAE) on videos, aiming to provide better temporal matching representations for Visual Object Tracking (VOT) and Video Object Segmentation (VOS);
π― What it does: Proposes DSFNet, which combines predictions from both image space and model space to achieve occlusion-robust 3D dense face alignment.
π― What it does: A deployable Transformer backbone DSVT for sparse point clouds is proposed, supporting efficient dynamic sparse window attention and 3D attention-based pooling.
π― What it does: A dual-path parameter-efficient transfer framework called DualPath is proposed, utilizing a frozen pre-trained image Transformer (ViT/Swin) to achieve video action recognition through spatial and temporal path adaptation.
DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium
Antyanta Bangunharcana (Korea Advanced Institute of Science and Technology), Kyung-Soo Kim (Korea Advanced Institute of Science and Technology)
CodePose EstimationDepth EstimationAutonomous DrivingSupervised Fine-TuningSimultaneous Localization and MappingImage
π― What it does: This work proposes a self-supervised multi-frame depth and pose estimation framework called DualRefine, which refines both depth and camera pose simultaneously using local epipolar line sampling matching and iterative updates.
DualVector: Unsupervised Vector Font Synthesis With Dual-Part Representation
Ying-Tian Liu (Tsinghua University), Song-Hai Zhang (Adobe)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: This paper studies an unsupervised vector font synthesis method called DualVector, which can generate high-quality vector glyphs solely through bitmap training, enabling font reconstruction and few-shot font generation.
π― What it does: A dynamic mask selection framework called DynaMask is designed to enhance instance segmentation quality through a dual-layer FPN, dynamically selecting the most suitable mask resolution for each instance.
π― What it does: This paper proposes a Dynamic Aggregation Network (DANet) that adaptively aggregates local motion patterns through Local Convolutional Mixture Blocks (LCMB) and a Global Motion Pattern Aggregator (GMPA) to construct robust global gait features.
π― What it does: This paper proposes a Dynamic Coarse-Fine Learning (DCFL) framework specifically designed to address the detection of inclined small targets with extreme shapes and scales.
Dynamic Focus-Aware Positional Queries for Semantic Segmentation
Haoyu He (Monash University), Bohan Zhuang (Monash University)
CodeObject DetectionSegmentationTransformerImage
π― What it does: A dynamic focus-aware position query (DFPQ) and high-resolution cross-attention (HRCA) have been designed and implemented to enhance the localization accuracy and detail recovery of DETR-style semantic segmentation models.
Dynamic Graph Enhanced Contrastive Learning for Chest X-Ray Report Generation
Mingjie Li (University of Technology Sydney), Xiaojun Chang (University of Technology Sydney)
CodeGenerationTransformerContrastive LearningImageTextMultimodalityElectronic Health Records
π― What it does: For the task of generating chest X-ray reports, a dynamic knowledge graph structure is proposed, which enhances visual features through graph encoding and graph-visual cross-attention, and trains the model using image-text contrastive learning and matching loss.
Dynamically Instance-Guided Adaptation: A Backward-Free Approach for Test-Time Domain Adaptive Semantic Segmentation
Wei Wang (Western University), Nicu Sebe (University of Trento)
CodeSegmentationDomain AdaptationImage
π― What it does: The paper proposes a test-time domain adaptive semantic segmentation method (DIGA) that achieves online real-time adaptation without backpropagation.
π― What it does: A 3D visual localization method (EDA) is proposed, which achieves more accurate target localization by decoupling natural language sentences into various semantic components and densely aligning each component with point cloud objects.
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
Yawei Li (ETH Zurich), Luc Van Gool (ETH Zurich)
CodeRestorationSuper ResolutionTransformerImage
π― What it does: A GRL network based on anchor stripe self-attention is proposed, achieving efficient explicit modeling of global, regional, and local features of images.
Efficient Loss Function by Minimizing the Detrimental Effect of Floating-Point Errors on Gradient-Based Attacks
Yunrui Yu (University of Macau), Cheng-Zhong Xu (University of Macau)
CodeOptimizationAdversarial AttackImage
π― What it does: A new loss function MIFPE is proposed, which can reduce gradient distortion caused by floating-point errors in gradient attacks, thereby more accurately assessing model robustness.
Efficient Mask Correction for Click-Based Interactive Image Segmentation
Fei Du (Alibaba Group), Fan Wang (Alibaba Group)
CodeSegmentationConvolutional Neural NetworkImage
π― What it does: An efficient click-based interactive image segmentation method is proposed, which quickly corrects the mask after each click through click-guided self-attention and correlation modules.
π― What it does: A Gradient Filtering method is proposed, which constructs a gradient mapping with fewer unique elements by block averaging the gradient map during backpropagation, significantly reducing the computational load and memory usage of backpropagation in convolutional layers.
π― What it does: A resolution-changing framework for compressed video semantic segmentation, AR-Seg, is proposed, which processes high-resolution keyframes and low-resolution non-keyframes in parallel and improves accuracy through cross-resolution feature fusion.
π― What it does: A three-dimensional multi-frame denoising and view synthesis method based on Multi-Plane Features (MPF) is proposed, which transfers Multi-Plane Images (MPI) to feature space, utilizing a learnable encoder-renderer to achieve cross-depth consistency, significantly improving denoising and synthesis quality.
π― What it does: This study investigates the fine-grained association between sound and vision in first-person perspective videos and proposes a self-supervised framework for locating sound-emitting objects.
EMT-NAS:Transferring Architectural Knowledge Between Tasks From Different Datasets
Peng Liao (East China University of Science and Technology), Wenli Du (East China University of Science and Technology)
CodeClassificationNeural Architecture SearchImageBiomedical Data
π― What it does: This paper proposes an Evolutionary Multi-Task Neural Architecture Search (EMT-NAS), which significantly improves the accuracy of various tasks and reduces search time by sharing network architecture knowledge between classification tasks on different datasets and training weights separately.
π― What it does: An end-to-end single-stage 3D fine-grained description framework called Vote2Cap-DETR is proposed, capable of simultaneously performing object detection and natural language description.
π― What it does: An energy-efficient adaptive 3D perception system is proposed and implemented, utilizing a camera to generate attention maps that project structured light only in the regions of interest, significantly reducing power consumption and enhancing eye safety while maintaining the same maximum measurement distance.
π― What it does: This paper proposes a Selective Query Recollection (SQR) training strategy to enhance the final stage performance of query-based object detectors.
π― What it does: A novel information bottleneck framework (NIB) is proposed, and an autoencoder version (AENIB) is implemented to enhance the model's performance on various robustness metrics by learning representations of negligible noise.
π― What it does: A transfer-based targeted attack method is proposed based on Self-Universality (SU), which generates perturbations that are insensitive to different local areas, more targeted, and do not require additional auxiliary networks by maximizing the feature similarity between the global image and the local image obtained from random cropping.
Enlarging Instance-Specific and Class-Specific Information for Open-Set Action Recognition
Jun Cen (Hong Kong University of Science and Technology), Qifeng Chen (Hong Kong University of Science and Technology)
CodeRecognitionContrastive LearningVideo
π― What it does: In the open set action recognition (OSAR) task, a Prototypical Similarity Learning (PSL) framework is proposed, which enhances instance specificity (IS) and class specificity (CS) information by preserving instance differences in the feature space and introducing video shuffling;
π― What it does: This paper studies a black-box attack method based on multi-model ensemble, which can generate targeted adversarial samples for object detection and semantic segmentation models.
CodePose EstimationAutonomous DrivingOptimizationGraph Neural NetworkGraphTime SeriesSequentialPhysics Related
π― What it does: This study presents EqMotion, a multi-agent motion prediction model that maintains the invariance of Euclidean geometric transformations and remains invariant in interactive reasoning.
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model With Knowledge-Enhanced Mixture-of-Denoising-Experts
Zhida Feng (Baidu Inc), Haifeng Wang (Baidu Inc)
CodeGenerationData SynthesisTransformerMixture of ExpertsDiffusion modelImageText
π― What it does: A Chinese text-to-image generation system based on diffusion models, ERNIE-ViLG 2.0, is proposed, which enhances image quality and text consistency through knowledge enhancement and multi-expert denoising techniques.
π― What it does: A 1B parameter ViT model called EVA was constructed and pre-trained, using a mask feature reconstruction MIM task on publicly available image data, and applied to various visual downstream tasks and cross-modal CLIP models.
Event-Based Blurry Frame Interpolation Under Blind Exposure
Wenming Weng (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
CodeRestorationVideo
π― What it does: This paper proposes a blind exposure blur frame interpolation method based on event cameras, which can recover high frame rate clear videos from low frame rate blurry videos without knowing the exposure time.
π― What it does: This paper proposes a Shape from Polarization (SfP) system utilizing a rotating polarizer and an event camera, which includes both physical modeling and deep learning estimation methods.
π― What it does: A Sparse-Dense Complementary Learning Network (SDCL) based on event cameras is proposed, which enhances video person re-identification performance by guiding video frame feature extraction through event streams.
π― What it does: Proposes Exact-NeRF, which uses pyramid-based precise volume parameterization to achieve accurate positional encoding for NeRF, replacing the traditional Gaussian approximation.
π― What it does: In this paper, the authors propose a conditional human motion generation method based on a latent space diffusion model (Motion Latent-Diffusion, MLD). This method first compresses the original motion sequences into low-dimensional latent variables using a Transformer-VAE, and then trains a conditional diffusion model in this latent variable space to achieve motion synthesis based on action categories or text descriptions.
Explaining Image Classifiers With Multiscale Directional Image Representation
Stefan Kolek (Ludwig-Maximilians-UniversitΓ€t MΓΌnchen), Ron Levie (Technion-Israel Institute of Technology)
CodeExplainability and InterpretabilityConvolutional Neural NetworkSupervised Fine-TuningImage
π― What it does: A mask interpretation method based on wavelet and shearlet transforms, called ShearletX, is proposed for interpreting the decisions of image classifiers.
π― What it does: In semi-supervised anomaly detection, the authors propose the BGAD model, which trains an anomaly detector by guiding semi-pull-and-push contrastive learning with a small number of known anomaly samples through explicit boundary separation.
Exploring the Effect of Primitives for Compositional Generalization in Vision-and-Language
Chuanhao Li (Beijing Institute of Technology), Yuwei Wu (Beijing Institute of Technology)
CodeRecognitionSegmentationTransformerVision Language ModelContrastive LearningVideoMultimodality
π― What it does: This paper proposes a self-supervised learning framework by analyzing the semantic and labeling effects of raw units such as words, image regions, and video frames on visual and language tasks. It utilizes masked generation to create both invariant and variant samples, training the model to learn semantic invariance and variance, thereby enhancing combinatorial generalization ability.
Extracting Class Activation Maps From Non-Discriminative Features As Well
Zhaozheng Chen (Singapore Management University), Qianru Sun (Singapore Management University)
CodeSegmentationImage
π― What it does: This paper proposes a new class activation map generation method called LPCAM, which can extract activation maps that fully cover the target object from classification models.
π― What it does: A framework is proposed that utilizes Inter-Frame Attention to unify the extraction of motion and appearance information in video frame interpolation.
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks
Xiao Han (University of Surrey), Tao Xiang (University of Surrey)
CodeClassificationGenerationRetrievalKnowledge DistillationTransformerVision Language ModelImageTextMultimodality
π― What it does: This paper proposes a multi-task vision-language model named FAME-ViL, which unifies the processing of four types of fashion tasks: cross-modal retrieval, text-guided retrieval, multi-modal classification, and image captioning.
FCC: Feature Clusters Compression for Long-Tailed Visual Recognition
Jian Li (Jilin University), Hao Xu (Jilin University)
CodeRecognitionCompressionImage
π― What it does: Proposes the Feature Clusters Compression (FCC) method, which increases feature clustering density by scaling down backbone features, thereby enhancing long-tail visual recognition performance.
Feature Aggregated Queries for Transformer-Based Video Object Detectors
Yiming Cui (University of Florida)
CodeObject DetectionTransformerVideo
π― What it does: A Transformer-based video object detection method based on query aggregation is proposed, implemented in two forms: vanilla and dynamic.
Feature Alignment and Uniformity for Test Time Adaptation
Shuai Wang (Tsinghua University), Rui Li (Tsinghua University)
CodeSegmentationDomain AdaptationKnowledge DistillationImageBiomedical Data
π― What it does: An online testing adaptive method based on feature unification and alignment is proposed to address the issue of model performance degradation under domain shift.
π― What it does: A Feature Separation and Recalibration (FSR) module is proposed, which enhances the model's adversarial robustness by separating intermediate features into robust and non-robust activations and recalibrating the non-robust activations.
π― What it does: The FedDM method is proposed, which constructs local proxy functions by generating synthetic data at each client, allowing the server to update the global model based on these proxy functions, thus achieving communication-efficient federated learning.
π― What it does: A Federated Domain Generalization (FedDG) global objective and Generalization Adjustment (GA) algorithm is proposed, which utilizes dynamic weight adjustment to reduce the variance of the generalization gap in the source domains, thereby enhancing cross-domain generalization ability.
π― What it does: The Federated Incremental Semantic Segmentation (FISS) problem is proposed, and the FBL (Forgetting-Balanced Learning) model is presented to implement federated incremental semantic segmentation.
Federated Learning With Data-Agnostic Distribution Fusion
Jian-hui Duan (Nanjing University), Sanglu Lu (Nanjing University)
CodeFederated LearningAuto EncoderImage
π― What it does: A federated learning aggregation method called FedFusion is proposed, which infers the global data distribution using virtual distribution components and dynamically adjusts aggregation weights;
π― What it does: The FEND framework is proposed to address the long-tail problem in trajectory prediction through future-enhanced distribution-aware contrastive learning and hypernetwork decoders.
π― What it does: This paper proposes a new framework based on class-aware bidirectional distillation and attention aggregation to address the issues of overfitting and catastrophic forgetting in few-shot incremental learning.
π― What it does: Proposes the FFC-Vision-Computer-Learning (FFC-V) library to eliminate data bottlenecks during training and accelerate model training.
π― What it does: A large-scale high-quality facial UV texture dataset FFHQ-UV is proposed, along with a complete automated generation pipeline and a GAN-based texture decoder, further enhancing the accuracy and texture quality of 3D facial reconstruction from a single image.
π― What it does: This paper proposes and implements the Fine-grained Audible Video Description (FAVD) task and constructs the first dataset for this task, FAVDBench.
π― What it does: A conditional generative adversarial network (FCGAN) capable of handling multi-condition missing data is proposed for three-dimensional radiotherapy dose prediction.
π― What it does: This paper proposes an efficient framework called FlowGrad, which enables controllable generation through gradient optimization in pre-trained ODE generative models, particularly for text-guided image editing.
π― What it does: This paper proposes a click-interaction-based image segmentation framework called FCFI, which uses the segmentation results from the previous interaction as feedback to guide subsequent clicks.
π― What it does: A video recognition framework capable of maintaining high performance at different frame rates (Frame Flexible Network, FFN) is proposed.
π― What it does: This paper proposes the task of Freestyle Layout to Image Synthesis (FLIS) and designs FreestyleNet, which combines a pre-trained text-to-image diffusion model (Stable Diffusion) with a new Rectified Cross-Attention (RCA) module to achieve high-fidelity image generation based on semantic masks and text.
π― What it does: A point cloud rendering pipeline based on frequency modulation is proposed, capable of achieving high-fidelity detail reconstruction, real-time rendering, and user-friendly editing.
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models
Jiaxian Guo (University of Sydney), Steven Hoi (Salesforce Research)
CodeRecognitionGenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: A Plug-and-Play module called Img2LLM is proposed, which utilizes existing visual models to generate question-answer pairs and descriptions related to images, directly injecting them as prompts into any large language model (LLM) to accomplish zero-shot visual question answering (VQA) tasks.
π― What it does: The PEER regularization method is proposed, which explicitly constrains the internal representations of the Q network and its target network to maintain distinguishability, thereby enhancing the performance and sample efficiency of deep reinforcement learning.
π― What it does: This paper proposes a completely self-supervised depth estimation framework that utilizes sparse focus stacks without requiring true labels for depth or panoramic focus images.
Fusing Pre-Trained Language Models With Multimodal Prompts Through Reinforcement Learning
Youngjae Yu (Allen Institute for Artificial Intelligence), Yejin Choi (OpenAI)
CodeGenerationTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodalityAudio
π― What it does: Proposes the ESPER framework, which utilizes reinforcement learning to extend pre-trained language models into text generators capable of handling multimodal inputs such as images and audio.
Fuzzy Positive Learning for Semi-Supervised Semantic Segmentation
Pengchong Qiao (Peking University), Jie Chen (Peking University)
CodeSegmentationImage
π― What it does: This paper proposes a Fuzzy Positive Learning (FPL) framework that utilizes multiple fuzzy positive class labels for each pixel to perform semi-supervised semantic segmentation, avoiding the misguidance of the model by a single pseudo-label.
Generalist: Decoupling Natural and Robust Generalization
Hongjun Wang (Peking University), Yisen Wang (Peking University)
CodeClassificationOptimizationAdversarial AttackConvolutional Neural NetworkMixture of ExpertsImage
π― What it does: A Generalist framework is proposed, which separately trains two base learners for natural classification and adversarial robustness, and then periodically aggregates their parameters to generate a global model, thereby improving adversarial robustness while maintaining high natural accuracy.
π― What it does: Pre-trained general local features for deformable 3D shapes and proposed a differentiable receptive field optimization method, allowing features to remain efficient in cross-category transfer tasks;
π― What it does: In multi-student online knowledge distillation, a Hybrid-Weight Model is formed through parameter mixing, guiding student learning with its supervised loss.
π― What it does: A unified 3D shape prior model, 3DQD, is proposed, which combines part-based discrete encoding, a discrete diffusion generator, and a multi-frequency fusion module to achieve high-quality and diverse shape generation, completion, and cross-modal generation.
Generalized Relation Modeling for Transformer Tracking
Shenyuan Gao (Hong Kong University of Science and Technology), Jun Zhang (Hong Kong University of Science and Technology)
CodeObject TrackingTransformerVideo
π― What it does: A general relation modeling method GRM is proposed, which achieves adaptive interaction between the template and search area in the Transformer tracker through learnable token partitioning.
π― What it does: Redefines the semantic segmentation problem as a mask generation task conditioned on images, using discrete latent variable learning to model the posterior distribution of the masks, and then learning the latent prior through an image encoder, allowing for the generation of segmentation masks given an input image.
π― What it does: A two-stage knowledge distillation framework G2SD is proposed, which first performs general knowledge distillation on a pre-trained masked autoencoder, and then conducts task-specific knowledge distillation on downstream tasks to enhance the performance of lightweight Vision Transformers.
GeoLayoutLM: Geometric Pre-Training for Visual Information Extraction
Chuwei Luo (Alibaba Group), Cong Yao (Alibaba Group)
CodeRecognitionTransformerVision Language ModelImageTextMultimodality
π― What it does: A multimodal pre-training framework named GeoLayoutLM is proposed, specifically designed to learn geometric layout representations for semantic entity recognition (SER) and relation extraction (RE) tasks in visual information extraction (VIE).
π― What it does: A self-supervised pre-training framework for 3D medical imaging based on geometric visual similarity learning is proposed, utilizing geometric matching to achieve cross-image semantic similarity learning and train consistent feature representations.
π― What it does: A category incremental learning framework for 3D point cloud semantic segmentation is proposed, which can gradually learn new categories and maintain the performance of old categories without storing old data.
π― What it does: This paper proposes GeoMVSNet, which explicitly injects coarse geometric information into the fine stage through a two-branch geometric fusion network and probabilistic volume embedding, enhancing the accuracy and completeness of multi-view stereo reconstruction.
π― What it does: This paper presents GlassesGAN, a personalized virtual try-on framework for optical lenses based on StyleGAN2, which can add and continuously adjust the appearance of glasses on high-resolution facial images.
π― What it does: A single-stage training framework GLMC is proposed, which enhances long-tail visual recognition using global and local mixed consistency loss and cumulative class-balanced reweighting loss.
π― What it does: Global structural pruning is performed on the Vision Transformer, reallocating dimensions of QKV, MLP, etc. within each layer to achieve efficient parameter utilization, resulting in the proposed NViT series models.
Global-to-Local Modeling for Video-Based 3D Human Pose and Shape Estimation
Xiaolong Shen (Zhejiang University), Yi Yang (Zhejiang University)
CodePose EstimationTransformerVideo
π― What it does: A video-based 3D human pose and shape estimation framework named GLoT is proposed, which utilizes global and local Transformers to model long-term and short-term information respectively, and achieves synergy between the two through cross-attention.
π― What it does: This paper proposes a general geometric model-based neural radiance field (GM-NeRF) framework that generates high-fidelity free-viewpoint images of arbitrary human shapes from sparse multi-view images.
π― What it does: A dual-branch model based on causal self-intervention (AIM) is proposed to automatically eliminate clothing bias and enhance the performance of clothing variation person re-identification.