🎯 What it does: A multi-modal tracking framework called ViPT based on visual prompts is proposed, which utilizes a small number of learnable prompts to achieve RGB + depth, thermal imaging, event, and other multi-modal tracking on a frozen RGB pre-trained model.
🎯 What it does: By performing prompt tuning on a generative visual Transformer pre-trained on a large-scale dataset, we achieve image synthesis transfer learning across various visual domains (natural, structured, specialized, few-shot, etc.).
🎯 What it does: This paper proposes Visual Query Tuning (VQT), which inserts learnable query tokens before each layer of the Vision Transformer. It uses only these tokens as queries to aggregate intermediate features while keeping the original features unchanged, achieving linear probing with a frozen backbone.
Visual-Language Prompt Tuning With Knowledge-Guided Context Optimization
Hantao Yao (Institute of Automation, Chinese Academy of Sciences), Changsheng Xu (Institute of Automation, Chinese Academy of Sciences)
CodeClassificationRecognitionDomain AdaptationOptimizationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: A prompt tuning method named Knowledge-guided Context Optimization (KgCoOp) is proposed to enhance the generalization performance of pre-trained vision-language models (such as CLIP) in downstream tasks, particularly in recognizing unseen categories.
Vita-CLIP: Video and Text Adaptive CLIP via Multimodal Prompting
Syed Talal Wasim (Mohamed bin Zayed University of Artificial Intelligence), Mubarak Shah (University of Central Florida)
CodeClassificationRecognitionRetrievalTransformerPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: A multi-modal prompt learning scheme called Vita-CLIP is proposed, which utilizes a frozen CLIP pre-trained model for efficient adaptation to video tasks, balancing supervised learning and zero-shot generalization.
ViTs for SITS: Vision Transformers for Satellite Image Time Series
Michail Tarasiou (Imperial College London), Stefanos Zafeiriou (Imperial College London)
CodeClassificationSegmentationTransformerImageTime Series
🎯 What it does: A spatiotemporal decomposed Vision Transformer (TSViT) is proposed, specifically designed for semantic segmentation and classification tasks of satellite image time series.
🎯 What it does: This paper proposes a Visual-Language Semantic Assisted Training (VL-SAT) framework to enhance 3D semantic scene graph prediction in point clouds.
VLPD: Context-Aware Pedestrian Detection via Vision-Language Semantic Self-Supervision
Mengyin Liu (University of Science and Technology Beijing), Xu-Cheng Yin (University of Science and Technology Beijing)
CodeObject DetectionVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Achieving explicit modeling of semantic context through a visual-language self-supervised method to enhance pedestrian detection performance.
🎯 What it does: This paper proposes a regularization method based on Von Neumann entropy (VNE) to directly control the eigenvalue distribution of the autocorrelation matrix of deep model representations, thereby improving representation quality.
🎯 What it does: A cross-scene implicit reconstruction method called VolRecon based on Signed Ray Distance Function is proposed, utilizing projection features and global voxel features, and achieving detail-rich surface reconstruction through view Transformer and ray Transformer.
🎯 What it does: In the text-video retrieval task, parameter-efficient prompt tuning of the pre-trained CLIP model is performed, and three types of video-specific prompts (position, context, function) are added to the visual encoder, forming the VoP framework.
🎯 What it does: This paper proposes VoxFormer, a two-stage camera-based 3D semantic scene completion framework that first generates sparse voxel queries using depth estimation and occupancy prediction, and then completes the entire 3D voxel grid through a sparse-to-dense Transformer similar to MAE.
VQACL: A Novel Visual Question Answering Continual Learning Setting
Xi Zhang (Chinese Academy of Sciences), Changsheng Xu (Tianjin University of Technology)
CodeRepresentation LearningTransformerImageVideo
🎯 What it does: A new framework called VQACL (Visual Question Answering Continual Learning) is proposed, defining a dual-layer task sequence and introducing combinatorial testing; at the same time, a representation learning method based on sample-specific (SS) and sample-invariant (SI) features is designed, significantly enhancing the continual learning and combinatorial reasoning capabilities of VQA.
🎯 What it does: This paper proposes a robust multimodal speech recognition framework that can maintain robustness even when audio and video inputs are simultaneously affected by noise, occlusion, and other interferences;
🎯 What it does: A weakly supervised monocular 3D object detection method is proposed, which trains the model using only 2D bounding boxes and directional annotations to achieve 3D bounding box prediction.
🎯 What it does: A variational model based on contrast mapping is proposed for weakly supervised segmentation of pathological images using only sparse point annotations, serving as supplementary supervision for deep segmentation networks.
🎯 What it does: This paper proposes a weakly supervised semantic segmentation framework based on adversarial learning between a classifier and a reconstructor (ACR). It improves the quality of pseudo-labels by training the classifier to generate more accurate Class Activation Maps (CAM) and allowing the reconstructor to utilize residual information to reconstruct missing areas.
🎯 What it does: This paper proposes a weakly supervised Cross-Modal Temporal Erasing Network to locate key frames and their contextual information in videos with only video-level emotion labels, thereby predicting the emotion categories of user-generated videos (UGV) more accurately.
🎯 What it does: This paper proposes a general framework that utilizes weak labels from the target domain (image-level, point-level, rough annotations) along with source domain annotations for weakly supervised domain adaptive semantic segmentation, and reduces the domain gap through prototype alignment.
🎯 What it does: A method for inferring past human actions based on thermal pixels is proposed, and the first indoor thermal-visible synchronous dataset, Thermal-IM, is constructed.
🎯 What it does: Under the condition of very few samples, a latent subspace optimization method is proposed using the continuity and interpretability of the StyleGAN latent space, achieving high-quality and diverse image generation for unseen categories.
Where We Are and What We're Looking At: Query Based Worldwide Image Geo-Localization Using Hierarchies and Scenes
Brandon Clark (University of Central Florida), Mubarak Shah (University of Central Florida)
CodeTransformerImageBenchmark
🎯 What it does: A query-based multi-level geographic localization framework based on a Transformer decoder is proposed, which learns query vectors for different geographic levels and scenarios to directly predict S2 fine-grained locations.
🎯 What it does: Utilizing the separation of smartphone flash and ambient light, combined with neural light fields and physical BRDF, to achieve inverse rendering of indoor object geometry and reflective properties.
🎯 What it does: This paper proposes the YOSO framework, which achieves semantic and instance panoptic segmentation with a single segmentation, suitable for real-time scenarios.
ZBS: Zero-Shot Background Subtraction via Instance-Level Background Modeling and Foreground Selection
Yongqi An (National Laboratory of Pattern Recognition Institute of Automation CAS), Jinqiao Wang (National Laboratory of Pattern Recognition Institute of Automation CAS)
🎯 What it does: A background subtraction framework based on zero-shot object detection (ZBS) is proposed, achieving instance-level background modeling and foreground selection.
🎯 What it does: A zero-shot dual-camera super-resolution framework ZeDuSR is proposed, which learns scene-specific SR models using only a single pair of dual-camera images during testing.
Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style
Fengyin Lin (Beijing University of Posts and Telecommunications), Yonggang Qi (Beijing University of Posts and Telecommunications)
CodeRetrievalExplainability and InterpretabilityTransformerImage
🎯 What it does: This paper addresses the zero-shot sketch-based image retrieval (ZS-SBIR) problem by proposing a unified cross-modal Transformer network that can perform retrieval in all scenarios, including cross-category, fine-grained, and cross-dataset, using a single model while providing interpretability.
🎯 What it does: The zero-shot object counting (ZSC) task is proposed, which allows counting instances of a target category in an image using only the category name, avoiding the reliance on human-annotated instances found in traditional methods.
Zero-Shot Referring Image Segmentation With Global-Local Context Features
Seonghoon Yu (Gwangju Institute of Science and Technology), Jeany Son (Gwangju Institute of Science and Technology)
CodeObject DetectionSegmentationTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: A zero-shot reference image segmentation framework is proposed, utilizing a pre-trained CLIP model and global-local context features to achieve the matching of text descriptions with pixel-level segmentation without any supervised training.