ICCV 2023 Papers — Page 20
IEEE/CVF International Conference on Computer Vision · 2156 papers
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
Kan Wu (Sun Yat-sen University), Han Hu (Sun Yat-sen University)
RetrievalCompressionKnowledge DistillationTransformerContrastive LearningImageTextMultimodality
🎯 What it does: For large-scale CLIP models, the TinyCLIP method is proposed to achieve cross-modal distillation, compressing the visual-text pre-trained model into a smaller and faster version.
TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration
Kehong Gong (National University of Singapore), Xinchao Wang (National University of Singapore)
GenerationTransformerAuto EncoderTextMultimodalityAudio
🎯 What it does: This paper proposes a 3D dance generation method TM2D that combines music and text in a dual-modal approach, capable of synchronously generating natural dance movements based on audio and text instructions.
TMA: Temporal Motion Aggregation for Event-based Optical Flow
Haotian Liu (Tongji University), Changjun Jiang (Tongji University)
Autonomous DrivingOptical FlowVideo
🎯 What it does: A new framework utilizing the temporal continuity of event cameras is proposed—Temporal Motion Aggregation (TMA), which enhances event optical flow estimation through event segmentation, linear searching, and motion pattern aggregation.
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
Mathis Petrovich (University Gustave Eiffel), Gül Varol (Max Planck Institute for Intelligent Systems)
GenerationRetrievalTransformerLarge Language ModelContrastive LearningTextMultimodality
🎯 What it does: The TMR model is proposed, which jointly trains text-to-motion retrieval and text-to-motion synthesis, and introduces a filtering strategy for negative samples.
To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation
Marc Botet Colomer (Univrses), Matteo Poggi (University of Bologna)
SegmentationDomain AdaptationTransformerImage
🎯 What it does: An online real-time domain adaptation framework named HAMLET is proposed, which can continuously fine-tune semantic segmentation models without requiring a large amount of additional computation, while maintaining a high frame rate.
Token-Label Alignment for Vision Transformers
Han Xiao (Tsinghua University), Jiwen Lu (Tsinghua University)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: A Token-Label Alignment (TL-Align) method is proposed, which aligns the labels generated by data mixing strategies such as CutMix layer by layer during the training of Vision Transformers, addressing the issue of label mismatch caused by token fluctuations due to self-attention.
Too Large; Data Reduction for Vision-Language Pre-Training
Alex Jinpeng Wang (Show Lab National University of Singapore), Mike Zheng Shou (Show Lab National University of Singapore)
RetrievalCompressionTransformerVision Language ModelImageTextMultimodality
🎯 What it does: Proposes the TL;DR algorithm, which aligns codebook quantization with the generation of new captions, selects representative samples, and compresses large-scale VLP datasets.
ToonTalker: Cross-Domain Face Reenactment
Yuan Gong (Shenzhen International Graduate School Tsinghua University), Yujiu Yang (Shenzhen International Graduate School Tsinghua University)
Image TranslationGenerationDomain AdaptationTransformerImageVideo
🎯 What it does: A cross-domain face reenactment method without paired data is proposed, utilizing the Transformer framework to align the motions of real and cartoon faces and achieve motion transfer.
TopoSeg: Topology-Aware Nuclear Instance Segmentation
Hongliang He (Peking University), Jie Chen (Peng Cheng Laboratory)
SegmentationSupervised Fine-TuningImageMagnetic Resonance Imaging
🎯 What it does: A topology-aware nuclear instance segmentation method called TopoSeg is proposed;
TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
Zhiyang Dou (University of Hong Kong), Wenping Wang (Texas A&M University)
Pose EstimationComputational EfficiencyTransformerMesh
🎯 What it does: This paper proposes two Transformer-based token reduction strategies (GTR and ITP), which significantly reduce the number of tokens during the HMR process by using only skeletal joint tokens and performing clustering pruning on image features.
Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis
Chonghyuk Song (Carnegie Mellon University), Deva Ramanan (Carnegie Mellon University)
GenerationPose EstimationNeural Radiance FieldVideo
🎯 What it does: This paper proposes a method for automatically reconstructing deformable scenes from monocular RGB-D videos, enabling realistic view synthesis and 3D filters from both first-person and third-person perspectives.
Toward Multi-Granularity Decision-Making: Explicit Visual Reasoning with Hierarchical Knowledge
Yifeng Zhang (University of Minnesota), Qi Zhao (University of Minnesota)
Object DetectionExplainability and InterpretabilityGraph Neural NetworkMultimodality
🎯 What it does: This paper proposes a Hierarchical Concept Graph (HCG) and a Hierarchical Concept Neural Module Network (HCNMN) to achieve explicit reasoning and explanation of multi-granularity knowledge in visual question answering.
Toward Unsupervised Realistic Visual Question Answering
Yuwei Zhang (University of California San Diego), Nuno Vasconcelos (University of California San Diego)
ClassificationRecognitionRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a 'Realistic VQA' framework that can simultaneously reject unanswerable questions (UQs) and accurately answer answerable questions (AQs).
Towards Attack-tolerant Federated Learning via Critical Parameter Analysis
Sungwon Han (KAIST), Meeyoung Cha (KAIST)
Federated LearningAdversarial AttackImage
🎯 What it does: This paper proposes a new federated learning defense strategy called FedCPA, which uses key parameter analysis to assess the normality of model updates, thereby defending against poisoning attacks from malicious clients.
Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond
Yang Zhao (Google), Matthias Grundmann (Google)
RestorationGenerationDiffusion modelImage
🎯 What it does: A real face restoration system based on an iterative diffusion model (IDM) is proposed, which automatically cleans training data through external iterative learning.
Towards Better Robustness against Common Corruptions for Unsupervised Domain Adaptation
Zhiqiang Gao (Duke Kunshan University), Jieming Ma (Xi'an Jiatong-Liverpool University)
Domain AdaptationGenerative Adversarial NetworkImage
🎯 What it does: A novel uncertainty robustness enhancement framework for unsupervised domain adaptation (UDA) called DDAR is proposed to enhance the model's robustness against common noise and distortions (RaCC).
Towards Building More Robust Models with Frequency Bias
Qingwen Bu (Shanghai Artificial Intelligence Laboratory), Heming Cui (University of Hong Kong)
OptimizationAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A Frequency Preference Control Module (FPCM) is proposed, which can dynamically reconfigure low-frequency and high-frequency information in the intermediate feature layers, thereby enhancing the robustness of adversarial training models.
Towards Content-based Pixel Retrieval in Revisited Oxford and Paris
Guoyuan An (KAIST), Sun-Eui Yoon
SegmentationRetrievalImageBenchmark
🎯 What it does: Proposes a pixel retrieval task and constructs two benchmark datasets: PROxford and PRParis.
Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learning
Junwen He (Dalian University of Technology), Xuansong Xie (DAMO Academy Alibaba Group)
SegmentationDepth EstimationAutonomous DrivingTransformerContrastive LearningImage
🎯 What it does: A deep unified framework for depth-aware panoramic segmentation is proposed, achieving both instance-level semantic segmentation and monocular depth estimation.
Towards Effective Instance Discrimination Contrastive Loss for Unsupervised Domain Adaptation
Yixin Zhang (University of Science and Technology of China), Zihan Lin (University of Science and Technology of China)
Domain AdaptationConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: An effective instance identification contrastive loss (EIDCo) specifically designed for domain adaptation is proposed, which learns unlabeled target domain features through low-confidence samples, class relationship enhanced features, and target-dominant cross-domain Mixup.
Towards Fair and Comprehensive Comparisons for Image-Based 3D Object Detection
Xinzhu Ma (Shanghai AI Lab), Wanli Ouyang (Shanghai AI Lab)
Object DetectionAutonomous DrivingImagePoint CloudBenchmark
🎯 What it does: A modular code library was constructed, unified training standards were established, and an error diagnosis tool was proposed to systematically evaluate 3D detection based on images.
Towards Fairness-aware Adversarial Network Pruning
Lei Zhang (Zhejiang University), Kui Ren (Zhejiang University)
CompressionGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes an end-to-end fairness-aware adversarial network pruning framework that jointly optimizes pruning and debiasing, significantly enhancing fairness while maintaining accuracy in the compressed model.
Towards General Low-Light Raw Noise Synthesis and Modeling
Feng Zhang (Huazhong University of Science and Technology), Nong Sang (Huazhong University of Science and Technology)
RestorationGenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageBenchmark
🎯 What it does: This paper proposes a low-light raw image noise synthesis method that separates signal-related noise from signal-independent noise. It achieves noise distribution alignment through a pre-trained denoising network and a Fourier Transformer discriminator, generating low-light raw images that are highly similar to real noise.
Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning
Yuanhao Zhai (University at Buffalo), Junsong Yuan (University at Buffalo)
Anomaly DetectionConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This study investigates a weakly supervised image tampering detection method that can complete tampering detection and localization using only binary image-level labels.
Towards Geospatial Foundation Models via Continual Pretraining
Matías Mendieta (University of Central Florida), Chen Chen (University of Central Florida)
ClassificationObject DetectionSegmentationKnowledge DistillationTransformerContrastive LearningImage
🎯 What it does: A geospatial foundation model (GFM) based on continuous pre-training is proposed, which combines ImageNet-22k weights with self-supervised MIM objectives for multi-task training.
Towards Grand Unified Representation Learning for Unsupervised Visible-Infrared Person Re-Identification
Bin Yang (Wuhan University), Mang Ye (Wuhan University)
RecognitionRetrievalRepresentation LearningContrastive LearningImageMultimodality
🎯 What it does: A unified representation learning framework for unsupervised visible-infrared person recognition is proposed, addressing the differences at the camera and modality levels;
Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images
Cuican Yu (Xi'an Jiaotong University), Hang Xu (Huawei Noah's Ark Lab)
GenerationData SynthesisGenerative Adversarial NetworkContrastive LearningImageTextMultimodality
🎯 What it does: A text-guided 3D face generation model TG-3DFace has been developed, which can generate high-quality, multi-view consistent 3D faces and textures based solely on text-2D face image data.
Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data
Gang Fu (Hong Kong Polytechnic University), Ping Li (Wuhan University)
RestorationConvolutional Neural NetworkImage
🎯 What it does: A three-stage single image specular highlight removal network is proposed, which can gradually eliminate visual defects such as color distortion, black blocks, and tone mismatches, generating high-quality highlight-free images.
Towards Improved Input Masking for Convolutional Neural Networks
Sriram Balasubramanian (University of Maryland), Soheil Feizi (University of Maryland)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerImage
🎯 What it does: A new Layer Masking method is proposed, utilizing neighbor padding and mask propagation, allowing CNNs to be unaffected by the color or shape of the mask when masking inputs, thus achieving more robust explanations and model diagnostics.
Towards Inadequately Pre-trained Models in Transfer Learning
Andong Deng (University of Central Florida), Cheng-Zhong Xu (University of Central Florida)
ClassificationRetrievalConvolutional Neural NetworkTransformerImage
🎯 What it does: The study investigates the impact of different checkpoints during the same pre-training process on transfer learning, finding that mid-term pre-trained models often outperform fully pre-trained models when used as feature extractors.
Towards Instance-adaptive Inference for Federated Learning
Chun-Mei Feng (Institute of High Performance Computing), Wangmeng Zuo (Harbin Institute of Technology)
Federated LearningTransformerSupervised Fine-TuningImage
🎯 What it does: This study investigates the issue of client internal heterogeneity in federated learning and proposes the FedIns algorithm, which reduces communication costs and improves model accuracy through instance adaptive reasoning and SSF pooling.
Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks
Qingyan Meng (Chinese University of Hong Kong), Zhi-Quan Luo (Chinese University of Hong Kong)
Spiking Neural NetworkImage
🎯 What it does: Proposes the Spatial Learning Through Time (SLTT) method, which improves the time and memory overhead of training SNNs using traditional BPTT+SG;
Towards Models that Can See and Read
Roy Ganz (Technion), Ron Litman (AWS AI Labs)
TransformerVision Language ModelImageTextMultimodality
🎯 What it does: The UniTNT model is proposed, enabling existing visual-language pre-training architectures to simultaneously understand visual information and scene text in images, thus achieving unified processing of 'seeing and reading';
Towards Multi-Layered 3D Garments Animation
Yidi Shao (Nanyang Technological University), Bo Dai (Shanghai AI Laboratory)
TransformerVideoMesh
🎯 What it does: This paper proposes LayersNet, a particle simulation network based on Transformer, which implements multi-level 3D clothing animation through a patch-level hierarchical structure, and introduces rotation-invariant attention and rotation-equivalent transformations to handle the interactions between external forces and multi-layer clothing.
Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction
Delin Qu (Fudan University), Xuelong Li (Northwestern Polytechnical University)
RestorationAutonomous DrivingTransformerOptical FlowVideo
🎯 What it does: A geometry-based quadratic rolling shutter motion solver (QRS) and a 3D video structure for rolling shutter correction in extreme occlusion scenarios, RSA-Net2, are proposed, capable of achieving high-quality global shutter image reconstruction under complex nonlinear motion and occlusion conditions.
Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization
Jungsoo Lee (Qualcomm Technologies Inc), Sungha Choi (Qualcomm Technologies Inc)
ClassificationSegmentationDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: In the task of Test-Time Adaptation (TTA), a sample selection method based on the confidence difference between the original model and the adaptive model is proposed to filter out noise loss caused by incorrect or unknown categories (open-set); at the same time, the open-set scenario is introduced in TTA for the first time to enhance long-term stability;
Towards Open-Vocabulary Video Instance Segmentation
Haochen Wang (University of Amsterdam), Efstratios Gavves (University of Amsterdam)
Object TrackingSegmentationTransformerVideo
🎯 What it does: This paper proposes the Open-Vocabulary Video Instance Segmentation (OV-VIS) task, which aims to achieve segmentation, tracking, and classification of any category in videos simultaneously.
Towards Real-World Burst Image Super-Resolution: Benchmark and Method
Pengxu Wei (Sun Yat-sen University), Liang Lin
RestorationSuper ResolutionTransformerImageBenchmark
🎯 What it does: A real scene flash group super-resolution dataset (RealBSR) and the FBAnet model are proposed, utilizing isomorphic alignment, federated similarity fusion, and Transformer decoding to achieve multi-frame super-resolution.
Towards Realistic Evaluation of Industrial Continual Learning Scenarios with an Emphasis on Energy Consumption and Computational Footprint
Vivek Chavan (Fraunhofer Institute for Production Systems and Design Technology), Clemens Briese (Fraunhofer Institute for Production Systems and Design Technology)
Computational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: A realistic evaluation of Class Incremental Learning (CIL) in industrial scenarios is conducted, proposing the RECIL framework and introducing the InVar-100 industrial object dataset, focusing on three-dimensional metrics: accuracy, energy consumption, and computational overhead.
Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild
Sungchan Park (Seoul National University), Joonseok Lee (Seoul National University)
Pose EstimationTransformerVideo
🎯 What it does: A sequence-to-sequence (seq2seq) 2D-to-3D enhancement network called POTR-3D is proposed, which generates an infinite number of training samples through geometry-aware data augmentation to address the issues of viewpoint robustness, occlusion, and jitter in monocular multi-person 3D pose estimation.
Towards Robust Model Watermark via Reducing Parametric Vulnerability
Guanhao Gan (Tsinghua University), Shu-Tao Xia (Tsinghua University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: A watermark embedding method based on min-max optimization (APP + c-BN) is proposed to enhance the robustness of model watermarks against parameter perturbations and watermark removal attacks.
Towards Saner Deep Image Registration
Bin Duan (Illinois Institute of Technology), Yan Yan (Illinois Institute of Technology)
Image TranslationOptimizationImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper introduces two regularization verification mechanisms, self-sanity and cross-sanity, to constrain the training of existing deep image registration models, thereby reducing folding transformations, enhancing inverse consistency and discriminative ability, and providing a theoretical error upper bound.
Towards Semi-supervised Learning with Non-random Missing Labels
Yue Duan (Nanjing University), Yinghuan Shi (Nanjing University)
ClassificationImage
🎯 What it does: This paper proposes a pseudo-label correction guidance method based on category transfer tracking (PRG) to address the non-random missing label (MNAR) problem in semi-supervised learning.
Towards Understanding the Generalization of Deepfake Detectors from a Game-Theoretical View
Kelu Yao (Zhejiang Laboratory), Chao Li (Institute of Computing Technology)
ClassificationRecognitionAnomaly DetectionConvolutional Neural NetworkGenerative Adversarial NetworkImageVideo
🎯 What it does: This paper proposes an explanation for the generalization ability of deepfake detectors from the perspective of multi-order interactions, providing corresponding evaluation metrics and an inference improvement strategy based on low-order interaction suppression.
Towards Unifying Medical Vision-and-Language Pre-Training via Soft Prompts
Zhihong Chen (Chinese University of Hong Kong), Xiang Wan (Shenzhen Research Institute of Big Data)
TransformerPrompt EngineeringVision Language ModelImageTextMultimodalityBiomedical Data
🎯 What it does: PTUnifier is proposed, unifying visual and language pre-training into a single model, using soft prompts to complete inputs in the absence of modalities, making the model compatible with three input formats: image, text, and image + text.
Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representations
Nikolaos-Antonios Ypsilantis (Czech Technical University in Prague), André Araujo (Google)
RetrievalRepresentation LearningTransformerImageBenchmark
🎯 What it does: A large-scale general image embedding dataset UnED, benchmark evaluation process, and a public industrial-grade general embedding challenge have been proposed, studying how to share a single embedding model across multiple domains.
Towards Universal LiDAR-Based 3D Object Detection by Multi-Domain Knowledge Transfer
Guile Wu (Huawei Noah's Ark Lab), Yuan Ren (Huawei Noah's Ark Lab)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: A multi-domain knowledge transfer and general feature transformation framework for LiDAR 3D object detection is proposed and implemented, capable of sharing and generalizing across multiple autonomous driving datasets.
Towards Unsupervised Domain Generalization for Face Anti-Spoofing
Yuchen Liu (Shanghai Jiao Tong University), Hongkai Xiong (Shanghai Jiao Tong University)
RecognitionDomain AdaptationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A framework for facial spoof detection based on unsupervised domain generalization (UDG-FAS) is proposed, which learns generalizable anti-spoofing features by utilizing a large amount of unlabeled facial data.
Towards Viewpoint Robustness in Bird's Eye View Segmentation
Tzofi Klinghoffer (MIT), Jose M. Alvarez (NVIDIA)
SegmentationAutonomous DrivingTransformerWorld ModelImageVideo
🎯 What it does: This study investigates the impact of different camera perspectives on Bird's Eye View (BEV) segmentation models and proposes a method called New View Synthesis (NVS) to transform source camera data into target camera perspectives, thereby enhancing the model's perspective robustness without the need to collect and annotate additional target camera data.
Towards Viewpoint-Invariant Visual Recognition via Adversarial Training
Shouwei Ruan (Beihang University), Xingxing Wei
ClassificationRecognitionAdversarial AttackConvolutional Neural NetworkTransformerNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: Enhancing the robustness of image classifiers to 3D viewpoint changes through viewpoint-invariant adversarial training (VIAT).
Towards Zero Domain Gap: A Comprehensive Study of Realistic LiDAR Simulation for Autonomy Testing
Sivabalan Manivasagam (Waabi), Raquel Urtasun (Waabi)
Domain AdaptationAutonomous DrivingSimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper proposes a 'paired-scenario' evaluation framework that conducts frame-by-frame comparisons between real-world data and LiDAR simulation results in its digital twin scenarios, systematically analyzing the impact of various LiDAR simulation phenomena (multi-echo, non-returning echoes, pseudo-echoes, noise, scanning calibration, rolling shutter, motion blur) and the construction methods of the virtual world on the performance of autonomous systems (perception, prediction, planning).
Towards Zero-Shot Scale-Aware Monocular Depth Estimation
Vitor Guizilini (Toyota Research Institute), Adrien Gaidon (Toyota Research Institute)
Depth EstimationAutonomous DrivingTransformerImage
🎯 What it does: Proposes the ZeroDepth framework, achieving zero-shot scale-aware monocular depth estimation.
Tracing the Origin of Adversarial Attack for Forensic Investigation and Deterrence
Han Fang (National University of Singapore), Ee-Chien Chang (National University of Singapore)
Adversarial AttackConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: A framework for adversarial sample tracing is proposed under the buyer-seller setting, divided into two phases: model separation and source tracking. It utilizes a parallel structure and a VAE-trained tracer to achieve model differentiation, and locates the source of the attack through the logit difference output by the tracer.
TrackFlow: Multi-Object tracking with Normalizing Flows
Gianluca Mancusi, Rita Cucchiara
Object TrackingRecurrent Neural NetworkTransformerFlow-based ModelVideo
🎯 What it does: This paper proposes a multimodal association cost estimator called TrackFlow based on regularized flows, aimed at enhancing detection-based multi-object tracking.
Tracking Anything with Decoupled Video Segmentation
Ho Kei Cheng (University of Illinois Urbana-Champaign), Joon-Young Lee (Adobe Research)
Object TrackingSegmentationVideo
🎯 What it does: We propose a decoupled video segmentation framework DEVA, which combines task-specific image-level segmentation models with a general temporal propagation model, utilizing bidirectional propagation to achieve tracking and segmentation of any object in the video.
Tracking by 3D Model Estimation of Unknown Objects in Videos
Denys Rozumnyi (ETH Zurich), Martin R. Oswald (University of Amsterdam)
Object TrackingSegmentationPose EstimationVideo
🎯 What it does: A model-free video object tracking method based on differentiable rendering is proposed, jointly estimating the unknown object's 3D shape, texture, and 6DoF pose, and using it to generate more accurate 2D segmentation.
Tracking by Natural Language Specification with Long Short-term Context Decoupling
Ding Ma (Harbin Institute of Technology), Xiangqian Wu (Harbin Institute of Technology)
Object TrackingTransformerContrastive LearningImageVideoText
🎯 What it does: A Transformer-based object tracking framework called DecoupleTNL has been designed and implemented. It extracts dynamic and static semantic information through two branches: Short-term Context Matching (SCM) and Long-term Context Perception (LCP), and integrates them into visual features using a Long-Short Modulation module (LSM) to achieve natural language description tracking.
Tracking Everything Everywhere All at Once
Qianqian Wang (Cornell University), Noah Snavely (Cornell University)
Object TrackingOptimizationOptical FlowVideoBenchmark
🎯 What it does: A testing optimization framework named OmniMotion is proposed to estimate dense motion trajectories of full duration for each pixel in a video, using quasi-3D canonical volumes and pixel-level bijective mapping.
Tracking without Label: Unsupervised Multiple Object Tracking via Contrastive Similarity Learning
Sha Meng (Northwestern Polytechnical University), Shan Gao (Northwestern Polytechnical University)
Object TrackingContrastive LearningVideo
🎯 What it does: A label-free multi-object tracking method UCSL is proposed, which utilizes contrastive similarity learning to train ReID embeddings under unsupervised conditions, achieving object association and tracking.
Traj-MAE: Masked Autoencoders for Trajectory Prediction
Hao Chen (Chinese University of Hong Kong), Pheng-Ann Heng (Chinese University of Hong Kong)
Autonomous DrivingRepresentation LearningTransformerAuto EncoderTime Series
🎯 What it does: A trajectory prediction framework called Traj-MAE based on Masked Autoencoder is designed and validated, utilizing self-supervised pre-training of trajectory and map encoders, and achieving multi-strategy learning through continuous pre-training.
Trajectory Unified Transformer for Pedestrian Trajectory Prediction
Liushuai Shi (Xi'an Jiaotong University), Gang Hua (Wormpex AI Research)
TransformerTime Series
🎯 What it does: A unified Transformer framework TUTR is proposed for post-processing-free pedestrian trajectory prediction;
TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses
Xuesong Chen (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
Object TrackingAutonomous DrivingTransformerPoint Cloud
🎯 What it does: This paper proposes TrajectoryFormer, which utilizes predicted trajectory hypotheses and multiple candidates from detection boxes to perform transformer-style association and refinement of 3D trajectories, achieving robust LiDAR 3D multi-object tracking.
TrajPAC: Towards Robustness Verification of Pedestrian Trajectory Prediction Models
Liang Zhang (Institute of Software), Lijun Zhang (Institute of Software)
Autonomous DrivingExplainability and InterpretabilityAdversarial AttackReinforcement LearningVideo
🎯 What it does: Proposed and implemented the TRAJPAC framework to validate the robustness of pedestrian trajectory prediction models, providing formal definitions of label robustness and pure robustness, and using the PAC method to complete robustness assessment and interpretability analysis of black-box models.
TransFace: Calibrating Transformer Training for Face Recognition from a Data-Centric Perspective
Jun Dan (Zhejiang University), Baigui Sun (Alibaba DAMO Academy)
RecognitionData-Centric LearningTransformerImage
🎯 What it does: A facial recognition model named TransFace is proposed, specifically optimized for the performance bottlenecks of Vision Transformer on large-scale datasets.
Transferable Adversarial Attack for Both Vision Transformers and Convolutional Networks via Momentum Integrated Gradients
Wenshuo Ma (Tsinghua University), Wei Xu (Beijing Jiaotong University)
Adversarial AttackConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes the Momentum Integrated Gradients (MIG) method, which combines integral gradients with momentum iteration to generate adversarial examples that exhibit high transferability between Vision Transformers (ViT) and Convolutional Neural Networks (CNN).
Transferable Decoding with Visual Entities for Zero-Shot Image Captioning
Junjie Fei (Southern University of Science and Technology), Feng Zheng (Southern University of Science and Technology)
GenerationRetrievalTransformerPrompt EngineeringVision Language ModelImageText
🎯 What it does: Proposes the ViECap model, which utilizes entity-aware hard prompts and soft prompts to achieve zero-shot image captioning.
TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering
Xiao Pan (Zhejiang University), Yi Yang (Zhejiang University)
GenerationPose EstimationTransformerNeural Radiance FieldMesh
🎯 What it does: The TransHuman framework is proposed, which uses Transformer to encode painted SMPL in the canonical space and capture global relationships, and then maps it back to the observation space through a deformable local light field, ultimately achieving generalizable neural human rendering.
TransIFF: An Instance-Level Feature Fusion Framework for Vehicle-Infrastructure Cooperative 3D Detection with Transformers
Ziming Chen (Beihang University), Jinrang Jia (Baidu Inc.)
Object DetectionDomain AdaptationAutonomous DrivingTransformerPoint Cloud
🎯 What it does: Proposes the TransIFF framework, which achieves low bandwidth high-precision detection through collaborative perception between vehicles and infrastructure via instance-level feature fusion.
Translating Images to Road Network: A Non-Autoregressive Sequence-to-Sequence Approach
Jiachen Lu, Li Zhang (Fudan University)
Image TranslationAutonomous DrivingTransformerImage
🎯 What it does: A unified road network sequence representation (RoadNet Sequence) and a RoadNetTransformer based on Transformer are proposed to extract road networks with high precision from multi-camera inputs.
Transparent Shape from a Single View Polarization Image
Mingqi Shao (Tsinghua Shenzhen International Graduate School), Xueqian Wang (Tsinghua Shenzhen International Graduate School)
Depth EstimationConvolutional Neural NetworkImage
🎯 What it does: A method for estimating the shape of transparent surfaces based on single-view polarized images is proposed.
TransTIC: Transferring Transformer-based Image Compression from Human Perception to Machine Perception
Yi-Hsin Chen (National Yang Ming Chiao Tung University), Wen-Hsiao Peng (National Yang Ming Chiao Tung University)
Object DetectionSegmentationCompressionTransformerPrompt EngineeringImage
🎯 What it does: A method is proposed that adapts a pre-trained Transformer image compression encoder to machine vision tasks without the need for fine-tuning.
Treating Pseudo-labels Generation as Image Matting for Weakly Supervised Semantic Segmentation
Changwei Wang (Chinese Academy of Sciences), Xiaopeng Zhang (Chinese Academy of Sciences)
SegmentationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper transforms the pseudo-label generation task in weakly supervised semantic segmentation into an image matting problem and proposes the Mat-Label framework.
Tree-Structured Shading Decomposition
Chen Geng, Jiajun Wu
GenerationOptimizationConvolutional Neural NetworkImage
🎯 What it does: A method is proposed to infer and reconstruct object shadows from a single image using a tree structure (Shade Tree), enabling editable and rearrangeable shadows.
Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields
Wenbo Hu (ByteDance), Yuewen Ma (ByteDance)
GenerationComputational EfficiencyNeural Radiance FieldImage
🎯 What it does: Proposes Tri-MipRF, which represents neural radiance fields through three-plane 3D pre-filtering mipmaps, achieving high-quality anti-aliasing rendering.
TripLe: Revisiting Pretrained Model Reuse and Progressive Learning for Efficient Vision Transformer Scaling and Searching
Cheng Fu (University of California San Diego), Jishen Zhao (Google)
Computational EfficiencyKnowledge DistillationNeural Architecture SearchTransformerSupervised Fine-TuningImage
🎯 What it does: A method called TripLe is proposed, which expands the width of a smaller pre-trained Vision Transformer (ViT) before training and gradually increases the depth during training, while copying the weights and AdamW's momentum state during the expansion to accelerate the convergence of large models and improve performance.
TRM-UAP: Enhancing the Transferability of Data-Free Universal Adversarial Perturbation via Truncated Ratio Maximization
Yiran Liu (Chongqing University of Technology), Di Ming (Chongqing University of Technology)
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A completely data-independent universal adversarial perturbation generation method TRM-UAP is designed, capable of quickly generating attack perturbations on any CNN.
Troubleshooting Ethnic Quality Bias with Curriculum Domain Adaptation for Face Image Quality Assessment
Fu-Zhao Ou (City University of Hong Kong), Sam Kwong (City University of Hong Kong)
Domain AdaptationGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes an Ethnic Quality Bias Mitigation framework (EQBM) based on curriculum-style domain adaptation, which adjusts the facial image quality assessment in the target domain by mapping the original regression targets to Likert quantized probabilities and incorporating a difficulty scheduler.
Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation
Xiangtai Li (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)
Object DetectionSegmentationTransformerContrastive LearningVideo
🎯 What it does: The Tube-Link framework is proposed, unifying the implementation of three major tasks: video semantic segmentation, instance segmentation, and panoptic segmentation.
Tubelet-Contrastive Self-Supervision for Video-Efficient Generalization
Fida Mohammad Thoker (University of Amsterdam), Cees G. M. Snoek (University of Amsterdam)
Representation LearningContrastive LearningVideo
🎯 What it does: A self-supervised video representation learning method is proposed, which achieves contrastive learning of motion dynamics by inserting synthetic motion trajectories (tubelets) into the video.
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
Jay Zhangjie Wu (Show Lab), Mike Zheng Shou (Show Lab)
GenerationData SynthesisTransformerDiffusion modelVideoText
🎯 What it does: This paper proposes Tune-A-Video, a method for text-driven video generation and editing that fine-tunes a pre-trained text-image diffusion model using only a single text-video pair.
Tuning Pre-trained Model via Moment Probing
Mingze Gao (Tianjin University), Jingbo Zhou (Baidu Research)
ClassificationRepresentation LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: The paper proposes a new feature distribution-based linear classifier called Moment Probing (MP), and designs a Partial Shared Recalibration Module (PSRP) based on it for efficient fine-tuning of pre-trained models.
Two Birds, One Stone: A Unified Framework for Joint Learning of Image and Video Style Transfers
Bohai Gu (Institute of Software Chinese Academy of Sciences), Libo Zhang (Institute of Software Chinese Academy of Sciences)
Image TranslationConvolutional Neural NetworkTransformerImageVideo
🎯 What it does: Proposes the UniST framework, which achieves arbitrary style transfer for both images and videos with a single training.
Two-in-One Depth: Bridging the Gap Between Monocular and Binocular Self-Supervised Depth Estimation
Zhengming Zhou (Chinese Academy of Sciences), Qiulei Dong (Chinese Academy of Sciences)
Depth EstimationDomain AdaptationKnowledge DistillationTransformerImage
🎯 What it does: This paper proposes a two-in-one network called TiO-Depth that can simultaneously perform monocular and binocular self-supervised depth estimation, and further enhances the prediction accuracy of both tasks through multi-stage joint training.
U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds
Yan Di (Technical University of Munich), Federico Tombari (Technical University of Munich)
RetrievalGraph Neural NetworkPoint Cloud
🎯 What it does: This paper proposes an unsupervised 3D shape retrieval and deformation framework called U-RED, which can retrieve and deform CAD models from noisy and partially observed point clouds to closely match the target.
UATVR: Uncertainty-Adaptive Text-Video Retrieval
Bo Fang (Institute of Information Engineering Chinese Academy of Sciences), Jingdong Wang (Baidu Inc)
RetrievalTransformerContrastive LearningVideoTextMultimodality
🎯 What it does: An uncertainty adaptive method for video text retrieval (UATVR) is proposed to address the multi-granularity uncertainty and one-to-many relationship issues in text-video matching.
UCF: Uncovering Common Features for Generalizable Deepfake Detection
Zhiyuan Yan (Chinese University of Hong Kong), Baoyuan Wu (Chinese University of Hong Kong)
ClassificationRecognitionConvolutional Neural NetworkContrastive LearningImageVideo
🎯 What it does: A multi-task decoupling framework is proposed, which decomposes images into content, method-specific forgery, and general forgery features, using general features to achieve deepfake detection across datasets.
UGC: Unified GAN Compression for Efficient Image-to-Image Translation
Yuxi Ren (ByteDance Inc), Xin Pan (ByteDance Inc)
Image TranslationCompressionKnowledge DistillationGenerative Adversarial NetworkImage
🎯 What it does: A unified GAN compression framework UGC is proposed, combining model compression and label compression;
UHDNeRF: Ultra-High-Definition Neural Radiance Fields
Quewei Li (Nanjing University), Yanwen Guo (Nanjing University)
GenerationData SynthesisSuper ResolutionComputational EfficiencyNeural Radiance FieldImagePoint Cloud
🎯 What it does: A NeRF framework suitable for 4K ultra-high-resolution scenes, called UHDNeRF, is proposed, which achieves new view synthesis through adaptive implicit-explicit scene representation.
UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework
Tianhang Wang (Tongji University), Changjun Jiang (Tongji University)
Autonomous DrivingOptimizationRecurrent Neural NetworkGraph Neural NetworkTime Series
🎯 What it does: This paper proposes a unified multi-agent collaborative perception framework (UMC) that simultaneously optimizes communication, collaboration, and reconstruction processes.
UMFuse: Unified Multi View Fusion for Human Editing Applications
Rishabh Jain (Adobe), Balaji Krishnamurthy (Adobe)
GenerationData SynthesisPose EstimationTransformerGenerative Adversarial NetworkImage
🎯 What it does: A multi-view fusion framework UMFuse is proposed, which generates high-quality portraits of a target pose from multiple images of the same person in different poses.
UMIFormer: Mining the Correlations between Similar Tokens for Multi-View 3D Reconstruction
Zhenwei Zhu (Macau University of Science and Technology), Yanyan Liang (Macau University of Science and Technology)
SegmentationGenerationTransformerPoint Cloud
🎯 What it does: This paper proposes UMIFormer, a Transformer network that alternates between intra-view and inter-view decoupled feature extraction for multi-view 3D reconstruction.
Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers
Abril Corona-Figueroa (Durham University), Chris G. Willcocks (Durham University)
Image TranslationGenerationData SynthesisTransformerDiffusion modelAuto EncoderImageBiomedical DataComputed Tomography
🎯 What it does: The research proposes a conditional diffusion model based on discrete vector quantization (VQ-VAE) encoding space, utilizing a Transformer to generate high-resolution 3D volumes from unaligned multi-view 2D images.
Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching
Junpeng Jing (Beihang University), Leonid Sigal (University of British Columbia)
Depth EstimationComputational EfficiencyConvolutional Neural NetworkRecurrent Neural NetworkOptical FlowImage
🎯 What it does: This paper proposes an uncertainty-guided adaptive Warping module called UGAC, and based on this, constructs CREStereo++ to achieve robust and efficient stereo matching.
Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecasting
Wentao Bao (Michigan State University), Yu Kong (OPPO US Research Center)
TransformerVideo
🎯 What it does: A model based on Uncertainty-Aware State Space Transformer (USST) is proposed to predict the future trajectory of the hand in 3D space from first-person RGB video, and an automated 3D trajectory annotation process is implemented.
Uncertainty-aware Unsupervised Multi-Object Tracking
Kai Liu (Zhejiang University), Jieping Ye (Alibaba DAMO Academy)
Object TrackingContrastive LearningVideo
🎯 What it does: This study proposes an unsupervised multi-object tracking framework U2MOT, which utilizes uncertainty information to improve the quality of pseudo trajectories and enhance feature consistency.
Uncertainty-guided Learning for Improving Image Manipulation Detection
Kaixiang Ji (Ant Group), Jingdong Chen (Ant Group)
ClassificationSegmentationAnomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper addresses the issues of data uncertainty and model uncertainty in image tampering detection, proposing a framework based on uncertainty-guided learning, which includes an Uncertainty Estimation Network (UEN), Dynamic Uncertainty Supervision (DUS), and Uncertainty-Guided Prediction Refinement (UPR), achieving precise localization of tampered areas.
Under-Display Camera Image Restoration with Scattering Effect
Binbin Song (University of Macau), Jiantao Zhou (University of Macau)
RestorationData SynthesisConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: A UDC image restoration framework SRUDC specifically designed for the scattering effects of translucent displays has been developed and the UDC image generation pipeline has been improved based on this framework.
Understanding 3D Object Interaction from a Single Image
Shengyi Qian (University of Michigan), David F. Fouhey (New York University)
Object DetectionSegmentationDepth EstimationRobotic IntelligenceTransformerImage
🎯 What it does: A Transformer-based model is proposed that can predict the 3D interaction attributes of objects (such as mobility, rigidity, joint type, executable actions, and operable point locations) based on query points in a single RGB image, along with their corresponding spatial localization.
Understanding Hessian Alignment for Domain Generalization
Sobhan Hemati (Huawei), Xi Chen (Huawei)
Domain AdaptationOptimizationImage
🎯 What it does: Proposed and validated a domain generalization method based on the alignment of the classifier head Hessian matrix.
Understanding Self-attention Mechanism via Dynamical System Perspective
Zhongzhan Huang (Sun Yat-sen University), Liang Lin (Sun Yat-sen University)
ClassificationObject DetectionImageOrdinary Differential Equation
🎯 What it does: This paper reinterprets the self-attention mechanism from the perspective of dynamic systems, viewing it as a step size adapter that can adaptively capture the stiffness of the network, and based on this, proposes the StepNet model.