CVPR 2024 Papers — Page 25
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts
Jiayi Chen (Northwestern Polytechnical University), Yong Xia (Northwestern Polytechnical University)
ClassificationSegmentationFederated LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: In the multi-center medical imaging federated learning scenario, this paper proposes a new federated evidence active learning framework (FEAL) to select the most informative unlabeled samples for labeling under domain transfer.
Three Pillars Improving Vision Foundation Model Distillation for Lidar
Gilles Puy (Valeo), Renaud Marlet (Valeo)
SegmentationAutonomous DrivingKnowledge DistillationTransformerContrastive LearningPoint Cloud
🎯 What it does: In the context of autonomous driving, the ScaLR method achieves high-quality 3D feature distillation through three main pillars: 3D backbone expansion, 2D backbone pre-training, and cross-dataset pre-training.
THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models
Prannay Kaul (VGG University of Oxford), Stefano Soatto (Amazon Web Services)
Object DetectionGenerationTransformerLarge Language ModelVision Language ModelImageTextBenchmark
🎯 What it does: This paper proposes the THRONE benchmark, which uses language models to automatically determine the existence of objects in free-form image descriptions, thereby assessing Type I hallucinations of LVLMs and conducting a comprehensive evaluation of Type II hallucinations in POPE. It then presents an object enumeration-based training enhancement method to significantly reduce both types of hallucinations.
TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models
Haomiao Ni (Pennsylvania State University), Tim K. Marks (Mitsubishi Electric Research Laboratories)
GenerationData SynthesisDiffusion modelImageVideoText
🎯 What it does: This paper proposes a zero-shot image-to-video generation framework TI2V-Zero, which enables unsupervised video generation based on a given image and text using a pre-trained text-to-video diffusion model without any additional training or fine-tuning.
TIGER: Time-Varying Denoising Model for 3D Point Cloud Generation with Diffusion Process
Zhiyuan Ren (Michigan State University), Xiaoming Liu (Michigan State University)
GenerationData SynthesisTransformerDiffusion modelPoint Cloud
🎯 What it does: This paper proposes a time-varying denoising diffusion model (TIGER) for 3D point cloud generation, achieving a dynamic fusion of convolution and Transformer in a dual-stream structure at different time steps of the diffusion process, significantly enhancing generation quality and diversity.
TIM: A Time Interval Machine for Audio-Visual Action Recognition
Jacob Chalk (University of Bristol), Dima Damen (Czech Technical University in Prague)
RecognitionTransformerVideoMultimodalityAudio
🎯 What it does: This paper proposes a multi-modal Transformer based on time interval queries (Time Interval Machine, TIM), which can simultaneously recognize audio and visual actions in long videos and achieve cross-modal context aggregation through unified time encoding.
Time- Memory- and Parameter-Efficient Visual Adaptation
Otniel-Bogdan Mercea (Google), Anurag Arnab (Google)
Domain AdaptationComputational EfficiencyTransformerSupervised Fine-TuningImageVideo
🎯 What it does: A method for training a lightweight parallel side network (LoSA) on a frozen large-scale visual pre-trained model is proposed to reduce training time, memory usage, and the number of learnable parameters.
Time-Efficient Light-Field Acquisition Using Coded Aperture and Events
Shuji Habuchi (Nagoya University), Hajime Nagahara (Nagoya University)
OptimizationComputational EfficiencyImage
🎯 What it does: This paper proposes a method for capturing light fields using a programmable aperture and an event camera within a single exposure, and reconstructing the light field through the combination of events and images.
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
Shuhuai Ren (Peking University), Lu Hou (Huawei)
GenerationCompressionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoMultimodality
🎯 What it does: This paper presents TimeChat, a time-sensitive multimodal large language model designed for long videos, capable of accurately locating events and generating summaries within long videos.
TiNO-Edit: Timestep and Noise Optimization for Robust Diffusion-Based Image Editing
Sherry X Chen (University of California), Pradeep Sen (University of California)
GenerationOptimizationDiffusion modelImageMultimodality
🎯 What it does: Using Stable Diffusion for image editing, the TiNO-Edit method is proposed to achieve various controllable editing tasks by automatically optimizing noise and diffusion step size.
Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer
Junyi Wu (Illinois Institute of Technology), Yan Yan (Illinois Institute of Technology)
Explainability and InterpretabilityTransformerImage
🎯 What it does: Proposes the TokenTM explanation method, which combines token transformation and attention weights of Vision Transformers to generate more reliable post-hoc explanations.
TokenCompose: Text-to-Image Diffusion with Token-level Supervision
Zirui Wang (Princeton University), Zhuowen Tu (University of California, San Diego)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: The TOKENCOMPOSE training framework is proposed, utilizing cross-attention token-level and pixel-level supervision to make text-to-image diffusion models more consistent in multi-category instance synthesis tasks.
TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation
Sai Kumar Dwivedi (Max Planck Institute for Intelligent Systems), Michael J. Black (Max Planck Institute for Intelligent Systems)
Pose EstimationTransformerMesh
🎯 What it does: This study investigates the recovery of 3D human pose and shape from a single image and proposes the TokenHMR method.
ToNNO: Tomographic Reconstruction of a Neural Network's Output for Weakly Supervised Segmentation of 3D Medical Images
Marius Schmidt-Mengin (TheraPanacea), Nikos Paragios (CentraleSupélec)
SegmentationConvolutional Neural NetworkBiomedical DataMagnetic Resonance ImagingComputed TomographyPositron Emission Tomography
🎯 What it does: The ToNNO method is proposed, utilizing a 2D encoder and inverse Radon transform for weakly supervised segmentation of 3D medical images, compatible with any 2D network.
ToonerGAN: Reinforcing GANs for Obfuscating Automated Facial Indexing
Kartik Thakral (Indian Institute of Technology Jodhpur), Richa Singh (Indian Institute of Technology Jodhpur)
RecognitionGenerationData SynthesisSafty and PrivacyGenerative Adversarial NetworkImage
🎯 What it does: De-identification of faces through the generation of cartoon avatars.
Total Selfie: Generating Full-Body Selfies
Bowei Chen (University of Washington), Steven M. Seitz (University of Washington)
GenerationData SynthesisPose EstimationDiffusion modelImageStochastic Differential Equation
🎯 What it does: This paper proposes a full-body selfie generation framework based on diffusion models—Total Selfie, which generates realistic full-body selfies using four selfies (face, upper body, lower body, shoes), a background image, and a target pose.
Total-Decom: Decomposed 3D Scene Reconstruction with Minimal Interaction
Xiaoyang Lyu (University of Hong Kong), Xiaojuan Qi (University of Hong Kong)
RecognitionObject DetectionSegmentationKnowledge DistillationNeural Radiance FieldImagePoint CloudMesh
🎯 What it does: By combining the Segment Anything Model (SAM) with a hybrid implicit-explicit surface representation and a mesh region growing method, 3D reconstruction and decomposition of indoor scenes from sparse pose multi-view images is achieved, requiring only a minimal number of human clicks (approximately 1.4) for object-level separation.
Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts
Jiawen Zhu (Singapore Management University), Guansong Pang (Singapore Management University)
Anomaly DetectionVision Language ModelImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: A general anomaly detection model, InCTRL, is designed to learn residual features for cross-domain unsupervised anomaly detection using a small number of normal images as contextual prompts.
Towards 3D Vision with Low-Cost Single-Photon Cameras
Fangzhou Mu (University of Wisconsin Madison), Yin Li (University of Wisconsin Madison)
Depth EstimationOptimizationNeural Radiance FieldPoint Cloud
🎯 What it does: A 3D shape reconstruction method based on analysis-synthesis is proposed using the time-domain histogram of a low-cost single-photon camera and neural implicit surface representation.
Towards a Perceptual Evaluation Framework for Lighting Estimation
Justine Giroux (Universit' e Laval), Jean-François Lalonde (Universit' e Laval)
Image
🎯 What it does: This paper evaluates the perceptual performance of lighting estimation methods in virtual object insertion through designed comparative experiments and psychological experiments, and proposes a perceptual evaluation framework based on learning from various image quality metrics.
Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation
Renshuai Liu (Xiamen University), Xuan Cheng (Xiamen University)
GenerationData SynthesisDiffusion modelImageMultimodality
🎯 What it does: A multi-modal personalized face generation framework is proposed, capable of simultaneously controlling identity, expression, and background, and achieving fine-grained expression synthesis.
Towards Accurate and Robust Architectures via Neural Architecture Search
Yuwei Ou (Sichuan University), Yanan Sun (Sichuan University)
Adversarial AttackNeural Architecture SearchImage
🎯 What it does: Design a search space for adversarial training and propose a differentiable search method based on multi-objective gradients to automatically search for network architectures that can maintain high natural accuracy while enhancing adversarial robustness.
Towards Accurate Post-training Quantization for Diffusion Models
Changyuan Wang (Tsinghua University), Jiwen Lu (Tsinghua University)
GenerationData SynthesisComputational EfficiencyDiffusion modelImage
🎯 What it does: A post-training quantization framework for diffusion models (APQ-DM) is proposed, which significantly improves the generation quality and inference efficiency of low-bit-width models through group quantization and active calibration set generation.
Towards Automated Movie Trailer Generation
Dawit Mureja Argaw (King Abdullah University of Science and Technology), Bernard Ghanem (King Abdullah University of Science and Technology)
GenerationTransformerVideo
🎯 What it does: A Transformer-based automatic movie trailer generation framework is proposed, which can select and order shots from the complete movie to generate a coherent trailer.
Towards Automatic Power Battery Detection: New Challenge Benchmark Dataset and Baseline
Xiaoqi Zhao (Dalian University of Technology), Huchuan Lu (Dalian University of Technology)
Object DetectionSegmentationConvolutional Neural NetworkImageBenchmark
🎯 What it does: A segmentation-based battery detection framework is proposed and implemented, capable of locating the cathode and anode endpoints in X-ray images to assess battery quality.
Towards Backward-Compatible Continual Learning of Image Compression
Zhihao Duan, Fengqing Zhu
CompressionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new method to address a specific problem in computer vision, with specific details not provided.
Towards Better Vision-Inspired Vision-Language Models
Yun-Hao Cao (Nanjing University), Ming Yang (Ant Group)
TransformerPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: A novel visual-language bridging module VIVL is proposed, which utilizes a Feature Pyramid Extractor (FPE) and Deep Visual Condition Prompting (DVCP) to fully leverage low-level visual information and deep visual-language interactions, enhancing the performance of visual-language models.
Towards Calibrated Multi-label Deep Neural Networks
Jiacheng Cheng (University of California), Nuno Vasconcelos (University of California)
ClassificationRetrievalConvolutional Neural NetworkTransformerImageMultimodality
🎯 What it does: This paper proposes a new strictly proper asymmetric loss (SPA) and a label pair regularizer (LPR) to enhance the probability calibration performance of multi-label deep neural networks, and validates its effectiveness in multi-label classification and retrieval tasks.
Towards CLIP-driven Language-free 3D Visual Grounding via 2D-3D Relational Enhancement and Consistency
Yuqi Zhang (Sichuan University), Yinjie Lei (Sichuan University)
Object DetectionSegmentationTransformerVision Language ModelContrastive LearningImagePoint Cloud
🎯 What it does: A language-free unsupervised 3D visual localization framework based on CLIP is proposed, which utilizes multi-view images to generate pseudo-language features for aligning 3D vision with text.
Towards Co-Evaluation of Cameras HDR and Algorithms for Industrial-Grade 6DoF Pose Estimation
Agastya Kalra (Intrinsic Innovation LLC), Michael Stark (Intrinsic Innovation LLC)
Pose EstimationMultimodality
🎯 What it does: Proposes an industrial-grade 6DoF pose estimation co-evaluation dataset IPD and provides a high-precision evaluation method based on robot consistency.
Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation
Junyan Wang (University of New South Wales), Yang Song (University of New South Wales)
GenerationData SynthesisPose EstimationDepth EstimationDiffusion modelImageText
🎯 What it does: A human-centered prior (HcP) layer is proposed in the text-to-image diffusion model, which directly embeds human priors such as pose or depth maps into the cross-attention key space during the fine-tuning stage, and enhances human structural information using alignment loss, allowing for the generation of more structurally accurate human images during inference without additional conditions.
Towards Efficient Replay in Federated Incremental Learning
Yichen Li (Huazhong University of Science and Technology), Guannan Zhang (Ant Group)
Federated LearningImage
🎯 What it does: In the scenario of federated incremental learning, a lightweight framework called Re-Fed is proposed, which utilizes both local and global information from each client to select important samples for caching, thereby alleviating catastrophic forgetting.
Towards Fairness-Aware Adversarial Learning
Yanghao Zhang (University of Liverpool), Wenjie Ruan (TrustAI)
OptimizationAdversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A Fairness-Aware Adversarial Learning (FAAL) framework is proposed, utilizing distributed robust optimization to enhance class fairness in adversarial training.
Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximation
Yuan Xiao (Nanjing University), Zhenyu Chen (University of Massachusetts Amherst)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper proposes MaxLin, a framework for reachability verification of CNNs with MaxPool layers, utilizing compact linear approximations to achieve stronger robustness lower bounds.
Towards Generalizable Multi-Object Tracking
Zheng Qin (Xi'an Jiaotong University), Wei Tang (University of Illinois)
Object TrackingVideo
🎯 What it does: This paper proposes GeneralTrack, a multi-object tracking framework that transitions from point-level relationships to instance-level associations, achieving high generality of targets in different scenarios.
Towards Generalizable Tumor Synthesis
Qi Chen (University of Science and Technology of China), Zongwei Zhou (Johns Hopkins University)
Object DetectionSegmentationGenerationData SynthesisDiffusion modelAuto EncoderGenerative Adversarial NetworkImageBiomedical DataComputed Tomography
🎯 What it does: A three-stage universal tumor synthesis framework named DiffTumor is proposed and implemented, which generates early tumors in multiple organs from a small number of annotated tumor samples in CT images, and enhances the performance of tumor detection/separation models through these synthetic data.
Towards Generalizing to Unseen Domains with Few Labels
Chamuditha Jayanga Galappaththige (Mohamed Bin Zayed University of AI), Muhammad Haris Khan (Mohamed Bin Zayed University of AI)
Domain AdaptationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A framework for Semi-Supervised Domain Generalization (SSDG) is proposed, which trains a model that can generalize to unknown domains using a limited number of labeled data and a large amount of unlabeled data.
Towards HDR and HFR Video from Rolling-Mixed-Bit Spikings
Yakun Chang (Beijing Jiaotong University), Boxin Shi (Peking University)
RestorationGenerationSpiking Neural NetworkOptical FlowVideo
🎯 What it does: Proposes a Rolling Mixed Bit (RMB) reading mechanism and RMB-Net network for reconstructing HDR and high frame rate video from single-bit and multi-bit time-varying pulses.
Towards High-fidelity Artistic Image Vectorization via Texture-Encapsulated Shape Parameterization
Ye Chen (Shanghai Jiao Tong University), Xuanhong Chen (Shanghai Jiao Tong University)
GenerationOptimizationNeural Radiance FieldImage
🎯 What it does: This paper proposes a texture-encapsulated shape vectorization representation, which decomposes an image into two parts: geometry (control points of Bezier curves) and texture (implicit vectors for each control point + lightweight MLP), and achieves self-supervised optimization of a single image through zero-shot learning.
Towards Language-Driven Video Inpainting via Multimodal Large Language Models
Jianzong Wu (Peking University), Chen Change Loy (Nanyang Technological University)
RestorationTransformerLarge Language ModelDiffusion modelVideoMultimodality
🎯 What it does: A video inpainting task based on natural language instructions is proposed, reducing the reliance on manually annotated masks.
Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training
Xiaoyang Wu (University of Hong Kong), Hengshuang Zhao (University of Hong Kong)
SegmentationDomain AdaptationRepresentation LearningPrompt EngineeringContrastive LearningPoint Cloud
🎯 What it does: A Point Prompt Training (PPT) framework is proposed to achieve collaborative training of 3D representation learning across multiple datasets, avoiding negative transfer.
Towards Learning a Generalist Model for Embodied Navigation
Duo Zheng (Chinese University of Hong Kong), Liwei Wang (Chinese University of Hong Kong)
Robotic IntelligenceTransformerLarge Language ModelPrompt EngineeringTextMultimodality
🎯 What it does: This paper presents NaviLLM, a general-purpose sensory navigation model based on large language models, which unifies various tasks into generative questions through schema-based instruction.
Towards Memorization-Free Diffusion Models
Chen Chen (University of Sydney), Chang Xu (University of Sydney)
GenerationData SynthesisSafty and PrivacyDiffusion modelImage
🎯 What it does: This paper proposes an Anti-Memory Guidance (AMG) framework that combines three targeted guidance strategies to eliminate the memory phenomenon of pre-trained diffusion models without compromising image quality.
Towards Modern Image Manipulation Localization: A Large-Scale Dataset and Novel Methods
Chenfan Qu (South China University of Technology), Lianwen Jin (South China University of Technology)
SegmentationAnomaly DetectionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a new paradigm for constrained image tampering localization (CIML) called CAAA, and based on this paradigm, constructs a large-scale, high-quality MIML dataset, further designing the APSC-Net model for image tampering localization.
Towards More Accurate Diffusion Model Acceleration with A Timestep Tuner
Mengfei Xia (Tsinghua University), Yong-Jin Liu (Tsinghua University)
GenerationOptimizationComputational EfficiencyDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper presents TimeTuner, a plug-in method that reduces truncation errors in the acceleration process of diffusion models by optimizing the time step size at each step.
Towards More Unified In-context Visual Understanding
Dianmo Sheng (University of Science and Technology of China), Nenghai Yu (University of Science and Technology of China)
SegmentationGenerationTransformerMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: A unified multimodal context learning framework is proposed, capable of performing multimodal visual understanding tasks such as image→image and image→text within a single model.
Towards Progressive Multi-Frequency Representation for Image Warping
Jun Xiao (Hong Kong Polytechnic University), Kin-Man Lam (Hong Kong Polytechnic University)
Image TranslationRestorationSuper ResolutionConvolutional Neural NetworkImage
🎯 What it does: A multi-frequency representation (MFR) network is proposed for image warping, achieving coarse-to-fine image reconstruction by learning features from different frequency bands layer by layer.
Towards Real-World HDR Video Reconstruction: A Large-Scale Benchmark Dataset and A Two-Stage Alignment Network
Yong Shu (Shanghai University), Zihao Zhou (Shanghai University)
RestorationConvolutional Neural NetworkVideoBenchmark
🎯 What it does: A new two-stage HDR video reconstruction network is proposed, and a large-scale real scene HDR video dataset Real-HDRV is constructed.
Towards Realistic Scene Generation with LiDAR Diffusion Models
Haoxi Ran, Yue Wang
GenerationData SynthesisAutonomous DrivingDiffusion modelAuto EncoderMultimodalityPoint Cloud
🎯 What it does: This paper proposes LiDAR Diffusion Models (LiDMs), a diffusion model for generating LiDAR scenes in latent space, supporting multimodal conditional generation (semantic maps, camera views, text prompts, etc.).
Towards Robust 3D Object Detection with LiDAR and 4D Radar Fusion in Various Weather Conditions
Yujeong Chae (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
Object DetectionAutonomous DrivingConvolutional Neural NetworkMultimodalityPoint Cloud
🎯 What it does: A 3D object detection framework for the fusion of LiDAR and 4D radar under different weather conditions is proposed.
Towards Robust 3D Pose Transfer with Adversarial Learning
Haoyu Chen (Computer and Machine Vision Society), Guoying Zhao (Stanford University)
GenerationPose EstimationAdversarial AttackGenerative Adversarial NetworkPoint Cloud
🎯 What it does: A 3D pose transfer framework based on adversarial learning is proposed, which can achieve end-to-end pose transfer on the original scanned point cloud and significantly enhance the robustness of the model.
Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach
Guoqiang Liang (Hong Kong University of Science and Technology), Lin Wang (Hong Kong University of Science and Technology)
RestorationTransformerImage
🎯 What it does: A large-scale real-world low-light event-image dataset (SDE) and an event-based low-light image enhancement framework called EvLight are proposed.
Towards Robust Learning to Optimize with Theoretical Guarantees
Qingyu Song (Chinese University of Hong Kong), Hong Xu (Chinese University of Hong Kong)
OptimizationRecurrent Neural NetworkTabular
🎯 What it does: A robust model for Learning to Optimize (L2O) is proposed, along with a theoretical analysis of convergence under out-of-distribution (OOD) conditions.
Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network
Wenqiao Li (ShanghaiTech University), Yingna Wu (ShanghaiTech University)
Anomaly DetectionContrastive LearningPoint CloudBenchmark
🎯 What it does: This paper proposes a 3D point cloud-based anomaly detection and localization framework called IMRNet, and constructs a scalable synthetic dataset named Anomaly-ShapeNet.
Towards Surveillance Video-and-Language Understanding: New Dataset Baselines and Challenges
Tongtong Yuan (Beijing University of Technology), Zhenzhen Jiao (Beijing University of Posts and Telecommunications)
Anomaly DetectionTransformerVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: We propose and construct UCA - the first multimodal surveillance video dataset, and conduct benchmark experiments on this dataset for four types of tasks (temporal sentence localization, video captioning, dense captioning, and cross-modal anomaly detection).
Towards Text-guided 3D Scene Composition
Qihang Zhang, Hsin-Ying Lee
GenerationOptimizationDiffusion modelScore-based ModelNeural Radiance FieldImageText
🎯 What it does: A text-based 3D scene synthesis method called SceneWiz3D is proposed, utilizing a hybrid representation of explicit (DMTet) and implicit (NeRF);
Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation
Xiaoyang Wang (Xidian University), Jimin Xiao (Beijing Jiaotong University)
SegmentationFlow-based ModelImage
🎯 What it does: This paper proposes a disturbance strategy based on low-density partitioning in feature space called Density-Descending Feature Perturbation (DDFP), which achieves stronger discriminative consistency learning by perturbing features in low-density directions within a semi-supervised semantic segmentation framework.
Towards Transferable Targeted 3D Adversarial Attack in the Physical World
Yao Huang (Beihang University), Xingxing Wei (Tsinghua University)
Adversarial AttackNeural Radiance FieldPoint CloudMesh
🎯 What it does: A framework called TT3D is proposed to generate transferable target 3D adversarial examples, utilizing a gridded NeRF to achieve dual optimization of texture and geometry;
Towards Understanding and Improving Adversarial Robustness of Vision Transformers
Samyak Jain (Indian Institute of Technology Banaras Hindu University Varanasi), Tanima Dutta (Indian Institute of Technology Banaras Hindu University Varanasi)
Adversarial AttackTransformerImage
🎯 What it does: This paper studies the gradient masking problem of Vision Transformer in adversarial attacks, finding that softmax computation can lead to floating-point underflow, thereby weakening the attack effect; subsequently, it proposes an Adaptive Attention Scaling (AAS) attack based on LPIPS and integrates this method into adversarial training to obtain AAS-AT.
Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing
Bingyan Liu (South China University of Technology), Jun Huang (Chinese University of Hong Kong)
Image TranslationGenerationTransformerDiffusion modelAuto EncoderImage
🎯 What it does: This paper proposes a parameter-free text-guided image editing method called FPE, which only modifies self-attention, through the detection and analysis of cross-attention and self-attention maps in Stable Diffusion.
Towards Variable and Coordinated Holistic Co-Speech Motion Generation
Yifei Liu (South China University of Technology), Changxing Ding (South China University of Technology)
GenerationData SynthesisAuto EncoderVideoAudio
🎯 What it does: A probabilistic framework named ProbTalk has been designed and implemented to generate synchronized overall 3D human motion (facial expressions, gestures, and body movements), incorporating various technical integrations to enhance the diversity and coordination of the movements.
Traceable Federated Continual Learning
Qiang Wang (Beijing University of Posts and Telecommunications), Yawen Li (Beijing University of Posts and Telecommunications)
Federated LearningKnowledge DistillationImageBenchmark
🎯 What it does: A traceable federated continual learning (TFCL) framework called TagFed is proposed to address the performance degradation caused by the repeated occurrence of tasks in a federated environment.
Traffic Scene Parsing through the TSP6K Dataset
Peng-Tao Jiang (Nankai University), Chunhua Shen (Nankai University)
Object DetectionSegmentationDomain AdaptationTransformerSupervised Fine-TuningImageBenchmark
🎯 What it does: A high-density dataset TSP6K aimed at traffic monitoring scenarios is proposed, and various scene parsing, instance segmentation, and unsupervised domain adaptation methods are evaluated on it. Furthermore, a detail refining decoder is designed to improve segmentation performance in monitoring scenarios.
Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning
Zichen Miao (Purdue University), Zicheng Liu (Advanced Micro Devices, Inc.)
GenerationData SynthesisOptimizationReinforcement LearningDiffusion modelImage
🎯 What it does: A diversity-oriented fine-tuning framework based on reinforcement learning is proposed, which defines a diversity reward at the image set level (MMD/mutual information) through a small number of unbiased reference images, achieving general diversity enhancement for diffusion models (both class-conditional and text-conditional).
Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifacts
Cansu Korkmaz (Koc University), Zafer Dogan (Koc University)
RestorationSuper ResolutionGenerative Adversarial NetworkImage
🎯 What it does: A GAN super-resolution training framework based on wavelet domain loss is proposed, utilizing adversarial and reconstruction losses from SWT subbands to suppress high-frequency artifacts and enhance detail fidelity.
Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation
Yunhe Gao (Rutgers University)
SegmentationTransformerImageBiomedical DataMagnetic Resonance ImagingComputed TomographyPositron Emission Tomography
🎯 What it does: A general medical image segmentation framework called Hermes is proposed, which can perform multi-position, multi-modal, and partially labeled multi-task segmentation within a single model.
Training Vision Transformers for Semi-Supervised Semantic Segmentation
Xinting Hu, Bernt Schiele
ClassificationSegmentationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a new computer vision method aimed at improving the accuracy of image recognition.
Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation
Luca Barsellotti (University of Modena and Reggio Emilia), Rita Cucchiara (Istituto Italiano di Tecnologia)
SegmentationRetrievalDiffusion modelImage
🎯 What it does: A training-free open vocabulary semantic segmentation method called FreeDA is proposed, which utilizes a diffusion model to offline generate visual prototypes and text keys, achieving pixel-level segmentation by combining local and global similarities during inference.
Training-Free Pretrained Model Merging
Zhengqi Xu (Zhejiang University), Jie Song (Zhejiang University)
Convolutional Neural NetworkTransformerImage
🎯 What it does: Proposes the MuDSC (Merging under Dual-Space Constraints) framework, which utilizes an untrained pre-trained model to achieve multi-task model merging through unit matching.
Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection
Zhiyuan Yan (Chinese University of Hong Kong Shenzhen), Baoyuan Wu (Chinese University of Hong Kong Shenzhen)
ClassificationRecognitionDomain AdaptationAnomaly DetectionKnowledge DistillationConvolutional Neural NetworkVideo
🎯 What it does: This paper proposes a deepfake detection method based on latent space augmentation, called LSDA, aimed at enhancing the model's generalization ability by expanding the forgery space.
Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary
Leheng Zhang (University of Electronic Science and Technology of China), Shuhang Gu (University of Electronic Science and Technology of China)
RestorationSuper ResolutionTransformerImage
🎯 What it does: Proposes the Adaptive Token Dictionary (ATD) Transformer, which utilizes a learnable Token Dictionary to integrate external priors and global information through cross-attention and self-attention, in order to enhance the super-resolution quality of single images.
Transcriptomics-guided Slide Representation Learning in Computational Pathology
Guillaume Jaume (Mass General Brigham), Faisal Mahmood (Mass General Brigham)
ClassificationRetrievalRepresentation LearningTransformerContrastive LearningMultimodalityBiomedical Data
🎯 What it does: TANGLE is proposed, a whole slide image (WSI) representation learning framework guided by gene expression data;
Transductive Zero-Shot and Few-Shot CLIP
Ségolène Martin (University Paris-Saclay), Ismail Ben Ayed (University Paris-Saclay)
ClassificationContrastive LearningImage
🎯 What it does: A transductive zero-shot and few-shot CLIP classification method based on the Dirichlet distribution is proposed, utilizing text-image probabilistic features for batch inference.
Transfer CLIP for Generalizable Image Denoising
Jun Cheng (Huazhong University of Science and Technology), Shan Tan (Huazhong University of Science and Technology)
RestorationConvolutional Neural NetworkContrastive LearningImageComputed Tomography
🎯 What it does: By utilizing the features extracted from the first four layers of the frozen CLIP ResNet encoder and concatenating them with noisy images, a lightweight convolutional decoder is used to restore clear images, achieving universal denoising for various types of noise.
Transferable and Principled Efficiency for Open-Vocabulary Segmentation
Jingxuan Xu (Simon Fraser University), Yunchao Wei (Beijing Jiaotong University)
SegmentationComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a transferable sparse subnetwork and an efficient fine-tuning method based on weight spectrum, achieving a significant reduction in model size and training costs for Open-Vocabulary Segmentation (OVS).
Transferable Structural Sparse Adversarial Attack Via Exact Group Sparsity Training
Di Ming (Chongqing University of Technology), Xin Feng (Chongqing University of Technology)
Object DetectionSegmentationAdversarial AttackConvolutional Neural NetworkTransformerGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a transferable structured sparse adversarial attack method EGS-TSSA, which can significantly improve the attack success rate on the target model while maintaining extremely low pixel perturbations and concentrating the perturbations in semantically relevant areas for classification.
TransLoc4D: Transformer-based 4D Radar Place Recognition
Guohao Peng (Nanyang Technological University), Danwei Wang (Nanyang Technological University)
RecognitionAutonomous DrivingTransformerPoint CloudBenchmark
🎯 What it does: This paper proposes an end-to-end venue recognition model TransLoc4D based on 4D millimeter-wave radar, capable of achieving robust localization in adverse weather conditions.
TransNeXt: Robust Foveal Visual Perception for Vision Transformers
Dai Shi
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: This paper proposes a bionic focus attention (Aggregated Attention) and Convolutional GLU, integrating them into a new visual backbone network called TransNeXt, aimed at addressing the degradation problem of traditional Vision Transformers in deep information mixing.
Tri-Modal Motion Retrieval by Learning a Joint Embedding Space
Kangning Yin (ShanghaiTech University), Zheng Tian (ShanghaiTech University)
RetrievalTransformerVision-Language-Action ModelContrastive LearningVideoTextMultimodality
🎯 What it does: This paper proposes a tri-modal learning framework called LAVIMO, which unifies text, video, and action modalities into the same embedding space to achieve cross-modal retrieval.
Tri-Perspective View Decomposition for Geometry-Aware Depth Completion
Zhiqiang Yan (Nanjing University of Science and Technology), Jian Yang (Nanjing University of Science and Technology)
RestorationDepth EstimationAutonomous DrivingConvolutional Neural NetworkSupervised Fine-TuningPoint Cloud
🎯 What it does: A three-view projection decomposition framework TPVD is proposed, which projects sparse depth point clouds into three 2D images: front view, top view, and side view, and achieves depth completion through 2D–3D–2D recursive aggregation.
TRINS: Towards Multimodal Language Models that Can Read
Ruiyi Zhang (Adobe Research), Tong Sun (Adobe Research)
RecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A text-rich image instruction dataset named TRINS has been constructed, and a multimodal language model called LaRA has been proposed, which combines OCR with a visual encoder, aiming to enhance the model's understanding and generation capabilities of text within images.
TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models
Zhongwei Zhang (University of Science and Technology of China), Tao Mei (HiDream.ai Inc.)
GenerationData SynthesisTransformerDiffusion modelImageVideo
🎯 What it does: A time residual learning framework based on image noise prior (TRIP) is proposed to improve the diffusion model from image to video, making the generated video more aligned with the original image and possessing better temporal consistency.
Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers
Zi-Xin Zou (Tsinghua University), Song-Hai Zhang (Tsinghua University)
TransformerGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes a Transformer-based single-view 3D reconstruction framework called TGS, which achieves fast and high-quality 3D reconstruction and view synthesis using a hybrid Triplane-Gaussian representation.
Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning
Siteng Huang (Zhejiang University), Donglin Wang (Westlake University)
ClassificationRecognitionTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: This paper addresses the problem of synthetic zero-shot learning and proposes a multi-path cross-modal coupling model called Troika, which explicitly models three types of semantics: state, object, and combination through three branches.
TTA-EVF: Test-Time Adaptation for Event-based Video Frame Interpolation via Reliable Pixel and Sample Estimation
Hoonhee Cho (KAIST), Kuk-Jin Yoon (KAIST)
Image TranslationRestorationDomain AdaptationKnowledge DistillationVideo
🎯 What it does: A testing time adaptation framework for event camera video frame interpolation (TTA‑EVF) is proposed, which can adapt the network online using only low frame rate videos and event streams in the target domain.
TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease
Kyungdo Kim (Duke University), Timothy W. Dunn (Duke University)
ClassificationPose EstimationVideoBiomedical Data
🎯 What it does: The TULIP dataset is proposed, recording multi-camera 3D videos of 25 UPDRS motor examination activities, rated by three clinical experts.
TULIP: Transformer for Upsampling of LiDAR Point Clouds
Bin Yang (ETH Zurich), Vaishakh Patil (ETH Zurich)
Super ResolutionAutonomous DrivingTransformerPoint Cloud
🎯 What it does: This paper proposes a Transformer-based LiDAR point cloud upsampling method called TULIP, which performs super-resolution processing directly on 2D distance maps.
Tumor Micro-environment Interactions Guided Graph Learning for Survival Analysis of Human Cancers from Whole-slide Pathological Images
Wei Shao (Nanjing University of Aeronautics and Astronautics), Peng Wan (Nanjing University of Aeronautics and Astronautics)
ClassificationSegmentationExplainability and InterpretabilityGraph Neural NetworkImageBiomedical Data
🎯 What it does: Developed a TMEGL model based on whole-slide images to predict the survival of human cancer patients using tumor microenvironment (TME) interactions.
TUMTraf V2X Cooperative Perception Dataset
Walter Zimmer (Technical University of Munich), Alois C. Knoll (Technical University of Munich)
Object DetectionAutonomous DrivingTransformerImageMultimodalityPoint Cloud
🎯 What it does: This paper presents the TUMTraf V2X multimodal V2X cooperative perception dataset for real traffic scenarios and develops the CoopDet3D cooperative 3D object detection model based on it.
Tune-An-Ellipse: CLIP Has Potential to Find What You Want
Jinheng Xie (National University of Singapore), Mike Zheng Shou (National University of Singapore)
Object DetectionOptimizationTransformerContrastive LearningImage
🎯 What it does: This paper proposes a differentiable visual prompt method (Tune-An-Ellipse) that utilizes the visual prompting capability of CLIP to achieve zero-shot referential expression localization by iteratively optimizing the parameters of an ellipse.
Tuning Stable Rank Shrinkage: Aiming at the Overlooked Structural Risk in Fine-tuning
Sicong Shen (Beihang University), Yan Xu (Beihang University)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: During the fine-tuning process of pre-trained models, it was found that existing methods failed to effectively reduce model complexity. The TSRS (Tuning Stable Rank Shrinkage) regularization is proposed, which utilizes noise sensitivity to constrain the model's stable rank, thereby reducing structural risk and enhancing generalization.
Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence
Ripon Kumar Saha (Arizona State University), Suren Jayasuriya (Arizona State University)
RestorationSegmentationTransformerOptical FlowVideo
🎯 What it does: A complete processing pipeline of 'first segmentation and then recovery' for dynamic video in atmospheric turbulence environments is proposed.
TurboSL: Dense Accurate and Fast 3D by Neural Inverse Structured Light
Parsa Mirdehghan (University of Toronto), Kiriakos N. Kutulakos (University of Toronto)
Depth EstimationOptimizationComputational EfficiencyNeural Radiance FieldImageVideo
🎯 What it does: This paper presents TurboSL, a neural network framework that treats structured light decoding as an inverse rendering problem, achieving sub-pixel accuracy in 3D reconstruction by utilizing SDF combined with forward/inverse rendering.
TutteNet: Injective 3D Deformations by Composition of 2D Mesh Deformations
Bo Sun (University of Texas at Austin), Noam Aigerman (University of Montreal)
Data SynthesisOptimizationNeural Radiance FieldMesh
🎯 What it does: A three-dimensional injective deformation network (TutteNet) is proposed, achieved through stacking two-dimensional Tutte embeddings, which can directly obtain reversible and non-self-intersecting three-dimensional deformations in learning and optimization.
Tyche: Stochastic In-Context Learning for Medical Image Segmentation
Marianne Rakic (Massachusetts Institute of Technology), Adrian V. Dalca (Massachusetts Institute of Technology)
SegmentationConvolutional Neural NetworkImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: An unsupervised, untrained medical image segmentation framework named Tyche is proposed, capable of generating diverse prediction results for unseen segmentation tasks given only a contextual sample set; it also supports generating multiple candidate segmentations during inference through noise or augmentation.
U-VAP: User-specified Visual Appearance Personalization via Decoupled Self Augmentation
You Wu (Institute of Computing Technology, Chinese Academy of Sciences), Jintao Li (Institute of Computing Technology, Chinese Academy of Sciences)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringDiffusion modelImageText
🎯 What it does: Under limited reference images, users can specify visual attributes through text descriptions to achieve refined visual appearance personalization.
UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusion
Junsheng Zhou (Tsinghua University), Zhizhong Han
GenerationData SynthesisDiffusion modelMesh
🎯 What it does: A 3D diffusion model UDiFF is developed, which uses unsigned distance fields (UDF) to generate textured 3D shapes with open and closed surfaces, and supports text or image conditional generation.
UFC-Net: Unrolling Fixed-point Continuous Network for Deep Compressive Sensing
Xiaoyang Wang (Northwestern Polytechnical University), Hongping Gan (Northwestern Polytechnical University)
RestorationCompressionConvolutional Neural NetworkTransformerImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This work proposes a deep unfolding network called UFC-Net based on fixed-point continuous algorithms for image compressed sensing and CS-MRI reconstruction;
UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity
Jialong Zuo (Huazhong University of Science and Technology), Changxin Gao (Huazhong University of Science and Technology)
RetrievalContrastive LearningImageTextBenchmark
🎯 What it does: A new text retrieval portrait dataset UFine6926 is proposed, focusing on ultra-fine-grained descriptions.
UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs
Yanwu Xu (Google), Tingbo Hou (Google)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageText
🎯 What it does: A first-order sampling text-to-image generation model called UFOGen is proposed, which integrates diffusion models and GAN objectives to generate high-quality images with a single forward inference.