🎯 What it does: A multi-language, multi-modal, and temporal social media popularity prediction benchmark (SMTPD) has been constructed, and a baseline model based on multi-modal feature fusion and LSTM regression has been proposed to predict the popularity time series of YouTube videos within 30 days.
🎯 What it does: A visual-driven 3D semantic scene completion method named SOAP is proposed, which combines occlusion-aware view projection and a scene-adaptive decoder;
🎯 What it does: The SocialMOIF model is proposed, which combines multi-order intent fusion, trajectory distribution approximator, and global trajectory optimizer to achieve multi-modal and parallel trajectory prediction for pedestrians (or vehicles, athletes).
🎯 What it does: This paper proposes SoftVQ-VAE, a differentiable continuous image tokenizer that significantly reduces the number of tokens to 32 or 64 through soft category posterior aggregation of multiple codewords, while maintaining high-quality reconstruction and improving the efficiency of generative models.
🎯 What it does: This paper proposes a parameter-efficient fine-tuning framework called SoMA based on singular value decomposition, aimed at adapting to domain transfer tasks while maintaining the generalization capability of visual foundation models.
🎯 What it does: We propose Sonata, a self-supervised learning framework for point clouds that achieves strong linear probing performance with very few trainable parameters and supports multi-scale representations and optional decoders.
SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models
Kevin Miller (Boston University), Venkatesh Saligrama (Boston University)
CodeClassificationRecognitionTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: This paper proposes a zero-shot multi-label recognition framework called SPARC, based on visual-language model (VLM) black-box scoring, which utilizes normalization to eliminate image and prompt biases, and enhances classification performance through composite prompts and adaptive rank fusion.
🎯 What it does: A method is proposed to directly predict 2D Gaussians from point clouds and perform lighting rendering, capable of generating high-quality images directly from sparse point clouds.
SpatialCLIP: Learning 3D-aware Image Representations from Spatially Discriminative Language
Zehan Wang (Zhejiang University), Zhou Zhao (Zhejiang University)
CodeRetrievalRepresentation LearningTransformerLarge Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes SpatialCLIP, which enhances CLIP's understanding of spatial relationships by improving the visual encoder and language supervision, and builds SpatialLLaVA based on this model to enhance the spatial intelligence of multimodal large models.
🎯 What it does: This paper proposes a self-supervised stereo video synthesis framework based on a video diffusion model—SpatialDreamer, which can generate high-quality stereo videos from monocular videos.
🎯 What it does: Proposed spherical manifold convolution (SMConv) and a diffusion model based on SMConv (SMGD) for generating panoramic images from text.
Spiking Transformer with Spatial-Temporal Attention
Donghyun Lee (Yale University), Priyadarshini Panda (Yale University)
CodeSpiking Neural NetworkTransformerImage
🎯 What it does: A block-based space-time attention mechanism STAtten is proposed for pulse Transformers, integrating spatial and temporal information to enhance feature representation.
SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking
Wenrui Cai (Beihang University), Yunhong Wang (Beihang University)
CodeObject TrackingTransformerMixture of ExpertsVideo
🎯 What it does: A visual tracker SPMTrack based on Mixture of Experts (TMoE) is proposed, which can adaptively handle different relationship modeling and introduce spatiotemporal context;
🎯 What it does: Proposes SSHNet, a framework that transforms unsupervised cross-modal homography estimation into two directly supervised subproblems and jointly trains through segmentation optimization.
🎯 What it does: This paper proposes an incremental facial forgery detection method that performs Aligned Feature Isolation in the latent space, utilizing Sparse Uniform Replay (SUR) and Latent Incremental Detector (LID) to achieve knowledge retention and new task learning.
🎯 What it does: A Transformer-style visual model SBM based on star-shaped operations and low-rank bilinear mapping is proposed, achieving global context modeling with linear complexity.
🎯 What it does: A collaborative training framework called SynFoC is proposed, which integrates a foundational model (MedSAM) with a traditional model (U-Net) to address the issues of domain shift and label scarcity in mixed-domain semi-supervised medical image segmentation (MiDSS).
Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks
Han Wang (University of Illinois Urbana-Champaign), Huan Zhang (University of Illinois Urbana-Champaign)
CodeAdversarial AttackTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper studies a defense method called ASTRA based on adaptive activation vectors, which utilizes visual attribution to identify attack features and dynamically shifts model activations during inference to resist jailbreak attacks on visual language models.
🎯 What it does: A new Sequential Tensor Estimation Framework (STEPS) is proposed to optimize discrete hard prompts in text-to-image diffusion models, making the model generate images that are more similar to the target image.
🎯 What it does: This paper proposes a two-stage framework named STEREO for achieving robust erasure of unwanted concepts in pre-trained text-to-image diffusion models.
🎯 What it does: Proposes the STiL framework, which uses semi-supervised learning to address the modality information gap in image-table multimodal classification.
STINR: Deciphering Spatial Transcriptomics via Implicit Neural Representation
Yisi Luo (Xi'an Jiaotong University), Deyu Meng (Xi'an Jiaotong University)
CodeRestorationSegmentationData SynthesisAuto EncoderBiomedical Data
🎯 What it does: This paper proposes a STINR framework based on Implicit Neural Representation (INR) for continuous modeling of spatial transcriptomics data, achieving tasks such as gene imputation, denoising, spatial domain detection, and cell type deconvolution.
🎯 What it does: This paper proposes an integrated Space-Time Dynamic Prompt (STOP) method, which achieves parameter-efficient fine-tuning for video tasks by incorporating learnable intra-frame spatial prompts and inter-frame temporal prompts into the CLIP pre-trained model.
🎯 What it does: A self-regressive text-to-long-video generation framework named StreamingT2V is proposed, capable of continuously generating high-quality, motion-rich, and temporally consistent videos of up to two minutes.
Structure-from-Motion with a Non-Parametric Camera Model
Yihan Wang (École Polytechnique Fédérale de Lausanne), Viktor Larsson (Lund University)
CodePose EstimationOptimizationSimultaneous Localization and MappingImage
🎯 What it does: This paper presents an incremental structure from motion (SfM) pipeline named GenSfM, which achieves joint reconstruction and calibration of cameras with arbitrary viewpoint distortion using an adaptive non-parametric camera model.
StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfer
Ruojun Xu (Zhejiang University), Zach Cheng
CodeImage TranslationDiffusion modelImage
🎯 What it does: This paper proposes StyleSSP, a training-free diffusion-based style transfer method that utilizes improved sampling starting points to enhance content preservation and prevent style image content leakage.
🎯 What it does: A super lightweight multimodal brain tumor segmentation network called SuperLightNet is proposed, aiming to significantly reduce the number of parameters and computational cost while maintaining segmentation accuracy.
🎯 What it does: A method named SURGEON is proposed to significantly reduce memory consumption during the Full Test-Time Adaptation (FTTA) process while maintaining or improving adaptation performance.
🎯 What it does: A video depth completion method called SVDC is proposed to address the sparse noisy depth maps generated by lightweight direct ToF (dToF) sensors for mobile devices, achieving more complete and coherent depth estimation.
SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering
Hanxiao Sun (Nankai University), Beibei Wang (Nanjing University)
CodeGaussian SplattingPoint Cloud
🎯 What it does: Introducing spatially varying Gaussian representations and a physically-based indirect lighting model in the inverse rendering task, achieving higher quality novel view synthesis and relighting.
Symbolic Representation for Any-to-Any Generative Tasks
Jiaqi Chen (Stanford University), Li-jia Li (LiveX AI)
CodeGenerationData SynthesisTransformerLarge Language ModelTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes a symbolic generation task description language called A-Language and a zero-shot inference engine based on pre-trained language models, which directly transforms natural language task descriptions into executable multimodal generation workflows.
SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization
Hongrui Jia (National Engineering Research Center for Software Engineering Peking University), Shikun Zhang (National Engineering Research Center for Software Engineering Peking University)
CodeOptimizationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringImageTextMultimodality
🎯 What it does: A method called Symbolic Direct Preference Optimization (SymDPO) is proposed and implemented, which replaces answers with non-semantic symbols in In-context Learning (ICL) to force the model to associate visual information with symbols, enhancing the understanding and utilization of image content.
🎯 What it does: A synchronous diffusion model SyncSDE based on a probabilistic framework is proposed to achieve collaborative generation among multiple diffusion trajectories.
SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding
Hao Li (Shanghai AI Laboratory), Jifeng Dai (Tsinghua University)
CodeRecognitionGenerationTransformerLarge Language ModelMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: A unified encoder-free multimodal large language model, SynerGen-VL, is proposed, capable of both image understanding and image generation.
Synthetic Data is an Elegant GIFT for Continual Vision-Language Models
Bin Wu (Wuhan University), Mang Ye (Wuhan University)
CodeGenerationData SynthesisKnowledge DistillationTransformerVision Language ModelDiffusion modelContrastive LearningImageMultimodality
🎯 What it does: A continuous fine-tuning framework named GIFT is proposed, which achieves knowledge retention and generalization balance for Vision-Language models (such as CLIP) by generating synthetic data while continuously learning new tasks.
🎯 What it does: This study achieves precise facial annotation in thermal imaging mode by constructing a large-scale synthetic thermal image dataset and training a heatmap facial keypoint detector.
T2SG: Traffic Topology Scene Graph for Topology Reasoning in Autonomous Driving
Changsheng Lv (Beijing University of Posts and Telecommunications), Huadong Ma (Beijing University of Posts and Telecommunications)
CodeAutonomous DrivingTransformerGraph
🎯 What it does: A unified Traffic Topology Scene Graph (TSG) is proposed for traffic scene understanding in autonomous driving, and the TopoFormer model is introduced to achieve one-stage TSG generation.
TAET: Two-Stage Adversarial Equalization Training on Long-Tailed Distributions
Wang Yu-Hang (Hefei University of Technology), Jian Liu (Hefei University of Technology)
CodeClassificationAdversarial AttackGenerative Adversarial NetworkImageBiomedical Data
🎯 What it does: A two-stage adversarial equalization training (TAET) framework is proposed to enhance the model's adversarial robustness and balanced accuracy under long-tail distributions.
🎯 What it does: This paper proposes a self-supervised, template-free framework for animatable model reconstruction of humans/animals, called TAGA. It utilizes explicit 3D Gaussian representations and skinning driven by skeletal structures to simultaneously learn geometry and skinning weights from sparse pose videos.
🎯 What it does: This paper proposes the L2S framework, which can automatically identify hard samples in medical images and use a specialized segmentation network for precise segmentation.
🎯 What it does: Using 3D Gaussian Splatting and a personalized SMPLX++ clothing extension template, we construct a fully-renderable Talking Avatar that can be rendered in real-time, achieving high-quality details through teacher-student distillation and lightweight Blend Shapes.
TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models
Xin Wang (Fudan University), Xingjun Ma (Fudan University)
CodeOptimizationAdversarial AttackTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: This paper proposes a method called Test-time Adversarial Prompt Tuning (TAPT) to enhance robustness in zero-shot inference of visual-language models like CLIP.
TAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justification
Dongyoon Yang (SK Hynix), Yongdai Kim (Seoul National University)
CodeDomain AdaptationAdversarial AttackImage
🎯 What it does: A theoretical framework and algorithm TAROT is proposed, which simultaneously considers adversarial robustness and unsupervised domain adaptation.
Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment
Ziang Yan (Shanghai AI Laboratory), Yi Wang (Shanghai Innovation Institute)
CodeSegmentationRetrievalOptimizationTransformerLarge Language ModelSupervised Fine-TuningImageVideoTextMultimodality
🎯 What it does: This paper proposes the Task Preference Optimization (TPO) method, which utilizes differentiable task preferences to couple various visual task heads with a multimodal large language model (MLLM), enhancing the model's capabilities in fine-grained visual understanding and dialogue.
🎯 What it does: This paper proposes a semi-supervised crowd counting framework TMTB, which enhances data through diffusion-based image filling in background areas and captures global context using a visual state space model to achieve high-precision counting.
Taxonomy-Aware Evaluation of Vision-Language Models
Vésteinn Snæbjarnarson (University of Copenhagen), Stella Frank (University of Copenhagen)
CodeClassificationGenerationTransformerVision Language ModelTextMultimodality
🎯 What it does: This paper proposes a hierarchical structure-based evaluation framework to measure the accuracy and specificity of open-ended text generated by visual language models (VLM) in fine-grained visual classification tasks.
🎯 What it does: This paper proposes a knowledge distillation method for spiking neural networks, utilizing temporal separation and entropy regularization to enhance the performance of the student network.
🎯 What it does: The TensoFlow method is proposed to learn spatial and directional importance samplers in inverse rendering, improving the Monte Carlo estimation of the rendering equation.
Test-Time Domain Generalization via Universe Learning: A Multi-Graph Matching Approach for Medical Image Segmentation
Xingguo Lv (Anhui University), Xuejun Li (Anhui University)
CodeSegmentationDomain AdaptationGraph Neural NetworkImageBiomedical Data
🎯 What it does: A testing domain transfer framework based on multi-graph matching is constructed, utilizing cosmic embedding to achieve cross-domain consistency and adaptive optimization in medical image segmentation.
🎯 What it does: This paper proposes a Visual Context Tuning (VICT) method for test-time, which utilizes a single test sample to perform self-supervised fine-tuning on visual context learning models, thereby enhancing robustness under distribution shifts.
Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
Wenxuan Guo (Tsinghua University), Jiwen Lu (Tsinghua University)
CodeObject DetectionComputational EfficiencyConvolutional Neural NetworkVision Language ModelTextPoint Cloud
🎯 What it does: A single-stage sparse convolution 3D visual localization framework TSP3D is designed, achieving efficient 3D visual localization through text-guided sparse voxel pruning and completion.
The Illusion of Unlearning: The Unstable Nature of Machine Unlearning in Text-to-Image Diffusion Models
Naveen George (Indian Institute of Technology), Konda Reddy Mopuri (Indian Institute of Technology)
CodeGenerationDiffusion modelImageText
🎯 What it does: The paper studies the machine unlearning methods for text-to-image diffusion models and reveals their instability, indicating that the concept of unlearning re-emerges after fine-tuning.
🎯 What it does: A CEM algorithm based on conditional entropy maximization is proposed, which enhances the robustness against model inversion attacks by performing Gaussian mixture estimation on intermediate features in collaborative reasoning, and can be seamlessly integrated into existing redundancy defense methods.
Three-view Focal Length Recovery From Homographies
Yaqing Ding (Czech Technical University in Prague), Zuzana Kukelova (Czech Technical University in Prague)
CodeOptimizationComputational EfficiencyImage
🎯 What it does: This paper proposes an efficient method to recover the camera focal length from the four-point correspondence observed by a three-view coplanar camera using single images from three different views.
🎯 What it does: For convolutional neural networks that include MaxPool, a verifier called Ti-Lin is proposed, which achieves robustness verification by constructing the tightest linear bounds at the neuron level.
🎯 What it does: To address the inference speed bottleneck of video diffusion models, a training-free caching strategy called TeaCache is proposed. It utilizes temporal embedding modulation of noise input to estimate the output difference of the model, thereby dynamically deciding whether to reuse cached outputs during inference, significantly improving speed.
🎯 What it does: We propose Cropr, a module that learns task relevance and efficiently prunes in Vision Transformers through cross-attention routing and auxiliary prediction heads, applicable to tasks such as classification, semantic segmentation, and object detection.
CodeGenerationData SynthesisDiffusion modelBiomedical Data
🎯 What it does: This paper proposes a cell topology generation method based on diffusion models, called TopoCellGen, to synthesize multi-class cell layouts while preserving realistic spatial and topological features.
🎯 What it does: The Tora framework is proposed to achieve controllable video generation based on trajectories, text, and images, maintaining motion consistency across various resolutions, frame rates, and long durations (up to 204 frames).
🎯 What it does: A unified multimodal medical image re-identification (MedReID) model called MaMI is proposed, which achieves the adaptation of multiple medical image modalities through a Continuous Modal Parameter Adapter (ComPA); at the same time, it utilizes a Medical Foundation Model (MFM) for medical prior alignment to enhance the discriminative power of features.
🎯 What it does: Fine-tuning a text-to-image diffusion model using reinforcement learning to address the issue of insufficient alignment between generated images and text prompts.
🎯 What it does: This paper studies the gradient conflict problem in multi-task learning and proposes ConsMTL, which consistently alleviates conflicts by simultaneously updating shared parameters and task-specific parameters through a dual-layer optimization.
Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients
Li Lun (Peking University), Xiaoxin Cui (Peking University)
CodeAdversarial AttackSpiking Neural NetworkImage
🎯 What it does: Proposed potential-dependent pseudo-gradient (PDSG) and sparse dynamic attack (SDA) to generate high success rate adversarial samples on inferable spiking neural networks.
Towards Fine-Grained Interpretability: Counterfactual Explanations for Misclassification with Saliency Partition
Lintong Zhang (Korea University), Seong-Whan Lee (Korea University)
CodeExplainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: A Fine-grained Visual Counterfactual Explanation (FG-VCE) framework is proposed to generate object-level and part-level fine-grained contrastive explanations for misclassified samples, helping humans understand model decisions.
Towards General Visual-Linguistic Face Forgery Detection
Ke Sun (Xiamen University), Rongrong Ji (Xiamen University)
CodeClassificationRecognitionExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringImageTextMultimodality
🎯 What it does: This study proposes a Facial Forgery Text Generator (FFTG) that utilizes masks and handcrafted features to generate original text annotations. It then refines these annotations through structured prompts to enhance GPT-4o-mini, followed by fine-tuning CLIP and multimodal LLMs with these high-quality texts to improve the performance and interpretability of facial forgery detection.
Towards Human-Understandable Multi-Dimensional Concept Discovery
Arne Grobrügge (Karlsruhe Institute of Technology), Philipp Spitzer (Karlsruhe Institute of Technology)
CodeSegmentationExplainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: A new interpretable method HU-MCD is proposed, which automatically discovers human-understandable concepts and provides reliable concept importance scores.
🎯 What it does: A side network decoder based on the CLIP image encoder is proposed, which includes spatial and temporal modules, and detects deepfake videos through Face Component Guidance (FCG) loss.
Towards Natural Language-Based Document Image Retrieval: New Dataset and Benchmark
Hao Guo (Institute of Information Engineering), Hailun Lin (Institute of Information Engineering)
CodeRetrievalTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextBenchmark
🎯 What it does: This paper studies the task of document image retrieval based on natural language queries (NL-DIR), proposing a corresponding benchmark dataset and an efficient retrieval framework that combines two-stage recall and re-ranking.
Towards Open-Vocabulary Audio-Visual Event Localization
Jinxing Zhou (Mohamed bin Zayed University of Artificial Intelligence), Meng Wang (Hefei University of Technology)
CodeClassificationRecognitionObject DetectionTransformerSupervised Fine-TuningVision Language ModelVideoMultimodalityBenchmark
🎯 What it does: This paper proposes the Open Vocabulary Audio-Video Event Localization (OV-AVEL) task and provides two baseline schemes: training with frozen parameters and fine-tuning.
Towards Training-free Anomaly Detection with Vision and Language Foundation Models
Jinjin Zhang (Beihang University), Di Huang (Beihang University)
CodeAnomaly DetectionTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: A training-free multimodal framework called LogSAD is proposed for simultaneously detecting structural and logical anomalies, combining GPT-4V to generate interests and combination rules, and achieving anomaly detection at different granularities (patch, interest set, combination matching).
Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation
Gianni Franchi (ENSTA Paris), Andrea Pilzer (Imperial College London)
CodeGenerationLarge Language ModelVision Language ModelDiffusion modelImageText
🎯 What it does: This paper proposes a Prompt-based UNCertainty (PUNC) method based on large visual language models to quantify the uncertainty of text-to-image (T2I) generation models when given prompts, and constructs a prompt dataset aimed at uncertainty assessment.
Towards Understanding How Knowledge Evolves in Large Vision-Language Models
Sudong Wang (Institute of Information Engineering, Chinese Academy of Sciences), Xiangyang Ji (Tsinghua University)
CodeKnowledge DistillationRepresentation LearningTransformerVision Language ModelMultimodality
🎯 What it does: This study investigates how the internal knowledge of large-scale vision-language models (LVLM) evolves across layers, revealing critical layers and mutation layers, as well as three evolutionary stages.
🎯 What it does: This paper proposes VIB-Net, a method that utilizes variational information bottleneck to filter CLIP multimodal features for general AI-generated image detection.
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning
Jiange Yang (Nanjing University), Limin Wang (Nanjing University)
CodeRobotic IntelligenceTransformerReinforcement LearningMixture of ExpertsVideoBenchmark
🎯 What it does: This paper proposes a trajectory prediction model called Tra-MoE, which is jointly trained on multi-domain video data, and based on this, achieves visual alignment and reinforcement of robot policies through an adaptive 2D trajectory mask.
🎯 What it does: Proposes the TraF-Align framework, which utilizes trajectory field prediction and trajectory-aware attention to align asynchronous multi-agent point clouds at the feature level, achieving delay compensation, spatial alignment, and semantic consistency, ultimately completing multi-agent collaborative perception.
🎯 What it does: A modular conditional image synthesis framework is proposed, achieving fine control over three basic conditions: text, layout, and drag-and-drop, through three dense alignment modules: Dense Concept Alignment, Dense Geometry Alignment, and Dense Motion Alignment.
Trajectory Mamba: Efficient Attention-Mamba Forecasting Model Based on Selective SSM
Yizhou Huang (Brunel University), Kezhi Wang (Brunel University)
CodeAutonomous DrivingComputational EfficiencyRecurrent Neural NetworkReinforcement LearningMultimodalityTime Series
🎯 What it does: A lightweight multimodal trajectory prediction framework called Tamba based on a Selective State Space Model (SSM) has been designed and implemented.
🎯 What it does: A unified model named UHD-Processor is proposed, capable of performing various denoising tasks such as denoising, dehazing, deburring, low-light enhancement, and rain/snow removal for ultra-high-resolution images all at once.
🎯 What it does: A large-scale multimodal dataset MINE was constructed, and the BEAR framework was proposed to achieve joint understanding of sentiment and intent.
Uncertainty Meets Diversity: A Comprehensive Active Learning Framework for Indoor 3D Object Detection
Jiangyi Wang (Singapore University of Technology and Design), Na Zhao (Singapore University of Technology and Design)
CodeObject DetectionPoint Cloud
🎯 What it does: A proactive learning framework for indoor 3D object detection is proposed, integrating uncertainty and diversity as two major sampling criteria.
🎯 What it does: A noise-weighted and improved diffusion model called UPSR, based on uncertainty guidance, is proposed for single image super-resolution tasks.
🎯 What it does: Proposes the UIGenMap model, which utilizes uncertainty-guided structural injection to achieve more generalized high-precision HD map vectorization construction.
CodeGenerationData SynthesisPose EstimationDiffusion modelGaussian SplattingSimultaneous Localization and MappingVideoPoint Cloud
🎯 What it does: This paper proposes and constructs a large-scale real object 3D dataset called uCO3D, and utilizes this dataset to train various 3D learning models, enhancing the performance of multi-view reconstruction, image synthesis, and text-to-3D generation.
🎯 What it does: This paper proposes HyperCLIP, a fine-tuning strategy for scaling the CLIP text encoder in hyperbolic space to achieve open-domain semantic segmentation.
CodeSegmentationDomain AdaptationConvolutional Neural NetworkLarge Language ModelPrompt EngineeringVideo
🎯 What it does: This paper proposes a Multi-Task Temporal Action Segmentation (MT-TAS) framework aimed at addressing the challenges posed by action interleaving, task switching, and background interference in single-person multi-task environments.
UNIALIGN: Scaling Multimodal Alignment within One Unified Model
Bo Zhou (Nanjing University of Science and Technology), Wenguan Wang (Zhejiang University)
CodeClassificationComputational EfficiencyKnowledge DistillationTransformerMixture of ExpertsContrastive LearningImageVideoMultimodalityPoint CloudAudio
🎯 What it does: A unified multimodal alignment model UNIALIGN is proposed, capable of aligning various modalities (images, text, audio, video, 3D point clouds, depth maps) within a single encoder and completing all tasks in one training phase.
UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming
Hao Lin (Nanjing University), Wu-Jun Li (Nanjing University)
CodeOptimizationTransformerText
🎯 What it does: Proposes the UniAP method, which unifies and simultaneously optimizes cross-layer (pipeline) and intra-layer (tensor, data, FSDP) parallel strategies through mixed-integer quadratic programming (MIQP) to achieve optimal throughput for distributed training.
🎯 What it does: A unified event camera scene reconstruction framework URSEE is proposed, which is divided into two stages for static scenes: convolutional integration + denoising, and an end-to-end video reconstruction for dynamic scenes using voxel-grid + parallel channels.
Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modeling
Guillem Capellera (Institut de Robótica i Informática Industrial), Antonio Agudo (Institut de Robótica i Informática Industrial)
CodeTransformerDiffusion modelTime Series
🎯 What it does: A unified uncertainty-aware diffusion model U2Diff has been designed and implemented, capable of predicting, interpolating, and recovering missing trajectories of multiple agents, while providing state-level uncertainty estimates at each step.
🎯 What it does: This paper proposes UniHOPE, a unified method for 3D hand and handheld object pose estimation that can simultaneously handle scenes with only hands and hand-object interactions.
UniNet: A Contrastive Learning-guided Unified Framework with Feature Selection for Anomaly Detection
Shun Wei (Nanjing University of Information Science and Technology), Xiaolong Xu (Nanjing University)
CodeAnomaly DetectionContrastive LearningImageVideoBiomedical Data
🎯 What it does: This paper proposes a unified contrastive learning-driven multi-domain anomaly detection framework called UniNet, which combines a student-teacher model, a lightweight multi-scale embedding module, domain-relevant feature selection, and feature similarity comparison with margin loss, further achieving robust anomaly score calculation through a weighted decision mechanism.