🎯 What it does: This paper presents MM-GRAPH, a multimodal graph learning benchmark that includes text and visual features, covering seven real-world graph datasets of different scales for node/edge prediction and knowledge graph completion.
MotionMap: Representing Multimodality in Human Pose Forecasting
Reyhaneh Hosseininejad (Ecole Polytechnique Federale de Lausanne), Alexandre Alahi (Ecole Polytechnique Federale de Lausanne)
CodePose EstimationRecurrent Neural NetworkDiffusion modelAuto EncoderMultimodalityTime Series
🎯 What it does: This paper proposes MotionMap, a multimodal human pose prediction framework based on 2D heatmaps, which can automatically capture different future motion patterns, provide confidence assessments, and support controllable predictions and uncertainty analysis.
🎯 What it does: A single-stage multi-person full-body motion capture system is proposed, capable of predicting the motion trajectories of all individuals directly from a monocular video in one go.
MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders
Jiajun Cao (Peking University), Shanghang Zhang (Peking University)
CodeKnowledge DistillationRepresentation LearningTransformerMixture of ExpertsVision Language ModelMultimodality
🎯 What it does: In large-scale visual language models, knowledge distillation integrates the unique advantages of multiple visual encoders into a single encoder, achieving more efficient visual representation.
MP-GUI: Modality Perception with MLLMs for GUI Understanding
Ziwei Wang (Zhejiang University), Jiajun Bu (Ant Group)
CodeTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: A multimodal large language model specifically designed for graphical user interfaces (GUI) called MP-GUI has been developed. It extracts screen information through three types of perceivers (graphic, text, spatial) and dynamically fuses these features based on task semantics using a Fusion Gate, achieving comprehensive understanding and application of GUIs.
🎯 What it does: This paper proposes a Multi-Focal Conditional Latent Diffusion Model (MCLD) for pose-guided portrait generation, enhancing detail and identity preservation by separating high-frequency areas such as facial and clothing textures as independent conditions.
🎯 What it does: This paper proposes and evaluates the Multi-Group Proportional Representation (MPR) metric to quantify the representation fairness of text-to-image models across multiple intersecting groups, using it as a fine-tuning objective.
Multi-modal Contrastive Learning with Negative Sampling Calibration for Phenotypic Drug Discovery
Jiahua Rao (Sun Yat-sen University), Yuedong Yang (Sun Yat-sen University)
CodeDrug DiscoveryGraph Neural NetworkMixture of ExpertsContrastive LearningMultimodalityBiomedical Data
🎯 What it does: The MINER framework is proposed, which achieves contrastive learning for weakly paired and missing drug phenotype data through negative sampling calibration and multimodal fusion, generating more robust molecular phenotype representations.
🎯 What it does: A self-supervised pre-training framework for multimodal brain MRI, BrainMVP, is proposed, utilizing cross-modal reconstruction, modality inductive distillation, and modality-aware contrastive learning to learn cross-modal representations.
🎯 What it does: This paper proposes MCA-Ctrl, a parameter-free framework that achieves high-quality image customization under text or image conditions, capable of simultaneously performing three types of tasks: subject generation, subject replacement, and subject addition.
Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation
Shahad Albastaki (Khalifa University of Science and Technology), Arif Mahmood
CodeClassificationSegmentationRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodalityBiomedical Data
🎯 What it does: A multi-resolution pathology-language pre-training model MR-PLIP was developed, utilizing whole slide images (WSI) to perform aligned training between images and automatically generated text descriptions at four magnification levels: 5×, 10×, 20×, and 40×.
Multimodal Autoregressive Pre-training of Large Vision Encoders
Enrico Fini (Apple), Alaaeldin El-Nouby (Apple)
CodeRecognitionObject DetectionGenerationRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A multi-modal autoregressive pre-training framework AIMV2 is proposed, which uses a single visual encoder and a multi-modal decoder, capable of simultaneously reconstructing image patches and text tokens, achieving general pre-training for large visual encoders.
Ruopeng Gao (Nanjing University), Limin Wang (Nanjing University)
CodeObject TrackingTransformerVideo
🎯 What it does: This paper rephrases multi-object tracking (MOT) as a 'context ID prediction' task, using an end-to-end DETR framework to directly predict the ID of targets in the current frame, rather than traditional matching or ReID schemes.
Multirate Neural Image Compression with Adaptive Lattice Vector Quantization
Hao Xu (McMaster University), Xi Zhang (Nanyang Technological University)
CodeCompressionDomain AdaptationImage
🎯 What it does: An Adaptive Lattice Vector Quantization (Adaptive LVQ) method is proposed, which can achieve variable bit rates and domain adaptation within the same network.
MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking
Haolin Qin (Beijing Institute of Technology), Jianan Li (Beijing Institute of Technology)
CodeObject TrackingTransformerVideo
🎯 What it does: The first multispectral drone single-target tracking dataset, MUST, is proposed, and based on this dataset, a unified spectral-spatial-temporal tracking framework, UNTrack, is introduced.
MuTri: Multi-view Tri-alignment for OCT to OCTA 3D Image Translation
Zhuangzhuang Chen (Hong Kong University of Science and Technology), Xiaomeng Li (Hong Kong University of Science and Technology)
CodeImage TranslationData SynthesisConvolutional Neural NetworkAuto EncoderContrastive LearningImageBiomedical Data
🎯 What it does: Proposes the MuTri multi-view triangular alignment framework, which transforms the 3D image translation from OCT to OCTA into a VQ-VAE model in a discrete finite space, utilizing three perspectives of 3D OCT, 3D OCTA, and 2D OCTA projections for multi-view guidance;
MVBoost: Boost 3D Reconstruction with Multi-View Refinement
Xiangyu Liu (Institute of Automation, Chinese Academy of Sciences), Zhen Lei
CodeRestorationGenerationDiffusion modelImage
🎯 What it does: This paper proposes a framework called MVBoost that utilizes a multi-view diffusion model to generate pseudo ground truth and enhances single image to 3D reconstruction, ultimately training a feedforward 3D reconstruction network without the need for real 3D data.
NADER: Neural Architecture Design via Multi-Agent Collaboration
Zekang Yang (SenseTime Research and Tetras.AI), Wentao Liu (SenseTime Research and Tetras.AI)
CodeNeural Architecture SearchGraph Neural NetworkLarge Language ModelAgentic AIImage
🎯 What it does: A neural network architecture design framework called NADER is proposed, which is based on multi-agent collaboration of large language models (LLM) and can automatically generate high-performance models in open spaces.
Chuanbo Tang (University of Science and Technology of China), Dong Liu (University of Science and Technology of China)
CodeCompressionOptical FlowVideo
🎯 What it does: A deep context modulation neural video compression framework named DCMVC is proposed, which utilizes bidirectional information from reference frames and reference features to generate higher quality temporal context, thereby improving video compression rates.
🎯 What it does: This paper proposes NightAdapter, which achieves efficient generalization of semantic segmentation in nighttime scenes by adapting the Vision Foundation Model in the frequency domain.
🎯 What it does: A neural network predictor called NN-Former, which integrates GNN and Transformer, has been designed and implemented for one-time predictions of accuracy and inference latency of neural architectures.
🎯 What it does: Proposes dual-buffer sub-significant patch selection and dual-attention multi-instance learning to enhance the accuracy and efficiency of high-resolution image recognition.
🎯 What it does: By statistically calibrating the noise in real STEM images, more realistic synthetic data is constructed, and a Spatial-Frequency Interaction Network (SFIN) is proposed to achieve denoising and enhancement of STEM images.
🎯 What it does: This paper reveals the security risks of generating harmful content through font attacks in visual modality inputs of image generation models and proposes a corresponding dataset.
Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models
Davide Berasi (Fondazione Bruno Kessler), Nicola Strisciuglio
CodeClassificationRepresentation LearningVision Language ModelDiffusion modelImageMultimodality
🎯 What it does: This paper studies the composability of visual embeddings in Visual-Language Models (VLM) and proposes a Geodesically Decomposable Embeddings (GDE) framework based on Riemannian geometry to decompose visual concepts and suppress noise and sparsity.
🎯 What it does: This paper proposes and trains an end-to-end pixel-space diffusion model called VIVID, which generates target views from a single input image at arbitrary angles, completely replacing traditional depth estimation, photometric mapping, and completion pipelines.
🎯 What it does: A nighttime dynamic thermal map reconstruction method based on 4D Gaussian Splatting, NTR-Gaussian, is proposed, which can predict the temperature distribution of outdoor scenes at different time points.
Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection
Le Yang (Xi'an Jiaotong University), Chao Shen (Xi'an Jiaotong University)
CodeObject DetectionGenerationOptimizationTransformerLarge Language ModelVision Language ModelTextMultimodality
🎯 What it does: To address the issue of object hallucination in large visual-language models, the Nullu method is proposed. It obtains a low-rank subspace HalluSpace through PCA/SVD decomposition of the feature differences between ground truth and hallucination samples, and then projects the model's MLP weights onto its orthogonal complement space to suppress hallucination information during inference.
Number it: Temporal Grounding Videos like Flipping Manga
Yongliang Wu (Southeast University), Xu Yang (Southeast University)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelVideoText
🎯 What it does: A lightweight method for video large language models (Vid-LLMs) is proposed to achieve precise video temporal grounding (VTG) by overlaying numerical identifiers (Number-Prompt, NumPro) on each frame.
Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset
Xiao Wang (Anhui University), Yonghong Tian (Peking University)
CodeObject DetectionConvolutional Neural NetworkTransformerMixture of ExpertsImageBenchmark
🎯 What it does: This paper proposes an event camera object detection framework based on Mixture-of-Experts thermal conduction (MvHeat-DET) and releases the EvDET200K dataset for the first time, which contains 10,054 entries and 200,000 bounding boxes.
Object-aware Sound Source Localization via Audio-Visual Scene Understanding
Sung Jin Um (Kyung Hee University), Jung Uk Kim (Kyung Hee University)
CodeRecognitionObject DetectionConvolutional Neural NetworkLarge Language ModelContrastive LearningMultimodalityAudio
🎯 What it does: This paper proposes a sound and visual source localization framework based on a multimodal large language model (MLLM), utilizing foreground/background text generated by MLLM for fine localization.
🎯 What it does: This paper proposes a network called OSGNet, which is used to accurately locate video segments in first-person (egocentric) videos based on natural language queries.
🎯 What it does: This paper proposes the first outdoor semantic occupancy prediction network, OccMamba, based on the Mamba state space model, which can directly and efficiently handle millions of dense voxels.
Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding
Wei Suo (Northwestern Polytechnical University), Yanning Zhang (Northwestern Polytechnical University)
CodeGenerationOptimizationTransformerReinforcement LearningVision Language ModelMultimodality
🎯 What it does: To address the hallucination problem in large vision-language models, this paper proposes a dynamic contrastive decoding framework called Octopus, which can identify the type of hallucination at each generation step and select different contrastive decoding strategies as needed.
🎯 What it does: A scene-specific 'Odd-One-Out' anomaly detection task is proposed, which constructs a 3D voxel representation through multi-view images, compares across instances to identify occasionally appearing anomalous objects, and releases two new benchmark datasets.
ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
Yahan Tu (Beijing Jiaotong University), Jitao Sang (Beijing Jiaotong University)
CodeGenerationData SynthesisGraph Neural NetworkTransformerLarge Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Proposed and implemented the ODE (Open-Set Dynamic Evaluation) protocol for dynamically generating open-set samples to evaluate the hallucinations of multimodal large language models (in terms of existence and attributes).
OffsetOPT: Explicit Surface Reconstruction without Normals
Huan Lei (AIML, University of Adelaide)
CodeOptimizationTransformerPoint CloudMesh
🎯 What it does: This paper proposes an explicit surface reconstruction method called OffsetOPT, which can directly reconstruct meshes from 3D point clouds without the need for point normals.
🎯 What it does: This paper proposes Omni-Scene, which utilizes a combination of pixel-level and voxel-level Omni-Gaussian representations to achieve high-quality 3D reconstruction and view synthesis in ego-centric scenes with limited view overlap for autonomous driving.
Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos
Chiara Plizzari, Federico Tombari
CodeTransformerLarge Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: This paper proposes and evaluates a temporal reasoning dataset specifically for first-person video question answering, called EgoTempo, and conducts systematic experiments on the performance of multimodal large language models (MLLM) on this task.
Kai Luo (Hunan University), Kailun Yang (Hunan University)
CodeObject TrackingImageVideo
🎯 What it does: A multi-object tracking framework called OmniTrack is proposed, specifically designed for panoramic images, addressing challenges such as geometric distortion, resolution loss, and uneven lighting in panoramic views.
OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning
Shihao Wang (NVIDIA), Jose M. Alvarez (NVIDIA)
CodeAutonomous DrivingTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Proposes the OmniDrive dataset and two LLM driving agent frameworks, Omni-L and Omni-Q, which achieve high-quality 3D visual-language question answering and planning data through adversarial reasoning.
🎯 What it does: OmniFlow is constructed as a unified multimodal generative model capable of performing generative tasks from any input to any output, such as text to image, text to audio, audio to image, etc.
🎯 What it does: We propose OmniGen, a unified image generation model based on VAE and Transformer, capable of performing various tasks such as text generation, image editing, and visual condition-based generation within a single framework.
🎯 What it does: Using the Yin-Yang grid to decompose the panoramic image into two quasi-perspective views, directly estimating the 3D Gaussian parameters on a pre-trained feedforward 3D Gaussian splatting network, and achieving rapid panoramic image reconstruction and new viewpoint synthesis through cross-view attention and Yin-Yang rasterization.
🎯 What it does: We propose OmniStereo, a panoramic depth estimation framework based on Cassini projection and a lightweight stereo matching network, capable of generating high-quality panoramic depth maps in real-time.
Dongyang Jin (Southern University of Science and Technology), Shiqi Yu (Southern University of Science and Technology)
CodeRecognitionDiffusion modelVideo
🎯 What it does: The DenoisingGait method is proposed, which uses knowledge-driven and geometry-driven feature matching through a diffusion model to denoise walking videos, enhancing pedestrian gait recognition accuracy.
On the Out-Of-Distribution Generalization of Large Multimodal Models
Xingxuan Zhang (Tsinghua University), Peng Cui (Tsinghua University)
CodeDomain AdaptationDrug DiscoveryTransformerLarge Language ModelVision Language ModelImageMultimodalityBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: The system evaluates and analyzes the zero-shot generalization ability of large multimodal models (LMM) in fields such as synthetic images, natural distribution drift, and medical and molecular imaging, finding that their performance significantly declines outside the training domain.
Once-Tuning-Multiple-Variants: Tuning Once and Expanded as Multiple Vision-Language Model Variants
Chong Yu (Fudan University), Zhongxue Gan (Fudan University)
CodeTransformerVision Language ModelMultimodality
🎯 What it does: A Once-Tuning-Multiple-Variants (OTMV) framework is proposed, which can dynamically expand to generate multiple variants of visual-language models after a single tuning.
One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception
Yuchen Xia (Beijing University of Posts and Telecommunications), Jinglin Li (Beijing University of Posts and Telecommunications)
CodeAutonomous DrivingExplainability and InterpretabilityPrompt EngineeringPoint Cloud
🎯 What it does: This paper proposes PolyInter, a polymorphic feature interpreter designed to align intermediate features from different perception networks to the semantic space of the target vehicle in immutable heterogeneous cooperative perception scenarios.
One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion
Chunyang Cheng (Jiangnan University), Josef Kittler (University of Surrey)
CodeTransformerAuto EncoderImage
🎯 What it does: This paper proposes GIFNet, a model that achieves unified multi-task image fusion and single-modal enhancement through the interaction of low-level tasks.
One-shot 3D Object Canonicalization based on Geometric and Semantic Consistency
Li Jin (Shandong University), Baoquan Chen (Peking University)
CodeObject DetectionPose EstimationOptimizationTransformerLarge Language ModelVision Language ModelPoint CloudMesh
🎯 What it does: This paper proposes a one-shot 3D object normalization framework that requires only a single prior model for each category to align the target object to a canonical coordinate system under any pose.
🎯 What it does: This paper studies a time-invariant unified encoder (TiUE) that implements a non-cyclic text-to-image diffusion model distillation, significantly improving inference speed while maintaining high-quality diversity.
Online Task-Free Continual Learning via Dynamic Expansionable Memory Distribution
Fei Ye (University of Electronic Science and Technology of China), Adrian G. Bors (University of York)
CodeData SynthesisAuto EncoderImage
🎯 What it does: A dynamic expandable memory distribution (DEMD) framework is proposed to address the problem of catastrophic forgetting in task-free continual learning without task or category information, combining dynamic memory systems with long-term memory systems for online learning.
🎯 What it does: The research proposes a testing-time OOD detection method named OODD, which achieves detection by dynamically maintaining an OOD dictionary during the inference phase, without the need for model fine-tuning or additional training.
Open Ad-hoc Categorization with Contextualized Feature Learning
Zilin Wang (University of Michigan), Liu Ren (Bosch Center for AI)
CodeClassificationExplainability and InterpretabilityRepresentation LearningTransformerVision Language ModelContrastive LearningImage
🎯 What it does: A framework for open ad-hoc categorization (OAK) is proposed, which can automatically discover and predict known and novel categories with only a small number of labeled samples and unlabeled images.
🎯 What it does: A publicly available VHR forest canopy height dataset covering 87,000 square kilometers in France was constructed, and the Open-Canopy and Open-Canopy-∆ benchmarks were proposed.
🎯 What it does: This study proposes a training-free open-world blind spot completion framework that utilizes text queries to achieve complete reconstruction of any occluded object, outputting RGBA layers that can be directly used for image editing and 3D reconstruction.
OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation
Pengfei Zhou (Shanghai AI Laboratory), Kaipeng Zhang (Shanghai AI Laboratory)
CodeGenerationTransformerLarge Language ModelImageTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes the OpenING intertwined image-text generation benchmark (5,400 examples, 56 tasks) and designs the IntJudge evaluation model to support large-scale assessment of intertwined generation.
🎯 What it does: This paper proposes a single-step diffusion model OSDFace, which extracts prior information from low-quality faces using a Visual Representation Embedder (VRE) and achieves high-quality restoration with good facial identity preservation by combining identity loss and GAN guidance.
OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP
Mohamad Hassan N C (Indian Institute of Technology Bombay), Biplab Banerjee (Indian Institute of Technology Bombay)
CodeGenerationDomain AdaptationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImage
🎯 What it does: A low-sample open-domain generalization task, LSOSDG, is proposed, and the OSLOPROMPT framework is designed to achieve domain-independent generalization and open-set detection with very few samples.
OSMamba: Omnidirectional Spectral Mamba with Dual-Domain Prior Generator for Exposure Correction
Gehui Li (Peking University), Jian Zhang (Peking University)
CodeRestorationDiffusion modelImage
🎯 What it does: This paper proposes a global frequency domain exposure correction network, OSMamba, based on a state space model, aimed at addressing the issues of illumination correction and structure recovery under extreme exposure.
CodeLarge Language ModelVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: This paper presents OVO-Bench, a benchmark specifically designed to evaluate the online understanding capabilities of video large language models, covering three modes: backward reasoning, real-time visual perception, and forward proactive response.
PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models
Mohamed Dhouib (Polytechnic Institute of Paris), Aymen Shabou (Credit Agricole)
CodeCompressionComputational EfficiencyTransformerVision Language ModelImageVideoMultimodality
🎯 What it does: To address the computational and memory consumption issues caused by a large number of visual tokens during inference in visual language models, this paper proposes pruning visual tokens and merging clusters in the early layers of the model, significantly reducing inference costs.
PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Models
Jenny Schmalfuss (University of Stuttgart), Jose M. Alvarez (NVIDIA)
CodeRecognitionData-Centric LearningTransformerPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: This study investigates the prompt sensitivity of visual language models (VLM) and proposes the PARC framework to systematically evaluate the model's performance under different prompt variations.
🎯 What it does: Low-bit post-training quantization is performed on a first-order diffusion super-resolution model, proposing PassionSR, which achieves 8/6 bit quantization while maintaining visual quality close to full precision.
🎯 What it does: Proposes PatchDEMUX, a multi-label adversarial patch defense framework that can be scaled to any single-label Certifiable Defense against Patch Attacks (CDPA);
Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Model
Ziyuan Yang (Sichuan University), Yi Zhang (Sichuan University)
CodeRestorationFederated LearningTransformerLarge Language ModelPrompt EngineeringImageBiomedical DataComputed Tomography
🎯 What it does: A dual-layer physically driven federated learning framework that combines scanning protocols and anatomical hints for low-dose CT image denoising is proposed.
CodeLarge Language ModelPrompt EngineeringImageTextMultimodalityBenchmark
🎯 What it does: Proposed the GeoMap-Bench benchmark and the GeoMap-Agent framework to achieve multimodal understanding and question answering of geological images.
🎯 What it does: This paper proposes a single-source domain generalization method called PEER (Parameter-Space Ensemble with Entropy Regularization), which alleviates mid-term OOD performance fluctuations caused by data augmentation and improves generalization to unknown target domains by averaging the parameters of the proxy model and the main model and maximizing the feature information of both through entropy regularization.
🎯 What it does: This paper proposes a 'World Feature Simulation' (WFS) framework that utilizes multi-layer perception, fragmented memory, and imagination mechanisms to synthesize virtual unknown features under the condition of no auxiliary data, thereby enhancing the robustness of object detectors against unknown objects in open domains.
PhD: A ChatGPT-Prompted Visual Hallucination Evaluation Dataset
Jiazhen Liu (Renmin University of China), Xirong Li (Renmin University of China)
CodeGenerationData SynthesisRetrievalTransformerLarge Language ModelPrompt EngineeringImageTextMultimodalityBenchmark
🎯 What it does: A large-scale multimodal visual illusion assessment benchmark PhD has been developed, which includes four assessment modes (base, sec, icc, ccs) and five visual tasks, constructed through a ChatGPT-assisted pipeline.
Phoenix: A Motion-based Self-Reflection Framework for Fine-grained Robotic Action Correction
Wenke Xia (Renmin University of China), Di Hu (Renmin University of China)
CodeRobotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningMultimodality
🎯 What it does: A self-reflection framework based on motion commands, named Phoenix, is proposed, which utilizes a multimodal large language model to transform semantic reflection into fine-grained action corrections and achieve lifelong learning.
CodePose EstimationRobotic IntelligenceTime SeriesPhysics Related
🎯 What it does: A pedestrian trajectory prediction framework based on physical feasibility assessment (Locomotion Embodiment) is proposed, which generates gait through physical simulation and evaluates trajectory feasibility using the LocoVal function, combined with EmLoco loss to train the prediction network.
🎯 What it does: A framework called PICO has been developed for recovering human-object interaction (HOI) 3D reconstruction from a single natural image, which includes the dataset PICO-db and the method PICO-fit.
🎯 What it does: A cross-view pose optimization network called PIDLoc, based on the idea of PID controllers, is proposed for precise vehicle positioning through satellite view images in GNSS-constrained environments.
PIDSR: Complementary Polarized Image Demosaicing and Super-Resolution
Shuangfan Zhou (Beijing University of Posts and Telecommunications), Imari Sato (National Institute of Informatics)
CodeRestorationSuper ResolutionImage
🎯 What it does: By using a jointly designed two-stage recursive network, we achieve demosaicing and super-resolution of polarization images directly from CPFA raw images, resulting in high-quality, high-resolution polarization images along with their Degree of Polarization (DoP) and Angle of Polarization (AoP).
🎯 What it does: This paper proposes a pixel-level and semantic-level adjustable super-resolution method called PiSA-SR, which utilizes the LoRA module to achieve decoupled and controllable inference for detail recovery and semantic enhancement on the pre-trained Stable Diffusion model.
Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy
Joonhyun Jeong (NAVER Cloud), Eunho Yang (Korea Advanced Institute of Science and Technology)
CodeAdversarial AttackTransformerLarge Language ModelReinforcement LearningImageTextMultimodality
🎯 What it does: This paper proposes an attack framework called JOOD that breaks through the security alignment of large language models and multimodal models by transforming harmful inputs through out-of-distribution (OOD) modification.
🎯 What it does: This paper proposes a pluggable point prompt optimizer (PPO) that utilizes deep reinforcement learning to automatically adjust the point prompts of SAM on a dual-space heterogeneous graph, thereby enhancing unsupervised/one-shot segmentation performance.
🎯 What it does: A parameter-efficient fine-tuning framework called Point Mamba Adapter (PMA) is proposed, which integrates all intermediate layer features on pre-trained point cloud models to enhance downstream task performance.
🎯 What it does: This paper proposes a posture-free 3D reconstruction method based on multi-view surface normals, PMNI, which can simultaneously recover high-quality surfaces of reflective and texture-less invalid surfaces along with camera poses without prior knowledge of camera poses.
🎯 What it does: This paper proposes a point cloud upsampling network PDANS based on a conditional diffusion model, which can suppress noise while preserving geometric details.
🎯 What it does: A training-free, hierarchical caching model called Point-Cache is proposed, constructed solely using point cloud data at the testing stage, to achieve adaptive and generalized point cloud recognition under distribution drift.
🎯 What it does: A parameter-efficient fine-tuning method for point cloud pre-training models, PointLoRA, is proposed, achieving fine-tuning with only 3.43% of trainable parameters by combining low-rank adaptation and multi-scale token selection.
Pose-Guided Temporal Enhancement for Robust Low-Resolution Hand Reconstruction
Kaixin Fan (Beijing University of Posts and Telecommunications), Jianxin Liao (Beijing University of Posts and Telecommunications)
CodePose EstimationTransformerVideo
🎯 What it does: This paper proposes a temporal enhancement framework for low-resolution hand reconstruction: it first captures temporal information using joint features from historical frames, then projects these sparse 3D joint information into dense 2D features through three-plane feature encoding, thereby enhancing the visual features of the current frame, and finally regresses the hand mesh from the enhanced features.
Positive2Negative: Breaking the Information-Lossy Barrier in Self-Supervised Single Image Denoising
Tong Li (Beijing Institute of Technology), Hua Huang (Beijing Normal University)
CodeRestorationImage
🎯 What it does: This paper studies a new self-supervised single-image denoising paradigm called Positive2Negative, which constructs multi-scale positive and negative noise images by predicting noise re-noising and trains the network with denoising consistency supervision, successfully breaking through the barrier of information loss.
🎯 What it does: This paper proposes CLIP-Refine, a lightweight post-pretraining method between CLIP pretraining and fine-tuning, aimed at aligning image and text features and reducing the gap between modalities.
POT: Prototypical Optimal Transport for Weakly Supervised Semantic Segmentation
Jian Wang (XJTLU), Jimin Xiao (XJTLU)
CodeSegmentationConvolutional Neural NetworkImage
🎯 What it does: A weakly supervised semantic segmentation framework based on multi-cluster prototypes and similarity-aware optimal transport (POT) is proposed, which can significantly enhance the completeness of class activation maps.
PQPP: A Joint Benchmark for Text-to-Image Prompt and Query Performance Prediction
Eduard Poesina (University of Bucharest), Radu Tudor Ionescu (University of Bucharest)
CodeGenerationRetrievalTransformerVision Language ModelDiffusion modelImageTextBenchmark
🎯 What it does: This paper proposes a joint benchmark (PQPP) to evaluate the difficulty of prompts/queries in text-to-image generation and retrieval tasks, and obtains real performance scores through human annotation.