π― What it does: A geometry-guided multimodal industrial defect detection framework G2SF is proposed, which integrates anomaly scores from 3D point clouds and RGB images.
π― What it does: Proposes the GameFactory framework, which utilizes pre-trained video diffusion models and a small number of game videos to achieve interactive game generation.
π― What it does: A Gaussian Prior-based World Model (GWM) is proposed, achieving an end-to-end unified framework for 4D occupancy prediction and future motion prediction from raw multimodal sensor inputs (cameras and LiDAR).
π― What it does: This paper presents GaussianReg, a fast 2D/3D registration framework for emergency surgery that can achieve high-precision registration in a matter of minutes.
π― What it does: This paper proposes a posture-free 3D surface reconstruction method based on Geometric Consistent Ray Diffusion (GCRayDiffusion), which can simultaneously achieve camera pose estimation and fine surface reconstruction under sparse viewpoint images.
π― What it does: A learning-based non-line-of-sight imaging method is proposed, which includes two main modules: learnable path compensation (LPC) and adaptive phase field (APF). This method can achieve high-quality 3D reconstruction under low signal-to-noise ratio (SNR) conditions and has good generalization ability for real data.
Generalizable Object Re-Identification via Visual In-Context Prompting
Zhizhong Huang (Michigan State University), Xiaoming Liu (Michigan State University)
CodeRecognitionRetrievalTransformerLarge Language ModelPrompt EngineeringImage
π― What it does: This paper proposes a Visual Context Prompting Framework (VICP) that utilizes large language models and visual foundation models to achieve object re-identification for unseen categories without the need for additional parameter fine-tuning.
π― What it does: A Gaussian Splatting-based arbitrary scale super-resolution model GSASR is proposed, which can convert low-resolution images into continuous Gaussian representations and generate high-resolution images at arbitrary magnification factors through differentiable GPU/CUDA rasterization.
π― What it does: This paper studies the task of Online Category Discovery (OCD) and proposes the DiffGRE framework, which generates virtual category samples by reorganizing attributes in latent space through a diffusion model, enhancing the generalization ability of the OCD model with these samples.
Generate, Transduct, Adapt: Iterative Transduction with VLMs
Oindrila Saha (University of Massachusetts), Subhransu Maji (University of Massachusetts)
CodeClassificationDomain AdaptationTransformerLarge Language ModelContrastive LearningImage
π― What it does: In the scenario of image classification with no labels or few labels, the GTA-CLIP framework is proposed, which utilizes large language models to dynamically generate category attributes, combining attribute-enhanced image-to-image transductive inference and adaptation of the CLIP encoder to form an iterative generation-transduction-adaptation process;
π― What it does: A unified generative framework called Generative Adversarial Diffusion (GAD) is proposed, which incorporates adversarial loss into the denoising process of latent diffusion models at each step, using a single U-Net to serve as both the generator and discriminator, enhancing training stability and image quality.
π― What it does: A general event boundary detection method based on the denoising diffusion model (DiffGEBD) is proposed, which can generate diverse and reasonable boundary predictions from the same video.
π― What it does: Proposes GenFlow3D, which jointly estimates the scene flow and future scene flow of point cloud sequences, utilizing recurrent networks and diffusion models for end-to-end learning.
GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices
Xudong Lu (vivo AI Lab), Hongsheng Li (Chinese University of Hong Kong)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextMultimodality
π― What it does: This paper proposes GenieBlue, a large language model (LLM) structure that balances pure language and multimodal capabilities on mobile devices. By freezing the original LLM parameters, copying Transformer blocks every four layers, and adding LoRA to the remaining blocks, it achieves multimodal training without compromising text capabilities, and employs a non-shared benchmark deployment strategy. It is deployed and evaluated on the Qualcomm Snapdragon 8 Elite NPU.
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
Muhammad Danish, Salman Khan (Australian National University)
CodeClassificationObject DetectionSegmentationTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmark
π― What it does: This paper presents GEOBench-VLM, a new benchmark specifically designed to evaluate visual-language models on geospatial tasks (comprising 31 fine-grained subtasks, 8 major categories, and over 10,000 manually verified instructions);
π― What it does: This paper proposes a self-distillation framework called GeoDistill based on FoV masking for weakly supervised cross-view localization.
π― What it does: This paper proposes an attention-based extended version of GFPack++, which utilizes attention-encoded geometric and relational networks to learn gradient fields for efficient packing of irregular 2D shapes.
GLEAM: Enhanced Transferable Adversarial Attacks for Vision-Language Pre-training Models via Global-Local Transformations
Yunqi Liu (Wuhan University), Xiaohui Cui (Wuhan University)
CodeRetrievalAdversarial AttackTransformerVision Language ModelImageTextMultimodality
π― What it does: A unified framework GLEAM is designed and implemented to generate highly transferable adversarial samples for visual language pre-trained models in a black-box setting.
π― What it does: A general framework called GREAT-Stereo is proposed, which integrates spatial, matching, and volumetric attention to improve iterative stereo matching.
Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data
Ke Fan (Shanghai Jiao Tong University), Jingbo Wang (Shanghai AI Laboratory)
CodeGenerationData SynthesisTransformerLarge Language ModelVideoTextBenchmark
π― What it does: A large-scale text-action pairing dataset called MotionMillion was constructed, and a scalable 7B parameter Transformer text-to-action generation model was trained based on this dataset. A zero-shot evaluation benchmark, MotionMillion-Eval, was introduced, demonstrating strong zero-shot generation capabilities.
Golden Noise for Diffusion Models: A Learning Framework
Zikai Zhou (Hong Kong University of Science and Technology Guangzhou), Zeke Xie (Hong Kong University of Science and Technology Guangzhou)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: A noise prompting network NPNet is proposed and trained to convert random Gaussian noise into 'golden noise', thereby enhancing the image quality and semantic consistency of text-to-image diffusion models.
π― What it does: This paper proposes an adversarial camouflage method GRAC for physical object detection models, which can successfully induce misjudgments or missed detections by the detector under multiple perspectives and different lighting conditions.
Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations
Dahee Kwon (KAIST), Jaesik Choi (KAIST)
CodeExplainability and InterpretabilityRepresentation LearningConvolutional Neural NetworkTransformerImage
π― What it does: This paper proposes the Granular Concept Circuit (GCC), which automatically identifies and constructs neuron circuits distributed across multiple layers to capture fine-grained concepts related to the query image in deep visual models.
GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions
Xiaomeng Chu (University of Science and Technology of China), Yanyong Zhang (University of Science and Technology of China)
CodeRobotic IntelligenceTransformerLarge Language ModelVision Language ModelMultimodalityBenchmarkChain-of-Thought
π― What it does: The GraspCoT framework is proposed, achieving 6-DoF grasp detection based on physical properties through chain-of-thought (CoT) reasoning and multimodal LLM integration.
Griffon v2: Advancing Multimodal Perception with High-Resolution Scaling and Visual-Language Co-Referring
Yufei Zhan (Chinese Academy of Sciences), Jinqiao Wang (Chinese Academy of Sciences)
CodeRecognitionObject DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: A high-resolution multimodal model Griffon v2 is proposed, supporting visual inputs of up to 1K resolution and achieving visual and language co-reference capabilities.
π― What it does: The GD-FAS framework is proposed, which jointly aligns the weights and biases in facial anti-spoofing to enhance cross-domain generalization capabilities.
Grouped Speculative Decoding for Autoregressive Image Generation
Junhyuk So (POSTECH), Eunhyeok Park (POSTECH)
CodeGenerationTransformerLarge Language ModelImage
π― What it does: Proposes the Grouped Speculative Decoding (GSD) method, which utilizes dynamically clustered visual token groups for untrained accelerated autoregressive image generation.
GS-LIVM: Real-Time Photo-Realistic LiDAR-Inertial-Visual Mapping with Gaussian Splatting
Yusen Xie (Hong Kong University of Science and Technology), Jun Ma (Hong Kong University of Science and Technology)
CodeAutonomous DrivingOptimizationRobotic IntelligenceGaussian SplattingSimultaneous Localization and MappingPoint Cloud
π― What it does: This paper presents GS-LIVM, a real-time lighting realistic LiDAR-inertial-visual fusion SLAM framework that utilizes efficient 3D Gaussian splatting for instant rendering and map construction in large-scale outdoor scenes.
π― What it does: This paper presents the GSOT3D benchmark dataset and the PROT3D general 3D single-object tracking method, aiming to advance the research of 3D single-object tracking in outdoor environments.
π― What it does: Using Stable Video Diffusion to generate multi-view latent variables, which are then transformed into renderable 3D representations through a Gaussian Splatting decoder, and incorporating geometric distillation (3D loss) during training to enhance multi-view consistency, achieving the generation of 3D objects from a single image.
GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices
Quanfeng Lu (Shanghai AI Laboratory), Ping Luo (University of Hong Kong)
CodeTransformerLarge Language ModelAgentic AIVision Language ModelMultimodality
π― What it does: This paper constructs a cross-application GUI navigation dataset called GUIOdyssey and designs OdysseyAgentβa multimodal navigation agent equipped with a historical resampling module based on this dataset.
π― What it does: A hybrid multi-view correspondence framework named H3R is proposed, which utilizes voxel implicit fusion and camera-aware Transformer to directly generate high-quality 3D Gaussian representations in a single forward pass, supporting variable numbers of views and high-resolution inputs.
Harnessing Input-Adaptive Inference for Efficient VLN
Dongwoo Kang (Oregon State University), Sanghyun Hong (Oregon State University)
CodeComputational EfficiencyTransformerMixture of ExpertsVision Language ModelMultimodality
π― What it does: A set of input-adaptive reasoning frameworks is proposed, which dynamically prunes unnecessary computations at runtime for the visual-language navigation (VLN) task, significantly reducing the computational power consumption of the visual encoder.
π― What it does: A 13M optical remote sensing image dataset, OpticalRS-13M, has been designed, and an efficient Masked Image Modeling pre-training method, SelectiveMAE, has been proposed for building remote sensing foundational models.
π― What it does: The PointSD framework is proposed, which replaces the Stable Diffusion text encoder with a 3D encoder. It first trains a diffusion model from point clouds to images, and then in the second stage aligns the intermediate features of SD with the features of the 3D backbone, achieving self-supervised pre-training of point clouds.
π― What it does: A single image dehazing framework called HazeFlow based on ODE is proposed, which reinterprets the atmospheric scattering model as a learnable dynamic equation.
π― What it does: This paper proposes a method for simultaneously predicting the 3D coordinates of the front and back surfaces of a target and performing dense sampling between the two surfaces, constructing an ultra-dense 2D-3D correspondence to improve the accuracy of pose estimation based on PnP.
HDR Image Generation via Gain Map Decomposed Diffusion
Yuanshen Guan (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: A framework for HDR image generation through diffusion models is proposed, which splits HDR images into SDR images and Gain Maps, and generates them jointly to obtain high dynamic range and wide color gamut images.
π― What it does: The paper proposes a lightweight label generation framework called HeLlO, which achieves online soft label generation during dataset distillation through an image-to-label projector, significantly reducing label storage costs.
Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models
Teng Ma (Sun Yat-Sen University), Wenqi Ren (Guangdong Key Laboratory of Information Security Technology)
CodeAdversarial AttackTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextMultimodality
π― What it does: A black-box multimodal jailbreak method called HIMRD is proposed, which first splits malicious prompts into text and image components and embeds them separately. It then constructs understanding-enhancing prompts and inducing prompts through heuristic search, inducing multimodal large language models to reconstruct malicious semantics and output violating content.
Hierarchical Cross-modal Prompt Learning for Vision-Language Models
Hao Zheng (South China Normal University), Zhenhua Huang (Shenzhen Polytechnic University)
CodeClassificationTransformerPrompt EngineeringVision Language ModelImageText
π― What it does: A hierarchical cross-modal prompt learning framework HiCroPL is proposed, which improves the interaction between text and visual prompts through bidirectional knowledge flow and hierarchical knowledge mapping, addressing the issues of modality isolation and hierarchical semantic decay in traditional methods.
Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Grounding
Minghang Zheng (Wangxuan Institute of Computer Technology, Peking University), Yang Liu (Peking University)
CodeRecognitionObject DetectionTransformerVideo
π― What it does: A hierarchical event memory framework is proposed, achieving accurate and low-latency predictions for online video temporal localization through event-level proposals and future prediction branches.
π― What it does: This paper proposes the Continuous Video Instance Segmentation (CVIS) task and designs a Hierarchical Visual Prompt Learning (HVPL) model.
π― What it does: This paper proposes the task of generating realistic clothing images from flat sketches and text prompts, termed FS2RG, and introduces the HiGarment framework based on this task.
π― What it does: Achieved text instruction-controlled infrared-visible image fusion through weakly supervised two-stage training, and realized instance-level target highlighting in the fusion results.
HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decoder
Yingqi Tang (Nullmax), Erkang Cheng (Nullmax)
CodeAutonomous DrivingTransformerPoint Cloud
π― What it does: A unified decoder end-to-end autonomous driving framework HiP-AD is proposed, integrating perception, prediction, and planning tasks, and achieving multi-granularity trajectory prediction through multi-scale planning queries and a deformable attention mechanism.
Hipandas: Hyperspectral Image Joint Denoising and Super-Resolution by Image Fusion with the Panchromatic Image
Shuang Xu (Northwestern Polytechnical University), Deyu Meng (Macau University of Science and Technology)
CodeRestorationSuper ResolutionImage
π― What it does: This paper proposes an end-to-end unsupervised framework called Hipandas, which jointly uses panchromatic images to denoise and recover super-resolution from low-resolution, noisy hyperspectral images.
HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding
Jiahe Zhao (Institute of Computing Technology, Chinese Academy of Sciences), Shiguang Shan (Institute of Computing Technology, Chinese Academy of Sciences)
CodeObject DetectionPose EstimationTransformerLarge Language ModelSupervised Fine-TuningMultimodalityPoint CloudBenchmark
π― What it does: Proposes the Human-In-Scene Question Answering (HIS-QA) task and the corresponding multimodal benchmark HIS-Bench, and designs the HIS-GPT model based on this task, which can simultaneously analyze 3D scenes and human movements;
HOLa: Zero-Shot HOI Detection with Low-Rank Decomposed VLM Feature Adaptation
Qinqian Lei (National University of Singapore), Robby T. Tan
CodeRecognitionObject DetectionTransformerLarge Language ModelVision Language ModelImageMultimodality
π― What it does: The HOLa method is proposed, which enhances zero-shot human-object interaction detection by adapting weights through low-rank decomposition of VLM text features, while combining LLM-generated action regularization and human-object tokens to improve unseen class generalization and action differentiation.
Holistic Tokenizer for Autoregressive Image Generation
Anlin Zheng (University of Hong Kong), Xiaojuan Qi (University of Hong Kong)
CodeImage TranslationRestorationGenerationTransformerLarge Language ModelImage
π― What it does: A global-local visual tokenizer named Hita is proposed, which is seamlessly integrated with autoregressive (AR) image generation models like Llama to achieve image generation, style transfer, and unsupervised restoration.
Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearning
Saemi Moon (POSTECH), Dongwoo Kim (POSTECH)
CodeGenerationData SynthesisLarge Language ModelVision Language ModelDiffusion modelImageTextBenchmark
π― What it does: Proposes the Holistic Unlearning Benchmark (HUB), a systematic evaluation of the concept unlearning effects of text-to-image diffusion models, covering six dimensions: authenticity, alignment, directionality, directionality accuracy, multilingual robustness, attack robustness, and efficiency.
How Do Multimodal Large Language Models Handle Complex Multimodal Reasoning? Placing Them in An Extensible Escape Game
Ziyue Wang (Tsinghua University), Yang Liu (Tsinghua University)
CodeTransformerLarge Language ModelReinforcement LearningVision-Language-Action ModelTextMultimodalityBenchmark
π― What it does: This paper proposes the MM-Escape benchmark and the EscapeCraft environment for evaluating the complete reasoning process of multimodal large language models (MLLMs) in escape room tasks.
Vedaant V Jain (University of Illinois Urbana-Champaign), Felipe dos Santos Alves Feitosa (University of SΓ£o Paulo)
CodeClassificationRecognitionData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageMultimodality
π― What it does: A multimodal visual humor dataset, HumorDB, containing pairs of control images was constructed and evaluated, utilizing human experiments and various visual/visual-language models for binary classification, ranking scoring, and comparative judgment tasks.
Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrieval
Bangxiang Lan (Renmin University of China), Xirong Li (Renmin University of China)
CodeRetrievalTransformerContrastive LearningVideo
π― What it does: This paper proposes the Hybrid-Tower framework and designs the Fine-grained Pseudo-query Interaction and Generation (PIG) method, which utilizes pseudo-queries to achieve fine-grained interaction before video retrieval, enhancing retrieval performance.
π― What it does: This paper proposes Hydra-NeXt, a multi-branch unified planning framework that integrates trajectory prediction, control prediction, and trajectory refinement into a single model to achieve closed-loop autonomous driving.
π― What it does: A Hyperbolic Diffusion Autoencoders (HypDAE) model is proposed, which combines hyperbolic space with diffusion models to generate high-quality, diverse, and category-consistent images from a very small number of samples.
CodeObject DetectionGraph Neural NetworkSimultaneous Localization and MappingPoint Cloud
π― What it does: A high-order geometric constraint method based on dynamic hypergraph neural networks, HyperGCT, is proposed for 3D point cloud registration.
π― What it does: This paper proposes HyPiDecoder, a hybrid pixel decoder that integrates a convolutional FPN structure with multi-scale linear attention to enhance the inference speed and accuracy of semantic, instance, panoptic segmentation, and object detection models.
π― What it does: The HyTIP framework is proposed, which mixes decoded frames with a small amount of implicit features in a buffer for mask-conditioned residual video coding, improving the rate-distortion performance.
I Am Big, You Are Little; I Am Right, You Are Wrong
David A. Kelly (Kings College London), Nathan Blake (Kings College London)
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkTransformerImage
π― What it does: This paper conducts a study on the minimum sufficient pixel set (MPS) of 15 different architectures of image classification models, comparing their size, location, and the differences in MPS size during misclassification on ImageNet.
π― What it does: Proposes the I2-World framework for 4D vehicle occupancy prediction, dividing the scene into intra-scene and inter-scene tokenizers, and using an encoder-decoder architecture to generate future 3D scenes.
π― What it does: A framework for invisible adversarial patch attacks based on perceptual adaptive positioning and perturbation optimization (IAP) is proposed.
ICE-Bench: A Unified and Comprehensive Benchmark for Image Creating and Editing
Yulin Pan (Alibaba Group), Yu Liu (Alibaba Group)
CodeGenerationTransformerVision Language ModelDiffusion modelImageBenchmark
π― What it does: A unified evaluation framework called ICE-Bench is proposed for a systematic assessment of 31 fine-grained tasks related to image generation and editing.
IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves
Ruofan Wang (Fudan University), Yu-Gang Jiang (Fudan University)
CodeGenerationAdversarial AttackTransformerVision Language ModelDiffusion modelImageTextMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper presents IDEATOR, a framework that utilizes Visual Language Models (VLM) and diffusion models to automatically generate image-text pairs for black-box jailbreak, and based on this framework, constructs the VLJailbreakBench evaluation benchmark.
Identity-aware Language Gaussian Splatting for Open-vocabulary 3D Semantic Segmentation
SungMin Jang (Konkuk University), Wonjun Kim (Konkuk University)
CodeSegmentationGaussian SplattingPoint Cloud
π― What it does: This paper proposes an Identity-Aware Language Gaussian Rendering (ILGS) method, which achieves high-precision predictions for open vocabulary 3D semantic segmentation by embedding language and identity embeddings into 3D Gaussian primitives, and introduces a progressive mask expansion strategy to refine boundaries.
IGD: Instructional Graphic Design with Multimodal Layer Generation
Yadong Qu (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
CodeGenerationData SynthesisLarge Language ModelDiffusion modelImageTextMultimodality
π― What it does: Based on natural language instructions, the Instructional Graphic Designer (IGD) has been designed and implemented to generate editable multimodal (text, images, graphics) layers in one go, outputting complete design files.
Image Intrinsic Scale Assessment: Bridging the Gap Between Quality and Resolution
Vlad Hosu (Sony AI), Dietmar Saupe (University of Konstanz)
CodeImage TranslationContrastive LearningImage
π― What it does: This paper introduces the concept of Image Intrinsic Scale (IIS) and studies the perceived quality of images at different scales as a new taskβImage Intrinsic Scale Assessment (IISA). It also constructs the first IISA dataset, IISA-DB, and proposes a weak label generation method, WIISA, to enhance the predictive performance of IQA models.
π― What it does: A template-based unsupervised 3D reconstruction method is proposed, which utilizes image color, gradient, and contour information combined with mesh non-stretch constraints to achieve object shape recovery.
IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance
Jiayi Guo (Georgia Tech), Humphrey Shi (Georgia Tech)
CodeGenerationData SynthesisLarge Language ModelDiffusion modelImageMultimodality
π― What it does: The Implicit Multimodal Guidance (IMG) framework is proposed to automatically identify and correct the misalignment between prompts and images during image generation with diffusion models, thereby enhancing the quality of multimodal alignment.
IMoRe: Implicit Program-Guided Reasoning for Human Motion Q&A
Chen Li (Institute of High Performance Computing), Basura Fernando (Nanyang Technological University)
CodePose EstimationTransformerVision Language ModelTextSequential
π― What it does: This paper proposes an implicit program-guided motion reasoning framework IMoRe, which utilizes structured program functions to uniformly handle multiple types of queries, dynamically selects multi-layer Vision Transformer motion features for reasoning, and ultimately achieves multi-step fine-grained question answering for motion sequences.
Improving Large Vision and Language Models by Learning from a Panel of Peers
Jefferson Hernandez (Rice University), Kushal Kafle (Adobe Research)
CodeGenerationOptimizationTransformerReinforcement LearningVision Language ModelImageTextMultimodalityBenchmark
π― What it does: This paper proposes the Panel-of-Peers (PoP) framework, which allows a group of visually language models with similar capabilities to generate answers for each other and evaluate one another, thereby achieving self-improvement without human annotation.
Incremental Few-Shot Semantic Segmentation via Multi-Level Switchable Visual Prompts
Maoxian Wan (Beihang University), Zhong Zhou (Beihang University)
CodeSegmentationTransformerPrompt EngineeringVision Language ModelImage
π― What it does: This paper proposes an incremental few-shot semantic segmentation framework based on Multi-layer Switchable Visual Prompts (MSVP), which utilizes visual language models and text semantics to separate foreground from background. It achieves learning of new categories while retaining memory of old categories through a three-layer knowledge base consisting of task-continuous prompts, stage-specific prompts, and region-unique prompts.
π― What it does: We propose InfGen, a generator that can decode fixed-size latent vectors at any resolution, enabling existing VAE-based diffusion models to achieve high-resolution generation while significantly reducing inference time.
π― What it does: A framework for identity-preserving image generation called InfiniteYou (InfU) based on Diffusion Transformer (FLUX) is proposed, capable of recreating specified character photos according to any text description while maintaining facial identity.
CodeTransformerLarge Language ModelVision Language ModelTextMultimodalityBenchmark
π― What it does: This paper proposes the principle of information density for evaluating multimodal large language models (MLLMs), defining four dimensions (error rate, difficulty, redundancy, diversity) to quantify the quality of benchmarks, and provides an executable evaluation pipeline.
INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance
Chenwei Lin (Fudan University), Jiebo Luo (University of Rochester)
CodeClassificationRecognitionData-Centric LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageMultimodalityBenchmark
π― What it does: INS-MMBench has been constructed, a hierarchical multimodal benchmark covering four types of insurance (auto insurance, property insurance, health insurance, and agricultural insurance), which includes 12,252 images, 10,372 question-and-answer pairs, covering 22 basic tasks, 12 meta-tasks, and 5 scenario tasks, and evaluates 11 large audiovisual language models.
π― What it does: A real-time text-guided image editing method called InstantEdit based on RectifiedFlow is proposed, capable of completing high-quality edits in just 8 sampling steps.
π― What it does: The INSTINCT framework is designed to achieve instance-level interaction and collaborative perception in a LiDAR-V2X environment, significantly reducing communication bandwidth while improving detection accuracy.
InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Models
Cong Wei (Tsinghua University), Yujiu Yang (Tsinghua University)
CodeSegmentationTransformerLarge Language ModelSupervised Fine-TuningImageVideoTextMultimodality
π― What it does: This paper proposes a unified multimodal large language model framework called InstructSeg, which can simultaneously perform four types of text-guided segmentation tasks (RES, ReasonSeg, R-VOS, ReasonVOS) in the image and video domains.
Integrating Biological Knowledge for Robust Microscopy Image Profiling on De Novo Cell Lines
Jiayuan Chen (Ohio State University), Ping Zhang (Ohio State University)
CodeClassificationSegmentationTransformerContrastive LearningImageBiomedical Data
π― What it does: In cellular microscopy image analysis, external biological knowledge is introduced to construct a perturbation relationship graph, combined with cell line transcriptome embeddings, to enhance perturbation prediction performance on unseen cell lines.
Integrating Visual Interpretation and Linguistic Reasoning for Geometric Problem Solving
Zixian Guo (Hong Kong Polytechnic University), Wangmeng Zuo (Harbin Institute of Technology)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
π― What it does: A decoupled visual language reasoning framework is proposed, where image parsing is handled by a vision-specific model and logical reasoning is managed by a language model, utilizing joint rewards for collaborative optimization.
CodeClassificationExplainability and InterpretabilityKnowledge DistillationConvolutional Neural NetworkTransformerLarge Language ModelImage
π― What it does: A black-box model intervention framework based on the concept bottleneck, CBM-HNMU, is proposed. It automatically extracts visual and natural language concepts, determines gradients, and prunes harmful concepts, then distills the improved knowledge back to the original model to enhance interpretability and classification accuracy.
π― What it does: This paper proposes a two-stage multimodal deepfake detection and temporal localization framework, which first captures the temporal consistency in videos through self-supervised audio-visual synchronization learning, and then utilizes pretrained features for deepfake detection and localization of local forged segments.
Is Less More? Exploring Token Condensation as Training-free Test-time Adaptation
Zixin Wang (University of Queensland), Yadan Luo (University of Queensland)
CodeCompressionDomain AdaptationTransformerVision Language ModelImage
π― What it does: A training-independent test-time adaptive method TCA is proposed, which enhances the zero-shot inference performance of VLMs like CLIP under distribution shift by dynamically compressing tokens in Vision Transformer and using domain-aware token pooling for logits self-correction.
Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency
Shiji Zhao (Institute of Artificial Intelligence, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University), Xingxing Wei (Institute of Artificial Intelligence, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University)
CodeSafty and PrivacyAdversarial AttackTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: This study investigates and utilizes multimodal large language models to understand the inconsistency between the comprehension and security mechanisms of shuffled (shuffle) malicious text and image instructions, proposing a query-driven black-box text-image SI-Attack method.
π― What it does: A continuous learning framework based on the Joint Diffusion Model (JDCL) is proposed, utilizing the same network to simultaneously perform generative replay and classification tasks, reducing model parameters and training time.
π― What it does: This paper proposes JointDiT, a model for RGB-Depth joint distribution modeling based on a diffusion Transformer, capable of performing joint generation, depth estimation, and depth-conditioned image generation at any noise level.
π― What it does: This paper proposes a learning-based codec-decoder JPNeO that is compatible with the existing JPEG standard, significantly improving image compression and reconstruction quality while maintaining compatibility with traditional JPEG.
Wujie Sun (Zhejiang University), Can Wang (Zhejiang University)
CodeKnowledge DistillationImage
π― What it does: A new knowledge distillation method called Refined Logic Distillation (RLD) is proposed to address the limitations of current logic distillation methods, particularly the impact of incorrect predictions from the teacher model on the learning of the student model.