π― What it does: A self-supervised differentiable 2D/3D X-ray and CT alignment framework, DiffPose, has been developed, achieving sub-millimeter registration without manual annotation.
π― What it does: This study investigates the 'natural attack capability' of diffusion models in generating images, proposing the NDD attack (Natural Denoising Diffusion Attack) and constructing the NDDA dataset for systematic evaluation.
π― What it does: A deep self-supervised clustering framework MVCAN is proposed to address the noise view defect (NVD) in multi-view clustering, aiming to achieve efficient and robust clustering in practical multi-view scenarios.
Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?
Zhiqi Li (Nanjing University), Jose M. Alvarez (NVIDIA)
CodeAutonomous DrivingPoint Cloud
π― What it does: A thorough analysis of open-loop end-to-end autonomous driving research is conducted, pointing out the dominance of ego state, dataset imbalance, and insufficient evaluation metrics, and proposing a new metric CCR as well as a lightweight baseline BEV-Planner that only uses BEV features and ego state.
π― What it does: A multi-modal 3D object detection framework named IS-FUSION is proposed, which enhances BEV features through instance-level and scene-level collaborative fusion.
π― What it does: The paper proposes a model called CONTHO that simultaneously reconstructs 3D human and object models from a single image, utilizing contact information between the human and the object for 3D refinement.
π― What it does: A joint sparsification-quantization framework called JointSQ is proposed, which treats sparsification as 0-bit quantization for gradient compression in distributed learning.
π― What it does: A Pose-Induced Video Transformer (Ο-ViT) is proposed, which injects 2D/3D keypoint information into the RGB representation of the video transformer by inserting two auxiliary modules, 2D-SIM and 3D-SIM, during the training phase, and does not require pose information during inference.
KITRO: Refining Human Mesh by 2D Clues and Kinematic-tree Rotation
Fengyuan Yang (National University of Singapore), Angela Yao (National University of Singapore)
CodePose EstimationMesh
π― What it does: Proposes the KITRO method, which refines 3D human meshes from 2D keypoints by solving skeletal orientations in a closed form and utilizing a motion chain-based decision tree;
π― What it does: A knowledge-enhanced dual-stream framework (KEDs) is proposed for zero-shot compositional image retrieval (ZSCIR) tasks without the need for triplet data.
KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation
Ruida Zhang (Tsinghua University), Xiangyang Ji (Tsinghua University)
CodeRetrievalPoint Cloud
π― What it does: A unified semantic keypoint-based 3D shape retrieval and deformation framework, KP-RED, is proposed, which can retrieve the most similar CAD models from global or partial scanned point clouds and guide fine-grained deformation through keypoints to achieve high-quality reconstruction results.
π― What it does: Two new types of point cloud convolution operations, KPConvD and KPConvX, have been introduced to improve the efficiency and performance of KPConv.
KTPFormer: Kinematics and Trajectory Prior Knowledge-Enhanced Transformer for 3D Human Pose Estimation
Jihua Peng (Hong Kong Polytechnic University), P. Y. Mok (Hong Kong Polytechnic University)
CodePose EstimationTransformerVideo
π― What it does: This paper proposes a Transformer network named KTPFormer, which enhances 3D pose estimation by incorporating two types of prior attention modulesβKinematics Prior Attention (KPA) and Trajectory Prior Attention (TPA)βutilizing prior information about human structure and motion trajectories.
π― What it does: This paper proposes a meta-learning-based 'self-guided' method (L2B) that dynamically adjusts the training loss by simultaneously learning the weights Ξ± and Ξ² of each sample's true labels and pseudo-labels, implicitly re-labeling and suppressing the impact of noisy labels.
CodeClassificationRecognitionVision Language ModelImageMultimodality
π― What it does: This paper proposes a new method for zero-shot classification using unlabeled dataβZLaP, which enhances the zero-shot recognition performance of VLM by performing label propagation on a bimodal graph composed of text class vectors and unlabeled image vectors.
π― What it does: A self-supervised learning framework LAFS based on facial key points is proposed, which pre-trains transferable facial representations using unlabeled facial images.
LAKE-RED: Camouflaged Images Generation by Latent Background Knowledge Retrieval-Augmented Diffusion
Pancheng Zhao (Nankai University), Jufeng Yang (Nankai University)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: This study proposes a latent background knowledge retrieval enhanced diffusion model (LAKE-RED) that automatically generates camouflage images without background input, capable of naturally integrating specified foreground objects into the generated background.
Language Model Guided Interpretable Video Action Reasoning
Ning Wang (Xidian University), Mohammed Bennamoun (University of Western Australia)
CodeRecognitionExplainability and InterpretabilityTransformerLarge Language ModelVideo
π― What it does: This paper proposes a video action reasoning framework called LaIAR, which utilizes language models to guide the process, achieving interpretable action recognition by aligning the logical reasoning knowledge of the language model with the video model.
Jang Hyun Cho (University of Texas Austin), Philipp KrΓ€henbΓΌhl (University of Texas Austin)
CodeObject DetectionTransformerVision Language ModelImageText
π― What it does: This paper proposes a language-conditioned detection Transformer (DECOLA) that can dynamically adjust the internal mechanisms of the detector based on the given text category, generating high-quality pseudo-labels using image-level labels, and then training an open-vocabulary detector with these pseudo-labels.
π― What it does: This paper proposes a language-driven anchor point adversarial training (LAAT) that achieves zero-shot adversarial robustness by utilizing text anchors generated by the CLIP text encoder.
Laplacian-guided Entropy Model in Neural Codec with Blur-dissipated Synthesis
Atefeh Khoshkhahtinat (West Virginia University), Nasser M. Nasrabadi (West Virginia University)
CodeCompressionTransformerDiffusion modelImage
π― What it does: A neural image compression decoder based on a conditional non-homogeneous variance diffusion model is proposed, incorporating an explicit frequency-induced bias;
Large Language Models are Good Prompt Learners for Low-Shot Image Classification
Zhaoheng Zheng (University of Southern California), Ram Nevatia (University of Southern California)
CodeClassificationKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringImage
π― What it does: This paper proposes the LLaMP framework, which utilizes the knowledge of large language models for low-shot image classification, generating category-specific prompts that are compatible with the CLIP text encoder.
π― What it does: A Latent Modulated Function (LMF) framework is proposed, which splits high-resolution (HR) high-dimensional decoding into shared decoding in low-resolution (LR) high-dimensional space and rendering in HR low-dimensional space, thus achieving efficient continuous image representation for arbitrary scale super-resolution (ASSR).
Learned Lossless Image Compression based on Bit Plane Slicing
Zhe Zhang (Wuhan University), Shan Liu (Tencent Media Lab)
CodeCompressionImage
π― What it does: The ArIB-BPS framework is proposed, achieving lossless image compression by slicing the bit plane and combining hierarchical latent variables with sub-image autoregression.
π― What it does: An end-to-end blind panoramic video quality assessment method is proposed, utilizing a differentiable scanning path generator and quality assessor trained jointly.
π― What it does: This paper proposes an Adaptive Spatial Consistency Correlation Learning (ASCCL) framework for Speech-Driven Facial Expression Manipulation (SPFEM), which learns the high correlation of local facial animations of the same spoken content under different emotions as additional supervision to guide the expression generation model.
π― What it does: This paper proposes a task of lifelong person ReID (RFL-ReID) that achieves this without re-indexing the original images, and introduces the Continual Compatible Representation (C2R) method.
Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis
Zicheng Zhang (University of Chinese Academy of Sciences), Ming Yang (Ant Group)
CodeGenerationData SynthesisVideoMesh
π― What it does: A hybrid representation called DynTet is proposed, which combines neural networks with tetrahedral meshes for high-quality, real-time speaker head synthesis.
π― What it does: For multi-instance point cloud registration, this paper proposes a coarse-to-fine instance-aware matcher MIRETR, which directly extracts instance-level correspondences from the scene point cloud and estimates transformations.
π― What it does: This paper proposes an ICGNet framework that introduces intra-view and inter-view geometric knowledge through interest point detectors and matchers to enhance stereo matching accuracy.
Learning Structure-from-Motion with Graph Attention Networks
Lucas Brynte (Chalmers University of Technology), Fredrik Kahl (Chalmers University of Technology)
CodePose EstimationOptimizationGraph Neural NetworkSimultaneous Localization and MappingOptical FlowImage
π― What it does: A method for learning structure from motion (SfM) without initialization based on graph attention networks is proposed, which can directly predict camera poses and 3D point coordinates from multi-view 2D keypoints.
π― What it does: The UnCounTR model is proposed, which learns reference-based counting tasks under completely unsupervised conditions using self-supervised generated Self-Collages.
Learning to Produce Semi-dense Correspondences for Visual Localization
Khang Truong Giang (KAIST), Sungho Jo (KAIST)
CodePose EstimationOptimizationTransformerSimultaneous Localization and MappingImage
π― What it does: A visual localization method based on semi-dense 2D-2D matching is proposed, which maps all detected 2D keypoints to 3D space through a Point Inference Network, generating a large number of 2D-3D correspondences.
π― What it does: A self-supervised image redundancy reduction framework LTRP has been developed, utilizing MAE to reconstruct differences and generate patch importance pseudo-labels, and selecting high-information image blocks through learned ranking.
Learning to Remove Wrinkled Transparent Film with Polarized Prior
Jiaqi Tang (Hong Kong University of Science and Technology), Ying-Cong Chen (Hong Kong University of Science and Technology)
CodeRestorationImage
π― What it does: This paper studies a new industrial image processing task - removing wrinkled transparent films to recover the image information obscured by the film.
π― What it does: This paper proposes a framework for selective view pipelining, which achieves efficient inference by dynamically selecting only 2-3 views in multi-view classification and detection.
π― What it does: This study investigates a multi-source audio localization method that utilizes visual-audio collaboration without requiring prior source number information.
π― What it does: A method called NegPrompt is proposed, which achieves unsupervised out-of-distribution (OOD) detection by learning negative prompts, utilizing the text-image alignment space of CLIP and relying solely on ID sample training.
π― What it does: The Triangular Distribution Transform (TDT) is proposed as a non-parametric plug-in that converts the nonlinear features extracted by CNN into features that linearly correspond to label differences, allowing regression tasks to be completed using only a linear head.
Learning Vision from Models Rivals Learning Vision from Data
Yonglong Tian (Google Research), Phillip Isola (Massachusetts Institute of Technology)
CodeSegmentationGenerationRepresentation LearningTransformerLarge Language ModelDiffusion modelContrastive LearningImageText
π― What it does: This paper proposes SynCLR, a method for visual representation learning that uses only synthetic text and images, without relying on any real data.
π― What it does: This paper proposes a method for fast sparse NeRF reconstruction using unreliable regions of pseudo-views from a limited number of perspectives.
LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising
Yuxing Duan (Huazhong University of Science and Technology)
CodeRestorationSpiking Neural NetworkImageVideo
π― What it does: A large-scale real event denoising dataset LED has been constructed, and a dual event denoising framework DED and a dynamic threshold LIF neuron-based SNN denoising model DTSNN have been proposed.
π― What it does: This paper proposes LeftRefill, a general framework for end-to-end reference-guided image inpainting (Ref-inpainting) and novel view synthesis (NVS) by horizontally stitching reference and target images, utilizing prompt tuning and self-attention.
π― What it does: The study utilizes LiDAR point clouds for person re-identification, proposing the ReID3D framework and constructing two datasets: LReID and LReID-sync.
π― What it does: This paper proposes the LiDAR4D framework, which utilizes neural fields to achieve space-time view synthesis of LiDAR point clouds in dynamic scenes.
LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
Gongwei Chen (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: LION enhances multimodal large language models through a dual-layer visual knowledge approach, thereby improving image understanding and reasoning capabilities.
π― What it does: This study proposes LiSA, which utilizes semantic information to enhance the Scene Coordinate Regression (SCR) method for LiDAR point cloud localization.
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning
Sijin Chen (Fudan University), Tao Chen (Fudan University)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityPoint Cloud
π― What it does: A 3D large language assistant LL3DA is proposed, which can simultaneously accept text instructions and visual interactions (such as clicks and box annotations) to understand, reason, and plan in complex 3D scenes.
LLM4SGG: Large Language Models for Weakly Supervised Scene Graph Generation
Kibum Kim (Korea Advanced Institute of Science and Technology), Chanyoung Park (Korea Advanced Institute of Science and Technology)
CodeObject DetectionGenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: This paper proposes a method for weakly supervised scene graph generation (LLM4SGG) using large language models (LLM), leveraging Chain-of-Thought and in-context few-shot prompting in the steps of triplet extraction and entity/predicate alignment, addressing the issues of semantic oversimplification caused by traditional rule parsers and low density due to dictionary matching.
π― What it does: A point cloud rotation-invariant learning framework called LocoTrans is proposed, which utilizes a Local Consistent Reference Frame (LCRF) and a Relative Pose Recovery (RPR) module to achieve rotation-invariant feature extraction.
π― What it does: This paper studies the data leakage problem in online mapping and proposes a geographically separated training/validation/testing split scheme (Near, Far), and re-evaluates mainstream methods.
Michail Tarasiou (Imperial College London), Stefanos Zafeiriou (Imperial College London)
CodeTransformerAuto EncoderMesh
π― What it does: A Local Adaptive Shape Model (LAMM) is proposed, which is an autoencoder framework that can directly manipulate 3D mesh with sparse control point displacements using a single forward pass;
LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model
Dongkai Wang (Peking University), Shiliang Zhang (Peking University)
CodePose EstimationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageText
π― What it does: A keypoint localization method based on large language models, LocLLM, is proposed, which utilizes a visual encoder to extract image features and inputs the image features along with text descriptions (including keypoint types, locations, and relationships) into a pre-trained LLM for inference, outputting keypoint coordinates.
π― What it does: A proactive speaker detection model called LoCoNet is proposed, which combines long-term intrinsic speaker context with short-term cross-speaker context to address the challenges of multiple speakers and small face scenarios.
Look-Up Table Compression for Efficient Image Restoration
Yinglong Li (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
CodeRestorationSuper ResolutionCompressionImage
π― What it does: This paper proposes a Look-Up Table (LUT)-based image recovery compression framework, utilizing diagonal re-indexing and non-diagonal subsampling (Diagonal-First Compression, DFC) to compress high-dimensional LUTs to smaller storage, and designs an SPF-LUT structure to enhance recovery performance.
π― What it does: A forward exploration method based on Hierarchical Neural Radiance Representation (HNR) is proposed, which predicts future environments in continuous visual-language navigation using multi-layer semantic features, thereby enhancing navigation planning.
π― What it does: By using multilingual audio tracks of dubbed movies to construct pairs of 'similar scenes with different voices', we improve audio-video self-supervised contrastive learning.
π― What it does: A backdoor attack framework called LOTUS is proposed, which first divides the samples of the victim class into multiple subsets, injects a unique trigger for each subset, and ultimately achieves a high success rate attack on the target class.
π― What it does: A residual-based low-rank reparameterization (RLRR) strategy is proposed for parameter-efficient fine-tuning of pre-trained Vision Transformers.
π― What it does: A 3D perception network LSK3DNet based on sparse large kernels is designed and implemented, utilizing Spatial Dynamic Sparsity (SDS) and Channel Weight Selection (CWS) to achieve efficient semantic segmentation and detection.
π― What it does: A multi-clothing virtual fitting and editing system M&M VTO is proposed, which allows trying on multiple garments on a single portrait and adjusting the clothing layout based on textual descriptions.
π― What it does: This paper proposes a new fetal heart structure detection framework called M3-UDA and constructs a cross-center FCS dataset to address the challenge of fetal ultrasound heart structure detection under unsupervised domain adaptation.
π― What it does: The MACE framework is proposed to achieve large-scale concept erasure in text-to-image diffusion models, capable of erasing up to 100 concepts at once.
MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer
Jianjian Cao (Fudan University), Tao Chen (Fudan University)
CodeRetrievalCompressionComputational EfficiencyTransformerVision Language ModelImageMultimodality
π― What it does: Designed and implemented the MADTP framework, which performs dynamic token pruning guided by multi-modal alignment for the visual-language Transformer, significantly reducing the model's computational load and GFLOPs.
MAFA: Managing False Negatives for Vision-Language Pre-training
Jaeseok Byun (Seoul National University), Taesup Moon (Seoul National University)
CodeRecognitionRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: This paper addresses the issue of false negative samples caused by the many-to-many correspondence between images and texts in visual-language pre-training. It proposes the MAFA method, which utilizes Efficient Connection Mining (ECM) to convert false negative samples into positive samples, and incorporates label smoothing (S-ITC) into the contrastive loss to mitigate the negative impact of false negatives on learning.
π― What it does: The EDITOR framework is proposed, which significantly enhances the feature representation and robustness of multi-modal object ReID through spatial-frequency token selection and hierarchical mask aggregation on Vision Transformer.
Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models
Gianni Franchi (Institut Polytechnique de Paris), Angela Yao (National University of Singapore)
CodeClassificationAnomaly DetectionImage
π― What it does: This paper proposes ABNN, which transforms a pre-trained DNN into a network capable of producing Bayesian posteriors by inserting Bayesian noise into the normalization layers and performing a small amount of fine-tuning, achieving posterior uncertainty estimation.
Making Visual Sense of Oracle Bones for You and Me
Runqi Qiao (Beijing University of Posts and Telecommunications), Honggang Zhang (Beijing University of Posts and Telecommunications)
CodeGenerationData SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageText
π― What it does: This paper proposes a visual guide system based on generative AI to help the public understand the relationship between oracle bone characters and their semantics, and designs a quantitative evaluation metric called TransOV.
π― What it does: This paper proposes a unified unsupervised domain adaptation framework MAPSeg, capable of voxel-level segmentation under different medical imaging domain shifts (cross-sequence, cross-site, cross-age, cross-modality).
π― What it does: A parallel decoding framework based on Masked AutoDecoder is designed, capable of uniformly handling various visual tasks such as object detection, instance segmentation, keypoint detection, and image captioning.
π― What it does: A dynamic soft masking method called MaxQ based on multi-axis queries is proposed to identify and enhance important weights in N:M sparse networks during the training phase.
Zequn Zeng (Xidian University), Zhengjue Wang (Xidian University)
CodeGenerationRetrievalTransformerVision Language ModelImageTextRetrieval-Augmented Generation
π― What it does: This paper proposes a memory-enhanced zero-shot image description framework called MeaCap, which retrieves and filters key concepts from external text memory, and then generates descriptions through a keyword-to-sentence language model, supporting both training-free and text-only training modes.
π― What it does: A feature transformation module (MSF-transformer) based on mean-shift updates is proposed to replace the existing Transformer module; PROBE projection and efficient grouped projection are introduced to further compress parameters.
π― What it does: This paper addresses the vulnerability of Test-Time Adaptation (TTA) methods when subjected to adversarial sample attacks, proposing a Median-based Batch Normalization (MedBN) that is seamlessly integrated into various existing TTA frameworks to enhance robustness against data poisoning attacks.
π― What it does: This paper proposes MemoNav, a working memory model for image target navigation that combines short-term memory (STM), long-term memory (LTM), and working memory (WM) in three scenarios.
Meta-Point Learning and Refining for Category-Agnostic Pose Estimation
Junjie Chen (Jiangxi University of Finance and Economics), Li Niu (Shanghai Jiao Tong University)
CodePose EstimationMeta LearningTransformerImage
π― What it does: A category-independent pose estimation framework based on meta-point learning and refinement is proposed, capable of predicting key points of any category with only a small number of labeled support images.
MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning
Yixin Liu (Lehigh University), Lichao Sun (Lehigh University)
CodeGenerationData SynthesisSafty and PrivacyAdversarial AttackMeta LearningDiffusion modelImage
π― What it does: Proposes the MetaCloak method, which prevents unauthorized personalized text-to-image diffusion model generation by adding robust adversarial perturbations to user photos;
MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction
Xiaolu Liu (Zhejiang University), Jianke Zhu (Zhejiang University)
CodeObject DetectionSegmentationAutonomous DrivingTransformerSimultaneous Localization and MappingImagePoint Cloud
π― What it does: This paper studies a mask-guided online high-definition map vectorization method called MGMap, aimed at accurately locating road features and achieving real-time generation.
π― What it does: The MiKASA Transformer is proposed for 3D visual localization tasks, achieving an end-to-end trained multimodal fusion model that can simultaneously process semantic and spatial information.
π― What it does: A new adversarial perturbation purification method based on diffusion models, called MimicDiffusion, is proposed to eliminate the impact of adversarial perturbations by mimicking the generation trajectory of undisturbed inputs.
π― What it does: A frequency distribution loss (FDL) is proposed for image transformation tasks, achieving better image restoration and style transfer without aligned training data.
π― What it does: This study investigates the cross-modal retrieval task in the presence of noisy correspondences and proposes the Geometrical Structure Consistency (GSC) framework, which identifies and corrects noisy correspondences by simultaneously maintaining the consistency of both cross-modal and intra-modal geometric structures.
π― What it does: This paper proposes a self-supervised learning framework based on object swapping, OESSL, to break the object dependency in indoor point cloud scenes and enhance feature robustness.
CodeObject DetectionGenerationTransformerVision Language ModelContrastive LearningImageMultimodality
π― What it does: A visual contrast decoding (VCD) method is proposed, which suppresses object hallucinations in large visual-language models by contrasting the outputs of the original image and the distorted image with added Gaussian noise.
MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning
Zhe Li (Huazhong University of Science and Technology), Stan Z. Li (Westlake University)
CodeRepresentation LearningAdversarial AttackContrastive LearningImageTextMultimodalityBiomedical Data
π― What it does: A multi-modal pre-training based medical visual representation framework MLIP is proposed, which enhances transferable visual features by utilizing cross-modal alignment of medical images and reports.
MMA: Multi-Modal Adapter for Vision-Language Models
Lingxiao Yang (Sun Yat-sen University), Xiaohua Xie (Sun Yat-sen University)
CodeClassificationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImageMultimodality
π― What it does: A multi-modal adapter (MMA) is proposed for efficient fine-tuning on few-shot generalization tasks using pre-trained vision-language models (such as CLIP).
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Xiang Yue (Ohio State University), Wenhu Chen (University of Waterloo)
CodeTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
π― What it does: A benchmark called MMMU has been constructed, covering 30 disciplines and 11.5K multimodal questions, to evaluate the multimodal understanding and reasoning capabilities of expert-level AI.
π― What it does: Designed and trained a series of efficient CLIP models for mobile devices, called MobileCLIP, and proposed a multimodal reinforcement training method to enhance the zero-shot classification and retrieval performance of small models.
ModaVerse: Efficiently Transforming Modalities with LLMs
Xinyu Wang, Qi Wu
CodeGenerationData SynthesisComputational EfficiencyTransformerLarge Language ModelAgentic AIVision Language ModelDiffusion modelImageVideoTextMultimodalityAudio
π― What it does: A multimodal large language model named ModaVerse is proposed, capable of understanding and generating multimodal content such as images, videos, and audio.
π― What it does: A MDP framework is designed to generate molecular pseudo-labels using various weakly supervised label functions based on graph kernels, fingerprints, and topological features, and these probabilistic pseudo-labels are used to train graph neural networks for molecular property classification.