π― What it does: A multi-granularity contrastive consistency learning framework MGCC is proposed for occluded text retrieval of portraits, and an occluded version of the T-ReID dataset is constructed using the occlusion generator OGor.
Text-Guided Molecule Generation with Diffusion Language Model
Haisong Gong (Chinese Academy of Sciences), Liang Wang (Chinese Academy of Sciences)
CodeGenerationDrug DiscoveryTransformerDiffusion modelTextBiomedical Data
π― What it does: A text-guided molecular generation method based on diffusion language models (TGM-DLM) is proposed, which generates SMILES molecules that meet text descriptions from random noise through a two-stage diffusion process.
Jiayi Liao (University of Science and Technology of China), Dongmei Zhang (Microsoft)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageText
π― What it does: A framework called TIAC based on a three-layer theory of artistic creation is proposed, specifically addressing the problem of generating abstract concepts from text to image (T2I);
Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries
Xinyi He (Xi'an Jiaotong University), Dongmei Zhang (Microsoft)
CodeLarge Language ModelTabularBenchmark
π― What it does: Developed the Text2Analysis benchmark, collecting 2249 table-query-code-result quadruples, covering advanced analytical tasks (such as prediction, chart generation, basic insights, etc.) as well as ambiguous user queries.
Yiming Qin (Shanghai Jiao Tong University), Rynson W.H. Lau (City University of Hong Kong)
CodeGenerationData SynthesisDiffusion modelText
π― What it does: A method for urban layout regeneration based on text descriptions and surrounding context, called Text2City, is proposed, which can jointly generate the road and building layouts of the target area in a single stage.
π― What it does: This paper addresses the Aspect-based Sentiment Analysis (ABSA) task and proposes a dual-view graph Transformer (TextGT), which alternately stacks graph convolutional layers and Transformer layers, aiming to capture both syntactic structure and global semantics while addressing the over-smoothing problem commonly encountered in traditional GNNs and Transformers.
π― What it does: This paper proposes TF-CLIP, a one-time trained, text-free CLIP-based video person re-identification framework that utilizes identity-specific sequence features instead of text features, and enhances temporal information modeling through Sequence-Specific Prompt (SSP) and Temporal Memory Diffusion (TMD) modules.
The Causal Impact of Credit Lines on Spending Distributions
Yijun Li (City University of Hong Kong), Zhixiang Huang (JD Digits)
CodeRecommendation SystemOptimizationExplainability and InterpretabilityComputational EfficiencyData-Centric LearningTabularFinance Related
π― What it does: This study investigates the causal impact of credit limits on the distribution of consumer spending and proposes a distributed causal estimation framework that views individual spending as a distribution.
Kiran Tomlinson (Cornell University), Jon Kleinberg (Cornell University)
Code
π― What it does: This paper explores the moderating effect of Instant Runoff Voting (IRV) on the position of the median candidate under a one-dimensional Euclidean voter preference model, and proves that it has an exclusion zone that rejects extreme candidates.
Theoretical and Empirical Analysis of Cost-Function Merging for Implicit Hitting Set WCSP Solving
Javier Larrosa (Universitat Politècnica de Catalunya), Emma Rollon (Universitat Politècnica de Catalunya)
CodeOptimization
π― What it does: This paper studies the application of the Implicit Hitting Set (HS) method in Weighted Constraint Satisfaction Problems (WCSP), theoretically analyzes the exponential iteration problem caused by core interchangeability, and proposes two reconstruction strategies, symbolic merging and numerical merging, to compress the core space.
Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning
Chengcheng Ma (Institute of Automation, Chinese Academy of Sciences), Changsheng Xu (Institute of Automation, Chinese Academy of Sciences)
CodeClassificationMixture of ExpertsImage
π― What it does: A multi-expert model CPE is proposed, which jointly trains three experts with different logit adjustment strengths, combined with class-level batch normalization, to generate high-quality pseudo-labels and improve model performance in long-tail semi-supervised learning.
π― What it does: This paper proposes a text-aware image mixing method called TiMix, which applies data augmentation techniques such as CutMix to self-supervised multimodal contrastive learning (SMCL), thereby enhancing the data efficiency of visual-language pre-training (VLP).
π― What it does: A high-order multi-objective polynomial regression model TMPNN based on Taylor mapping decomposition is proposed, which achieves high-order polynomial fitting by iteratively sharing low-order polynomial coefficients, avoiding the curse of dimensionality.
π― What it does: This paper proposes a framework based on token-level contrastive learning and modality-aware prompts (TCL-MAP) to enhance multimodal intent recognition performance.
TOP-ReID: Multi-Spectral Object Re-identification with Token Permutation
Yuhao Wang (Dalian University of Technology), Huchuan Lu (Jiangsu University)
CodeRecognitionRetrievalTransformerImage
π― What it does: This paper proposes a multi-spectral object re-identification framework called TOP-ReID, which extracts RGB, NIR, and TIR features using a visual Transformer. Based on this, a cyclic token permutation module and a complementary reconstruction module are designed to achieve spatial alignment and distribution alignment of multi-spectral features.
π― What it does: This paper proposes an unsupervised graph contrastive learning framework called TopoGCL, which achieves stronger representation learning through topology-topology contrast between two graph views.
Towards Automated Chinese Ancient Character Restoration: A Diffusion-Based Method with a New Dataset
Haolong Li (Tongji University), Chen Ye (Tongji University)
CodeRestorationDiffusion modelImage
π― What it does: A large-scale real ancient character erasure dataset ARMCD was constructed, and the DiffACR diffusion model was proposed to achieve automatic restoration of ancient Chinese characters.
Towards Automated RISC-V Microarchitecture Design with Reinforcement Learning
Chen Bai (Chinese University of Hong Kong), Martin D. F. Wong
CodeOptimizationReinforcement Learning
π― What it does: A reinforcement learning-based automated design framework for RISC-V microarchitecture has been designed and implemented, capable of automatically generating microarchitecture configurations under given performance/power/area (PPA) targets.
π― What it does: A Modal-Enhanced Semantic Modeling (MESM) framework is proposed for Video Moment Retrieval (VMR), enhancing video and text features at both the frame-word and paragraph-sentence levels to address the modality imbalance issue.
π― What it does: A continuous knowledge graph embedding framework named IncDE is proposed, which can maintain old knowledge while incrementally learning new triples.
π― What it does: A learning-free framework for graph models named MEGU is proposed, which achieves efficient deletion of nodes, features, and edges while maintaining the original model's predictive performance through the mutual evolution of two modules (the prediction module and the forgetting module);
π― What it does: This paper proposes InstDiffEdit, a no-training, no-manual-operation method that utilizes diffusion models and cross-modal attention to instantaneously generate masks for semantic image editing.
Towards Evidential and Class Separable Open Set Object Detection
Ruofan Wang (Fudan University), Rui Feng (Fudan University)
CodeObject DetectionContrastive LearningImage
π― What it does: Proposes the Evidential Object Detector (EOD), which estimates class uncertainty and identifies unknown objects in open-set object detection through an evidence deep learning framework.
π― What it does: This paper proposes a complete angle annotation dataset RMOS based on rendering, which combines Laplace smoothing labels and weighted Smooth-L1 loss, using the OEFormer model with local window self-attention, significantly improving the fine-grained accuracy of human body pose estimation.
π― What it does: A Data Augmentation Domain Adaptation (DADA) framework is proposed, which utilizes adversarial domain adaptation and a mixed feature intermediate domain to align the distributions of samples and proxies, enhancing the efficiency of proxy-based deep metric learning.
Towards Learning and Explaining Indirect Causal Effects in Neural Networks
Abbavaram Gowtham Reddy (Indian Institute of Technology Hyderabad), Satyanarayan Kar (Honeywell)
CodeExplainability and InterpretabilityComputational EfficiencyTabularTime Series
π― What it does: A causal explanation method is proposed, which adds feedforward connections based on causal graphs to the input layer of neural networks, enabling the model to learn and explain the direct and indirect causal effects of input features on outputs during the training process.
Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQA
Chengen Lai (Xidian University), Guangneng Hu (Xidian University)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityChain-of-Thought
π― What it does: A multi-layer contrastive learning-based natural language explanation model (MCLE) is proposed, which generates explanations and answers through chain-of-thought (COT) and enhances the reliability and consistency of explanations using semantic-level, image-level, and instance-level contrastive learning.
π― What it does: A multi-modal robust tensor ring decomposition (ORTRD) method is proposed to recover low-rank tensors from high-order tensors contaminated by multi-modal outliers and noise.
Towards Real-World Test-Time Adaptation: Tri-net Self-Training with Balanced Normalization
Yongyi Su (South China University of Technology), Kui Jia (Chinese University of Hong Kong)
CodeDomain AdaptationSupervised Fine-TuningImage
π― What it does: The research addresses both global and local class imbalance as well as continuous domain shift in Test-Time Adaptation (TTA) in real-world testing, and proposes the TRIBE method.
π― What it does: This paper proposes a robust attack and adaptive adversarial training for image stitching, enhancing the resistance of deep stitching models to adversarial perturbations.
π― What it does: This paper proposes a Sequential Deformation method to address visual defects of sleeve and waist shrinkage in high-resolution virtual fitting.
Towards the Robustness of Differentially Private Federated Learning
Tao Qi (Tsinghua University), Yongfeng Huang (Tsinghua University)
CodeFederated LearningSafty and PrivacyAdversarial AttackImage
π― What it does: This paper first proposes a novel poisoning attack method for differential privacy federated learning (DP-FL) called Attack-DPFL, and subsequently designs a robust aggregation algorithm called Robust-DPFL based on the detection of different distribution characteristics between poisoned gradients and clean gradients, aimed at enhancing the model's resistance to poisoning attacks while maintaining differential privacy.
Traffic Flow Optimisation for Lifelong Multi-Agent Path Finding
Zhe Chen (Monash University), Peter J. Stuckey (Monash University)
CodeOptimizationRobotic IntelligenceTabular
π― What it does: A traffic congestion-aware guiding path is proposed, improving the PIBT and LaCAM* algorithms, enabling low latency and high throughput in large-scale MAPF (Multi-Agent Path Finding) in high-density environments.
π― What it does: A framework called TAN is proposed, which combines knowledge transfer and feature alignment for the task of General Category Discovery (GCD);
π― What it does: This paper proposes an attack method based on 'salient feature suppression and neighborhood feature amplification' (OSFD). By using data augmentation (rotation, scaling, blurring) and a specially designed loss function, it perturbs the backbone features of the target detector, achieving high transferability in untargeted black-box attacks.
π― What it does: The TLEG method is proposed, which generates different depth Transformers using two sets of shared parameters through a linear formula, and learns learngene through knowledge distillation.
π― What it does: A novel NR-IQA model SaTQA that combines supervised contrastive learning with Transformer is proposed, which uses pre-training to obtain distortion features and integrates perceptual information to predict image quality.
π― What it does: An end-to-end Transformer-based gaze object prediction method called TransGOP is proposed, which can simultaneously perform object detection, gaze estimation, and gaze object localization.
π― What it does: Utilizing a single high-speed moving spike camera (Spike Camera) to achieve background de-occlusion tasks through continuous spike streams;
Transition-Informed Reinforcement Learning for Large-Scale Stackelberg Mean-Field Games
Pengdeng Li (Nanyang Technological University), Bo An (Nanyang Technological University)
CodeOptimizationReinforcement Learning
π― What it does: A framework for solving large-scale average field games based on reinforcement learning (RL) is proposed, which enhances data efficiency and training stability through Transfer Information Reinforcement Learning (TIRL) and regularization methods.
π― What it does: A Unified Graph Transformation Network (UGT) is proposed, which integrates local and global structural information in the graph attention mechanism by constructing features such as virtual edges, structural identity, structural distance, and transfer probabilities. It uses self-supervised pre-training to obtain universal graph embeddings that can be directly applied to downstream tasks such as node clustering, node classification, and graph-level classification.
Trust Region Methods for Nonconvex Stochastic Optimization beyond Lipschitz Smoothness
Chenghan Xie (Fudan University), Yinyu Ye (Stanford University)
CodeOptimizationImage
π― What it does: A trust-region-based stochastic non-convex optimization algorithm is proposed, achieving convergence to first-order and second-order approximate extremum points under general (L, L0, 1) smoothness and its second-order extension.
TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients
Mengdi Wang (Technical University of Munich), Enkelejda Kasneci (Technical University of Munich)
CodeOptimizationFederated LearningImage
π― What it does: TurboSVM-FL is proposed, an aggregation strategy that utilizes SVM for selective aggregation and maximum margin diffusion regularization in federated learning.
π― What it does: In continuous conditional generation tasks, the Dual-NDA method is proposed, which utilizes two types of negative samples (real images with inconsistent labels and visually unrealistic pseudo-images) to improve the training of Continuous Conditional GAN (CcGAN), enhancing the visual quality and label consistency of generated images.
π― What it does: Proposes the TCBC (Twice Class Bias Correction) method, which provides a unified correction for model bias and pseudo-label bias in class-imbalanced semi-supervised learning;
π― What it does: A unified multimodal information extraction model, UMIE, is proposed, which integrates MNER, MRE, and MEE tasks into a generative task and achieves cross-task reasoning through instruction fine-tuning.
CodeOptimizationConvolutional Neural NetworkAuto EncoderTime SeriesPhysics Related
π― What it does: The LE-PDE-UQ framework is proposed, which uses latent vectors for solving PDEs with time evolution and uncertainty quantification, supporting forward prediction and inverse optimization;
Uncertainty Quantification in Heterogeneous Treatment Effect Estimation with Gaussian-Process-Based Partially Linear Model
Shunsuke Horii (Waseda University), Yoichi Chikahara (NTT Communication Science Laboratories)
CodeTabular
π― What it does: A Gaussian process prior is proposed for the non-parametric components in partially linear models, resulting in a closed-form posterior distribution of CATE, enabling Bayesian quantification of heterogeneous treatment effects.
Kai Ye (University of Pittsburgh), Liang Zhan (University of Pittsburgh)
CodeDepth EstimationOptimizationImageTabular
π― What it does: This study addresses and solves the problem of zero gradients in the Evidential Regression Network (ERN) in high uncertainty areas (HUA), which hinders learning, and proposes an uncertainty regularization method.
Uncertainty-Aware Yield Prediction with Multimodal Molecular Features
Jiayuan Chen (Ohio State University), Xiangliang Zhang (University of Notre Dame)
CodeDrug DiscoveryGraph Neural NetworkTransformerMixture of ExpertsContrastive LearningMultimodalityGraph
π― What it does: A multimodal model for uncertainty-aware perception, UAM, is proposed to predict chemical reaction yields using multimodal molecular features.
Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolution
Emily McMilin (Independent Researcher)
CodeTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
π― What it does: This study investigates the under-specification problem in language model tasks, revealing how it induces spurious correlations with irrelevant features such as gender and time/location, and proposes two lightweight black-box evaluation methods to detect and quantify these associations.
π― What it does: The researchers revealed three learning stages (approximately constant, decreasing, and increasing) during the training process through real-time monitoring of the reconstruction error of deep network layer outputs.
π― What it does: An algorithm capable of finely adjusting biological colors in underwater images is proposedβCECF, which utilizes the decomposition of color and content encoding and guides the image to achieve colorful enhancement;
UniCell: Universal Cell Nucleus Classification via Prompt Learning
Junjia Huang (Sun Yat-sen University), Guanbin Li (Chinese University of Hong Kong)
CodeClassificationObject DetectionSegmentationTransformerPrompt EngineeringContrastive LearningImageBiomedical Data
π― What it does: This paper presents UniCell, an end-to-end universal cell nucleus detection and classification framework capable of uniformly processing multiple heterogeneous datasets.
Unifying Decision and Function Queries in Stochastic Boolean Satisfiability
Yu-Wei Fan (National Taiwan University), Jie-Hong R. Jiang (National Taiwan University)
Code
π― What it does: A new formalization of SSAT(Ξ) is proposed, which incorporates threshold quantifiers into the prefix of the original SSAT to unify decision-making and function queries, along with the corresponding solver ClauSSat(Ξ).
π― What it does: A unified visual and visual-language tracking framework UVLTrack is proposed, which can simultaneously support three target reference methods: BBOX, NL, and NL+BBOX.
π― What it does: This paper proposes the Union Subgraph Neural Network (UnionSNN), which enhances the expressive power of GNNs by combining the unified subgraph of 1-hop neighbors with the shortest path matrix.
π― What it does: A MAP (Max Agreement Prediction) method based on deep network ensemble is proposed, which utilizes the prediction consistency of models across different training epochs to mitigate overfitting, showing excellent performance especially on datasets with label noise.
π― What it does: Proposes Unknown Awareness Graph Regularization (UAG) for open semi-supervised learning, which can robustly learn from unlabeled data containing unknown classes.
Unraveling Batch Normalization for Realistic Test-Time Adaptation
Zixian Su (Xi'an Jiaotong-Liverpool University), Kaizhu Huang (Duke Kunshan University)
CodeDomain AdaptationImageBenchmark
π― What it does: A test-time adaptive method based on batch normalization is proposed, utilizing Test-time Exponential Moving Average (TEMA) and hierarchical normalization correction to enhance the model's robustness under different batch sizes.
Unsupervised Action Segmentation via Fast Learning of Semantically Consistent Actoms
Zheng Xing (Chinese University of Hong Kong), Weibing Zhao (Shenzhen MSU-BIT University)
CodeRecognitionSegmentationVideo
π― What it does: This paper proposes a completely unsupervised action segmentation framework called Split-and-Merge (SaM), which identifies local minima by calculating the subspace similarity between adjacent frames to partition them into semantically consistent actoms, and then merges actoms of the same action using spatio-temporal similarity, ultimately achieving complete video action segmentation results.
π― What it does: This paper proposes a prototype-based optimal transport framework, ProtoOT, to address the unsupervised cross-domain image retrieval task, balancing intra-domain feature learning and inter-domain alignment.
Unsupervised Layer-Wise Score Aggregation for Textual OOD Detection
Maxime Darrin (International Laboratory on Learning Systems), Pierre Colombo (Universite Paris-Saclay)
CodeAnomaly DetectionTransformerTextBenchmark
π― What it does: An unsupervised hierarchical score aggregation method is proposed, which automatically combines the anomaly scores from all encoder layers to enhance text OOD detection performance.
π― What it does: A deep graph convolution kernel machine (GCKM) is proposed, which achieves recursive aggregation and classification of node features through multi-layer unsupervised kernel PCA and semi-supervised kernel spectral clustering.
π― What it does: This paper proposes a multi-view urban area embedding model called ReCP based on a consistency learning paradigm, which jointly learns area representations using POI attribute views and human mobility views, emphasizing consistency between different views rather than just late-stage fusion.
Using Artificial Populations to Study Psychological Phenomena in Neural Models
Jesse Roberts (Vanderbilt University), Douglas Fisher (Cornell University)
CodeTransformerLarge Language ModelText
π― What it does: This paper proposes PopulationLM, which constructs an artificial model family by applying hierarchical MC dropout on the Transformer, thereby systematically studying the cognitive behavior of language models in terms of typicality and structural derivation effects.
Using Stratified Sampling to Improve LIME Image Explanations
Muhammad Rashid (University of Torino), Damiano Verda (Rulex Innovation Labs)
CodeObject DetectionExplainability and InterpretabilityConvolutional Neural NetworkImage
π― What it does: This study investigates the use of stratified sampling in LIME Image as a replacement for Monte Carlo sampling to reduce the inaccuracies in explanations caused by downsampling.
CodeAnomaly DetectionTransformerVision Language ModelContrastive LearningVideo
π― What it does: This paper proposes VadCLIP, a dual-branch weakly supervised video anomaly detection framework that utilizes a frozen CLIP model to directly perform video feature extraction and anomaly detection without the need for pre-training or fine-tuning.
π― What it does: In response to the vulnerability of graph neural networks when facing attacks, a cost-based graph defense framework called RisKeeper is proposed, which enhances robustness against structural attacks by learning node costs.
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Taylor Sorensen (University of Washington), Yejin Choi (University of Washington)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper first constructs a large-scale multi-value dataset called ValuePrism, which contains 218k entries of values, rights, and obligations for 31k real-life scenarios. Based on this dataset, the Value Kaleidoscope (Kaleido) model is trained to generate, explain, and evaluate the relevance and positive/negative tendencies of values, rights, and obligations in given contexts.
π― What it does: A Variational Hybrid Attention Framework (VHAF) is proposed to address the category detection problem in multi-label few-shot learning.
Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding
Taolin Zhang (Tsinghua University), Shu-Tao Xia (Harbin Institute of Technology)
CodeRecognitionObject DetectionTransformerVision Language ModelContrastive LearningMultimodalityPoint Cloud
π― What it does: A 3DVLP pre-training framework is proposed, achieving unified representation and transfer of 3D vision-language through object-level contrastive learning.
CodeObject DetectionComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextChain-of-Thought
π― What it does: A Visual Chain Thinking Prompt (VCTP) framework is proposed, which implements knowledge-based visual reasoning through an iterative see-think-confirm three-stage interaction.
Delong Chen (Xiaobing AI), Baoyuan Wang (Xiaobing AI)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: This paper proposes Polite Flamingo, a multimodal response rewriter that transforms short, unformatted answers from the original visual-language dataset into polite, structured, high-quality replies, thereby generating the PF-1M dataset for training multimodal LLMs (Clever Flamingo).
VITA: βCarefully Chosen and Weighted Lessβ Is Better in Medication Recommendation
Taeri Kim (Hanyang University), Sang-Wook Kim (Hanyang University)
CodeRecommendation SystemDrug DiscoveryGraph Neural NetworkTransformerBiomedical DataElectronic Health Records
π― What it does: This paper proposes a framework for drug recommendation called VITA, which automatically selects the most relevant historical visit records related to the current visit and weights them to more accurately represent the patient's current health status and recommend medications.
π― What it does: We propose ViTEraser, a single-stage scene text removal model based on Vision Transformer, and design the SegMIM pre-training method.
CodeClassificationExplainability and InterpretabilityRepresentation LearningTransformerImage
π― What it does: A single-path neural tree called ViTree is designed and implemented for fine-grained visual classification, combining visual Transformers for feature extraction, achieving progressive representation learning through hard patch selection in the tree, and ultimately outputting classification results via a single path.
VIXEN: Visual Text Comparison Network for Image Difference Captioning
Alexander Black (University of Surrey), John Collomosse (University of Surrey)
CodeRecognitionGenerationTransformerLarge Language ModelContrastive LearningImageText
π― What it does: VIXEN is an image difference subtitle generation model that can describe the visual changes between two images with brief text, helping to identify forged content.
VLCounter: Text-Aware Visual Representation for Zero-Shot Object Counting
Seunggu Kang (Sungkyunkwan University), Jae-Pil Heo (Sungkyunkwan University)
CodeObject DetectionVision Language ModelContrastive LearningImage
π― What it does: A one-stage zero-shot object counting framework called VLBase is proposed, and based on it, semantic condition prompt tuning (SPT), learnable affine transformation (LAT), and segmented skip connections (SaSC) are added to construct VLCounter, eliminating the traditional two-stage example discovery process.
VLM2Scene: Self-Supervised Image-Text-LiDAR Learning with Foundation Models for Autonomous Driving Scene Understanding
Guibiao Liao (Peking University), Xiaoqing Ye (Baidu Inc.)
CodeSegmentationAutonomous DrivingTransformerVision Language ModelContrastive LearningImagePoint Cloud
π― What it does: This paper proposes VLM2Scene, which achieves unsupervised training of 3D scene representation through self-supervised learning by combining CLIP, BLIP-2, and SAM in image-text-lidar contrastive learning.
π― What it does: A few-shot font generation framework called VQ-Font has been developed, which is based on VQGAN codebook and structure-level style enhancement, capable of transferring fine-grained strokes and structural styles from a small number of reference glyphs to the target glyph.
VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models
Ziyi Yin (Pennsylvania State University), Fenglong Ma (Pennsylvania State University)
CodeAdversarial AttackTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: A transferable adversarial attack method based on pre-trained vision-language models, VQAttack, is proposed to generate adversarial samples that perturb both images and text without accessing the target VQA model.
π― What it does: A night scene image restoration framework based on vector quantization codebook, VQCNIR, is proposed, along with the design of an Adaptive Illumination Enhancement Module and a Deformable Bi-directional Cross-Attention module to enhance the details and lighting consistency of low-light and blurred images.
π― What it does: A visual-spatial fusion Transformer (VSFormer) is proposed for two-view correspondence pruning and camera pose estimation, significantly improving performance in high outlier rate scenarios.
VVS: Video-to-Video Retrieval with Irrelevant Frame Suppression
Won Jo (Sejong University), Yukyung Choi (Sejong University)
CodeRetrievalTransformerVideo
π― What it does: A VVS framework is proposed, which first removes easily recognizable interference frames, then generates suppression weights based on temporal saliency and thematic relevance, ultimately producing video-level features to enhance content video retrieval performance.
Chengyi Yang (Shanghai Institute of AI for Education), Aimin Zhou (Shanghai Institute of AI for Education)
CodeSafty and PrivacyConvolutional Neural NetworkImage
π― What it does: A differential privacy framework WDP based on Wasserstein distance is proposed, and a Wasserstein accountant is implemented for privacy budget calculation in deep learning.
π― What it does: A WaST framework based on 3D wavelet transform is proposed, utilizing a time-frequency aware decoder to learn spatial-temporal dependencies in multi-scale wavelet space, thereby achieving non-recursive video prediction.
π― What it does: A weakly supervised multimodal method WSMA is proposed, which transfers appearance human-computer interaction images and textual knowledge to first-person perspective images to locate functional areas.
π― What it does: This paper proposes a weakly supervised semantic segmentation framework for driving scenarios, utilizing CLIP to generate pseudo-masks and enhancing segmentation performance through global-local perspective training and Consistency-Aware Region Balancing (CARB).
Weakly-Supervised Mirror Detection via Scribble Annotations
Mingfeng Zha (University of Electronic Science and Technology of China), Heng Tao Shen (University of Electronic Science and Technology of China)
CodeObject DetectionSegmentationTransformerImage
π― What it does: This paper proposes a weakly supervised mirror detection framework based on sparse graffiti annotations and constructs the first graffiti-style mirror dataset.
π― What it does: A weakly supervised temporal action localization framework is proposed, which generates high-quality pseudo-labels by inferring significant segment features and uses these pseudo-labels to train the action localization network.
WebVLN: Vision-and-Language Navigation on Websites
Qi Chen (Australian Institute for Machine Learning), Qi Wu (Australian Institute for Machine Learning)
CodeTransformerLarge Language ModelVision Language ModelTextMultimodality
π― What it does: A web-based visual and language navigation task called WebVLN is proposed, along with the corresponding WebVLN-v1 dataset and baseline model WebVLNNet.
π― What it does: We propose WeditGAN, a model transfer method for cross-domain few-shot image generation that learns a fixed latent space offset Ξw.
Well, Now We Know! Unveiling Sarcasm: Initiating and Exploring Multimodal Conversations with Reasoning
Gopendra Vikram Singh (Indian Institute of Technology Bombay), Pushpak Bhattacharyya (Indian Institute of Technology Bombay)
CodeRecognitionGenerationTransformerLarge Language ModelVideoTextMultimodalityAudio
π― What it does: This paper proposes the task of identifying the starting point of sarcasm and generating reasons in multimodal dialogue (SIRC), and constructs the corresponding SIRD dataset;
What Do Hebbian Learners Learn? Reduction Axioms for Iterated Hebbian Learning
Caleb Schultz Kisby (Indiana University), Lawrence S. Moss (Indiana University)
Code
π― What it does: This paper proposes a framework that maps dynamic logic operators [Ο] to iterative Hebbian learning, providing corresponding reduction axioms and proving the completeness of this learning strategy.
π― What it does: This paper studies the generalization problem of visual reinforcement learning in unknown environments, proposing a Truncated Return Prediction (TRP) auxiliary task and implementing multi-policy cross-domain consistency using a Transformer network.
π― What it does: A collaborative framework CMiMC is proposed to construct high-quality collaborative views by maximizing mutual information between intermediate collaborative views and individual views.
When Model Meets New Normals: Test-Time Adaptation for Unsupervised Time-Series Anomaly Detection
Dongmin Kim (Korea Advanced Institute of Science and Technology), Jaegul Choo (Korea Advanced Institute of Science and Technology)
CodeAnomaly DetectionRecurrent Neural NetworkAuto EncoderTime Series
π― What it does: This paper proposes a test-time adaptation method for unsupervised time series anomaly detection based on trend estimation and self-supervised model updates, which can learn new 'normal' patterns in real-time during distribution drift and reduce false positives.