🎯 What it does: This paper proposes a self-supervised transferable graph dataset compression framework ST-GCond, which generates synthetic graph datasets that are extremely small in size but rich in information while maintaining test performance.
Stabilized Neural Prediction of Potential Outcomes in Continuous Time
Konstantin Hess (Munich Center for Machine Learning), Stefan Feuerriegel (Munich Center for Machine Learning)
CodeTabularBiomedical DataElectronic Health RecordsStochastic Differential Equation
🎯 What it does: A neural network method for estimating Conditional Average Potential Outcomes (CAPO) in continuous time, called SCIP-Net, is proposed, which can make accurate inferences in the presence of time-varying confounding.
🎯 What it does: This paper proposes the Stable Hadamard Memory (SHM) — a memory augmentation network that utilizes the Hadamard product for memory calibration and updating, aimed at addressing memory management issues in RL under long time sequences and partially observable environments.
Qi Fan (Nanjing University), Chi-Keung Tang (Hong Kong University of Science and Technology)
CodeSegmentationSupervised Fine-TuningImage
🎯 What it does: Stable-SAM is proposed to enhance segmentation stability under low-quality prompts (inaccurate boxes or sparse points) by adjusting the mask attention of SAM.
STAFF: Speculative Coreset Selection for Task-Specific Fine-tuning
Xiaoyu Zhang (Xi'an Jiaotong University), Yang Liu (Nanyang Technological University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: A task-specific LLM fine-tuning Coreset selection method named STAFF is proposed, which utilizes sibling small models to quickly estimate sample importance and validate on the target LLM, dynamically allocating selection budgets while balancing importance and diversity, significantly improving data efficiency and reducing selection costs.
🎯 What it does: A scalable and task-agnostic multi-vehicle collaborative perception framework called STAMP is proposed, utilizing lightweight adapter-recovery modules to achieve cross-heterogeneous vehicle BEV feature alignment and fusion.
Weronika Ormaniec (ETH Zurich), Andreas Krause (ETH Zurich)
CodeGenerationData Synthesis
🎯 What it does: This paper proposes a structural causal model with internal standardization for each variable during the generation process (iSCM) to eliminate artificial traces such as variance and correlation sorting (Var-sortability, R2-sortability) that arise in synthetic data from traditional SCM.
STAR: Stability-Inducing Weight Perturbation for Continual Learning
Masih Eskandar (Northeastern University), Jennifer Dy (Northeastern University)
CodeOptimizationImage
🎯 What it does: Proposes the STAR regularization method, which minimizes the KL divergence under worst-case perturbations in the parameter space to enhance the output stability of the model on learned samples in continual learning.
Start Smart: Leveraging Gradients For Enhancing Mask-based XAI Methods
Buelent Uendes (Vrije Universiteit Amsterdam), Mark Hoogendoorn (Vrije Universiteit Amsterdam)
CodeOptimizationExplainability and InterpretabilityRecurrent Neural NetworkImageTime Series
🎯 What it does: A gradient-based mask initialization method called StartGrad is proposed and implemented, significantly improving the optimization speed and final performance of mask-based XAI methods.
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
Peijie Dong (Hong Kong University of Science and Technology), Xiaowen Chu (Hong Kong University of Science and Technology)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Perform structured binary compression with sub 1-bit precision on large language models, significantly reducing memory and computational load.
CodeClassificationSafty and PrivacyAdversarial AttackContrastive LearningImage
🎯 What it does: This paper studies the defense against black-box model reverse attacks and proposes a post-processing method called Stealthy Shield Defense (SSD).
🎯 What it does: A generative model for flow matching on the Stiefel manifold has been developed to accurately recover three-dimensional molecular structures from molecular formulas and moments of inertia.
Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold
Hoang Phuc Hau Luu (University of Helsinki), Arto Klami (University of Helsinki)
CodeOptimizationTabular
🎯 What it does: A variance-reduced estimator for Gaussian variational inference on the Bures-Wasserstein manifold is proposed, significantly reducing the noise of single-sample Monte Carlo gradients, thereby achieving faster and more stable optimization.
🎯 What it does: A single forward prediction method is proposed in Bayesian deep learning, achieved through local linearization and local Gaussian approximation, which can obtain an approximate posterior predictive distribution without sampling.
StringLLM: Understanding the String Processing Capability of Large Language Models
Xilong Wang (Duke University), Neil Zhenqiang Gong (Duke University)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the StringLLM method, constructs a large-scale string processing benchmark dataset called StringBench, and systematically evaluates the performance of multiple LLMs on string processing tasks.
StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization
Zhuoqun Li (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Yongbin Li (Tongyi Lab)
CodeTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the StructRAG framework, which can automatically select the most suitable structured form (table, graph, algorithm, directory, or block) in knowledge-intensive reasoning tasks and convert the original document into that structured knowledge. It then generates answers through question decomposition, knowledge extraction, and reasoning.
🎯 What it does: This paper proposes a sample selection method based on structural entropy (SES), aimed at efficiently selecting samples that are both informative and representative of the overall distribution in machine learning tasks.
Structure Language Models for Protein Conformation Generation
Jiarui Lu (Mila Quebec AI Institute), Jian Tang (Mila Quebec AI Institute)
CodeGenerationProtein Structure PredictionTransformerDiffusion modelAuto EncoderBiomedical Data
🎯 What it does: A structural language model (SLM) framework is proposed, which quantizes protein three-dimensional conformations into latent corpora using discrete variational autoencoders. Subsequently, new conformations are generated in the latent space using conditional language models or masked diffusion, and are restored to three-dimensional space through a structural decoder.
Structuring Benchmark into Knowledge Graphs to Assist Large Language Models in Retrieving and Designing Models
Hanmo Liu (Hong Kong University of Science and Technology), Lei Chen (Hong Kong University of Science and Technology)
CodeRetrievalNeural Architecture SearchGraph Neural NetworkLarge Language ModelGraphBenchmark
🎯 What it does: Construct a Knowledge Benchmark Graph (KBG) that structures data, models, and performance information into a graph, and uses this graph to retrieve the most suitable neural network models for unseen datasets.
🎯 What it does: This study investigates the complementarity and information specialization of the actor and critic in deep reinforcement learning when sharing and separating representations.
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
Benjamin Feuer (New York University), John P Dickerson
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper systematically evaluates the reliability of using LLM-judge for alignment benchmarks and finds that its implicit preferences tend towards style rather than factuality or safety. It proposes SOS-BENCH, a unified alignment evaluation consisting of 19 standard benchmarks, and demonstrates through large-scale experiments that data scale and prompt diversity are the main drivers for improving alignment during the post-training SFT phase, while a decline in world knowledge is observed during the PO phase.
Subgraph Federated Learning for Local Generalization
Sungwon Kim (KAIST), Chanyoung Park (KAIST)
CodeFederated LearningSafty and PrivacyGraph Neural NetworkGraph
🎯 What it does: A subgraph federated learning framework named FedLoG is proposed, which utilizes reliable head node (head degree and head category) information to generate global synthetic data, and adapts missing knowledge during local training through a local generalization phase, thereby reducing local overfitting and enhancing generalization ability to unseen data.
Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
Wenkai Yang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeReinforcement LearningTabular
🎯 What it does: This paper studies the security issue of strong model alignment under weak supervision—weak-to-strong deception—and quantifies and evaluates this phenomenon in a multi-objective alignment scenario.
SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction
Ling Yang (Peking University), Shuicheng YAN
CodeOptimizationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: To address the difficulties in error localization and self-correction of small LLMs in complex mathematical reasoning, a two-stage SupERCORRECT framework is proposed, significantly improving the model's reasoning accuracy.
🎯 What it does: Proposes Supervised Diffusion Maps (SDM) and Semi-Supervised Diffusion Maps (SSDM), which utilize label information to construct a multi-view adjacency matrix based on traditional Diffusion Maps, and achieve label-driven diffusion through multiplicative interpolation to obtain more task-relevant low-dimensional embeddings.
Support is All You Need for Certified VAE Training
Changming Xu (University of Illinois Urbana-Champaign), Gagandeep Singh (University of Illinois Urbana-Champaign)
CodeGenerationAnomaly DetectionAuto EncoderImage
🎯 What it does: Proposes the CIVET method, which conducts provably robust training for Variational Autoencoders (VAE) by transforming the worst-case error into deterministic decoder error using a support set;
Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation
Xinpeng Wang (Ludwig Maximilian University of Munich), Barbara Plank (Bocconi University)
CodeTransformerLarge Language ModelText
🎯 What it does: This paper proposes a surgical method to reduce the false refusal rate of language models by truncating a single vector (i.e., the false refusal vector) in the Transformer activation stream.
SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?
John Yang (Stanford University), Ofir Press (Meta AI)
CodeAI Code AssistantTransformerLarge Language ModelImageVideoTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: A new benchmark called SWE-bench Multimodal (SWEbench M) was constructed and evaluated to test the performance of code generation models on front-end JavaScript software engineering tasks that include images and visual elements.
SWEb: A Large Web Dataset for the Scandinavian Languages
Tobias Norlund (AI Sweden), Magnus Sahlgren (AI Sweden)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: SWEb is proposed and released, a pre-trained dataset covering Swedish, Danish, Norwegian, and Icelandic with a scale of 1 trillion tokens, along with a model-based Markdown text extractor.
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
Heming Xia (Hong Kong Polytechnic University), Wenjie Li
CodeGenerationOptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A self-reflective reasoning acceleration method called SWIFT is proposed, which utilizes the hierarchical sparsity of LLM to dynamically skip intermediate layers during inference, forming a lightweight draft model for Draft-Verify.
🎯 What it does: A self-supervised learning framework called Sylber is proposed, which maps raw audio to clearly structured syllable-level embeddings, achieving linear-time syllable segmentation and low-bitrate reconstruction.
🎯 What it does: This paper proposes an unsupervised method that extracts coarse semantic units of speech (similar to syllables) by analyzing the loss distribution of pre-trained self-supervised models, and refines the feature space using iterative distillation (SylBoost), ultimately generating low-bitrate discrete labels for training the Speech Language Model (SyllableLM), achieving significant improvements in training and inference speed.
🎯 What it does: This paper proposes SYMDIFF, a method for constructing equivariant diffusion models based on random symmetrization, and applies it to E(3) equivariant molecular generation.
SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models
Daniel Levy (McGill University), Siamak Ravanbakhsh (Intel Labs)
CodeGenerationData SynthesisComputational EfficiencyDiffusion modelGraphPhysics Related
🎯 What it does: A diffusion model is proposed that utilizes asymmetric units and point group symmetry information to achieve precise generation of crystal space group symmetries.
SymmetricDiffusers: Learning Discrete Diffusion on Finite Symmetric Groups
Yongxing Zhang (University of Waterloo), Renjie Liao (University of British Columbia)
CodeOptimizationTransformerDiffusion modelImage
🎯 What it does: A discrete diffusion model called SymmetricDiffusers is proposed on the finite symmetric group $S_n$ to learn permutation distributions and solve sorting, jigsaw, and traveling salesman problems.
🎯 What it does: A recognizable decoupled representation learning framework is proposed, combining sufficient variation and sparse mixture assumptions, and implementing two models: CG-VAE and CG-GAN.
🎯 What it does: A generative model named SynFlowNet is proposed, which constructs molecules using chemical reaction templates and purchasable materials, and recursively generates target molecules in the reaction space through GFlowNet.
Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo
João Loula (Massachusetts Institute of Technology), Timothy J. O'Donnell (McGill)
CodeGenerationOptimizationComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: Using the Sequential Monte Carlo (SMC) method to implement syntactic and semantic constraint generation in large language models allows for the flexible incorporation of different constraint signals during inference and dynamic adjustment of computational resources.
🎯 What it does: This study proposes a diffusion model based on hypergraph hierarchical structure and multifractal dynamics for generating realistic functional magnetic resonance imaging (fMRI) time series, aiming to reduce scanning time and improve data quality.
Zitong Yang (Stanford University), Tatsunori Hashimoto (Stanford University)
CodeData SynthesisRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This study investigates a method to transform small-scale proprietary corpora into large-scale diversified synthetic corpora through Synthetic Continued Pretraining, and continues pretraining on this data to achieve efficient learning of small domain knowledge by pretrained language models.
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A SysBench benchmark was constructed to systematically evaluate the ability of large language models to follow system messages in multi-turn dialogues.
System 1.x: Learning to Balance Fast and Slow Planning with Language Models
Swarnadeep Saha (University of North Carolina at Chapel Hill), Mohit Bansal (University of North Carolina at Chapel Hill)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a system called System1.x Planner that can dynamically switch between fast and slow planning modes in long-term planning tasks, significantly reducing search costs while maintaining accuracy.
Yongqi An (Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences), Jinqiao Wang (Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences)
CodeTransformerLarge Language ModelText
🎯 What it does: A systematic analysis of activation, weight, and attention outliers in LLMs is conducted, demonstrating that they originate from softmax and act as context-aware scaling factors.
Systematic Relational Reasoning With Epistemic Graph Neural Networks
Irtaza Khalid (Cardiff University), Steven Schockaert (Cardiff University)
CodeGraph Neural NetworkContrastive LearningGraph
🎯 What it does: The Epistemic GNN (EpiGNN) model is proposed for systematic relationship reasoning, capable of learning reasoning rules during training and generalizing to longer reasoning chains during testing.
Systems with Switching Causal Relations: A Meta-Causal Perspective
Moritz Willig (Technical University of Darmstadt), Kristian Kersting (Eindhoven University of Technology)
CodeTabular
🎯 What it does: This paper proposes and empirically validates the concept of 'Meta-Causal Models (MCM)' to capture the structural switching of causal graphs in different contexts or states. By extending traditional Structural Causal Models (SCM) through meta-causal states, an algorithm is provided for inferring meta-causal states from observational data. The EM+LO-RANSAC method is used to estimate the number of different mechanisms in a two-variable linear model, and the visualization and interpretation of meta-causal states are demonstrated in an idealized stress-fatigue system. Finally, MCM is compared with classical SCM, and the performance of the algorithm is evaluated on synthetic data.
🎯 What it does: Designed and implemented a joint hybrid-type diffusion model TABDIFF for generating high-quality tabular data, supporting seamless integration of continuous and discrete features;
🎯 What it does: This paper studies and proposes a TabM model based on parameter-efficient ensemble for supervised learning on tabular data, and systematically evaluates its performance and efficiency on 46 public datasets.
🎯 What it does: This paper proposes the TabReD benchmark, which constructs and evaluates eight industry-level datasets for tabular data in real industrial scenarios, particularly focusing on time drift and feature richness.
🎯 What it does: Designed and implemented the TabWak watermarking scheme, which embeds invisible watermarks during the sampling phase of table diffusion models, supporting row-level detection while maintaining data quality.
Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling
Jiawei Xu (Chinese University of Hong Kong), Lei Han (Tencent Robotics X)
CodeTransformerReinforcement LearningSequential
🎯 What it does: This study investigates the robustness of offline reinforcement learning when data is corrupted by random or adversarial noise, and proposes a robust sequence modeling algorithm called RDT.
🎯 What it does: A variant of Mixup based on similarity kernel is proposed—Similarity Kernel Mixup, which dynamically adjusts the linear interpolation coefficient to reduce the deviation of mixed samples from the original class manifold, thereby enhancing the model's predictive performance and confidence calibration.
🎯 What it does: Evaluate the turn-taking ability of audio foundation models (FM) in natural conversations and propose a novel evaluation protocol based on a supervised turn prediction model; simultaneously conduct user studies and benchmarking on existing full-duplex and cascaded speech dialogue systems.
Taming Overconfidence in LLMs: Reward Calibration in RLHF
Jixuan Leng (Carnegie Mellon University), Jiaxin Huang (Washington University in St. Louis)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: To address the issue of overconfidence in LLMs trained with RLHF, two calibration methods without additional golden labels are proposed—PPO-M (calibrated reward model) and PPO-C (calibrated reward computation), which are seamlessly integrated into the existing PPO framework.
🎯 What it does: Proposes the TANGO framework, which utilizes retrieval + diffusion generators to create high-quality body posture videos synchronized with target speech from a single speaker's video.
TASAR: Transfer-based Attack on Skeletal Action Recognition
Yunfeng Diao (Hefei University of Technology), He Wang (UCL Centre for Artificial Intelligence, Department of Computer Science, University College London)
🎯 What it does: A transfer-based adversarial attack method for skeleton action recognition, TASAR, is proposed, and the first large-scale S-HAR robustness benchmark, RobustBenchHAR, is constructed.
TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types
Jiankang Chen (Kuaishou Technology), Di ZHANG
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: TaskGalaxy is proposed, a large-scale multimodal instruction fine-tuning dataset that includes 19,227 hierarchical task types and 413,648 samples, aimed at addressing the issue of insufficient task diversity in existing datasets.
TAU-106K: A New Dataset for Comprehensive Understanding of Traffic Accident
Yixuan Zhou (Alibaba Cloud Computing), Heng Tao Shen (Tongji University)
CodeRecognitionObject DetectionAutonomous DrivingTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageVideoMultimodality
🎯 What it does: This paper constructs a large-scale multimodal traffic accident dataset TAU-106K, and based on this, trains a specialized multimodal large language model TABot to achieve tasks such as traffic accident identification, description, spatiotemporal localization, and spatial localization.
TC-MoE: Augmenting Mixture of Experts with Ternary Expert Choice
Shen Yan (Peking University), Zhouchen Lin (Peking University)
CodeMixture of ExpertsText
🎯 What it does: By expanding the expert space by multiplying each expert by the ternary set {-1, 0, 1}, TC-MoE is constructed to improve expert activation efficiency without changing the routing mechanism.
Zhihao Zhu (University of Science and Technology of China), Defu Lian (University of Science and Technology of China)
CodeComputational EfficiencyData-Centric LearningConvolutional Neural NetworkTransformerLarge Language ModelImageTextMultimodalityTabularBenchmark
🎯 What it does: A large-scale training data detection benchmark named TDDBench has been constructed to systematically evaluate the performance and resource consumption of 21 TDD algorithms across 13 cross-modal datasets (images, tables, text) and 41 target models (including LLMs).
🎯 What it does: A text-video retrieval method named TempMe is proposed, which freezes the model based on the pre-trained CLIP and only trains the LoRA parameters. It utilizes a stepwise multi-granularity spatiotemporal token merging (ImgMe and ClipMe) to compress video tokens, thereby reducing computational complexity.
Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness
Kangrui Du (University of Electronic Science and Technology of China), Shi Gu (University of Electronic Science and Technology of China)
CodeSpiking Neural NetworkImage
🎯 What it does: Proposed and implemented Mixed Temporal Training (MTT), allowing for the random allocation of different time steps during training, enabling SNNs to adapt to various time step configurations during inference.
Temporal Heterogeneous Graph Generation with Privacy, Utility, and Efficiency
Xinyu He (University of Illinois Urbana-Champaign), Jingrui He (University of Illinois Urbana-Champaign)
CodeGenerationSafty and PrivacyGraph Neural NetworkTransformerGenerative Adversarial NetworkGraphTime Series
🎯 What it does: A temporal heterogeneous graph generation framework named THEPUFF is proposed, which provides differential privacy protection while maintaining data usability.
TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data
Jeremy Andrew Irvin (Stanford University), Stefano Ermon (Stanford University)
CodeClassificationRecognitionSegmentationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTime SeriesSequential
🎯 What it does: This paper proposes and implements the first visual language assistant TEOChat capable of processing time series Earth observation images. It constructs a spatiotemporal instruction tuning dataset TEOChatlas, containing 554,071 instruction-image-response triplets based on four mainstream EO datasets, to train TEOChat for various spatial and temporal reasoning tasks.
🎯 What it does: This paper proposes an online testing adaptive method called TCR for Query Shift, aimed at improving the performance of cross-modal retrieval models in scenarios with inconsistent real distributions.
Test-time Adaptation for Regression by Subspace Alignment
Kazuki Adachi (NTT Corporation), Tomoki Hamagami (Yokohama National University)
CodeDomain AdaptationImageTabular
🎯 What it does: This paper proposes a Test-Time Adaptation (TTA) method for regression models called SSA (Significant-Subspace Alignment), which fine-tunes a regression network pre-trained on the source domain using only unlabeled target data under an unknown target distribution.
🎯 What it does: A testing method based on Sequential Monte Carlo (SMC) sampling called DAS is proposed to align diffusion models with arbitrary reward functions, optimizing rewards while maintaining diversity and cross-reward generalization, thus avoiding over-optimization.
CodeGenerationAI Code AssistantLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Designed and released the TESTGENEVAL benchmark to evaluate the unit test generation and test completion capabilities of large language models in real-world Python projects, providing execution metrics such as coverage and mutation scores;
TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshes
Minghao Guo (Massachusetts Institute of Technology), Wojciech Matusik (Massachusetts Institute of Technology)
CodeGenerationOptimizationGaussian SplattingMesh
🎯 What it does: This paper proposes TetSphere splatting, a Lagrangian geometric representation method that utilizes deformable tetrahedral spheres to generate high-quality 3D shapes.
🎯 What it does: Proposes the use of the pre-trained Rectified Flow model as a plug-and-play prior for text-to-3D generation, image inversion, and editing;
Text4Seg: Reimagining Image Segmentation as Text Generation
Mengcheng Lan (Nanyang Technological University), Wayne Zhang (SenseTime Research)
CodeSegmentationGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageText
🎯 What it does: Reformulating the image segmentation problem as a text generation task, a decoder-free framework called Text4Seg is proposed, which implements segmentation using semantic descriptors and row-level run-length encoding (R-RLE).
TexTailor: Customized Text-aligned Texturing via Effective Resampling
Suin Lee (KAIST), Daeshik Kim
CodeGenerationData SynthesisDiffusion modelMesh
🎯 What it does: This work proposes the TexTailor method, which can generate consistent and view-seamless textures for 3D mesh models based on text descriptions.
🎯 What it does: This paper proposes TFG-Flow, a training-free guidance method for multi-modal flow models that allows generated molecules to meet specified properties without additional training.
🎯 What it does: A theoretical framework based on Bayesian inference is proposed for probabilistic reasoning between unseen cross-modal (A↔C) during training, providing a closed-form condition for direct comparison ('Law of Unconscious Contrast Learners') and a Monte Carlo LogSumExp approximation method when this law is not satisfied.
🎯 What it does: Evaluated the effectiveness of the discrete curvature-based graph reconnection method on real-world benchmark datasets and analyzed the impact of hyperparameter search on performance improvement.
The Hidden Cost of Waiting for Accurate Predictions
Ali Shirali (University of California), Rediet Abebe (Max Planck Institute for Intelligent Systems)
CodeOptimizationReinforcement LearningTabular
🎯 What it does: The study investigates the impact of collecting more observations on ranking quality and intervention timing in prediction-driven resource allocation, providing theoretical explanations and optimal allocation algorithms.
The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modeling
Andre Cornman (Tatta Bio), Yunha Hwang (Tatta Bio)
CodeTransformerLarge Language ModelMultimodality
🎯 What it does: An open mixed-modal meta-genomic corpus (OMG) consisting of 3.1 Tbp and 3.3 billion protein-coding sequences has been created and made publicly available, and the first mixed-modal gene language model gLM2 has been trained based on this.
The Pitfalls of Memorization: When Memorization Hurts Generalization
Reza Bayat (Mila), Pascal Vincent (Mila)
CodeClassificationDomain AdaptationImageText
🎯 What it does: This paper studies the interaction between 'memorization' and 'false association' that occurs during the learning process of neural networks, proving that the combination of the two can lead to model generalization failure under distribution shift. It proposes a novel training method called Memorization-Aware Training (MAT), which uses the predicted probabilities of retained samples to adaptively shift logits, thereby guiding the model to learn robust features across distributions.
The Ramanujan Library - Automated Discovery on the Hypergraph of Integer Relations
Itay Beit Halachmi, Ido Kaminer (Technion - Israel Institute of Technology)
CodeLarge Language Model
🎯 What it does: A 'Ramanujan Library' structured as a hypergraph has been constructed, and 75 previously unknown constant relationships have been discovered in this library using an automated search for integer relations method.
🎯 What it does: Without retraining, new multimodal or multi-condition samples are generated by 'superposition' of existing pre-trained diffusion models during the inference phase;
🎯 What it does: This paper provides an in-depth theoretical and empirical analysis of the sigmoid self-attention mechanism, demonstrating its effectiveness as an alternative in transformer architectures, and introduces FLASHSIGMOID, a hardware-aware and memory-efficient implementation.
Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models
Chengyu Du (Fudan University), Yanghua Xiao (Fudan University)
CodeTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: The Progressive Thought Refinement (PTR) framework is proposed, enabling large language models to iteratively improve answers through multi-round reasoning.
Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation
Shengjie Ma (International Digital Economy Academy), Jian Guo (Renmin University of China)
CodeGenerationRetrievalTransformerLarge Language ModelTextFinance RelatedRetrieval-Augmented Generation
🎯 What it does: This paper presents Think-on-Graph 2.0 (ToG-2), a tightly coupled KG×Text RAG framework that iteratively combines knowledge graphs and document retrieval to enhance the deep reasoning of LLMs.
🎯 What it does: A query-driven KV cache channel pruning method called THINK is proposed, which significantly reduces memory consumption during long sequence inference.
TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
Lijie Yang (Carnegie Mellon University), Zhihao Jia (Carnegie Mellon University)
CodeGenerationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The TidalDecode framework is proposed, utilizing position-persistent sparse attention to achieve fast and high-quality generation in LLM decoding.
🎯 What it does: A lightweight time-frequency domain speech separation model called TIGER is proposed, and a more realistic EchoSet dataset is constructed.
CodeOptimizationAdversarial AttackTransformerMixture of ExpertsImageText
🎯 What it does: This paper proposes an Adaptive Clustering Router (AC router) and the corresponding ACMoE layer, which can match tokens with experts in a feature-weighted transformation space, significantly improving the convergence speed, robustness, and overall performance of sparse mixture of experts models.
Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
Xiaoming Shi (Xiaohongshu Inc), Ming Jin (Griffith University)
CodeTransformerMixture of ExpertsTime Series
🎯 What it does: A time series Transformer architecture called TIME-MOE based on sparse Mixture-of-Experts is proposed to build a scalable foundational model for time series.
CodeClassificationConvolutional Neural NetworkTransformerImageBiomedical DataComputed TomographyElectronic Health Records
🎯 What it does: This paper proposes and implements a time-to-event (TTE) pre-training framework that utilizes large-scale longitudinal electronic health records (EHR) supervision to enhance the performance of 3D medical imaging models in predicting future disease risks.
TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecasting
Songtao Huang (Shanghai Artificial Intelligence Laboratory), LEI BAI
CodeConvolutional Neural NetworkTime Series
🎯 What it does: The TimeKAN model is proposed, achieving efficient prediction of long sequences through multi-layer moving average preprocessing, frequency decomposition, deep convolution, and multi-order KAN learning.
Timer-XL: Long-Context Transformers for Unified Time Series Forecasting
Yong Liu (Tsinghua University), Mingsheng Long (Tsinghua University)
CodeTransformerTime Series
🎯 What it does: This paper presents Timer-XL, a causal Transformer capable of handling long contexts and unifying multivariate time series forecasting, using multivariate next-token prediction to unify different types of forecasting tasks.
TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning
Xiangyu Zeng (Nanjing University), Limin Wang (Nanjing University)
CodeRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningVideoText
🎯 What it does: This work proposes TimeSuite, which utilizes Token Shuffle, TAPE, and time-based instruction tuning to transform short video MLLM for long video understanding and temporal localization.
🎯 What it does: A general image-text pre-training model named TIPS is proposed, which combines contrastive learning, self-distillation, and masked image modeling to significantly improve the performance of dense visual tasks and global tasks.
TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights
Aiwei Liu (Tsinghua University), Meng Cao (Apple)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: Proposes Token-level Importance Sampling based Direct Preference Optimization (TIS-DPO), assigning weights to each token and improving the optimization process.
🎯 What it does: This paper studies the impact of gradient clipping on learning dynamics for the least squares problem under high-dimensional stochastic gradient descent (SGD) streaming training, providing theoretical analysis and empirical validation.
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Zayne Rea Sprague, Greg Durrett (University of Texas at Austin)
CodeTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper explores the effectiveness of prompt-based Chain-of-Thought (CoT) through systematic meta-analysis and experimental evaluation across different tasks, finding that it significantly enhances performance primarily in mathematical and symbolic reasoning tasks.
To Tackle Adversarial Transferability: A Novel Ensemble Training Method with Fourier Transformation
Wanlin Zhang (University of Science and Technology of China), Hu Ding (University of Science and Technology of China)
CodeAdversarial AttackImage
🎯 What it does: This paper proposes an integrated training method based on frequency domain transformation (FDT), which significantly enhances adversarial robustness by introducing random noise or targeted attack noise at low amplitude frequencies to allocate the vulnerable directions of sub-models.
CodeRecommendation SystemOptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes integrating a three-level grading system (excellent, good, poor) into preference modeling, improving the Bradley-Terry model to TOBT, and designing the TODO algorithm based on this to enhance the preference alignment effect of LLMs.