SConU: Selective Conformal Uncertainty in Large Language Models
Zhiyuan Wang (University of Electronic Science and Technology of China), Kaidi Xu (Drexel University)
CodeExplainability and InterpretabilityComputational EfficiencyLarge Language ModelText
🎯 What it does: Proposes the SConU framework, which eliminates confidence outliers that violate the exchangeability assumption through significance testing and calculates the minimum manageable risk level to achieve reliable uncertainty estimation for large language models.
SCOP: Evaluating the Comprehension Process of Large Language Models from a Cognitive View
Yongjie Xiao (Sichuan University), Wenqiang Lei (Sichuan University)
CodeExplainability and InterpretabilityLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed the SCOP framework to evaluate five key skills of large language models (LLMs) in the reading comprehension process from a cognitive perspective.
SDD: Self-Degraded Defense against Malicious Fine-tuning
ZiXuan Chen, Ziqian Zeng (South China University Of Technology)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a new self-degradation defense framework (SDD), which defends against malicious fine-tuning attacks by letting LLMs generate high-quality but irrelevant answers under harmful prompts;
SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings
Weikai Lu (South China University of Technology), Ziqian Zeng (South China University of Technology)
CodeData SynthesisOptimizationRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningVideoTextMultimodalityBenchmarkAudio
🎯 What it does: Propose the SEA (Synthetic Embedding Augmented Safety Alignment) method, which utilizes gradient optimization to generate additional modal embeddings to replace real multi-modal data for achieving low-resource multi-modal safety alignment, while simultaneously constructing the VA-SafetyBench video/audio safety evaluation benchmark.
SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation
Zijun Yao (Tsinghua University), Juanzi Li (Tsinghua University)
CodeGenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose SEAKR—a self-aware knowledge retrieval framework that quantifies uncertainty through LLM internal states to determine when to retrieve and dynamically integrate retrieval results, achieving adaptive retrieval-augmented generation.
SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
Changhun Lee (Pohang University of Science and Technology), Eunhyeok Park (Pohang University of Science and Technology)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose a method to enhance the performance of large language models in long-text retrieval tasks by weighting attention heads or channels (SEAL).
Trisha Das (University of Illinois Urbana-Champaign), Jimeng Sun (University of Illinois Urbana-Champaign)
CodeRetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningTextBiomedical Data
🎯 What it does: Propose a semi-supervised clinical trial protocol similarity search framework called SECRET, which uses LLM to generate question-answer pairs to compress long documents, followed by contrastive learning at the question-answer level and trial level to obtain high-quality embeddings for retrieving similar trials.
CodeTransformerLarge Language ModelTextBenchmarkAgriculture Related
🎯 What it does: Constructed SeedBench—a multi-task LLM evaluation benchmark for seed science, covering three stages: gene information retrieval, gene function and regulation, and variety breeding with agronomic traits. It includes 2,264 expert-verified question-answer pairs, supporting 0-shot and 1-shot testing.
Seeking Rational Demonstrations for Large Language Models: A Domain Generalization Approach to Unsupervised Cross-Domain Keyphrase Generation
Guangzhen Zhao (Nanjing University of Posts and Telecommunications), Zhenjiang Dong (Nanjing University of Posts and Telecommunications)
CodeGenerationDomain AdaptationTransformerLarge Language ModelContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Proposes the Seeking Rational Demonstrations (SRD) method, which leverages source domain labeled samples to provide effective examples for large language models (LLMs) in unsupervised cross-domain keyword generation tasks.
Segment First or Comprehend First? Explore the Limit of Unsupervised Word Segmentation with Large Language Models
Zihong Zhang (Wuhan University), Bo Du (Wuhan University)
CodeSegmentationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper utilizes large language models for unsupervised word segmentation, proposing the LLM-WS framework and the LLACA method to evaluate and enhance segmentation performance.
Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Models
Zhuojun Ding (Huazhong University of Science and Technology), Chenghao Fan (Huazhong University of Science and Technology)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the SaM framework, which dynamically selects and merges multi-domain expert models to generate task-specific NER models tailored for the target domain.
Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models
Sungjae Lee (POSTECH), Jungseul Ok (POSTECH)
CodeComputational EfficiencyLarge Language ModelTextChain-of-Thought
🎯 What it does: Proposed a framework named SEAG, integrating adaptive gating, semantic exploration, and early stopping, to enhance the efficiency and accuracy of large language models in multi-step reasoning tasks.
Semantic-Eval : A Semantic Comprehension Evaluation Framework for Large Language Models Generation without Training
Shusheng Li (Northeastern University), Wenjun Tan (Northeastern University)
CodeComputational EfficiencyRepresentation LearningLarge Language ModelTextBenchmark
🎯 What it does: Proposed Semantic-Eval, a training-free semantic understanding evaluation framework for assessing the quality of text generated by large language models.
SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection
Yi-Fan Lu (Beijing Institute of Technology), Heyan Huang (Beijing Institute of Technology)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: Constructed an scalable and reliable semantic evaluation framework named SEOE, which includes a more representative benchmark (564 event types covering 7 domains) and a semantic F1 evaluation method based on large language models.
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers
Clément Dumas (ENS Paris-Saclay), Robert West (EPFL)
CodeExplainability and InterpretabilityRepresentation LearningTransformerPrompt EngineeringText
🎯 What it does: Through activation patching techniques in multilingual translation tasks, causal analysis of Transformer internal representations demonstrates the existence of language-agnostic concept representations, and verifies that averaging concept vectors across different languages can improve translation quality and definition generation performance.
🎯 What it does: Propose a self-guided iterative calibration framework named SGIC, which dynamically corrects answers in retrieval-augmented generation (RAG) by leveraging the context reasoning capability of large language models and uncertainty scores.
SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script
Eunwon Kim (Sogang University), Buru Chang (Korea University)
CodeGenerationData SynthesisTransformerLarge Language ModelText
🎯 What it does: Constructed a long-term dialogue dataset named SHARE, extracting character features, personal events, and shared memories from movie scripts, and proposed the EPISODE framework to generate more coherent and engaging long-term dialogues by leveraging shared memories.
🎯 What it does: Propose a set-based retrieval method called SETR, which leverages chain-of-thought reasoning to identify query information needs and select a complete subset of documents from the candidate list, replacing traditional top-k sorting;
Guangzhi Sun (University of Cambridge), Mark Gales (University of Cambridge)
CodeClassificationTransformerLarge Language ModelText
🎯 What it does: Proposed a no-reference multi-LLM evaluation aggregation method called SkillAggregation, which can learn to dynamically weight and fuse judgments from multiple LLMs into more accurate decisions.
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Benjamin Warner (Answer Ai), Iacopo Poli (Lighton)
CodeClassificationRetrievalAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: This paper proposes ModernBERT, an efficient encoder-only Transformer architecture designed for long contexts (8192) and tailored for retrieval, classification, and code-related tasks.
🎯 What it does: Built a bilingual script-based social intelligence evaluation benchmark named SOCIALEVAL, systematically assessing the goal achievement and interpersonal capability performance of large language models (LLMs) in social interactions using manually crafted 'World Tree' scripts;
🎯 What it does: Designed and verified a SoftCoT framework that achieves efficient continuous-space chain-of-thought reasoning by generating soft thinking tokens through an auxiliary model and mapping them into the LLM representation space.
SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
Zexiong Ma (Peking University), Bing Xie (Peking University)
CodeAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: Proposed a subtask-based reinforcement fine-tuning method called SoRFT, which enhances the performance of open-source LLMs in software problem-solving tasks by using rule-based rewards.
Speaking Beyond Language: A Large-Scale Multimodal Dataset for Learning Nonverbal Cues from Video-Grounded Dialogues
Youngmin Kim (Yonsei University), Youngjae Yu (Yonsei University)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelAuto EncoderVideoTextMultimodality
🎯 What it does: Proposed a large-scale multimodal dialogue dataset VENUS and trained a multimodal language model MARS capable of understanding and generating text along with non-linguistic cues
SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models
Zhen Wan (Kyoto University), Sadao Kurohashi (Kyoto University)
CodeLarge Language ModelBenchmarkAudio
🎯 What it does: Propose SpeechIQ, a multidimensional evaluation framework based on Bloom's cognitive hierarchy, to assess the memory, understanding, and application capabilities of large speech understanding models (LLMVoice);
SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers
Zicong Tang (Wuhan University), Ping Wang (Wuhan University)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose a KV cache compression method called SpindleKV, which addresses redundancy elimination in both shallow and deep layers, employing attention-weight-based pruning and codebook-based replacement techniques.
Stealing Training Data from Large Language Models in Decentralized Training through Activation Inversion Attack
Chenxi Dai (Huazhong University of Science and Technology), Pan Zhou (Huazhong University of Science and Technology)
CodeFederated LearningSafty and PrivacyAdversarial AttackLarge Language ModelText
🎯 What it does: This paper proposes Activation Inversion Attack (AIA), which steals private training data by exploiting intermediate activation values during decentralized training, demonstrating its feasibility and efficiency in a multi-layer pipeline parallel training framework.
🎯 What it does: Systematically detect and define 'sticky tokens' in text embedding models, propose an efficient detection method called STD, and identify 868 sticky tokens across 40 checkpoints and 14 model families.
Structural Reasoning Improves Molecular Understanding of LLM
Yunhui Jang (KAIST), Sungsoo Ahn (KAIST)
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningTextGraphChain-of-Thought
🎯 What it does: Propose a molecular structure reasoning (MSR) framework that enhances performance on molecular tasks by reasoning about molecular structure details before large language models (LLMs) generate answers.
Structure-aware Domain Knowledge Injection for Large Language Models
Kai Liu (Zhejiang University), Jieping Ye (Alibaba Cloud)
CodeDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: In the paper, the authors propose a two-stage structured knowledge injection method (SCPT+SSFT), embedding the hierarchical structure of domain knowledge into large language models to achieve efficient domain adaptation.
STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
Jaeseong Lee (Snowflake AI Research), Yuxiong He (Snowflake AI Research)
CodeComputational EfficiencyLarge Language ModelMixture of ExpertsText
🎯 What it does: Propose a structured-then-unstructured pruning method (STUN), first removing redundant experts via expert-level structured pruning, then performing fine-grained unstructured pruning within the remaining experts;
SURVEYFORGE : On the Outline Heuristics, Memory-Driven Generation, and Multi-dimensional Evaluation for Automated Survey Writing
Xiangchao Yan (Shanghai Artificial Intelligence Laboratory), Bo Zhang (Shanghai Artificial Intelligence Laboratory)
CodeLarge Language ModelAgentic AITextReview/Survey PaperBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes the SURVEYFORGE framework to automate the generation of academic review papers, incorporating heuristic outline generation and memory-based content generation, and constructs the SurveyBench evaluation benchmark.
Synergizing LLMs with Global Label Propagation for Multimodal Fake News Detection
Shuguo Hu (Inner Mongolia University), Huaiwen Zhang (Inner Mongolia University)
CodeClassificationGraph Neural NetworkLarge Language ModelVision Language ModelContrastive LearningMultimodality
🎯 What it does: This paper proposes a multi-modal fake news detection framework named GLPN-LLM, which combines a global label propagation network with pseudo labels generated by large language models (LLMs). It leverages high-confidence pseudo labels from LLMs and enhances detection accuracy by fusing multi-modal features through global label propagation and a mask mechanism.
Synergizing Unsupervised Episode Detection with LLMs for Large-Scale News Events
Priyanka Kargupta (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
CodeSegmentationTransformerLarge Language ModelText
🎯 What it does: This paper proposes an unsupervised episode detection method that can automatically identify and segment different episode paragraphs from news corpora based on key events.
🎯 What it does: Proposed a text-to-image model framework called SYNTHIA based on text prompts, for generating visually novel and functionally consistent innovative design concepts.
TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models
Xinyi He (Xi'an Jiaotong University), Dongmei Zhang (Microsoft)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTabular
🎯 What it does: Proposed and implemented the TableLoRA module, enabling large language models to better understand and process tabular data under parameter-efficient fine-tuning (PEFT).
Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling
Yang Yuguang (Ximalaya Inc), Jianjun Zhao (Kyushu University)
CodeGenerationTransformerFlow-based ModelAudio
🎯 What it does: This paper proposes a zero-shot emotional voice conversion framework named Takin-VC, which can convert speech into any unseen speaker while preserving the content and emotional features of the source speech.
CodeData SynthesisTransformerLarge Language ModelPrompt EngineeringGraphTabularRetrieval-Augmented Generation
🎯 What it does: Propose the TARGA framework, which can automatically synthesize highly relevant logical forms and natural language questions for each test problem, serving as a demonstration for unsupervised semantic parsing.
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes a grammar error correction (GEC) evaluation paradigm based on Targeted Syntactic Evaluation (TSE), and constructs a minimal pair evaluation set CTSEG corresponding to CEFR-J, enabling fine-grained assessment for each grammatical item.
Task-Specific Information Decomposition for End-to-End Dense Video Captioning
Zhiyue Liu (Guangxi University), Jinyuan Liu (Guangxi University)
CodeGenerationTransformerVision Language ModelContrastive LearningVideoText
🎯 What it does: Propose the DDVC framework, which splits shared event queries into localization queries and description queries, and enhances dense video captioning through contrastive learning and joint label assignment.
🎯 What it does: Proposed and implemented TC-RAG, a Turing-complete retrieval-augmented generation framework aimed at enhancing the reliability and accuracy of medical large language models.
🎯 What it does: Designed and implemented the TEACH framework, unifying classical Chinese word sense disambiguation with sentence translation, and constructing a confidence-annotated knowledge base; achieved stepwise reasoning through zero-shot Chain-of-Thought prompting; applied contrastive knowledge self-adaptive distillation to transfer large model reasoning and translation style to small models.
Teaching an Old LLM Secure Coding: Localized Preference Optimization on Distilled Preferences
Mohammad Saqib Hasan (Stony Brook University), Niranjan Balasubramanian (Stony Brook University)
CodeData SynthesisOptimizationSafty and PrivacyKnowledge DistillationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: A synthetic dataset named DiSCo, containing unsafe code, secure code, and their secure reasoning pairs, was constructed, along with a localized preference optimization algorithm (LPO) specifically designed to address local differences in security-related code, further enhancing the performance of large language models in generating secure code.
Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions
Pu Jian (Institute of Automation Chinese Academy of Sciences), Jiajun Zhang (Institute of Automation Chinese Academy of Sciences)
CodeTransformerSupervised Fine-TuningVision Language ModelMultimodalityBenchmark
🎯 What it does: This paper proposes an interactive clarification framework for addressing ambiguity in visual question answering (VQA) and systematically evaluates the performance of vision-language models (VLMs) in this scenario by constructing the ClearVQA benchmark dataset.
TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competition
Tianwei Lin (Zhejiang University), Yueting Zhuang (Zhejiang University)
CodeComputational EfficiencyTransformerMixture of ExpertsTextMultimodalityBenchmark
🎯 What it does: Propose TeamLoRA, a parameter-efficient fine-tuning method that treats LoRA experts as a team, enhancing the effectiveness and efficiency of multi-task learning through collaboration and competition mechanisms.
TeRDy: Temporal Relation Dynamics through Frequency Decomposition for Temporal Knowledge Graph Completion
Ziyang Liu (Tsinghua University), Chaokun Wang (Tsinghua University)
CodeRepresentation LearningGraphTime Series
🎯 What it does: Propose the TeRDy method, which captures long-term and short-term relationship dynamics in temporal knowledge graphs through frequency decomposition to achieve knowledge graph completion.
Text is All You Need: LLM-enhanced Incremental Social Event Detection
Zitai Qiu (Macquarie University), Jian Yang (Macquarie University)
CodeClassificationGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: This paper proposes a novel social event detection framework, LSED, which utilizes large language models (LLMs) to normalize, expand, and translate short texts. The processed texts are then encoded with SBERT, time-vectorized, and projected into hyperbolic space for clustering, with the entire process not relying on graph structures.
The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate It
Aaron Nicolson (CSIRO), Bevan Koopman (CSIRO)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningImageTextMultimodalityTabularElectronic Health Records
🎯 What it does: In automatic chest X-ray report generation, the authors integrated multimodal patient data from MIMIC-CXR and MIMIC-IV-ED (such as demographics, medications, medical history, etc.) and injected them into a language model in the form of embeddings, achieving more accurate report generation.
The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generation
Xiaoyu Zhang (Xi'an Jiaotong University), Yang Liu (Nanyang Technological University)
CodeData-Centric LearningAI Code AssistantLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigate provider bias in code generation by large language models (LLMs), build and publicly release an automated dataset construction pipeline, and conduct systematic experiments on seven mainstream LLMs;
The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects
Yixin Wan (University of California, Los Angeles), Kai-Wei Chang (University of California, Los Angeles)
CodeGenerationLarge Language ModelPrompt EngineeringDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the Paired Stereotype Test (PST) framework to systematically evaluate gender bias in text-to-image models under dual-agent scenarios, and designs 1952 prompts based on occupation and organizational power, using Stereotype Score to quantify bias; significant male stereotypes are found on DALLE-3, and a FairCritic method driven by LLM is proposed to eliminate bias through judgment and feedback.
The Noisy Path from Source to Citation: Measuring How Scholars Engage with Past Research
Hong Chen (University of Michigan), David Jurgens (University of Michigan)
CodeTransformerLarge Language ModelText
🎯 What it does: Built a large-scale computational pipeline to automatically assess citation fidelity on 13,000,000 pairs of cited sentences, revealing systematic patterns of information loss and misrepresentation in academic citations.
The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights
Yufang Liu (East China Normal University), Xunliang Cai (Meituan Inc)
CodeRepresentation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: Investigate the practical role of the visual modality in multimodal math reasoning, propose the HC-M3D benchmark dataset, and systematically evaluate the robustness of existing models under image perturbations.
The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Models
Chen Qian (Renmin University of China), Jing Shao (Shanghai Artificial Intelligence Laboratory)
CodeSafty and PrivacyExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose a post-processing method called SPIN, which addresses the fairness-privacy trade-off in LLMs during SFT by identifying and suppressing neurons that simultaneously affect fairness and privacy.
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
Jeffrey Li (University of Washington), Fartash Faghri (Apple)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed TiC-LM, a benchmark for time-continuous pretraining of large language models, covering 114 months of Common Crawl data and multi-domain dynamic evaluations;
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Developed the TigerLLM series of Bangla language models and open-sourced a high-quality Bangla-TextBook corpus and 100k instruction-response pairs (Bangla-Instruct).
🎯 What it does: Proposes a two-stage MixORE framework for open relation extraction (OpenRE) tasks involving a mixture of known and unknown relations, simultaneously achieving classification of known relations and clustering of unknown relations.
Towards Better Evaluation for Generated Patent Claims
Lekang Jiang (University of Cambridge), Stefan Goetz
CodeTransformerContrastive LearningTextBenchmark
🎯 What it does: Proposed an automatic evaluation method for patent claims called PatClaimEval, and constructed the first patent claim evaluation benchmark named Patent-CE;
Towards Better Value Principles for Large Language Model Alignment: A Systematic Evaluation and Enhancement
Bingbing Xu (Renmin University of China), Xiaofeng Meng (Renmin University of China)
CodeExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This study proposes a systematic framework for evaluating value principles and constructs a hierarchical set of value principles called HiVaP based on this framework to enhance value alignment in large language models.
Towards Comprehensive Argument Analysis in Education: Dataset, Tasks, and Method
Yupei Ren (East China Normal University), Xiaopeng Bai (East China Normal University)
CodeClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed the CEAMC-v2 dataset containing 226 high school English argumentative essays, and proposed 14 fine-grained longitudinal and lateral argumentation relation annotation schemes to comprehensively characterize argument structures; conducted experiments on three tasks: argument component identification, relation prediction, and automatic essay scoring on this dataset.
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages
Hyangsuk Min (Korea Advanced Institute of Science and Technology), Hwanjun Song (Korea Advanced Institute of Science and Technology)
CodeGenerationTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: Proposes a multidimensional, multi-domain, bilingual (English-Chinese) summarization evaluation benchmark called MSumBench, covering domain-specific key facts and multi-agent debate-style human annotations.
Towards Reward Fairness in RLHF: From a Resource Allocation Perspective
Sheng Ouyang (Renmin University of China), Yong Liu (Renmin University of China)
CodeReinforcement Learning from Human FeedbackReinforcement LearningTextBenchmark
🎯 What it does: This paper proposes a reward fairness framework, treating reward bias as a resource allocation problem, and designs two methods: fair regularization and fair coefficient, achieving fair reward models and strategies in two stages of reward learning and reinforcement learning in RLHF.
Towards Robust Universal Information Extraction: Dataset, Evaluation, and Solution
Jizhao Zhu (Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences), Xueqi Cheng (Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences)
CodeAdversarial AttackData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes a benchmark dataset called RUIE-Bench for evaluating the robustness of a universal information extraction (UIE) model, covering 14 types of adversarial perturbations, and comprehensively evaluates existing UIE models, traditional IE models, and LLMs on this dataset; meanwhile, a loss-guided data augmentation (LDA) method is proposed to enhance model robustness with a small number of difficult samples.
Xin Zhang (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeRetrievalLarge Language ModelVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: Proposed the text-image interleaved retrieval (TIIR) task, constructed a TIIR benchmark dataset based on WikiHow tutorials, and introduced a Matryoshka Multimodal Embedder (MME) with compressible visual tokens to enhance interleaved retrieval performance.
Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
Andrei Mircea (Mila Quebec AI Institute), Ekaterina Lobacheva (Mila Quebec AI Institute)
CodeOptimizationTransformerLarge Language ModelText
🎯 What it does: Investigate the impact of language model scale on training dynamics, discovering and explaining the phenomena of loss deceleration and zero-sum learning (ZSL).
CodeGraph Neural NetworkTransformerLarge Language ModelTextGraphPhysics Related
🎯 What it does: Proposed the Tree-KG framework, which constructs and iteratively expands knowledge graphs by leveraging textual book structures and large language models.
Tree-of-Debate: Multi-Persona Debate Trees Elicit Critical Thinking for Scientific Comparative Analysis
Priyanka Kargupta (University of Illinois at Urbana Champaign), Jiawei Han (University of Illinois at Urbana Champaign)
CodeTransformerLarge Language ModelAgentic AITextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the Tree-of-Debate framework, which uses multi-role LLMs to conduct structured debate trees between papers, generating fine-grained comparative summaries.
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Enhancing the performance of large language models in mathematical and code reasoning tasks by introducing tree search and process supervision in reinforcement learning training.
TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs
Lanxiang Hu (University of California, San Diego), Hao Zhang (University of California, San Diego)
CodeCompressionDomain AdaptationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Compress large language models and enhance inference speed by layer-wise elimination combined with sparse fine-tuning on domain-specific tasks.
TripleFact: Defending Data Contamination in the Evaluation of LLM-driven Fake News Detection
Cheng Xu (University College Dublin), Nan Yan (Georgia Institute of Technology)
CodeData-Centric LearningTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the TripleFact evaluation framework, integrating human adversarial testing (HAPT), real-time network verification (RTW-AV), and entity-agnostic environment (ECVE), to assess the real-world capabilities of large language models in fake news detection, resisting benchmark data contamination (BDC) issues.
TROVE: A Challenge for Fine-Grained Text Provenance via Source Sentence Tracing and Relationship Classification
Junnan Zhu (State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences), Chengqing Zong (State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences)
CodeClassificationRetrievalLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper constructs the TROVE challenge, which requires tracing each sentence in the target text back to specific sentences in the source text and performing fine-grained classification of their relationships.
Truth Knows No Language: Evaluating Truthfulness Beyond English
Blanca Calvo Figueras (University of Basque Country), Rodrigo Agerri (University of Basque Country)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextMultimodalityBenchmark
🎯 What it does: Professionally translate the original English TruthfulQA dataset into Basque, Catalan, Galician, and Spanish, and conduct truthfulness evaluations on 12 open-source LLMs (Llama 3, Llama 3.1, Gemma 2, and their various size versions) across five languages using human evaluation, the MC2 multiple-choice metric, and LLM-as-a-Judge assessment.
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningAgentic AITextTabularFinance RelatedRetrieval-Augmented Generation
🎯 What it does: Proposes a dynamic perception two-stage text-to-table framework TST based on type recognition, which can accurately extract multi-instance dynamic fields under a known domain schema.
🎯 What it does: This study constructs a multilingual multitask language understanding benchmark called TUMLU and releases its mini version TUMLU-mini; meanwhile, it systematically evaluates multiple open-source and proprietary large language models using this benchmark.
Mingze Wang (University of Science and Technology of China), Fuli Feng (Academy of Cyber)
CodeRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITabularSequential
🎯 What it does: Proposes an adjustable LLM-driven proactive recommendation agent (T-PRA), which generates recommendations under real-time user feedback through the Actor-Advisor framework and uses Critic to evaluate long-term rewards, combined with DPO for agent fine-tuning.
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
Xianzhen Luo (Harbin Institute of Technology), Dongliang Xu (Du Xiaoman Science Technology Co Ltd)
CodeComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposes an untrained Token Recycling method that stores candidate tokens and accelerates LLM inference by leveraging adjacency matrices and tree attention.
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
Boyang Xue (Chinese University of Hong Kong), Kam-Fai Wong (Chinese University of Hong Kong)
CodeRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: By embedding two uncertainty estimates, confidence and semantic entropy, as knowledge boundary information into prompts, the reward model and policy model are trained, enabling LLMs to more reliably answer known questions and reject unknown ones in knowledge Q&A.
Uncovering Visual-Semantic Psycholinguistic Properties from the Distributional Structure of Text Embedding Space
Si Wu (Northeastern University), Sebastian Bruch (Northeastern University)
CodeRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Proposed and verified an unsupervised measure based on the neighborhood stability in semantic embedding space (Neighborhood Stability Measure, NSM) to estimate the imagineability and concreteness of vocabulary from text (especially image captions).
Understanding Impact of Human Feedback via Influence Functions
Taywon Min (KAIST), Kimin Lee (KAIST)
CodeExplainability and InterpretabilityComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Propose an scalable influence function method to quantify the impact of human feedback on reward models, and use this method to detect label bias and guide labelers to improve strategies.
Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu
Renhao Pei (Center for Information and Language Processing, LMU Munich), Hinrich Schuetze
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: The study explores the use of large language models (LLMs) for context-based machine translation in the low-resource language Manchu, and systematically evaluates the impact of resources such as dictionaries, parallel examples, grammar books, and chain-of-thought (CoT) on translation performance.
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook
Yidi Jiang (National University of Singapore), Haizhou Li (Chinese University of Hong Kong)
CodeCompressionDomain AdaptationTransformerMixture of ExpertsAudio
🎯 What it does: Proposed a unified audio codec called UniCodec, designed to support multi-domain audio data, including speech, music, and sound, using a single codebook.
Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights
Sooyung Choi (Sungkyunkwan University), JinYeong Bak (Microsoft Research Asia)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This study conducts a systematic safety evaluation of large language models (LLMs) aligned with the Schwartz value system, exploring the association between different value dimensions and specific safety risks (such as hate speech, sexual content, political agitation, etc.), and proposes a simple strategy to reduce harmful behaviors by suppressing risk-related values in prompts.
🎯 What it does: By engineering the internal representations of LLMs, we propose a no-training method called GLoRE, which activates the model's general long-chain reasoning (long CoT) capability.
Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
Haobo Zhang (University of Michigan), Jiayu Zhou (University of Michigan)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a data-driven constraint on the LoRA subspace before fine-tuning (OSRM) to reduce output interference between different tasks, thereby improving the merging effectiveness of multi-task models.
Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models
Atsuyuki Miyai (University of Tokyo), Kiyoharu Aizawa (University of Tokyo)
CodeExplainability and InterpretabilitySupervised Fine-TuningPrompt EngineeringMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Propose the Unsolvable Problem Detection (UPD) task, constructing the MM-UPD Bench to evaluate the refusal capability of multi-modal large models in multiple-choice QA scenarios involving missing answers, invalid answer sets, and image-text mismatches
Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
Junfeng Tian (East China Normal University), Debing Zhang (East China Normal University)
CodeComputational EfficiencyData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Proposed and implemented a data augmentation strategy called Untie the Knots (UtK) to efficiently enhance the long-text context capability of large language models without altering the original data mixture.
Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View
Yanran Wu (Purdue University), Yi Ding (Purdue University)
CodeComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose the FUEL framework, which standardizes the evaluation of carbon emissions in LLM services using functional units (FU), and validate its feasibility through three case studies on model size, quantization, and hardware.
Unveiling the Potential of BERT-family: A New Recipe for Building Scalable, General and Competitive Large Language Models
Yisheng Xiao (Soochow University), Min Zhang (Soochow University)
CodeTransformerLarge Language ModelText
🎯 What it does: Proposed BiGLM—a bidirectional universal language model based on the BERT family—and designed new pre-training tasks and multiple enhancement strategies, demonstrating that encoder-only models can achieve comparable performance to autoregressive LLMs.
Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
Zheheng Luo (University of Manchester), Peng Cheng (University of Manchester)
CodeDomain AdaptationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the Velocitune framework, which dynamically adjusts domain sampling ratios during continuous pre-training based on the learning speeds of each domain to balance multi-domain learning progress;
VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos
Tingyu Song (University of Chinese Academy of Sciences), Yilun Zhao (Yale University)
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposes the VF-EVAL benchmark to evaluate the feedback generation and reasoning capabilities of multimodal large language models on AI-generated videos.
VISA: Retrieval Augmented Generation with Visual Source Attribution
Xueguang Ma (University Of Waterloo), Jimmy Lin (University Of Waterloo)
CodeGenerationRetrievalSupervised Fine-TuningVision Language ModelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose a retrieval-augmented generation (RAG) system called VISA for visual source attribution, which simultaneously generates answers on retrieved document screenshots and provides bounding boxes highlighting the evidence supporting the answers, enabling visual evidence localization.
VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service
Xiasi Wang (Hong Kong University of Science and Technology), Yuan Yao (Hong Kong University of Science and Technology)
CodeComputational EfficiencyAdversarial AttackVision Language ModelImage
🎯 What it does: Propose VLMInferSlow, a black-box efficiency attack method that significantly increases VLM inference time by generating imperceptible perturbations through gradient-free optimization.