ACL 2024 Papers — Page 5
Annual Meeting of the Association for Computational Linguistics · 940 papers
Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue
Jian Wang (Hong Kong Polytechnic University), Xiaoyong Wei
TransformerSupervised Fine-TuningText
🎯 What it does: Proposed a multi-round interactive dialogue fine-tuning framework called MIDI-Tuning, which constructs LoRA adapters separately for the agent and user, and achieves context interaction through round-level memory caching;
Instruction Fusion: Advancing Prompt Evolution through Hybridization
Weidong Guo (Tencent), Di Niu (University of Alberta)
AI Code AssistantTransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Through Instruction Fusion technology, two different code instructions are merged into a single more complex and diverse training instruction for fine-tuning code generation large language models.
Instruction-tuned Language Models are Better Knowledge Learners
Zhengbao Jiang (Carnegie Mellon University), Srinivasan Iyer (FAIR at Meta)
RetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the Pre-instruction Tuning (PIT) method, which performs instruction tuning on question-answer pairs before continuing pre-training with new documents, thereby enhancing the LLM's ability to absorb and retrieve new knowledge.
InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
Zeyuan Wang (Zhejiang University), Huajun Chen (Zhejiang University)
Drug DiscoveryProtein Structure PredictionLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Proposed a large language model called InstructProtein, achieving bidirectional generation between human language and protein sequences;
Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation
Xinglin Wang (Beijing Institute of Technology), Kan Li (Beijing Institute of Technology)
GenerationLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose Fine-Grained Self-Consistency (FSC), which extracts and integrates paragraph-level consensus from multiple LLM-generated samples to produce higher-quality free-text outputs.
Interactive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approach
Saehyung Lee (Seoul National University), Sungroh Yoon (Seoul National University)
RetrievalTransformerLarge Language ModelVision Language ModelImageTextChain-of-Thought
🎯 What it does: Designed PlugIR, a plug-and-play interactive text-to-image retrieval framework that leverages LLM to format dialogues and generate non-redundant questions for retrieval candidate images, enabling efficient retrieval of target images without fine-tuning the retrieval model.
Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models
Guanming Xiong (Peking University), Wen Zhao (Peking University)
TransformerLarge Language ModelAgentic AIPrompt EngineeringTextGraphBenchmarkChain-of-Thought
🎯 What it does: Proposed an interactive KBQA framework that leverages large language models to generate SPARQL queries through multi-round interactions with knowledge base tools, achieving semantic parsing.
Interpretability of Language Models via Task Spaces
Lucas Weber (University Pompeu Fabra), Dieuwke Hupkes (Meta)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Researchers interpret and visualize the language processing of large language models by constructing linguistic task spaces and similarity probing methods;
Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Models
Ying-Chun Lin (Purdue University), Jaime Teevan (Microsoft Corporation)
Explainability and InterpretabilityKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Built a framework for interpretable user satisfaction estimation (SPUR) based on large language models, achieving satisfaction determination through three steps: supervised extraction, rule aggregation, and satisfaction evaluation.
Interpreting Conversational Dense Retrieval by Rewriting-Enhanced Inversion of Session Embedding
Yiruo Cheng (Renmin University of China), Zhicheng Dou (Renmin University of China)
Explainability and InterpretabilityLarge Language ModelContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes the CONVINV method, which converts session embeddings in dialogue-based dense retrieval models into interpretable text while尽量 maintaining the original retrieval performance.
InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers
Yakir Yehuda (Microsoft), Noam Koenigstein (Tel-Aviv University)
Anomaly DetectionTransformerLarge Language ModelText
🎯 What it does: Propose a zero-resource hallucination detection method called InterrogateLLM, which determines whether an answer contains hallucinations by first generating an answer and then reconstructing the question in reverse.
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning
Yutao Zhu, Zhicheng Dou (Renmin University of China)
RetrievalTransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper constructs an instruction tuning dataset called INTERS for search tasks, covering 20 tasks and 43 public IR datasets. Instruction tuning is performed on multiple open-source LLMs (Falcon-1B, Minima-2-3B, Mistral-7B, LLaMA-2-7B), and their performance is evaluated on IR tasks such as query understanding, document understanding, and query-document relationship analysis.
Intrinsic Task-based Evaluation for Referring Expression Generation
Guanyi Chen (Central China Normal University), Kees Van Deemter
GenerationTextBenchmark
🎯 What it does: This paper supplements traditional scoring methods by introducing two new meta-tasks (judging referential success and rewrite suggestions) for intrinsic task-based human evaluation of Reference Expression Generation (REG) models.
Intuitive or Dependent? Investigating LLMs’ Behavior Style to Conflicting Prompts
Jiahao Ying (Singapore Management University), Yongbin Liu (University of South China)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Investigate the behavioral patterns of large language models when facing conflicts between prompts and internal memory, and propose a comprehensive evaluation framework to measure factual robustness and decision-making styles;
Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models
Weihong Zhong (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
Large Language ModelPrompt EngineeringVision Language ModelMultimodalityBenchmark
🎯 What it does: This paper studies how large vision-language models (LVLM) can be misled when encountering previous hallucinations in dialogues. It constructs the MMHalSnowball evaluation framework and a dataset containing 4,973 manually annotated hallucinatory dialogues, and proposes an untrained residual visual decoding (RVD) method to alleviate the hallucination avalanche phenomenon.
Investigating Cultural Alignment of Large Language Models
Badr AlKhamissi (EPFL), Mona Diab (Carnegie Mellon University)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper simulates the World Values Survey in Egypt and the United States, using multilingual prompts and combinations of pre-trained language models (LLMs) to evaluate the consistency of their responses with actual survey results across different cultural contexts, thereby measuring the models' cultural alignment.
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
Tianjie Ju (Shanghai Jiao Tong University), Gongshen Liu (Shanghai Jiao Tong University)
Explainability and InterpretabilityTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Investigated the phenomenon of large language models leveraging fact shortcuts in multi-hop reasoning and assessed the risks to knowledge editing.
IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators
Indraneil Paul (Technische Universität Darmstadt), Iryna Gurevych (Technische Universität Darmstadt)
Representation LearningAI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: Align source code from multiple programming languages using LLVM Intermediate Representation (IR), continuing pre-training existing Code-LM models to obtain a series of multilingual code generation models named IRCoder.
Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval
Peter Baile Chen (Massachusetts Institute of Technology), Dan Roth (University of Pennslyvania)
RetrievalOptimizationLarge Language ModelTabularBenchmark
🎯 What it does: Propose a join-aware multi-table retrieval framework that uses mixed integer programming (MIP) to re-rank retrieval results, comprehensively considering table-query relevance and table-table compatibility, thereby enhancing multi-table retrieval and end-to-end question answering performance.
Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Non-Literal Intent Resolution in LLMs
Akhila Yerukola (Carnegie Mellon University), Maarten Sap (Carnegie Mellon University)
GenerationTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Propose a generative dialogue chain-based evaluation framework to measure large language models' (LLM) understanding of non-literal expressions (indirect speech, sarcasm, violations of Grice's maxims, metaphors) and their ability to generate context-appropriate responses.
Isotropy, Clusters, and Classifiers
Timothee Mickus (University of Helsinki), Joseph Attieh (University of Helsinki)
ClassificationRepresentation LearningTransformerText
🎯 What it does: This paper proves through theoretical derivation and experimental validation that isotropy and clusterability in the embedding space are fundamentally mutually exclusive;
ItD: Large Language Models Can Teach Themselves Induction through Deduction
Wangtao Sun (Institute of Automation Chinese Academy of Sciences), Kang Liu (Institute of Automation Chinese Academy of Sciences)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposes the Induction through Deduction (ItD) framework, which leverages the reasoning capabilities of large language models (LLMs) to first self-generate inductive data, then fine-tune and decode the model using Naive Bayesian inference, thereby significantly enhancing the inductive performance of LLMs.
Iterative Forward Tuning Boosts In-Context Learning in Language Models
Jiaxi Yang (Shenzhen Institute of Advanced Technology Chinese Academy of Sciences), Yongbin Li (Shenzhen Institute of Advanced Technology Chinese Academy of Sciences)
TransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Designed a two-stage Deep-Thinking framework to enhance in-context learning (ICL) performance in large language models through multi-round forward reasoning and iterative key-value matrix updates.
Jailbreak Open-Sourced Large Language Models via Enforced Decoding
Hangfan Zhang (Pennsylvania State University), Dinghao Wu (Pennsylvania State University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose the EnDec attack method, which manipulates the generation process of open-source aligned LLMs through enforced decoding to generate harmful or sensitive content.
JumpCoder: Go Beyond Autoregressive Coder via Online Modification
Mouxiang Chen (Zhejiang University), Jianling Sun (Zhejiang University)
GenerationAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: We propose the JUMPCODER framework, enabling existing code LLMs to perform online modifications and non-sequential generation during the code generation process, thereby improving code generation quality.
KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models
Zhuohao Yu (Peking University), Shikun Zhang (Microsoft Research)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Proposes KIEval, a knowledge-based interactive evaluation framework that leverages multi-round dialogues generated by LLMs to assess the knowledge understanding and generation capabilities of large language models.
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction
Zixuan Li (Chinese Academy of Sciences), Xueqi Cheng (Chinese Academy of Sciences)
Representation LearningAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Develop KnowCoder, an LLM framework that unifies different information extraction tasks through code-style schema representation and completes extraction via code generation; construct a code-style schema library covering over 30k knowledge types, and design a two-phase learning framework (code pre-training + instruction tuning) to enhance the LLM's schema understanding and adherence capabilities.
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Chenxin An (Fudan University), Xipeng Qiu (Fudan University)
Data-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes L-Eval, a standardized evaluation framework for long-context language models (LCLMs), comprising 20 subtasks, 508 long documents, and over 2000 human-annotated QA pairs, covering multiple domains, lengths (3k–200k tokens), and task types.
Label Augmentation for Zero-Shot Hierarchical Text Classification
Lorenzo Paletto (University of Turin), Roberto Esposito (University of Turin)
ClassificationTransformerLarge Language ModelText
🎯 What it does: This paper proposes HiLA, a label expansion method based on large language models, which enriches the label hierarchy in strict zero-shot hierarchical text classification by adding sub-labels at the deepest level; subsequently, the UP (Upward Score Propagation) algorithm is used for zero-shot hierarchical classification.
Label-Efficient Model Selection for Text Generation
Shir Ashury Tahan, Eyal Shnarch (Bar-Ilan University)
GenerationData-Centric LearningTransformerTextBenchmark
🎯 What it does: Proposed the DiffUse method, which selects the most informative examples by leveraging difference vectors from clustering models to achieve label-efficient selection in text generation models;
Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translation
Keqi Deng (University of Cambridge), Phil Woodland
TransformerLarge Language ModelTextAudio
🎯 What it does: This paper proposes LS-Transducer-SST, an end-to-end synchronous speech translation model based on a label-synchronous neural transducer, which can adaptively emit translated words under streaming input;
LaMP: When Large Language Models Meet Personalization
Alireza Salemi (University of Massachusetts Amherst), Hamed Zamani (University of Massachusetts Amherst)
ClassificationGenerationRecommendation SystemLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed the LaMP benchmark to evaluate the personalization capabilities of large language models in text classification and generation tasks; proposed retrieval-enhanced personalization methods (IPA and FiD) and conducted experiments on multiple tasks.
LANDeRMT: Dectecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation
Shaolin Zhu (Tianjin University), Deyi Xiong (Tianjin University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the LANDeRMT framework, which enhances machine translation performance by detecting and routing language-aware neurons to selectively fine-tune large language models (LLM).
Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models
Kun Luo (Institute of Automation Chinese Academy of Sciences), Kang Liu (Institute of Automation Chinese Academy of Sciences)
RetrievalComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose Landmark Embedding, a chunking-free, position-aware embedding method for language models to retrieve enhanced long-text context.
LangBridge: Multilingual Reasoning Without Multilingual Supervision
Dongkeun Yoon (KAIST), Minjoon Seo (KAIST)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes the LANGBRIDGE method, which bridges a multilingual encoder with a specialized reasoning language model through a small number of trainable parameters, achieving zero-shot multilingual reasoning without multilingual supervision.
Language Complexity and Speech Recognition Accuracy: Orthographic Complexity Hurts, Phonological Complexity Doesn’t
Chihiro Taguchi (University of Notre Dame), David Chiang (University of Notre Dame)
RecognitionTransformerSupervised Fine-TuningAudio
🎯 What it does: Fine-tune automatic speech recognition (ASR) models for 25 languages and 15 writing systems, and analyze the relationship between orthographic complexity and speech recognition accuracy.
Language Model Adaption for Reinforcement Learning with Natural Language Action Space
Jiangxing Wang (Peking University), Zongqing Lu (Peking University)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Propose a strategy optimization framework called MIPO based on mutual information regularization, dynamically adjusting the prior of pre-trained language models as the action space, thus achieving implicit and scalable compression of the natural language action space.
Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic
Rishabh Bhardwaj (Singapore University of Technology and Design), Soujanya Poria (Singapore University of Technology and Design)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose the RESTA method, which restores the safety of fine-tuned LLMs by adding a safety vector to their weights, and release a multilingual safety evaluation dataset named CATQA.
Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks
Anwoy Chatterjee (Indian Institute of Technology Delhi), Tanmoy Chakraborty (Indian Institute of Technology Delhi)
ClassificationData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Studies how to leverage the self-learning capabilities of large language models (LLM) by utilizing cross-task examples from different tasks to help models complete new, data-scarce tasks without target task examples, and explores further performance improvements through pseudo-label generation.
Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks
Andrew Gambardella (University of Tokyo), Yutaka Matsuo (University of Tokyo)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Perform MC Dropout error analysis on LLM's performance in arithmetic tasks, revealing its confidence in predicting the highest digit of multiplication but difficulty in predicting the lowest digit.
Language Models Don’t Learn the Physical Manifestation of Language
Bruce Lee, Jaehyuk Lim
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringImageMultimodalityBenchmarkChain-of-ThoughtAudio
🎯 What it does: Propose the H-TEST evaluation framework to systematically examine the blind spots of language models in understanding the visual and auditory attributes of text, verifying the perceptual gaps caused by text-only training;
Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models
Tianyi Tang (Renmin University of China), Ji-Rong Wen (Microsoft Research Asia)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: This paper studies language-specific neurons in large language models and proposes the Language Activation Probability Entropy (LAPE) method to detect and analyze these neurons.
Large Language Models Are No Longer Shallow Parsers
Yuanhe Tian (University of Science and Technology of China), Yan Song (University of Science and Technology of China)
TransformerPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: This paper systematically evaluates the performance of current state-of-the-art large language models (LLMs) on constituency parsing and proposes a three-step splitting process: first, let the LLM perform shallow chunking on the sentence; second, filter out longer blocks that may contain noise based on block length thresholds; finally, use the remaining blocks as soft constraints to guide the LLM in generating complete bracketed syntactic trees in the prompt, adding chain-of-thought (CoT) prompts when necessary to further improve performance.
Large Language Models are not Fair Evaluators
Peiyi Wang (Peking University), Zhifang Sui (Peking University)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: The study found significant positional information bias when using large language models (e.g., GPT-4) as evaluators, and proposes a calibration framework based on multi-evidence, balanced positions, and human-computer interaction;
Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
Keming Lu (Alibaba Inc), Jingren Zhou (Alibaba Inc)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes DITTO, a method that enhances the role-playing capabilities of large language models through self-alignment;
Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
Zekun Li (University of California Santa Barbara), Paul A. Crook (Meta)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: By reconstructing the task-oriented dialogue state tracking task as function calls, the FNCTOD framework was constructed to achieve zero-shot dialogue state tracking across multiple large language models (LLMs).
Large Language Models Can Learn Temporal Reasoning
Siheng Xiong (Georgia Institute of Technology), Faramarz Fekri (Georgia Institute of Technology)
Representation LearningGraph Neural NetworkSupervised Fine-TuningContrastive LearningTextGraphBenchmarkChain-of-Thought
🎯 What it does: By converting text into a Temporal Graph and performing Chain-of-Thought (CoT) reasoning on this graph, the temporal reasoning performance of large language models is enhanced.
Latxa: An Open Language Model and Evaluation Suite for Basque
Julen Etxaniz (University of the Basque Country), Aitor Soroa (University of the Basque Country)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper develops an open-source large language model series called Latxa for Basque, providing three pre-trained model versions (7B, 13B, 70B), a high-quality Basque corpus with up to 4.3M documents and 4.2B tokens, and four multiple-choice evaluation datasets (EusProficiency, EusReading, EusTrivia, EusExams).
Layer-Condensed KV Cache for Efficient Inference of Large Language Models
Haoyi Wu (ShanghaiTech University), Kewei Tu (ShanghaiTech University)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose Layer-Condensed KV Cache to improve the Transformer decoder, enabling all layers to use only the KV from the top layer for attention, significantly reducing KV cache memory usage while maintaining model performance through warmup layers.
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
Mostafa Elhoushi (FAIR at Meta), Carole-Jean Wu (FAIR at Meta)
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the LayerSkip training and inference framework, achieving early exit and self-speculation decoding in LLMs through hierarchical dropout and early exit loss, significantly accelerating inference.
Learn from Failure: Fine-tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic Proving
Chenyang An (University of California, San Diego), Jingbo Shang (University of California, San Diego)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes to fine-tune large language models by leveraging information from failed search paths (trial-and-error) to enhance their reverse search and strategy generation capabilities in intuitive propositional logic proofs.
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models
Junhao Zheng (South China University of Technology), Qianli Ma (South China University of Technology)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper re-examines the catastrophic forgetting phenomenon of pre-trained language models (PLMs) in incremental learning, systematically evaluates the memory degradation of PLMs and classifiers using probe techniques, and proposes a lightweight SEQ* baseline method;
Learnable Privacy Neurons Localization in Language Models
Ruizhe Chen (Zhejiang University), Zuozhu Liu (Zhejiang University)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Investigated a learnable binary neuron mask that locates sparse neurons responsible for memorizing personally identifiable information (PII) in large language models through adversarial training;
Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks
Yingji Zhang (University of Manchester), Andre Freitas
GenerationExplainability and InterpretabilityRepresentation LearningTransformerSupervised Fine-TuningFlow-based ModelAuto EncoderText
🎯 What it does: Proposes a sentence semantic separation framework that integrates reversible neural networks (INN) with Transformer-based language autoencoders, achieving explainability and controllable generation of sentence semantics.
Learning Geometry-Aware Representations for New Intent Discovery
Kai Tang (Zhejiang University), Haobo Wang (Zhejiang University)
ClassificationRepresentation LearningTransformerContrastive LearningText
🎯 What it does: Propose the GeoID method, which leverages the equiangular tight frame (ETF) of the neural collapse phenomenon to learn geometry-aware representations, combined with dual pseudo-labeling (optimal transport + semi-supervised clustering) and contrastive learning, achieving high-quality discovery of known and new intents in unlabelled corpora.
Learning Global Controller in Latent Space for Parameter-Efficient Fine-Tuning
Zeqi Tan (Zhejiang University), Yueting Zhuang (Zhejiang University)
Computational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a global controller (GloC) that interacts with large language models (LLMs) in the latent space using a small number of latent units, achieving parameter-efficient fine-tuning.
Learning or Self-aligning? Rethinking Instruction Fine-tuning
Mengjie Ren (Chinese Information Processing Laboratory), Le Sun (Chinese Information Processing Laboratory)
Large Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Investigate the Instruction Fine-Tuning (IFT) mechanism, proposing a knowledge intervention framework that separates knowledge injection from behavioral norm transfer, and verify through multiple experiments that IFT is primarily self-aligning rather than knowledge injection.
Learning Relational Decomposition of Queries for Question Answering from Tables
Raphaël Mouravieff (Sorbonne Université), Sylvain Lamprier (Université d'Angers)
TransformerLarge Language ModelSupervised Fine-TuningTabular
🎯 What it does: Propose an intermediate supervision framework that enables the model to learn both generating SQL-like operations and directly outputting answers by partially executing SQL execution graphs;
Learning Task Decomposition to Assist Humans in Competitive Programming
Jiaxin Wen (Tsinghua University), Minlie Huang (Tsinghua University)
AI Code AssistantLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose using automatic task splitting to help humans fix model-generated programs more quickly and effectively in competitive programming
Learning to Decode Collaboratively with Multiple Language Models
Zejiang Shen, David Sontag (Massachusetts Institute of Technology)
GenerationTransformerLarge Language ModelAgentic AIText
🎯 What it does: Train a small base language model to learn whether to let a larger or more specialized assistant model generate the next token during decoding, enabling multi-model collaborative generation.
Learning to Edit: Aligning LLMs with Knowledge Editing
Yuxin Jiang (Hong Kong University of Science and Technology), Wei Wang (Huawei)
Computational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose a two-stage framework called Learning to Edit (LTE) to efficiently and dynamically inject new knowledge into large language models and retrieve updated information in real-time during inference, avoiding knowledge confusion caused by traditional methods that rely solely on memory;
Learning to Generate Answers with Citations via Factual Consistency Models
Rami Aly (University of Cambridge), George Karypis (Amazon Web Services)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes a weakly supervised fine-tuning framework called CaLF based on a Fact Consistency Model (FCM) to improve the citation accuracy of generated answers in long-text question answering.
Learning to Plan and Generate Text with Citations
Constanza Fierro (University of Copenhagen), Mirella Lapata (Google DeepMind)
GenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This study explores enhancing the verifiability and authenticity of long-text answers within the Retrieval-Augmented Generation framework by leveraging a problem-based blueprint model, and implements a complete process for automatically annotating blueprints and citations.
Legal Case Retrieval: A Survey of the State of the Art
Yi Feng (Nanjing University), Vincent Ng (University of Texas at Dallas)
RetrievalExplainability and InterpretabilityRecurrent Neural NetworkTransformerLarge Language ModelTextReview/Survey Paper
🎯 What it does: This paper reviews the research progress in the field of Legal Case Retrieval (LCR), elaborating on problem definitions, seven modeling challenges, commonly used datasets, evaluation metrics, and major methods (traditional IR, machine learning, deep learning, knowledge enhancement, explainability, interaction, etc.).
LEMON: Reviving Stronger and Smaller LMs from Larger LMs with Linear Parameter Fusion
Yilong Chen (Institute of Information Engineering Chinese Academy of Sciences), Hua Wu (Baidu Inc)
Knowledge DistillationTransformerLarge Language ModelText
🎯 What it does: In this study, the authors propose a linear parameter fusion method called LEMON, which utilizes parameters from large models (e.g., LLaMA 2-7B) to construct initial parameters for smaller models (1.3B/2.7B) using hierarchical and dimensional fusion operators, followed by further pretraining on this basis;
LePaRD: A Large-Scale Dataset of Judicial Citations to Precedent
Robert Mahari (MIT), Alex Pentland
RetrievalTransformerTextBenchmark
🎯 What it does: Constructed LePaRD, a large-scale judicial citation context-precedent paragraph dataset, encompassing approximately 1.8 million target paragraphs and corresponding contexts.
Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective
Zihao Yue (Renmin University of China), Qin Jin (Renmin University of China)
GenerationData-Centric LearningSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: This paper investigates the root cause of hallucinations in multimodal large models as excessive detail in training data, proposing two methods—Selective EOS Supervision and Scoring EOS Supervision—to enhance the model's EOS decision-making ability by leveraging the mechanism of assessing the completeness of generated text and visual information during EOS prediction, thereby significantly reducing hallucinations.
Let’s Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation
Se Park, Yong Ro
GenerationTransformerLarge Language ModelVision Language ModelGenerative Adversarial NetworkVideoTextMultimodalityAudio
🎯 What it does: Built a face-to-face speech dialogue model that directly processes audio-visual inputs and outputs audio-visual responses, and released the MultiDialog audio-visual corpus with 340 hours of dialogue containing approximately 9,000 utterances.
Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification
Yibo Hu (Georgia Institute of Technology), Vito D’Orazio
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study proposes two zero-shot political relationship classification methods, leveraging existing codebook knowledge through NLI (ZSP model) and ChatGPT (GPT-3.5/4) to automatically categorize source-target relationships in event encoding.
Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling
Hang Jiang (Massachusetts Institute of Technology), Jad Kabbara (Massachusetts Institute of Technology)
TransformerLarge Language ModelText
🎯 What it does: Leverage large language models (LLMs) to generate stories about legal concepts and corresponding reading comprehension questions, constructing the LEGALSTORIES dataset, and evaluate the effectiveness of stories in helping non-professional readers learn legal concepts through randomized controlled trials (RCTs).
Leveraging Machine-Generated Rationales to Facilitate Social Meaning Detection in Conversations
Ritam Dutt (Carnegie Mellon University), Carolyn Rose
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Generate rationales containing intent, assumptions, and implicit information using large language models (LLMs) and use them as text augmentation to identify social meanings in dialogues (emotion recognition and anti-persuasion strategies).
LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition
Xiaomeng Zhu (Yale University), Robert Frank (Yale University)
RecognitionTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the LIEDER dataset for fine-grained evaluation of the semantic knowledge of language models in discourse entity recognition.
Lightweight reranking for language model generations
Siddhartha Jain (NVIDIA), Bing Xiang (Goldman Sachs)
GenerationAI Code AssistantLarge Language ModelTextBenchmark
🎯 What it does: Proposed a lightweight re-ranking method that leverages n-gram similarity between generated texts (particularly word-level consistency scoring) to select the optimal or near-optimal generation from diverse sampling, without requiring additional inference or training auxiliary models.
Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition
Matteo Bortoletto (University of Stuttgart), Andreas Bulling (University of Stuttgart)
Graph Neural NetworkTransformerTextGraphBenchmark
🎯 What it does: Propose a graph-based plan representation and candidate sampling method for conversational collaborative plan acquisition, and systematically evaluate the role of Theory of Mind (ToM) features in this task.
Linear Transformers with Learnable Kernel Functions are Better In-Context Models
Yaroslav Aksenov (Tinkoff), Daniil Gavrilov (Tinkoff)
TransformerText
🎯 What it does: Propose the ReBased model, an improved linear Transformer that uses a learnable quadratic kernel and layer normalization to enhance attention effectiveness.
Linear-time Minimum Bayes Risk Decoding with Reference Aggregation
Jannis Vamvas (University of Zurich), Rico Sennrich (University of Zurich)
GenerationComputational EfficiencyTransformerText
🎯 What it does: Proposes a method to accelerate Minimum Bayes Risk (MBR) decoding through reference aggregation, significantly reducing the computational complexity of pairwise evaluation metrics.
Linguistically Conditioned Semantic Textual Similarity
Jingxuan Tu (Brandeis University), James Pustejovsky (Brandeis University)
Data-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper re-annotated the C-STS validation set, discovering that approximately 55% of instances contained annotation errors, and proposed a QA-based automatic error identification and conditional information extraction method; by decomposing C-STS into a two-step process of first generating answers and then evaluating answer similarity, model performance was significantly improved; simultaneously, a method using Typed Feature Structure (TFS) was proposed to construct more semantically interpretable conditions.
ListT5: Listwise Reranking with Fusion-in-Decoder Improves Zero-shot Retrieval
Soyoung Yoon (Seoul National University), Seung-won Hwang (Seoul National University)
RetrievalTransformerLarge Language ModelText
🎯 What it does: Proposed LISTT5, a listwise re-ranking model based on Fusion-in-Decoder, which can process multiple candidate passages simultaneously during training and inference, and generate an index list sorted in ascending order of relevance.
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?
Zhaochen Su (Soochow University), Min Zhang (Soochow University)
TransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Proposed the COTEMPQA benchmark dataset, specifically designed to evaluate the co-temporal reasoning capabilities of large language models.
LLaMA Pro: Progressive LLaMA with Block Expansion
Chengyue Wu (University of Hong Kong), Ping Luo (University of Hong Kong)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed the Block Expansion method, inserting duplicated Transformer layers into the pre-trained LLaMA model, training only the newly added layers to build LLAMA PRO and LLAMA PRO - INSTRUCT, balancing general and domain-specific capabilities.
Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval
Zheng Liu (Beijing Academy of Artificial Intelligence), Defu Lian (University of Science and Technology of China)
RetrievalDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Propose Llama2Vec, an unsupervised method to adapt large language models (LLMs) into dense retrieval encoders;
LLM in a flash: Efficient Large Language Model Inference with Limited Memory
Keivan Alizadeh (Apple), Mehrdad Farajtabar (Apple)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Enable large model inference on storage-constrained devices by storing LLM weights in flash memory and loading them on demand.
LLM Knows Body Language, Too: Translating Speech Voices into Human Gestures
Chenghao Xu (Xidian University), Cheng Deng (A*STAR)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderTextMultimodalityAudio
🎯 What it does: Propose the GesTran framework, which utilizes LLM to convert speech into gestures and employs VQ-VAE to quantize gestures into discrete symbols for training.
LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback
Timon Ziegenbein (Leibniz University Hannover), Henning Wachsmuth (Leibniz University Hannover)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: To address inappropriate arguments in online discussions, this paper proposes a rewriting method based on reinforcement learning: first, an instruction-tuned LLM generates initial rewrites through prompting, and then utilizes existing appropriateness and semantic similarity classifiers as reward functions. The rewriting strategy is trained using PPO to ensure the generated arguments are both more appropriate and retain their original content.
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts
Helia Hashemi (Microsoft), Chris Kedzie (Microsoft)
Large Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper constructs the LLM-RUBRIC framework by first using a large language model (LLM) to ask questions based on a manually designed multi-dimensional evaluation rubric, obtaining the probability distribution for each question, then using a small feedforward network to perform personalized calibration on these distributions, ultimately predicting each human rater's scores across all questions, including overall satisfaction.
LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environments
Junzhe Chen (Tsinghua University), Lijie Wen (Tsinghua University)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: Built a multi-agent dynamic game benchmark called LLMARENA based on PettingZoo to evaluate large language models' spatial reasoning, strategic planning, numerical reasoning, risk assessment, communication, opponent modeling, and team collaboration abilities in seven different game environments.
LLMEmbed: Rethinking Lightweight LLM’s Genuine Function in Text Classification
Chun Liu (Systems Engineering Institute, AMS), Lin Yang (Systems Engineering Institute, AMS)
ClassificationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Explored the application of lightweight LLMs in text classification, proposing LLMEmbed which achieves transfer learning by extracting and fusing multi-layer LLM embeddings.
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error
Boshi Wang (Ohio State University), Yu Su (Ohio State University)
Reinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningAgentic AITextChain-of-Thought
🎯 What it does: This paper proposes a tool learning framework based on 'Simulated Trial and Error' (STE), leveraging the imagination, interaction, and memory mechanisms of LLMs to enable the model to autonomously explore and improve tool usage through API interactions.
LLMs Learn Task Heuristics from Demonstrations: A Heuristic-Driven Prompting Strategy for Document-Level Event Argument Extraction
Hanzhang Zhou (Nanyang Technological University), Kezhi Mao (Nanyang Technological University)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: In the document-level event argument extraction task, a heuristic-based linked analogy prompting strategy (HD-LoA) based on large language models (LLMs) is proposed, significantly improving few-shot reasoning performance by explicitly constructing heuristic examples and chain analogy reasoning.
LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
Mihir Parmar (Arizona State University), Chitta Baral (Arizona State University)
Supervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes the LogicBench dataset, systematically evaluating the ability of large language models to perform single logical reasoning rules (totaling 25 types, covering propositional logic, first-order logic, and non-monotonic logic) in natural language reasoning, and designs two task formats: BQA and MCQA;
LogogramNLP: Comparing Visual and Textual Representations of Ancient Logographic Writing Systems for NLP
Danlu Chen (University Of California San Diego), Taylor Berg-Kirkpatrick (University Of California San Diego)
ClassificationConvolutional Neural NetworkTransformerVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Proposed the LogogramNLP benchmark, collecting and uniformly processing four ancient logographic systems (Linear A, Egyptian hieroglyphs, cuneiform, bamboo slips), and performing NLP tasks such as machine translation, dependency parsing, and attribute classification on them.
Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models
Longze Chen (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), Min Yang (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: The ProLong framework filters documents with strong semantic long-range dependencies by measuring long dependency (LDS) on training samples, and uses these high-quality long texts for fine-tuning large language models (LLMs).
Long-Context Language Modeling with Parallel Context Encoding
Howard Yen (Princeton University), Danqi Chen (Princeton University)
RetrievalComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposed the CEPE framework, which utilizes a small encoder and cross-attention to extend the context window of large language models from 8K to over 128K while keeping the original decoder frozen.
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Yushi Bai (Tsinghua University), Juanzi Li (Tsinghua University)
RetrievalCompressionTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed and implemented LongBench—the first bilingual, multi-task, long-text understanding benchmark, covering 21 tasks, 6 categories of application scenarios, and providing a unified format with an automated evaluation process;
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
Huiqiang Jiang (Microsoft Corporation), Lili Qiu (Microsoft Corporation)
CompressionComputational EfficiencyLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes LongLLMLingua, a prompt compression method for long-text contexts, which enhances LLM performance while reducing cost and latency through question-aware coarse-to-fine compression, document re-ranking, dynamic compression ratio, and subsequence recovery.
LooGLE: Can Long-Context Language Models Understand Long Contexts?
Jiaqi Li (National Key Laboratory of General Artificial Intelligence, BIGAI), Muhan Zhang (National Key Laboratory of General Artificial Intelligence, BIGAI)
Large Language ModelTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Constructed the LooGLE benchmark for systematic evaluation of large language models' comprehension and reasoning capabilities on ultra-long texts.
LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
Hanqing Wang (Shanghai University of Finance and Economics), Maosong Sun (Tsinghua University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Designed and implemented a method called LoRA-Flow, which combines existing LoRA modules in generation tasks using dynamic fusion weights to enhance multi-task adaptation capabilities.
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Shihan Dou (Fudan University), Xuanjing Huang (Fudan University)
TransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: During the supervised fine-tuning (SFT) of large language models, the main model is frozen, and a Mixture-of-Experts (MoE) plugin composed of multi-ported experts in LoRA form is inserted at the FFN position of each Transformer layer. A router automatically assigns tasks; a local balance constraint is introduced to make some experts focus on retaining world knowledge while others focus on downstream tasks.
LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models
Jiaqi Zhao (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposes LR Quant — a post-training quantization (PTQ) framework for large language models, leveraging learnable smooth parameters and negative log cosine similarity loss to achieve efficient, robust low-bit quantization.
M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions
Zheng Wang (Huawei Technologies Co Ltd), Wei Shi (Huawei Technologies Co Ltd)
GenerationRetrievalTransformerLarge Language ModelReinforcement LearningAgentic AITextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed a multi-partition retrieval-augmented generation (M-RAG) framework, leveraging two reinforcement learning agents (Agent-S for partition selection and Agent-R for memory refinement) to enhance large language models through retrieval, significantly improving text generation quality.