Conference on Empirical Methods in Natural Language Processing Β· 593 papers
M-Wanda: Improving One-Shot Pruning for Multilingual LLMs
Rochelle Choenni (University of Amsterdam), Ivan Titov (University of Amsterdam)
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: Investigated the impact of sparsification on multilingual large language models (LLMs) and proposed an M-Wanda one-shot unstructured pruning method based on language-aware activation statistics and hierarchical sparse allocation, aiming to significantly preserve multilingual performance while drastically reducing model size.
MA-GTS: A Multi-Agent Framework for Solving Complex Graph Problems in Real-World Applications
Zike Yuan (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeOptimizationComputational EfficiencyLarge Language ModelAgentic AITextGraphBenchmarkChain-of-Thought
π― What it does: Proposed the MA-GTS multi-agent framework, which can automatically convert text descriptions from real-world scenarios into structured graph models and achieve efficient graph problem solving through multi-layer collaboration;
MADAWSD: Multi-Agent Debate Framework for Adversarial Word Sense Disambiguation
Kaiyuan Zhang (Qilu University of Technology (Shandong Academy of Sciences)), Wenpeng Lu (Qilu University of Technology (Shandong Academy of Sciences))
CodeLarge Language ModelAgentic AITextBenchmarkChain-of-Thought
π― What it does: Proposed the MADAWSD framework based on multi-agent debate, utilizing LLM agents to perform word sense disambiguation in contexts containing adversarial information;
MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning
Justin Chen, Mohit Bansal (UNC Chapel Hill)
CodeExplainability and InterpretabilityComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AITextBenchmark
π― What it does: Proposed an adaptive multi-agent coarse-to-fine hierarchical refinement framework called MAGICORE, which dynamically allocates computational resources and iteratively improves answers in LLM inference tasks based on problem difficulty.
MahΔnΔma: A Unique Testbed for Literary Entity Discovery and Linking
Sujoy Sarkar (Indian Institute of Technology Kharagpur), Pawan Goyal (Indian Institute of Technology Kharagpur)
CodeRecognitionRetrievalTransformerTextBenchmark
π― What it does: Constructed and publicly released 'MahΒ― anΒ―ma a'βa large-scale Sanskrit entity discovery and linking dataset based on the Indian epic 'MahΒ― abhΒ― arata,' covering 109K entity mentions, 5.5K unique entities, and associations with English knowledge bases;
MAKAR: a Multi-Agent framework based Knowledge-Augmented Reasoning for Grounded Multimodal Named Entity Recognition
Xinkui Lin (Chinese Academy of Sciences), Hongbo Xu (Chinese Academy of Sciences)
CodeRecognitionTransformerLarge Language ModelReinforcement LearningAgentic AIVision Language ModelMultimodalityRetrieval-Augmented GenerationChain-of-Thought
π― What it does: This paper proposes the MAKAR multi-agent framework, which leverages internal and external knowledge enhancement to achieve multi-modal named entity recognition and localization.
Making VLMs More Robot-Friendly: Self-Critical Distillation of Low-Level Procedural Reasoning
Chan Young Park (University of Washington), Yejin Choi (Stanford University)
CodeKnowledge DistillationRobotic IntelligenceVision Language ModelImageVideoChain-of-Thought
π― What it does: Propose SelfReVisionβa self-improvement framework based on a cycle of self-critique, revision, and verificationβto enhance the execution feasibility of low-capacity vision-language models in robotic program planning;
Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
Yuto Harada (NII LLMC), Yu Takagi (Nagoya Institute of Technology)
CodeData-Centric LearningLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Conduct systematic experiments on over 1,000 SFT models to investigate how training data, layers, and training methods affect the alignment quality of LLMs.
MAVL: A Multilingual Audio-Video Lyrics Dataset for Animated Song Translation
Woohyun Cho (Yonsei University), Youngjae Yu (Yonsei University)
CodeTransformerLarge Language ModelVideoTextMultimodalityBenchmarkChain-of-ThoughtAudio
π― What it does: This paper proposes the multilingual audio-video lyrics dataset MAVL, and develops the SylAVL-CoT model based on this dataset, achieving multimodal lyric translation.
Measuring Risk of Bias in Biomedical Reports: The RoBBR Benchmark
Jianyou Wang (University of California San Diego), Leon Bergen (University of California San Diego)
CodeClassificationRetrievalTransformerLarge Language ModelBiomedical DataBenchmarkRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Created the RoBBR benchmark to evaluate model performance in the task of assessing risk of bias in biomedical research, and designed a main task and two subtasks (Support Sentence Retrieval (SSR) and Support Judgment Selection (SJS)).
MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering
Teng Lin (Hong Kong University of Science and Technology), Nan Tang (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelTextTabularBenchmarkRetrieval-Augmented Generation
π― What it does: Propose MEBench, a cross-document multi-entity question answering benchmark, to evaluate the capabilities of LLMs and RAG systems in information retrieval, merging, and reasoning.
π― What it does: Propose the MedLinkDE method, which implements a two-step workflow for German MedDRA entity linking: retrieval embedding + Guided CoT re-ranking based on coding guidelines.
MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models
Zixin Chen (Beijing University of Posts and Telecommunications), Jing Ma (Hong Kong Baptist University)
CodeAgentic AIPrompt EngineeringVision Language ModelMultimodalityBenchmarkChain-of-Thought
π― What it does: Propose the MemeArena framework, which utilizes multiple agents under contextual simulation and multi-perspective fusion to conduct unbiased, context-aware evaluation of harmful content understanding in multimodal large language models.
MemeIntel: Explainable Detection of Propagandistic and Hateful Memes
Mohamed Bayan Kmainasi (Qatar Computing Research Institute), Firoj Alam
CodeClassificationExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextMultimodality
π― What it does: Constructed the MemeXplain dataset, providing labels and natural explanations for Arabic promotional memes and English hate memes; proposed a multi-stage optimization training scheme to simultaneously enhance the performance of vision-language models (VLMs) in detection and explanation generation.
Memorization or Reasoning? Exploring the Idiom Understanding of LLMs
Jisu Kim (Hanyang University), Taeuk Kim (Hanyang University)
CodeTransformerLarge Language ModelTextBenchmarkChain-of-Thought
π― What it does: Proposed the MIDAS multilingual idiom dataset and evaluated the understanding of large language models (LLMs) of idioms using multiple dimensions;
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBiomedical DataChain-of-Thought
π― What it does: Construct a Chinese social media mental health explainable instruction dataset, C-IMHI, and perform two-stage fine-tuning on the GLM open-source LLM, resulting in the MentalGLM series, which achieves multi-task mental health analysis and explainable outputs.
π― What it does: Created a verifiable, code-based benchmark called MCBench for evaluating complex instruction following, mathematical reasoning, and long-range consistency in large language models.
MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning
Jingyan Shen (New York University), Han Zhao (University of Illinois Urbana-Champaign)
CodeRecommendation SystemReinforcement Learning from Human FeedbackLarge Language ModelMixture of Experts
π― What it does: This paper proposes the MiCRo two-stage framework, which learns personalized preferences from large-scale binary preference data through hybrid modeling and context-aware routing.
Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
Zinan Tang (OpenDataLab), Lijun Wu (OpenDataLab)
CodeOptimizationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposed and implemented Middo, a closed-loop dynamic data optimization framework based on model self-diagnosis for supervised fine-tuning of large language models.
π― What it does: Propose MiLQ, the first publicly available multilingual query benchmark, and evaluate the performance of multilingual IR models in cross-lingual retrieval on this benchmark.
MIRROR: Multimodal Cognitive Reframing Therapy for Rolling with Resistance
Subin Kim (KT Corporation), Gary Lee (POSTECH)
CodeData SynthesisTransformerVision Language ModelMultimodality
π― What it does: Studied a cognitive restructuring therapy that integrates visual and textual modalities, utilizing the synthetic multimodal dataset MIRROR to train vision-language models for identifying and addressing client resistance.
Mitigating Biases in Language Models via Bias Unlearning
Dianqing Liu (University of Science and Technology of China), Zhendong Mao (State Key Laboratory of Communication Content Cognition People's Daily Online)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Propose the BiasUnlearn framework, which employs a dual-path unlearning mechanism (forgetting stereotypes + retaining anti-stereotypes) to debias large language models; meanwhile, adversarial forgetting sets and dynamic dataset exchange are used to prevent bias polarization reversal.
Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
Wei Huang (Ant Group), Yinggui Wang (Ant Group)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose a forget-aware pruning metric (FAPM) that alleviates catastrophic forgetting in LLM fine-tuning by simultaneously considering magnitude and relative change ratio on task vectors, without modifying the training process or model architecture.
Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data
Shenglai Zeng (Michigan State University), Jiliang Tang (University of Arizona)
CodeData SynthesisRetrievalSafty and PrivacyLarge Language ModelAgentic AITextBiomedical DataRetrieval-Augmented Generation
π― What it does: Propose a two-stage synthetic data generation framework called SAGE, which uses synthetic data instead of real retrieval data to reduce privacy leakage risks in RAG systems.
MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
Tuan-Luc Huynh (Monash University), Thanh-Toan Do (Monash University)
CodeRetrievalTransformerMixture of ExpertsText
π― What it does: Propose the MixLoRA-DSI framework, achieving a non-replayable, dynamically scalable generative retrieval index that appends only necessary LoRA experts when new documents are added.
MLWQ: Efficient Small Language Model Deployment via Multi-Level Weight Quantization
Chun Hu (Wuhan University), Qingan Li (Wuhan University)
CodeComputational EfficiencyTransformerText
π― What it does: Propose a multi-layer weight quantization (MLWQ) method for efficiently deploying small language models on resource-constrained devices.
MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines
Lei Gao (University of Southern California), Murali Annavaram (University of Southern California)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose MobiZO, a zeroth-order optimization-based on-device LLM fine-tuning framework, which utilizes the MP-LoRA module to achieve multi-perturbation parallelism and combines outer-loop and inner-loop parallelization, ultimately realizing efficient, low-memory on-device fine-tuning.
Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores
Ashwin Ramaswamy (Independent), Ermal Rrapaj (Lawrence Berkeley National Laboratory)
CodeLarge Language ModelTextBenchmarkChain-of-Thought
π― What it does: Propose a low-cost model intelligence estimation metric called Consistency Score by measuring the consistency (variance) of LLMs in contrast tasks.
Model-based Large Language Model Customization as Service
Zhaomin Wu (National University Of Singapore), Qiang Yang (National University Of Singapore)
CodeData SynthesisSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelTabular
π― What it does: Propose a framework called Llamdex, allowing clients to upload pre-trained domain expert models instead of sensitive data, thereby enabling customized services for large language models (LLMs).
Molecular String Representation Preferences in Pretrained LLMs: A Comparative Study in Zero- & Few-Shot Molecular Property Prediction
George Arthur Baker (University of Colorado Boulder), Katharina von der Wense (University of Colorado Boulder)
CodeRepresentation LearningDrug DiscoveryTransformerLarge Language ModelBiomedical DataBenchmarkChain-of-Thought
π― What it does: Evaluate the zero-shot and few-shot reasoning performance of four mainstream large language models in molecular property prediction tasks using five molecular string representations: SMILES, DeepSMILES, SELFIES, InChI, and IUPAC names.
MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval
Xixi Wu (Chinese University of Hong Kong), Hong Cheng (Chinese University of Hong Kong)
CodeRetrievalGraph Neural NetworkVision Language ModelMultimodalityGraphRetrieval-Augmented Generation
π― What it does: Propose MoLoRAG, a logic-aware multimodal multi-page document retrieval framework, which achieves efficient document page retrieval using graph traversal methods and feeds the retrieval results into an LVLM for question answering.
CodeRetrievalMixture of ExpertsTextBenchmarkRetrieval-Augmented Generation
π― What it does: Designed and implemented the Mixture of Retrievers (MoR) framework, which dynamically assigns and fuses results from multiple sparse/dense retrievers or even human retrievers for each query in retrieval-augmented generation (RAG) tasks.
π― What it does: Constructed and annotated a new 'Moral Framework' corpus (MFiP) from German parliamentary debate texts, with fine-grained labels for moral framework types, narrative roles, and moral foundations for each text segment; trained and evaluated models for framework identification, type classification, and moral foundation classification based on this corpus, further exploring the effectiveness of data augmentation and contrastive learning in this task.
MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations
Genglin Liu (University of California, Los Angeles), Saadia Gabriel (University of California, Los Angeles)
CodeExplainability and InterpretabilityTransformerLarge Language ModelAgentic AITextTabular
π― What it does: Proposed and implemented MOSAICβa multi-agent social network simulation framework based on large language models (LLMs)βto study content propagation, user engagement, and misinformation diffusion;
MoVa: Towards Generalizable Classification of Human Morals and Values
Ziyu Chen (Australian National University), Lexing Xie (Australian National University)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: This study constructs the MoVa resource, aiming to perform generalizable classification of human morality and values in text, and provides a multi-framework, multi-dataset evaluation benchmark.
MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs
Chong Jun Rong Brian (National University of Singapore), Anthony Kum Hoe Tung (National University of Singapore)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Designed and implemented a multi-round character-conditioned generation framework, MPCG, using large language models (LLMs) to simulate the gradual evolution and reconstruction of misinformation across different political stances.
MUCAR: Benchmarking Multilingual Cross-Modal Ambiguity Resolution for Multimodal Large Language Models
Xiaolong Wang (Tsinghua University), Yang Liu (Tsinghua University)
CodeAgentic AIPrompt EngineeringVision Language ModelMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper constructs MUCAR, a multilingual, cross-modal ambiguity resolution benchmark, evaluates and compares the ambiguity resolution performance of 19 large language models in image-text context, and proposes an explicit reasoning framework based on agents.
Multi-Document Event Extraction Using Large and Small Language Models
Qingkai Min (Zhejiang University), Yue Zhang (Tsinghua University)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Proposed a collaborative framework that first uses a small language model (SLM) to complete local subtasks such as event triggering, coreference, and argument extraction, and then employs a large language model (LLM) for multi-step reasoning, ultimately achieving cross-document event aggregation and normalization.
Nitay Calderon (Technion), Roi Reichart (Technion)
CodeExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a fully automated multi-domain preference explanation framework: first, using LLM to discover concepts and generate concept definitions, then representing each query-reply triplet as a concept vector, followed by learning shared and domain-specific concept weights through a hierarchical multi-domain regression (HMDR) model, thereby providing local and global explanations for 12 preference mechanisms such as human preferences, LLM judgments, and reward models.
Multi-Frequency Contrastive Decoding: Alleviating Hallucinations for Large Vision-Language Models
Bingqian Liu (Northeastern University), Jingwei Cheng (Northeastern University)
CodeExplainability and InterpretabilityVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: Propose the Multi-Frequency Contrastive Decoding (MFCD) method, leveraging the differences between high-frequency and low-frequency image features to suppress object hallucinations in large vision-language models.
Multi-perspective Analysis of Large Language Model Domain Specialization: An Experiment in Accounting Audit Procedures Generation
Yusuke Noro
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextFinance RelatedRetrieval-Augmented Generation
π― What it does: This paper conducts experiments on the accounting audit procedure generation task (AAPG), comparing three domain-specialization methods: supervised fine-tuning (SFT-IT, SFT-CV), context learning (ICL), and combinations of both, to explore their differences in text generation characteristics.
Multi-view-guided Passage Reranking with Large Language Models
Jeongwoo Na (Sungkyunkwan University), Jongwuk Lee (Sungkyunkwan University)
CodeRetrievalComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a non-generative large language model re-ranker called MVP based on multi-perspective soft prompts and anchor decoding, to efficiently evaluate all candidate paragraphs in one go.
Multilingual Prompting for Improving LLM Generation Diversity
Qihan Wang (New York University), Emily Black (New York University)
CodeGenerationTransformerPrompt EngineeringText
π― What it does: Propose a multilingual prompting method that significantly enhances the diversity of LLM-generated outputs by repeatedly asking questions under different language and cultural contexts and aggregating the answers.
MultiLogicNMR(er): A Benchmark and Neural-Symbolic Framework for Non-monotonic Reasoning with Multiple Extensions
Yeliang Xiu (Sun Yat-sen University), Yongmei Liu (Sun Yat-sen University)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: Constructed the multi-extension non-monotonic reasoning dataset MultiLogicNMR and its OOD and NL variants, and proposed the neural-symbolic framework MultiLogicNMRer.
MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification
Iustin Sirbu (National University of Science and Technology POLITEHNICA Bucharest), Traian Rebedea (National University of Science and Technology POLITEHNICA Bucharest)
CodeClassificationTransformerTextMultimodality
π― What it does: Propose MultiMatch, a semi-supervised text classification framework combining multi-head co-training, consistency regularization, and pseudo labels, and design a pseudo label weighting module for selection, filtering, and weighting.
MultiMed-ST: Large-scale Many-to-many Multilingual Medical Speech Translation
Khai Le-Duc (University of Toronto), Thanh Nguyen-Tang (New Jersey Institute of Technology)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical DataAudio
π― What it does: The paper systematically constructs a large-scale multilingual medical speech translation dataset called MultiMed-ST and conducts in-depth experiments and analysis in this field.
π― What it does: Construct the MFCIG-CSS system, which models word-level semantics and prosody in multi-modal dialogue history using two fine-grained interaction graphs (Semantic Interaction Graph SIG and Prosody Interaction Graph PIG), and injects the encoded interaction features into the speech synthesizer to generate more natural and conversation-like speech.
MULTIVOX: A Benchmark for Evaluating Voice Assistants for Multimodal Interactions
Ramaneswaran Selvakumar (University of Maryland), Dinesh Manocha (University of Maryland)
CodeVision Language ModelImageVideoMultimodalityBenchmarkAudio
π― What it does: Propose the MULTIVOX benchmark, collecting 1000 professionally recorded voice queries with corresponding images/videos to evaluate the context-awareness and answer quality of multimodal language models in integrating speech and vision.
MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
Siyu Yan (East China Normal University), Chenjuan Guo (East China Normal University)
CodeSafty and PrivacyAdversarial AttackReinforcement Learning from Human FeedbackLarge Language ModelText
π― What it does: Proposed the MUSE framework, which includes MUSE-A (based on framework semantics and MCTS) for multi-round jailbreak attacks, and MUSE-D (fine-grained security alignment) for multi-round security defense;
Ali Sarosh Bangash (University of South Florida), Raiyan Abdul Baten (University of South Florida)
CodeTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Proposes MUSESCORER, a fully automated and scalable originality scoring system that leverages LLMs as judges and external retrieval to perform bucketing of creative responses and calculate frequency-based originality scores.
MusKGC: A Flexible Multi-source Knowledge Enhancement Framework for Open-World Knowledge Graph Completion
Xin Song (National University of Defense Technology), Bin Zhou (National University of Defense Technology)
CodeTransformerLarge Language ModelTextGraphRetrieval-Augmented Generation
π― What it does: Proposed a multi-source knowledge-enhanced framework called MusKGC based on large language models to complete knowledge graphs under the open-world assumption;
π― What it does: Proposed a non-verbal speech separation representation learning framework called N-CORE based on multi-view consistency regularization, which can separate emotional and speaker information in non-verbal speech.
CodeAI Code AssistantTransformerLarge Language ModelAgentic AIPrompt EngineeringTextBenchmark
π― What it does: Propose the NESTFUL benchmark to evaluate the performance of large language models on nested API call sequences, covering over 1800 executable multi-level nested calls.
Neural Topic Modeling via Contextual and Graph Information Fusion
Jiyuan Liu (Sun Yat-sen University), Yanghui Rao (Sun Yat-sen University)
CodeRepresentation LearningAuto EncoderTextGraph
π― What it does: This paper proposes a variational autoencoder framework (CGTM) that integrates contextual and graph information for unsupervised topic modeling.
NILE: Internal Consistency Alignment in Large Language Models
Minda Hu (Chinese University of Hong Kong), Irwin King (Chinese University of Hong Kong)
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Propose the NILE framework, which extracts, revises, and filters the IFT dataset using internal knowledge to enhance the fine-tuning effectiveness of LLMs.
NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communities
Abdellah El Mekki (University of British Columbia), Muhammad Abdul-Mageed (University of British Columbia)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
π― What it does: Propose a multi-stage framework that generates pre-trained corpora aligned with Egyptian and Moroccan Arabic dialects and their cultural values through machine translation, controlled synthesis generation, and retrieval-augmented data augmentation, and trains a 3B-parameter NileChat LLM based on this data.
Non-Existent Relationship: Fact-Aware Multi-Level Machine-Generated Text Detection
Yang Wu (China Telecom Cloud Computing Research Institute), Jie Wu (China Telecom Cloud Computing Research Institute)
CodeClassificationGraph Neural NetworkTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: This paper proposes a multi-layer feature fusion model (FAML) based on the authenticity of entity relationships for detecting machine-generated text.
Nullspace Disentanglement for Red Teaming Language Models
Yi Han (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
CodeAdversarial AttackTransformerLarge Language ModelText
π― What it does: Proposed a black-box red teaming technique called NDR based on nullspace, which can separate and reconstruct success information and semantic information from test cases to generate more aggressive test cases.
NUTMEG: Separating Signal From Noise in Annotator Disagreement
Jonathan Ivey (Johns Hopkins University), David Jurgens (University of Michigan)
CodeData-Centric LearningTransformerText
π― What it does: Propose NUTMEG, a Bayesian model that separates noise and systematic disagreements based on annotator background information, generating subgroup-level true labels; subsequently, these labels are used to learn from disagreement, improving downstream task performance.
OG-RAG: Ontology-grounded retrieval-augmented generation for large language models
Kartik Sharma (Georgia Institute of Technology), Yunqing Li (Lenovo)
CodeRetrievalTransformerLarge Language ModelTextGraphAgriculture RelatedRetrieval-Augmented Generation
π― What it does: Propose OG-RAG, an ontology-based retrieval-augmented generation framework that constructs hypergraphs using domain ontologies and retrieves minimal hyperedge contexts during LLM generation;
π― What it does: Built a comprehensive RAG evaluation benchmark called OmniEval for the financial domain, covering task-topic matrices, automated data generation, phased evaluation, and multi-layer metrics;
π― What it does: Propose the OmniThink framework, which simulates human slow thinking processes in machine writing. It utilizes information trees and concept pools for information acquisition, reflection, and expansion, ultimately generating high-quality long-form articles.
CodeTransformerLarge Language ModelReinforcement LearningText
π― What it does: Proposed a single policy planner PADPP that is adaptable to multi-objective, goal-oriented dialogues, capable of dynamically adjusting goal weights during inference without requiring retraining.
π― What it does: This paper proposes OpenNER 1.0, a standardized named entity recognition dataset collection containing 36 manually annotated datasets across 52 languages;
π― What it does: Constructed the OpenTuringBench benchmark and the OTBDetector framework for training and evaluating the detection and authorship attribution of text generated by open large language models (OLLM).
OWL: Probing Cross-Lingual Recall of Memorized Texts via World Literature
Alisha Srivastava (University of Massachusetts), Mohit Iyyer (University of Maryland)
CodeData SynthesisRetrievalTransformerLarge Language ModelTextMultimodalityBenchmarkAudio
π― What it does: Constructed the OWL dataset, aligning 20 English novels across 10 languages (including 6 low-resource languages), and evaluated LLMs' multilingual and cross-lingual memory capabilities through three tasks (direct retrieval, name filling, and prefix continuation).
PakBBQ: A Culturally Adapted Bias Benchmark for QA
Abdullah Hashmat (Lahore University of Management Sciences), Agha Ali Raza (Lahore University of Management Sciences)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Constructed the PakBBQ datasetβa QA bias benchmark tailored for Pakistani culture, containing 214 templates and 17,180 Chinese-English QA pairs.
π― What it does: Propose parallel continuous chain-of-thought reasoning (PCCoT), which improves the training and inference efficiency of continuous CoT by parallelizing the update of implicit thought tokens through Jacobi iteration.
Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning
Senjie Jin (Fudan University), Xuanjing Huang (Fudan University)
CodeReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Designed the Parrot training pipeline, enhancing the reasoning performance of program chains and natural language chains through three collaborative training steps: information retrieval, program reasoning, and paradigm conversion.
PERSEVAL: A Framework for Perspectivist Classification Evaluation
Soda Marem Lo (University of Turin), Davide Bernardi (Amazon)
CodeClassificationTransformerPrompt EngineeringMixture of ExpertsText
π― What it does: Propose the PERSEVAL framework for unified evaluation of perspectivist text classification models, focusing on the separation between annotators during training and test users, and evaluating at four levels: global, text, user, and feature.
CodeClassificationAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Studied the conflicts between achieving personalized content moderation on social media and legal boundaries, and proposed a framework that ensures legal compliance through boundary constraint mechanisms.
Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PD
Bryan Chen Zhengyu Tan (Singapore University of Technology and Design), Roy Ka-Wei Lee (Singapore University of Technology and Design)
CodeSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose the DuET-PD framework to evaluate stance shifts of LLMs in multi-round persuasive dialogues, and develop the Holistic DPO training method to enhance robustness and adaptability against misleading and corrective persuasion.
Phi: Preference Hijacking in Multi-modal Large Language Models at Inference Time
Yifan Lan (Pennsylvania State University), Jinghui Chen (Pennsylvania State University)
CodeOptimizationAdversarial AttackTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
π― What it does: This paper studies attack methods that achieve preference hijacking by optimizing images during inference in multimodal large language models.
Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis
Haoming Huang (Hong Kong University of Science and Technology), Xuming Hu (Hong Kong University of Science and Technology)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Proposed and implemented the PHANTOMCIRCUIT framework for systematic analysis and detection of knowledge obscuring phenomena in large language models.
PIIvot: A Lightweight NLP Anonymization Framework for Question-Anchored Tutoring Dialogues
Matthew Zent (Eedi), Simon Woodhead
CodeSafty and PrivacyData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Developed the PIIvot framework, which leverages potential PII tagging and context-aware LLM alternatives to achieve lightweight PII anonymization, and released the largest real-world Q&A tutoring dialogue dataset QATD 2k based on this.
Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts
Michal Golovanevsky (Brown University), Carsten Eickhoff (University of TΓΌbingen)
CodeExplainability and InterpretabilityPrompt EngineeringVision Language ModelMultimodality
π― What it does: This paper proposes the Visual CounterFact dataset and designs the Pixels Versus Priors (PvP) activation-level intervention method to control the dependency of multimodal large language models between visual information and world knowledge priors.
PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models
Lorenzo Molfetta (University of Bologna), Gianluca Moro (University of Bologna)
CodeRetrievalTransformerLarge Language ModelContrastive LearningTextBenchmark
π― What it does: Developed a training framework for tool retrieval based on LLM called PORTS, designed to pre-select the most relevant tool documents for queries.
POSITION BIAS MITIGATES POSITION BIAS: Mitigate Position Bias Through Inter-Position Knowledge Distillation
Yifei Wang (State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences), Daniel Dajun Zeng (State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences)
π― What it does: Propose the Pos2Distill framework, which mitigates position bias (PB) in large models through inter-positional knowledge distillation, and designs two instances, Pos2Distill-R1 and Pos2Distill-R2, for retrieval and reasoning tasks, respectively.
Pragmatic Inference Chain (PIC) Improving LLMsβ Reasoning of Authentic Implicit Toxic Language
Xi Chen (Nanyang Technological University), Shuo Wang (University of Macau)
CodeClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: Developed a 'Pragmatic Inference Chain (PIC)' prompting method based on pragmatics and linguistics, and created a real-world dataset containing 3,097 Chinese examples of implicit harmful language to train and evaluate large language models (LLMs) in reasoning about implicit harmful language.
Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers
Marek KadlΔΓk (Faculty of Informatics, Masaryk University), Michal Spiegel (Faculty of Informatics, Masaryk University)
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Investigate the precision of numerical embeddings in pre-trained language models and propose a sine-based probe to achieve near-perfect numerical decoding.
Precise In-Parameter Concept Erasure in Large Language Models
Yoav Gur-Arieh (Tel Aviv University), Mor Geva (Tel Aviv University)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelAuto EncoderText
π― What it does: Proposes the PISCES method, which directly locates and precisely removes knowledge of specified concepts in the parameter space of language models.
Predicate-Guided Generation for Mathematical Reasoning
Jiajun Chen (New York University), Yik-Cheung Tam (New York University)
CodeGenerationData SynthesisAI Code AssistantTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
π― What it does: Constructed the Prolog-MATH dataset using a two-phase automated process to first generate mathematical predicates and then synthesize complete Prolog programs, further enhancing problem-solving coverage through GRPO reinforcement learning.
PRIM: Towards Practical In-Image Multilingual Machine Translation
Yanzhi Tian (Beijing Institute of Technology), Yuhang Guo (Beijing Institute of Technology)
CodeImage TranslationTransformerVision Language ModelImageTextMultimodality
π― What it does: Studied multilingual image translation in real-world scenarios, proposed the first real-world multilingual IIMT dataset PRIM, and designed an end-to-end model called VisTrans;
PrimeX: A Dataset of Worldview, Opinion, and Explanation
Rik Koncel-Kedziorski (Apple), Tim Paek (Apple)
CodeData-Centric LearningTransformerLarge Language ModelTextBenchmark
π― What it does: Constructed and released the PRIMEX dataset, containing public opinions, free-text explanations, and Primal World Beliefs survey results from 858 American respondents.
CodeKnowledge DistillationRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsTextBenchmark
π― What it does: This study constructs a complete dataset system named PRIMUS, covering three stages of LLM training in the field of cybersecurity: pre-training, instruction fine-tuning, and reasoning distillation. Continuous pre-training, instruction fine-tuning, model fusion, and reasoning distillation were implemented based on Llama-3.1-8B-Instruct, ultimately releasing a series of LLMs tailored for cybersecurity.
Probabilistic Soundness Guarantees in LLM Reasoning Chains
Weiqiu You (University of Pennsylvania), Eric Wong (University of Pennsylvania)
CodeExplainability and InterpretabilityTransformerLarge Language ModelTextBenchmarkChain-of-Thought
π― What it does: Proposes Autoregressive Reasoning Entailment Stability (ARES), a probabilistic reasoning framework that evaluates the credibility of each step in the LLM's generated reasoning chain and provides statistical safety guarantees.
Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Induction
Jinwook Park (Gwangju Institute of Science and Technology), Kangil Kim (Gwangju Institute of Science and Technology)
CodeRepresentation LearningText
π― What it does: Propose the probability distribution collapse (PDC) problem and design a collapse-relaxing neural parameterization (CRNP) to alleviate this bottleneck, achieving more compact and accurate unsupervised neural syntax induction;
Probing and Boosting Large Language Models Capabilities via Attention Heads
Dezhi Zhao (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
π― What it does: Explore the correspondence between attention heads in large language models and five fundamental capabilities (mathematical reasoning, reading comprehension, common-sense reasoning, scientific reasoning, professional expertise), and utilize this correspondence for targeted instruction fine-tuning.
Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding
Yun-Shiuan Chuang (University of Wisconsin-Madison), Timothy T. Rogers (University of Wisconsin-Madison)
CodeTransformerLarge Language ModelWorld ModelTextBenchmarkChain-of-Thought
π― What it does: Constructed three guesstimation datasets (MARBLES, FUTURE, ELECPRED) and evaluated the estimation capabilities of large language models on these tasks.
Probing Logical Reasoning of MLLMs in Scientific Diagrams
Yufei Wang (University of Pittsburgh), Adriana Kovashka (University of Pittsburgh)
CodeData SynthesisExplainability and InterpretabilityLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelMultimodalityBenchmarkChain-of-Thought
π― What it does: Constructed a visual question-answering dataset based on food web/food chain images, and generated a large number of logical reasoning questions using seven logical templates to evaluate the logical reasoning ability of multimodal large language models on scientific charts.
Procedural Environment Generation for Tool-Use Agents
Michael Sullivan (Saarland University), Alexander Koller (Saarland University)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: Propose a pipeline named RandomWorld for automatically generating synthetic environments containing interactive tools and nonlinear composite tasks to train LLM tool-use agents.
ProLongVid: A Simple but Strong Baseline for Long-context Video Instruction Tuning
Rui Wang (Fudan University), Yu-Gang Jiang (Fudan University)
CodeData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoText
π― What it does: Construct a large-scale long video instruction dataset and propose a progressive video instruction fine-tuning strategy to achieve efficient understanding of long videos by multimodal large models.
Pun Unintended: LLMs and the Illusion of Humor Understanding
Alessandro Zangari (Ca' Foscari University of Venice), Jose Camacho-Collados (Cardiff University)
CodeRecognitionExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: The paper systematically evaluates the robustness and interpretability of large language models in identifying and explaining puns by constructing new pun evaluation datasets (PunnyPattern and PunBreak) and improving existing datasets.