EMNLP 2024 Papers — Page 12
Conference on Empirical Methods in Natural Language Processing · 1268 papers
The Empirical Variability of Narrative Perceptions of Social Media Texts
Joel Mire (Carnegie Mellon University), Maarten Sap (Carnegie Mellon University)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Collected and analyzed 2,496 narrative perception data points from 255 American participants regarding 502 social media texts, constructing a dataset named STORYPERCEPTIONS, and generated 30 descriptive features and discourse codes through open coding.
The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark and Fact-Augmented Intervention
Yixin Wan (University of California, Los Angeles), Kai-Wei Chang (University of California, Los Angeles)
GenerationData SynthesisLarge Language ModelPrompt EngineeringDiffusion modelImageTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Investigated the distortion of racial and gender distributions of historical figures (i.e., 'fact tax') caused by diversity interventions in text-to-image (T2I) models, and proposed a benchmark called DoFaiR to measure this issue, while introducing a method called Fact-Augmented Intervention (FAI) to enhance factual accuracy under diversity interventions.
The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models
Siyang Liu (University of Michigan), Rada Mihalcea (University of Michigan)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: By systematically asking and collecting answers from different age groups (18–24, 25–34, 35–44, 45–54, 55–64, 65+) and LLMs' responses to questionnaire items from the World Value Survey (WVS) covering 13 value categories, the study quantifies the value gap between LLMs and age groups, and evaluates the effectiveness of incorporating age identity information in prompts to mitigate this gap.
The Greatest Good Benchmark: Measuring LLMs’ Alignment with Utilitarian Moral Dilemmas
Giovanni Franco Gabriel Marraffini (Universidad De Buenos Aires), Luciano Del Corro (Universidad De Buenos Aires)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: By constructing the 'Greatest Good Benchmark (GGB)', the Oxford Utilitarianism Scale (OUS) was expanded and adapted for large language models (LLMs), evaluating and comparing the judgments of 15 LLMs of different scales and origins in moral dilemmas.
The Illusion of Competence: Evaluating the Effect of Explanations on Users’ Mental Models of Visual Question Answering Systems
Judith Sieker (Bielefeld University), Sina Zarrieß (Honda Research Institute Europe)
Explainability and InterpretabilityVision Language ModelMultimodality
🎯 What it does: Conduct experimental research on whether users can more accurately build mental models of a system's capabilities and limitations when receiving natural language explanations in visual question-answering systems. The experiment controls system inputs (color vs. grayscale images) to artificially induce model defects, comparing two experimental settings with and without explanations.
The Instinctive Bias: Spurious Images lead to Illusion in MLLMs
Tianyang Han (Hong Kong Polytechnic University), Tong Zhang (University of Illinois at Urbana-Champaign)
Data SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Study and quantify the 'instinctive bias' (visual hallucination) in multimodal large language models (MLLMs) when confronted with pseudo-relevant images, and construct the first such benchmark—CorrelationQA.
The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?
Alexander S. Choi (George Mason University), Antonios Anastasopoulos (George Mason University)
TransformerLarge Language ModelText
🎯 What it does: Conduct topic modeling experiments on AI policy documents to evaluate the interactive effects of efficiency gains and cognitive biases when experts collaborate with large language models (LLMs).
The Lou Dataset - Exploring the Impact of Gender-Fair Language in German Text Classification
Andreas Waldis (Technical University of Darmstadt), Iryna Gurevych (Lucerne University of Applied Sciences and Arts)
ClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper constructs the Lou dataset to systematically evaluate the impact of gender-fair language (Gender-Fair Language) on German text classification tasks.
The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
Aakanksha (Cohere For AI), Sara Hooker (Cohere For AI)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Constructed and released the first multilingual red-teaming dataset (Aya Red-teaming), and compared alignment techniques such as supervised fine-tuning (SFT) and offline preference optimization (DPO) across six languages, evaluating their effectiveness in mitigating global and local harms.
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis
Yuxiang Zhou (King's College London), Yulan He (King's College London)
Explainability and InterpretabilityTransformerLarge Language ModelTextReview/Survey Paper
🎯 What it does: This paper systematically reviews the theoretical mechanisms and empirical factors of in-context learning (ICL) in large language models, providing a comprehensive framework from mechanisms to factors.
The Mystery of the Pathological Path-star Task for Language Models
Arvid Frydenlund (University of Toronto)
Explainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelGraph
🎯 What it does: Investigated the learning challenges of the path-star task and conducted experiments on various models and training methods to explore the impact of teacher forcing, causal constraints, and graph representations on model performance.
The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning
Shaobo Cui (EPFL), Boi Faltings (EPFL)
Review/Survey PaperBenchmark
🎯 What it does: This paper reviews the classification, benchmarks, acquisition methods, qualitative and quantitative reasoning techniques for common-sense causality, and proposes future research directions.
The Zeno’s Paradox of ‘Low-Resource’ Languages
Hellina Hailu Nigatu (UC Berkeley), Monojit Choudhury (MBZUAI)
TextReview/Survey Paper
🎯 What it does: Conducted a qualitative analysis of 150 papers involving 'low-resource languages,' extracting four dimensions defining 'low-resource languages' and proposing corresponding terminology recommendations.
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability
Xinyu Hu (Peking University), Xiaojun Wan (Peking University)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose a large-scale no-reference NLG evaluation corpus, NLG-Eval, and train a specialized evaluation LLM, Themis, to achieve flexible and interpretable assessments.
TheoremLlama: Transforming General-Purpose LLMs into Lean4 Experts
Ruida Wang (Hong Kong University of Science and Technology), Tong Zhang (University of Illinois Urbana Champaign)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Train a general-purpose large language model to become an expert in Lean4 and construct an NL-FL aligned dataset
Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting
Stephen Meisenbacher (Technical University of Munich), Florian Matthes (Technical University of Munich)
Safty and PrivacyData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper conducts a case study on DP-PROMPT, comparing strict differential privacy (DP), loose DP (Quasi-DP), and non-DP text rewriting methods, evaluating their semantic similarity, readability, and privacy protection effects.
Thoughts to Target: Enhance Planning for Target-driven Conversation
Zhonghua Zheng (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
GenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes the EnPL two-stage framework, first leveraging LLM to extract natural language plans from goal-driven dialogue corpora and constructing the ConvPlan dataset through entity consistency filtering; then generating high-quality dialogue plans for new goals via example-driven context learning.
Threshold-driven Pruning with Segmented Maximum Term Weights for Approximate Cluster-based Sparse Retrieval
Yifan Qiao (University of California at Santa Barbara), Tao Yang (University of California at Santa Barbara)
RetrievalComputational EfficiencyText
🎯 What it does: This paper proposes ASC ((µ,η)-approximate control scheme), a cluster-level pruning method based on threshold-driven, segmented maximum term weight, which can significantly accelerate retrieval in sparse retrieval while maintaining high relevance.
TimeR^4 : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering
Xinying Qian (Tiangong University), Kehui Song (Tiangong University)
TransformerSupervised Fine-TuningPrompt EngineeringContrastive LearningGraphTime SeriesRetrieval-Augmented Generation
🎯 What it does: Designed and implemented a time-aware retrieval-rewrite-retrieval-re-ranking (TimeR4) framework, which injects temporal constraints from time knowledge graphs (TKG) into large language models through retrieval and rewriting strategies to enhance their reasoning capabilities in time knowledge graph question answering (TKGQA).
TinyChart: Efficient Chart Understanding with Program-of-Thoughts Learning and Visual Token Merging
Liang Zhang (Renmin University of China), Fei Huang (Alibaba Group)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: Developed TinyChart, a multimodal large language model with only 3B parameters, for efficient chart understanding.
TKGT: Redefinition and A New Way of Text-to-Table Tasks Based on Real World Demands and Knowledge Graphs Augmented LLMs
Peiwen Jiang (Shanghai Jiao Tong University), Jinhua Cheng (Shanghai Jiao Tong University)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextTabularBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper redefines the text-to-table task in the long-text domain, proposing a two-stage TKGT pipeline: first generating domain-specific knowledge graph classes using hybrid IE (rule-based, statistical, and deep learning), then utilizing Hybrid-RAG with dynamic prompting to guide LLMs to populate tables based on KG classes; additionally, it constructs a challenging CPL dataset with legal judgment texts.
TL-CL: Task And Language Incremental Continual Learning
Shrey Satapara (Indian Institute of Technology Hyderabad), P. K. Srijith (Indian Institute of Technology Hyderabad)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the task and language incremental continual learning (TLCL) problem, proposed Task and Language-Specific Adapters (TLSA) adapter solution, and compared various continual learning methods under a multi-task and multi-language setting.
To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models
Junyan Lin (Eastern Institute of Technology), Xiaoyu Shen (Eastern Institute of Technology)
TransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Investigate the impact of different types of connectors (feature-preserving and feature-compressing) on the performance of multi-modal large language models (MLLMs) in coarse-grained perception, fine-grained perception, and reasoning tasks, and provide systematic experimental guidelines.
To Word Senses and Beyond: Inducing Concepts with Contextualized Language Models
Bastien Liétard (University of Lille), Mikaela Keller (University of Lille)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Propose an unsupervised concept induction task, leveraging two-layer clustering to learn concept sets of words from context
Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
Sheridan Feucht (Northeastern University), David Bau (Northeastern University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: This paper studies how multi-subword tokens in large language models are combined into meaningful lexical units, proposing to infer the model's latent vocabulary by observing the 'erasure' effect of the final token in multi-token words.
Tokenization Is More Than Compression
Craig W Schmidt, Chris Tanner (Kensho Technologies)
CompressionRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: This paper introduces the PATHPIECE tokenizer and systematically evaluates the impact of each stage of tokenization (pre-tokenization, vocabulary construction, segmentation) on LLM training and downstream task performance, verifying that compression rate is not a decisive factor;
TokenVerse: Towards Unifying Speech and NLP Tasks via Transducer-based ASR
Shashi Kumar (Idiap Research Institute), Aravind Ganapathiraju (Uniphore)
ClassificationRecognitionTransformerTextFinance RelatedAudio
🎯 What it does: Propose TokenVerse, a unified model based on the Transducer architecture, capable of simultaneously performing automatic speech recognition (ASR), speaker change detection (SCD), semantic endpoint detection (ENDP), and named entity recognition (NER).
ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models
Yuxiang Zhang (Waseda University), Hayato Yamana (Waseda University)
Large Language ModelTextBenchmark
🎯 What it does: Proposed the ToolBH benchmark for multi-level diagnosis of tool-enhanced hallucination issues in large models;
ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback
Qinzhuo Wu (XiaoMi AI Lab), Bin Wang (XiaoMi AI Lab)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITextBenchmark
🎯 What it does: Construct a multi-granularity instruction dataset MGToolBench and propose a two-stage reinforcement learning framework ToolPlanner to enhance task completion and instruction following capabilities of tool-augmented LLMs under multi-granularity instructions.
Tools Fail: Detecting Silent Errors in Faulty Tools
Jimin Sun (CohereAI), Yonatan Bisk (Carnegie Mellon University)
Anomaly DetectionLarge Language ModelPrompt EngineeringTextMultimodalityChain-of-Thought
🎯 What it does: This paper constructs a tool error classification framework and conducts a systematic study on detecting tool 'silent errors' (i.e., tool failures without explicit error signals) in LLMs, proposing three adversarial strategies (declaration, confidence, checklist), and verifies their effectiveness in arithmetic calculator and multimodal robot tasks.
Topic-Oriented Open Relation Extraction with A Priori Seed Generation
Linyi Ding (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
ClassificationTransformerLarge Language ModelPrompt EngineeringTextBiomedical Data
🎯 What it does: Proposes a zero-shot topic-oriented open relation extraction method called PriORE, which improves the quality of relation extraction by leveraging prior seed generation and dynamic expansion of the dictionary.
TopViewRS: Vision-Language Models as Top-View Spatial Reasoners
Chengzu Li (University of Cambridge), Ivan Vulić (University of Cambridge)
ClassificationRecognitionPrompt EngineeringVision Language ModelImageTextBenchmarkChain-of-Thought
🎯 What it does: Propose the TOPVIEWRS dataset to evaluate the spatial reasoning capabilities of vision-language models (VLMs) from top-down views, and conduct fine-grained assessment through four progressively complex tasks (identification, localization, static reasoning, dynamic reasoning).
Toward Compositional Behavior in Neural Models: A Survey of Current Views
Kate McCurdy (Universität des Saarlandes), Jianfeng Gao (Microsoft Research)
TransformerTextReview/Survey Paper
🎯 What it does: Proposed a conceptual framework for combinatorial behavior (CB) in neural models, designed and conducted a survey among active researchers, and systematically organized and quantified the academic community's consensus and divergences regarding CB definitions, evaluation methods, and implementation approaches.
Towards a Greek Proverb Atlas: Computational Spatial Exploration and Attribution of Greek Proverbs
John Pavlopoulos (Athens University of Economics and Business), Panagiotis Filos (Athens University of Economics and Business)
ClassificationTransformerTextBenchmark
🎯 What it does: Constructed the first public, machine-processable large-scale dataset of Greek proverbs, and performed computational spatial exploration of their geographic distribution; achieved prediction of proverb location attribution through text classification and regression algorithms, and localized unattributed proverbs;
Towards a Similarity-adjusted Surprisal Theory
Clara Meister (ETH Zürich), Tiago Pimentel (ETH Zürich)
TransformerLarge Language ModelText
🎯 What it does: Proposed 'similarity-regulated surprise'—incorporating word similarity into traditional surprise to measure predictability in context, thereby enabling a more refined quantification of sentence comprehension cost.
Towards Aligning Language Models with Textual Feedback
Saüc Abadal Lloret (ETH Zurich), Mrinmaya Sachan (ETH Zurich)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the ALT (ALignment with Textual feedback) method, which aligns language models using textual feedback, directly achieved through conditional language model training
Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs
Simone Conia (Sapienza University of Rome), Yunyao Li (Adobe)
GenerationTransformerContrastive LearningTextGraphBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes evaluation and improvement methods for cross-cultural machine translation, constructing a benchmark dataset XC-Translate specifically for texts containing culturally diverse entity names, and introduces the KG-MT translation framework that utilizes multilingual knowledge graphs for retrieval-augmented generation;
Towards Difficulty-Agnostic Efficient Transfer Learning for Vision-Language Models
Yongjin Yang (KAIST AI), Se-Young Yun (KAIST AI)
Domain AdaptationRepresentation LearningPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: The study investigates efficient transfer learning of vision-language models under different transfer difficulties, proposing the APEX method that combines visual prompts (VPT) + text adapters (TA) + adaptive weighted ensemble to achieve automatic adjustment for target domains.
Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs
Mihir Parmar (Arizona State University), Trung Bui (Adobe Research)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed an extractive summary dataset containing natural language user intent feedback, and used this dataset to fine-tune large language models to improve the coherence of summaries.
Towards Faithful Knowledge Graph Explanation Through Deep Alignment in Commonsense Question Answering
Weihe Zhai (Harbin Institute of Technology), Yalong Zhao (XtalPi Innovation Center, XtalPi Beijing)
Explainability and InterpretabilityGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: This paper studies how to improve the interpretability of language model and knowledge graph fusion models in commonsense question answering, proposing the LKDA training framework and introducing the Fidelity metric to quantify the interpretability of graph neural networks.
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
Euiin Yi (KAIST AI), Se-Young Yun (KAIST AI)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a voting-based reasoning method that combines small, language-specific 'drafter' models trained for each language with large models to accelerate the inference speed of multilingual LLMs.
Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale
Junying Chen (Shenzhen Research Institute of Big Data), Benyou Wang (Shenzhen Research Institute of Big Data)
TransformerLarge Language ModelVision Language ModelMultimodalityBiomedical DataBenchmark
🎯 What it does: By extracting medical image-text pairs from PubMed and using a multi-modal LLM (GPT-4V) for unblinded formatting, we generated 1.3 million high-quality medical VQA data points, constructing the PubMedVision dataset. Based on this, we trained a 34B parameter medical multi-modal LLM called HuatuoGPT-Vision.
Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
Michael Lan (Apart Research), Fazl Barez (University of Oxford)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigate shared circuits in large language models for sequence continuation tasks (Arabic numerals, number words, months), analyzing and comparing the sub-circuit structures of GPT-2 Small and Llama-2-7B.
Towards Low-Resource Harmful Meme Detection with LMM Agents
Jianzhao Huang (Beijing University of Posts and Telecommunications), Jing Ma (Beijing University of Posts and Telecommunications)
ClassificationTransformerLarge Language ModelAgentic AIVision Language ModelImageMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose a low-resource harmful meme detection framework, LOREHM, which leverages large multimodal models (LMM) to achieve detection under extremely limited labeled samples, and enhances model reasoning capabilities through retrieval augmentation and self-reflection.
Towards Measuring and Modeling “Culture” in LLMs: A Survey
Muhammad Farid Adilazuarda (MBZUAI), Monojit Choudhury (MBZUAI)
Explainability and InterpretabilityTransformerLarge Language ModelTextReview/Survey Paper
🎯 What it does: Conduct a systematic review of nearly 90 papers on cultural performance and bias in large language models (LLMs), constructing two classification systems: cultural agents (demographic and semantic) and detection methods (discriminative and generative), and identifying existing research gaps and directions for improvement.
Towards Online Continuous Sign Language Recognition and Translation
Ronglai Zuo (Hong Kong University of Science and Technology), Brian Mak (Hong Kong University of Science and Technology)
RecognitionConvolutional Neural NetworkTransformerVideoText
🎯 What it does: Proposed an end-to-end online continuous sign language recognition (CSLR) framework. First, a pre-trained CTC model is used for sign segmentation to build a dictionary. Then, an improved ISLR model is trained on the dictionary, and real-time recognition is achieved through sliding windows and post-processing. The framework can also integrate with a gloss-to-text network to realize online sign language translation (SLT), and performance of offline CSLR models can be enhanced through lightweight adapters.
Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights
Hao Yang (Monash University), Reza Haf
Safty and PrivacyLarge Language ModelPrompt EngineeringMultimodalityBenchmarkAudio
🎯 What it does: Proposed a risk classification system for speech and constructed an evaluation dataset containing synthetic speech, systematically assessing the ability of large-scale multimodal models to identify risks in speech based on acoustic cues such as tone and emotion.
Towards Robust Speech Representation Learning for Thousands of Languages
William Chen (Carnegie Mellon University), Shinji Watanabe (Carnegie Mellon University)
Representation LearningTransformerAuto EncoderContrastive LearningAudio
🎯 What it does: Developed the XEUS cross-lingual speech representation learning model, pre-trained on a large-scale dataset with over 1 million hours of speech across 4,057 languages.
Towards Tool Use Alignment of Large Language Models
Zhi-Yuan Chen (Gaoling School of Artificial Intelligence, Renmin University of China), Yankai Lin (Gaoling School of Artificial Intelligence, Renmin University of China)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed the alignment principles H2A (Helpfulness, Harmlessness, Autonomy) for LLMs in tool-use scenarios, constructed the ToolAlign dataset, and trained the AlignToolLLaMA model;
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
Yuping Lin (Michigan State University), Jiliang Tang (Michigan State University)
Representation LearningAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Conduct a representation space analysis of jailbreak attacks in large language models (LLMs), introduce the concept of 'acceptance direction,' and incorporate a new optimization objective into existing white-box jailbreak methods (GCG, AutoDAN) to improve attack success rates.
Towards Verifiable Text Generation with Evolving Memory and Self-Reflection
Hao Sun (Peking University), Dawei Yin (Baidu Inc)
GenerationLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposes the VTG framework, integrating evolutionary long short-term memory, dual-layer validator, and evidence retrieval to support verifiable text generation
Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderators through a User-Centric Method
Yang Trista Cao (University of Maryland College Park), Hal Daumé III (University of Maryland College Park)
TransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: A systematic evaluation of the applicability of existing models across various community rules was conducted, with experiments on rule violation detection in the AskHistorians subreddit using GPT-4 and Llama-2, alongside collecting user feedback from volunteer moderators through questionnaires.
ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations
Yunze Xiao (Carnegie Mellon University Qatar), Roy Ka-Wei Lee (Singapore University Of Technology And Design)
ClassificationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Investigated the robustness of Chinese hate speech detection models against homonym and emoji obfuscation attacks, proposed the ToxiCloakCN dataset, and conducted evaluations.
Tracking the perspectives of interacting language models
Hayden Helm (Nomic AI), Carey Priebe (Johns Hopkins University)
Representation LearningData-Centric LearningGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextGraph
🎯 What it does: Constructed a directed graph network for interactive language models and defined 'Perspective Space' to quantify the relative response differences of multiple models on a fixed query set, thus systematically studying the impact of different communication structures on information diffusion and model perspective evolution.
Training-free Deep Concept Injection Enables Language Models for Video Question Answering
Xudong Lin (Columbia University), Shih-Fu Chang (Columbia University)
TransformerPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposed a training-free deep concept injection (Deep Concept Injection, DCI) method, enabling pre-trained language models (PLM) to directly perform cross-modal tasks (e.g., video question answering) without requiring cross-modal pre-training or additional projection layers.
TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities
Ming Zhang (Fudan University), Xuanjing Huang (Fudan University)
Domain AdaptationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed a Chinese multi-domain task-oriented dialogue dataset called TransferTOD spanning 30 life service scenarios with multi-turn information collection, and fine-tuned the TransferTOD-7B model based on this dataset;
Transformers are Multi-State RNNs
Matanel Oren (Hebrew University of Jerusalem), Roy Schwartz (Hebrew University of Jerusalem)
CompressionRecurrent Neural NetworkTransformerLarge Language ModelText
🎯 What it does: This paper reconsiders decoder-only Transformer as a multi-state RNN (MSRNN) and proves that they can be transformed into bounded MSRNN through compression strategies.
TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering
Chuyi Shang (University of California Berkeley), Roei Herzig (University of California Berkeley)
RetrievalLarge Language ModelAgentic AIVision Language ModelVideoTextMultimodality
🎯 What it does: Propose a modular framework called TraveLER based on multi-agent systems to achieve video question answering without fine-tuning; the framework iteratively constructs plans, locates key frames, asks questions to obtain details, evaluates answers, and re-plans through four stages: Planner, Retriever, Extractor, and Evaluator;
Tree of Problems: Improving structured problem solving with compositionality
Armel Randy Zebaze (Inria), Rachel Bawden
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose the Tree of Problems (ToP) framework, which decomposes complex reasoning problems into similar subproblems. It first uses LLMs to solve leaf nodes and then recursively merges them to obtain the final answer.
Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering
Chang Zong (Zhejiang University), Yueting Zhuang (Zhejiang University)
Large Language ModelAgentic AIPrompt EngineeringTextBenchmark
🎯 What it does: Propose the Triad framework, which utilizes multi-role LLM agents to complete four stages of knowledge base question answering (question parsing, URI linking, query construction, answer generation), achieving zero/few-shot learning;
TroL: Traversal of Layers for Large Language and Vision Models
Byung-Kwan Lee (Korea Advanced Institute of Science and Technology), Yong Man Ro (Korea Advanced Institute of Science and Technology)
TransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Proposes TroL, a 1.8B/3.8B/7B-scale LLVM family, which enhances learning capabilities by reusing layers through layer traversing technology without increasing parameters.
TRoTR: A Framework for Evaluating the Re-contextualization of Text Reuse
Francesco Periti (University of Milan), Dominik Schlechtweg (University of Stuttgart)
ClassificationRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningTextBenchmark
🎯 What it does: This paper proposes the TRoTR framework to evaluate thematic relevance in text recontextualization, based on two tasks TRiC and TRaC for experiments.
Turn Waste into Worth: Rectifying Top-k Router of MoE
Zhiyuan Zeng (Fudan University), Xipeng Qiu (Fudan University)
Computational EfficiencyTransformerLarge Language ModelMixture of ExpertsTextBenchmark
🎯 What it does: Propose Rectify-Router, addressing the token loss and padding issues caused by top-k routing in Sparse MoE, designing two post-processing schemes: Intra-GPU Rectification (re-allocating lost tokens within the same GPU to avoid cross-GPU communication) and Fill-in Rectification (replacing padding with high-scoring tokens from k+1 experts).
TV-TREES: Multimodal Entailment Trees for Neuro-Symbolic Video Reasoning
Kate Sanders (Johns Hopkins University), Benjamin Van Durme (Johns Hopkins University)
Explainability and InterpretabilityTransformerLarge Language ModelVision Language ModelVideoMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the TV-TREES multimodal evidence tree generator, which uses an interpretable tree structure to reason about TV video question answering;
Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gaps
Giuseppe Attanasio (Instituto de Telecomunicações), Dirk Hovy (Bocconi University)
RecognitionExplainability and InterpretabilityTransformerAudio
🎯 What it does: Systematic evaluation of Whisper and SeamlessM4T multilingual ASR models across 19 languages and 3 datasets, quantifying the recognition error gap between males and females;
Uncertainty in Language Models: Assessment through Rank-Calibration
Xinmeng Huang, Edgar Dobriban (University of Pennsylvania)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose a Rank-Calibration based evaluation framework to measure the quality of uncertainty and confidence metrics in language models.
Understanding “Democratization” in NLP and ML Research
Arjun Subramonian (University of California Los Angeles), Zeerak Talat (Mohamed Bin Zayed University of Artificial Intelligence)
TextReview/Survey Paper
🎯 What it does: A large-scale mixed-method analysis of the term 'democratization' in NLP/ML literature, exploring its conceptualization, objectives, methodologies, and intersections with democratic theory.
Understanding and Mitigating Language Confusion in LLMs
Kelly Marchisio (Cohere), Sebastian Ruder (Cohere)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Construct a language confusion benchmark and evaluate the language confusion issue in LLMs during monolingual and cross-lingual generation
Understanding Higher-Order Correlations Among Semantic Components in Embeddings
Momose Oyama (Kyoto University), Hidetoshi Shimodaira (Kyoto University)
Explainability and InterpretabilityRepresentation LearningLarge Language ModelText
🎯 What it does: The study uses higher-order correlation measures in ICA-transformed word embeddings to quantify and visualize non-independence, interpreting it as semantic associations;
Understanding Slang with LLMs: Modelling Cross-Cultural Nuances through Paraphrasing
Ifeoluwa Wuraola (University of Hull), Daniel Marciniak (University of Hull)
ClassificationRecognitionTransformerLarge Language ModelText
🎯 What it does: Investigate the emotional recognition capabilities of large language models in cross-cultural slang paraphrasing, using climate-related tweets from Nigeria and the UK as examples.
UNICORN: A Unified Causal Video-Oriented Language-Modeling Framework for Temporal Video-Language Tasks
Yuanhao Xiong (UCLA), Jie Lei (Meta)
GenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelVideoText
🎯 What it does: Proposes the Unified Causal Video-Language Model Framework UNICORN, achieving temporal video-language tasks through instruction tuning.
UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation
Xiangyu Zhao (Hong Kong Polytechnic University), Xiao-Ming Wu (Hong Kong Polytechnic University)
GenerationRetrievalTransformerVision Language ModelDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose UniFashion, a unified framework that integrates multiple multimodal tasks in the fashion domain, including cross-modal retrieval, combinatorial image retrieval, image captioning, and image generation, into a single model to achieve bidirectional collaboration between retrieval and generation.
Unifying Multimodal Retrieval via Document Screenshot Embedding
Xueguang Ma (University of Waterloo), Jimmy Lin (University of Waterloo)
RetrievalTransformerSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposes a Document Screenshot Embedding (DSE) approach, directly encoding document screenshots into dense vectors for retrieval, eliminating traditional text/image splitting and preprocessing steps.
UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation
Juhwan Choi (Chung-Ang University), YoungBin Kim (Chung-Ang University)
ClassificationData SynthesisDomain AdaptationKnowledge DistillationRecurrent Neural NetworkTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Propose the UNIGEN method, which generates domain-agnostic training data from large language models using a general prompt, and trains a lightweight task model (TAM) to achieve zero-shot, lightweight sentiment classification.
Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning
Shuai Zhao (Nanyang Technological University), Jinming Wen (Guangzhou University)
Adversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes a backdoor attack method called ICLAttack that does not require fine-tuning, inducing large language models to generate predefined labels by implanting trigger words in the in-context learning demonstration context;
Unknown Claims: Generation of Fact-Checking Training Examples from Unstructured and Structured Data
Jean-Flavien Bussotti (EURECOM), Paolo Papotti (EURECOM)
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextTabular
🎯 What it does: Built a framework called UNOWN that can automatically generate training samples for fact-checking from both structured and unstructured data (text and tables).
Unlabeled Debiasing in Downstream Tasks via Class-wise Low Variance Regularization
Shahed Masoudian, Markus Schedl (Thomson Reuters Labs)
ClassificationSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose an unlabeled debiasing method based on category variance regularization, enabling the encoder LM to generate representations that are sensitive to category information but have low sensitivity to protected attribute information in downstream tasks.
Unleashing the Power of Emojis in Texts via Self-supervised Graph Pre-Training
Zhou Zhang (Fudan University), Jiarong Xu (Fudan University)
ClassificationGraph Neural NetworkLarge Language ModelContrastive LearningTextGraph
🎯 What it does: Propose a self-supervised pre-training framework based on heterogeneous graphs that jointly learns representations for posts, words, and emojis, and integrates the learned emoji vectors into downstream tasks;
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding
Lifu Tu (Salesforce AI Research), Yingbo Zhou (Salesforce AI Research)
GenerationTransformerLarge Language ModelText
🎯 What it does: Proposed a method to formalize text generation as a future constraint generation problem, aiming to reduce undesirable behaviors and ensure instruction fidelity.
Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering
Yifei Yuan (University of Copenhagen), Mohammad Aliannejadi (University of Amsterdam)
Data-Centric LearningTransformerLarge Language ModelTextBenchmarkFinance RelatedRetrieval-Augmented Generation
🎯 What it does: Proposes a multilingual cross-market product question answering task (MCPQA) and constructs the McMarket dataset, which contains approximately 7 million question-answer pairs and 52 million evaluations, while using GPT-4 to generate subsets for training and evaluation.
Unlocking Memorization in Large Language Models with Dynamic Soft Prompting
Zhepeng Wang (George Mason University), Yanfu Zhang (William And Mary)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: A framework based on dynamic soft prompts was studied and implemented to extract and measure the training data memorized by large language models (LLMs), significantly improving the detection accuracy of the model's internal memory.
Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models
Tianyi Men (Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences), Jun Zhao (Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigate the look-ahead planning mechanism of large language models (LLMs) in the Blocksworld task, using information flow and internal representation techniques to conduct interpretive analysis of the model's decision-making process.
UNO Arena for Evaluating Sequential Decision-Making Capability of Large Language Models
Zhanyue Qin (Harbin Institute of Technology), Dianbo Sui (Harbin Institute of Technology)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextSequentialBenchmark
🎯 What it does: Built a dynamic evaluation platform called UNO Arena based on the UNO card game, and designed TUTRI reflective players to improve the performance of large language models in sequential decision-making.
Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applications
Weize Liu (Zhejiang University), Jian Wu (Zhejiang University)
Computational EfficiencyTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: By converting large language models into a fine-grained Mixture of Experts structure and visualizing expert activation frequency heatmaps, the study investigates activation patterns within LLMs across different languages, language family associations, the impact of instruction fine-tuning, and leverages activation differences to achieve sparse activation and pruning.
Unsupervised Discrete Representations of American Sign Language
Artem Abzaliev (University of Michigan), Rada Mihalcea (University of Michigan)
RecognitionRepresentation LearningTransformerSequential
🎯 What it does: Developed a sign language tokenizer based on Residual Vector Quantization (RVQ), converting continuous joint coordinate sequences of American Sign Language (ASL) gestures into discrete tokens.
Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel
Brendan King (University of California, Santa Cruz), Jeffrey Flanigan (University of California, Santa Cruz)
GenerationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Built an end-to-end task-oriented dialogue system that relies solely on unlabeled user-agent dialogue text and API schema, utilizing LLM to infer hidden dialogue states, API calls, and system behaviors through a noisy channel model and expectation maximization (Hard-EM), thereby completing dialogue generation.
Unsupervised Extraction of Dialogue Policies from Conversations
Makesh Narsimhan Sreedhar (NVIDIA), Christopher Parisien (NVIDIA)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Automatically extract dialogue strategies from task-oriented dialogue corpora by leveraging normalized expressions generated by LLMs and constructing a weighted directed graph for flow path search.
Unsupervised Human Preference Learning
Sumuk Shashidhar (University of Illinois Urbana-Champaign), Dilek Hakkani Tur
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Studied a method that uses a small preference proxy model to guide large pre-trained language models to achieve personalized output by generating natural language rules.
Unsupervised Named Entity Disambiguation for Low Resource Domains
Debarghya Datta (Indian Institute of Technology, Bhilai), Soumajit Pramanik (Indian Institute of Technology, Bhilai)
Representation LearningData-Centric LearningGraph Neural NetworkTextGraph
🎯 What it does: This study proposes an unsupervised group Steiner tree method (GST-NED) for named entity disambiguation in low-resource domains.
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
Xin Zhou (Fudan University), Xuanjing Huang (Fudan University)
Explainability and InterpretabilityComputational EfficiencyMixture of ExpertsTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Studied the role of expert activation within Mixture-of-Expert (MoE) large models in retrieval-augmented generation (RAG), and proposed identifying core experts (CEAI) by comparing differences in expert activation under adversarial scenarios
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
Yifei Wang (Chinese Academy of Sciences), Daniel Dajun Zeng (Chinese Academy of Sciences)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper studies the recall behavior of internal factual knowledge in large language models during multi-hop reasoning tasks, and quantifies and analyzes their factual memory at each step through Knowledge Neurons (KN).
Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism
Anhao Zhao (Southwest Jiaotong University), Xiaoyu Shen (Digital Twin Institute Eastern Institute of Technology)
Explainability and InterpretabilityLarge Language ModelPrompt EngineeringText
🎯 What it does: Constructed a 2D coordinate system using task identification and example similarity as axes, systematically analyzing the in-context learning (ICL) mechanisms of large language models (LLMs), and proposed the PIRE metric to evaluate task identification capability.
Unveiling Multi-level and Multi-modal Semantic Representations in the Human Brain using Large Language Models
Yuko Nakagi (Osaka University), Yu Takagi (Osaka University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelVision Language ModelVideoTextMultimodalityBiomedical DataMagnetic Resonance ImagingAudio
🎯 What it does: In an fMRI experiment, six participants watched 8.3 hours of TV shows and movies, and the videos were annotated with multi-level (speech, objects, background story, summary, spatiotemporal) semantic annotations; subsequently, these annotations were processed through LLMs (e.g., Llama 2) and multimodal models (e.g., LLaVA-v1.5) to extract latent representations, a linear encoding model was built to predict brain activity, and variance partitioning analysis was conducted to examine the unique contributions of different levels and modalities.
Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement
Pengwei Zhan (Institute of Information Engineering, Chinese Academy of Sciences), Ru Xie (Institute of Information Engineering, Chinese Academy of Sciences)
OptimizationTransformerPrompt EngineeringText
🎯 What it does: This paper investigates the high sensitivity of large language models (LLMs) to subtle variations in prompt vocabulary and proposes a black-box method called COPLE based on combinatorial optimization, which iteratively fine-tunes the vocabulary level of prompts to significantly improve model performance across multiple downstream tasks.
Unveiling the mystery of visual attributes of concrete and abstract concepts: Variability, nearest neighbors, and challenging categories
Tarun Tater (University of Stuttgart), Diego Frassinelli (Ludwig-Maximilians-Universität Munich)
ClassificationObject DetectionConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: The study uses approximately 1,000 nouns with high abstraction and high concreteness, extracts multiple visual features from two image datasets (Bing and YFCC100M), classifies image visual diversity, conducts nearest neighbor consistency analysis, and manually categorizes the causes of visual diversity.
Unveiling the Role of Pretraining in Direct Speech Translation
Belen Alastruey (FAIR, Meta), Marta R. Costa-jussà (Universitat Politècnica de Catalunya)
Explainability and InterpretabilityTransformerTextAudio
🎯 What it does: Compared the training dynamics of Speech Translation models using pre-trained Encoders and those trained from scratch, and proposed improving the decoder's cross-attention through weighted residual connections and layer normalization (WeRC), significantly enhancing the performance of models without pre-training;
UOUO: Uncontextualized Uncommon Objects for Measuring Knowledge Horizons of Vision Language Models
Xinyu Pi (University of California San Diego), Zhiting Hu (University of California San Diego)
Vision Language ModelImageBenchmark
🎯 What it does: Constructed a million-scale 'context-free rare objects' UOUO benchmark for systematically evaluating visual language models on long-tail rare objects.
Updating CLIP to Prefer Descriptions Over Captions
Amir Zur (Pr(Ai) R Group 2), Atticus Geiger (Pr(Ai) R Group 2)
ClassificationRetrievalSupervised Fine-TuningVision Language ModelContrastive LearningImageText
🎯 What it does: Update the CLIP model to prefer descriptions (description) over captions (caption) in image-text matching, thereby enhancing the reliability of accessibility assessments.
User Inference Attacks on Large Language Models
Nikhil Kandpal (University of Toronto), Zheng Xu (Google)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigated privacy leakage of user data during fine-tuning of large language models (LLMs), proposing a likelihood ratio-based user inference attack, and conducting experiments on the GPT-Neo model using Reddit comments, CC News articles, and Enron email datasets.
Using Language Models to Disambiguate Lexical Choices in Translation
Josh Barua (University of California, Berkeley), Alane Suhr (University of California, Berkeley)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper studies the use of language models in translation to address lexical choice ambiguity and constructs a cross-lingual lexical choice dataset called DTAiLS.