ACL 2024 Papers — Page 10
Annual Meeting of the Association for Computational Linguistics · 940 papers
VisDiaHalBench: A Visual Dialogue Benchmark For Diagnosing Hallucination in Large Vision-Language Models
Qingxing Cao (Sun Yat-sen University), Liang Lin (Sun Yat-sen University)
Explainability and InterpretabilityTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Constructed VisDiaHalBench, a vision-dialogue benchmark based on GQA, to systematically diagnose hallucinations in large-scale vision-language models (LVLMs) under multi-round visual and textual inputs.
VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval
Junjie Zhou (Beijing University of Posts and Telecommunications), Yongping Xiong (Beijing Academy of Artificial Intelligence)
RetrievalRepresentation LearningTransformerVision Language ModelDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a unified embedding model VISTA, which is built upon a frozen strong text encoder and uses Vision Transformer as an image tokenizer, capable of handling text, images, and their combinations. The model achieves multimodal retrieval capability through a two-stage training process.
Visualization Recommendation with Prompt-based Reprogramming of Large Language Models
Xinhang Li (University of Science and Technology of China), Enhong Chen (University of Science and Technology of China)
Recommendation SystemTransformerLarge Language ModelPrompt EngineeringTabular
🎯 What it does: This paper proposes an HTP framework based on hierarchical table prompts for LLM reprogramming, used for automatically recommending visualization charts.
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Jing Yu Koh (Carnegie Mellon University), Daniel Fried (Carnegie Mellon University)
Large Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposed the VisualWebArena benchmark to evaluate the ability of multimodal agents in visually dependent real-world web tasks.
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild
Puyuan Peng (University of Texas at Austin), David Harwath (University of Texas at Austin)
GenerationData SynthesisTransformerAuto EncoderAudio
🎯 What it does: Propose VOICECRAFT, a Transformer-based neural encoder language model for voice editing and zero-shot TTS, capable of generating natural and recognizable speech in real-world audio;
VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model
Tianyu Chen (Peking University), Tao Xie (Peking University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Propose a generative framework called VulLibGen based on large language models (LLMs) to automatically identify the names of affected software packages in security vulnerability reports.
WARDEN: Multi-Directional Backdoor Watermarks for Embedding-as-a-Service Copyright Protection
Anudeex Shetty (University of Melbourne), Qiongkai Xu (University of Melbourne)
Safty and PrivacyText
🎯 What it does: To address copyright protection in Embedding-as-a-Service (EaaS), the paper proposes the CSE three-step attack (clustering, selection, elimination) to remove the EmbMarker watermark while preserving embedding quality; subsequently, it designs the WARDEN multi-directional watermark scheme, using multiple watermark vectors to enhance robustness against CSE attacks.
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
Shangqing Tu (Tsinghua University), Juanzi Li (Tsinghua University)
Safty and PrivacyLarge Language ModelTextBenchmark
🎯 What it does: Propose the WaterBench benchmark for evaluating watermarks in large language models, designing a unified watermark strength, task diversity, and GPT4-Judge evaluation framework.
WatME: Towards Lossless Watermarking Through Lexical Redundancy
Liang Chen (Chinese University of Hong Kong), Kam-Fai Wong (Chinese University of Hong Kong)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper proposes WatME, a watermarking method based on lexical redundancy, which can embed detectable watermarks into large language model texts without significantly compromising generation quality;
Wav2Gloss: Generating Interlinear Glossed Text from Speech
Taiqi He (Carnegie Mellon University), Lori Levin (Johns Hopkins University)
GenerationTransformerLarge Language ModelTextAudio
🎯 What it does: Propose a new task, WAV2GLOSS, which directly generates aligned annotated text (IGT) containing transcription, basic forms, morphological annotations, and free translations from raw speech.
WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning
Zhaojian Yu (Tsinghua University), Qiufeng Yin (Microsoft)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose WaveCoder, which utilizes multi-task, scalable high-quality instruction data (CodeSeaXDataset) for instruction tuning of Code LLMs, and constructs an LLM-based generation-discrimination framework to achieve automatic generation and quality control of instruction data.
WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations
Haolin Deng (Harbin Institute of Technology), Ruifeng Xu (Harbin Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed the WebCiteS dataset and proposed and experimented with the Chinese Web Search Result Query-Focused Summarization (AQFS) task.
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
Hongliang He (Zhejiang University), Dong Yu (Tencent AI Lab)
TransformerLarge Language ModelAgentic AIPrompt EngineeringVision-Language-Action ModelMultimodality
🎯 What it does: Designed and implemented WebVoyager, a large multimodal model (LMM)-based system capable of end-to-end completion of interactive tasks on real websites, using screenshots and webpage element text jointly to guide decision-making and execute actions such as clicking, typing, and scrolling.
What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects
Verena Blaschke (LMU Munich), Barbara Plank (LMU Munich)
TextReview/Survey Paper
🎯 What it does: An online questionnaire was conducted with 327 German dialect speakers to assess their attitudes and needs toward various language technologies (e.g., speech-to-text, virtual assistants, machine translation, spell checking, etc.).
What Do Language Models Hear? Probing for Auditory Representations in Language Models
Jerry Ngo (Massachusetts Institute of Technology), Yoon Kim (Massachusetts Institute of Technology)
RetrievalRepresentation LearningTransformerContrastive LearningTextMultimodalityAudio
🎯 What it does: By aligning text model sentence embeddings with audio model acoustic embeddings through a contrastive probe, the study examines whether language models can implicitly encode sound characteristics when trained only on text, and tests their zero-shot generalization capability on unseen categories.
What Do Language Models Learn in Context? The Structured Task Hypothesis.
Jiaoda Li (ETH Zurich), Ryan Cotterell (ETH Zurich)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigating the context learning mechanisms of large language models
What Does Parameter-free Probing Really Uncover?
Tommi Buder-Gröndahl (University of Helsinki)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Apply a parameter-agnostic perturbed masking technique to the BERT model for bottom-up syntactic probing, generating dependency trees and comparing them with English Universal Dependencies (UD) annotations to analyze differences in argument structure, noun phrase structure, modifiers, and prepositional phrases.
What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection
Shangbin Feng (University of Washington), Yulia Tsvetkov (University of Washington)
Anomaly DetectionTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextMultimodalityGraphBenchmark
🎯 What it does: Explore opportunities and risks of applying large language models (LLM) to social media bot detection, propose a hybrid heterogeneous expert framework to enhance detection performance, and investigate LLM-driven text and structural information tampering strategies to evade detection.
What Evidence Do Language Models Find Convincing?
Alexander Wan (University of California Berkeley), Dan Klein (University of California Berkeley)
RetrievalExplainability and InterpretabilityLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed the CONFLICTINGQA dataset to evaluate the persuasiveness of retrieval-augmented language models when facing controversial questions, and conducted experiments and comparative analysis within this framework.
What is the Best Way for ChatGPT to Translate Poetry?
Shanshan Wang (University of Macau), Lidia Chao (University of Macau)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Investigates the capability of ChatGPT in translating modern English-Chinese poetry and proposes a two-step translation approach (EAPMT) based on poetic interpretation assistance.
What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages
Nadav Borenstein (Københavns Universitet), Ryan Cotterell (ETH Zürich)
Explainability and InterpretabilityRepresentation LearningRecurrent Neural NetworkTransformerSequential
🎯 What it does: This paper evaluates the learnability of recurrent neural networks (RNNs) and Transformer language models (LMs) in learning regular language distributions by training on samples from randomly generated deterministic probabilistic finite state automata (DPFSAs).
When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
Norah Alzahrani (National Center for AI), Haidar Khan (Saudi Data and AI Authority)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Investigate how small perturbations in multiple-choice question (MCQ) benchmarks affect large language model (LLM) leaderboards, systematically evaluating sensitivity to answer order, symbols, scoring methods, and prompts;
When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP
Sara Papi (Fondazione Bruno Kessler), Matteo Negri (Fondazione Bruno Kessler)
TransformerTextAudio
🎯 What it does: Analyzed and fixed three categories of bugs in the widely used Conformer implementation, and through experiments demonstrated the potential misleading effects of these bugs on ASR and ST results; proposed the pangoliNN library for neural network unit testing, and provided a code quality checklist to enhance the verifiability of research software.
When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
Ziru Chen (Ohio State University), Huan Sun (Ohio State University)
GenerationAI Code AssistantLarge Language ModelSupervised Fine-TuningGenerative Adversarial NetworkTextTabular
🎯 What it does: The study investigates the effectiveness of tree search and iterative correction planning methods for LLM language agents within a generator-discriminator framework in solving multi-step problems, and explores the decisive role of discriminative accuracy.
When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality
Brielen Madureira (University of Potsdam), David Schlangen (University of Potsdam)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Studied the internal state update mechanism of restart-incremental Transformer when processing locally ambiguous sentences, and proposed an interpretable analysis method to compare the performance of different models during the ambiguity revision process.
When Phrases Meet Probabilities: Enabling Open Relation Extraction with Cooperating Large Language Models
Jiaxin Wang (Xi'an Jiaotong University), Jun Liu (National University of Singapore)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose an open relational extraction framework called ORELLM based on large language models (LLMs), which uses two LLMs to generate relation phrases and estimate similarity probabilities, respectively, and achieves clustering through collaborative loops;
Where Do People Tell Stories Online? Story Detection Across Online Communities
Maria Antoniak (Allen Institute for AI), Andrew Piper (McGill University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: This study proposes and implements the task of cross-online community story detection, developing and publicly releasing the StorySeeker toolkit, which includes 502 annotated Reddit posts and comments with story and event span annotations, a story codebook for social media, and directly usable document- and span-level detection models.
Who Wrote this Code? Watermarking for Code Generation
Taehyun Lee (Seoul National University), Gunhee Kim (Seoul National University)
AI Code AssistantLarge Language ModelText
🎯 What it does: Proposes a selective watermarking method called SWEET based on entropy thresholds for watermark embedding and detection in code generation LLMs
Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback
Maria Lerner, Naman Goel (University Of Oxford)
Explainability and InterpretabilityData-Centric LearningReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningTextTabular
🎯 What it does: Systematically investigate the fairness preferences of human annotators with different demographic backgrounds in content moderation tasks, and study how these differences affect AI fairness models based on human feedback.
Why are Sensitive Functions Hard for Transformers?
Michael Hahn (Saarland University), Mark Rofin (Saarland University)
Explainability and InterpretabilityTransformerSequential
🎯 What it does: Investigated why learning highly input-space-sensitive functions (e.g., PARITY) in Transformers is challenging, proving that high input-space sensitivity leads to extremely steep minima in the parameter space, creating a learning bias characterized by low sensitivity and low polynomial degree.
Why Don’t Prompt-Based Fairness Metrics Correlate?
Abdelrahman Zayed (Polytechnique Montreal), Sarath Chandar (IBM Research)
Large Language ModelPrompt EngineeringText
🎯 What it does: Research and improve the correlation between fairness evaluation metrics based on prompts, proposing the CAIRO method that maximizes the Pearson correlation coefficient between metrics through multi-model prompt enhancement and combination selection.
Word Embeddings Are Steers for Language Models
Chi Han (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
GenerationExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Investigate and utilize linear transformations in language model output embeddings, proposing the LM-Steer method to control the style of generated text in an interpretable, transferable, and efficient manner, primarily applied to toxicity elimination and sentiment control, supporting continuous and composite adjustments.
Word Matters: What Influences Domain Adaptation in Summarization?
Yinghao Li (Beijing Institute of Technology), Yang Gao (Beijing Institute of Technology)
GenerationDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigate the impact of words in domain adaptation for the abstract task, propose a dataset learning difficulty coefficient (the product of compression rate and abstraction level), and use cross-domain word overlap to measure the similarity between the source and target domains, verifying its linear relationship with model performance improvement.
WRP: Weight Recover Prune for Structured Sparsity
Zhendong Tan (Xi'an Jiaotong University), Zheng Wei (Xi'an Jiaotong University)
CompressionComputational EfficiencyTransformerText
🎯 What it does: Proposed the Weight Recover Prune (WRP) method, which recovers a small number of critical weights on the basis of 2:4 structured sparsity to improve the accuracy of large language models (LLMs) while maintaining compression effects
XCodeEval: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
Mohammad Abdullah Matin Khan (Islamic University of Technology), Shafiq Joty (Nanyang Technological University)
GenerationRetrievalTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose XCODEEVAL—a large-scale executable multi-task benchmark covering multiple languages and seven tasks (code understanding, generation, translation, retrieval), and implement ExecEval, an execution engine supporting 44 compilers/interpreters.
XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts
Yifeng Ding (University of Illinois Urbana Champaign), Lingming Zhang (University of Illinois Urbana Champaign)
Computational EfficiencyAI Code AssistantTransformerSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: Proposed the XFT training framework, which first upgrades a pre-trained dense code LLM into a Mixture-of-Experts (MoE) model, then fine-tunes it on instruction data, and finally learns to merge the MoE back into a dense model, thereby enhancing instruction-following performance without increasing inference costs.
XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception
HyoJung Han (University of Maryland), Changhan Wang (Meta AI)
RecognitionRepresentation LearningConvolutional Neural NetworkTransformerVideoMultimodalityAudio
🎯 What it does: Propose a cross-lingual audio-visual speech representation model, XLAVS-R, for noise-robust speech recognition and translation across more than 100 languages.
Your Transformer is Secretly Linear
Anton Razzhigaev (AIRI), Andrey Kuznetsov (AIRI)
Computational EfficiencyKnowledge DistillationRepresentation LearningTransformerText
🎯 What it does: This paper analyzes and reveals the near-linear characteristics of the Transformer decoder, explores its dynamics during pre-training and fine-tuning stages, and proposes cosine similarity regularization along with hierarchical pruning and distillation methods to enhance efficiency and performance.
Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation
Xiang Luo (Yunnan University), Xuejie Zhang (Yunnan University)
Domain AdaptationComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose the DualLoRA framework, which utilizes dual low-rank adapters (Context LoRA and Prompt LoRA) to achieve zero-shot cross-domain dialogue state tracking, and ensures inference latency remains unchanged by fusing adapter weights through pre-computation.
Zero-Shot Cross-Lingual Reranking with Large Language Models for Low-Resource Languages
Mofetoluwa Adeyemi (University of Waterloo), Jimmy Lin (University of Waterloo)
RetrievalTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigated the zero-shot listwise re-ranking effectiveness of large language models (LLMs) in low-resource African languages (Hausa, Somali, Swahili, Yoruba), and systematically evaluated across three scenarios: cross-lingual, monolingual (original language), and self-translation.