EMNLP 2024 Papers — Page 6
Conference on Empirical Methods in Natural Language Processing · 1268 papers
GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generation
Shihao Cai (University of Science and Technology of China), Bo Zheng (Alibaba Group)
GenerationData SynthesisTransformerLarge Language ModelVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: Propose an automated pipeline based on GPT-4V and GPT-4 to generate simplified geometry problems highly aligned with images, and construct the GeoGPT4V dataset.
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners
Shimao Zhang (National Key Laboratory for Novel Software Technology, Nanjing University), Shujian Huang (National Key Laboratory for Novel Software Technology, Nanjing University)
ClassificationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Instruction-tune LLMs using answer-free question-translation alignment data, and evaluate their cross-lingual performance and internal mechanisms across 20 languages.
Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection
Abhishek Purushothama (Georgetown University), Katharina Von Der Wense (Georgetown University)
TransformerAuto EncoderContrastive LearningText
🎯 What it does: This paper explores the use of unsupervised secondary tasks (autoencoding and masked denoising) in low-resource morphology tasks, combining them with pretraining-finetuning and multitask learning to investigate the impact of different training settings on Transformer model performance.
GLaPE: Gold Label-agnostic Prompt Evaluation for Large Language Models
Xuanchang Zhang (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
OptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a gold-label-free prompt evaluation method called GLaPE, aiming to automatically evaluate and optimize prompts for large language models without relying on human-annotated answers.
Global Reward to Local Rewards: Multimodal-Guided Decomposition for Improving Dialogue Agents
Dong Won Lee (Massachusetts Institute of Technology), Louis-Philippe Morency (Carnegie Mellon University)
Knowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningMultimodality
🎯 What it does: Propose a framework (GELI) that decomposes a single global explicit reward into local episode-level rewards, leveraging multimodal implicit feedback to guide the decomposition process and improve long-horizon dialogue agents.
GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization
Yangfan Ye (Harbin Institute of Technology), Bing Qin (Du Xiaoman Financial)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed a unified multilingual, cross-lingual, multi-document news summarization task (MCMS) and constructed the corresponding benchmark dataset GLOBESUMM;
GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text
Michael Ginn (University of Colorado Boulder), Lori Levin (Carnegie Mellon University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed the largest multilingual interlinear glossed text (IGT) corpus, containing approximately 451,000 example sentences, covering 1,800 languages, with 80% of the grammatical tags standardized using UniMorph; continuously pre-trained the ByT5 model on this corpus for automatically generating IGT.
Glue pizza and eat rocks - Exploiting Vulnerabilities in Retrieval-Augmented Generative Models
Zhen Tan (Arizona State University), Huan Liu (Arizona State University)
Adversarial AttackTransformerTextRetrieval-Augmented Generation
🎯 What it does: This paper demonstrates the security vulnerabilities of Retrieval-Augmented Generative (RAG) systems under gray-box attacks by constructing adversarial documents that are prioritized by retrieval systems and induce generative models to output erroneous or harmful content, and proposes LIAR (a bilevel optimization attack framework) for the attack.
Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs
Chengyuan Liu (Zhejiang University), Fei Wu (Zhejiang University)
Domain AdaptationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the use of vocabulary expansion in domain-specific LLMs and proposed an adaptive method called VEGAD to select the most valuable subset of vocabulary
GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory
Wei Fan (Hong Kong University Of Science And Technology), Yangqiu Song (Hong Kong University Of Science And Technology)
Data SynthesisSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the GOLDCOIN framework, which generates synthetic cases of compliance and non-compliance using situational integrity theory to guide instruction fine-tuning of LLMs in privacy law judgment tasks.
GOME: Grounding-based Metaphor Binding With Conceptual Elaboration For Figurative Language Illustration
Linhao Zhang (Chinese Academy of Sciences), Guangluan Xu (Chongqing University)
GenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextChain-of-Thought
🎯 What it does: Proposes a system named GOME for visualizing language metaphors from a grounding perspective. It first generates refined visual interpretations using LLMs (triggered by chain-of-thought prompts to elicit rhetorical knowledge), then employs cross-attention constraints during the Stable Diffusion process to bind metaphor attributes to target objects, resulting in images more aligned with metaphorical meanings.
GottBERT: a pure German Language Model
Raphael Scheible (Technical University Munich), Martin Boeker (Technical University Munich)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Proposed and trained the first German monolingual RoBERTa model, GottBERT (base and large versions), pre-trained on both unfiltered and filtered German OSCAR corpora.
GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning
Aleksander Ficek (NVIDIA), Oleksii Kuchaiev (NVIDIA)
RetrievalComputational EfficiencyTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Compared multiple parameter-efficient fine-tuning (PEFT) methods (P-tuning, Adapters, LoRA) under the retrieval-augmented generation (RAG) framework, evaluating the performance of GPT and RETRO architectures across different model sizes (823M~48B) and multi-task datasets.
GPT-4 Jailbreaks Itself with Near-Perfect Success Using Self-Explanation
Govind Ramesh (Georgia Institute of Technology), Wei Xu (Georgia Institute of Technology)
Adversarial AttackLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the IRIS method, which leverages LLMs to autonomously generate and iteratively refine jailbreak prompts and outputs, achieving black-box level jailbreak.
Granular Privacy Control for Geolocation with Vision Language Models
Ethan Mendes (Georgia Institute of Technology), Alan Ritter (Georgia Institute of Technology)
Data SynthesisSafty and PrivacySupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Built a new conversational geolocation privacy benchmark, GPTGEOCHAT, collecting 1,000 Stock images and generating 1,000 dialogues with GPT-4v, annotated at five levels of granularity (country, city, community, location name, and GPS coordinates); simultaneously generated synthetic dialogues for GPTGEOCHATSynthetic to fine-tune. Explored multiple VLMs and their fine-tuned multimodal moderation agents, evaluating their privacy leakage and retention effects at different granularities.
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction
Sizhe Zhou (University of Illinois Urbana Champaign), Jiawei Han (University of Illinois Urbana Champaign)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed a zero-shot relation extraction framework REPAL based solely on relation definitions.
GRASS: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
Aashiq Muhamed (Carnegie Mellon University), Virginia Smith (Carnegie Mellon University)
OptimizationComputational EfficiencyLarge Language ModelTextBenchmark
🎯 What it does: Propose a subspace optimization method called GRASS that utilizes sparse projection to significantly reduce the memory consumption of optimizer states and gradients during full-parameter training of LLMs, while maintaining approximately the same training effectiveness.
GRIZAL: Generative Prior-guided Zero-Shot Temporal Action Localization
Onkar Kishor Susladkar (Yellow.ai), Sparsh Mittal (IIT Roorkee)
Object DetectionTransformerLarge Language ModelVision Language ModelContrastive LearningOptical FlowImageVideoTextMultimodality
🎯 What it does: In zero-shot video action localization, the GRIZAL framework is proposed, which utilizes generative text and image augmentation combined with visual-language embeddings and optical flow features to achieve precise localization.
Grounding Language in Multi-Perspective Referential Communication
Zineng Tang (University of California, Berkeley), Alane Suhr (University of California, Berkeley)
Reinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelContrastive LearningTextMultimodalityPoint Cloud
🎯 What it does: In a shared 3D scene, the study investigates how two agents with different perspectives generate and understand referential expressions, and constructs a controllable difficulty multi-agent scene generation platform and corresponding dataset.
GuardBench: A Large-Scale Benchmark for Guardrail Models
Elias Bassani (European Commission Joint Research Centre), Ignacio Sanchez (European Commission Joint Research Centre)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Propose GuardBench, a benchmark consisting of 40 security evaluation datasets, to systematically assess and compare guardrail models of generative AI.
HalluMeasure: Fine-grained Hallucination Measurement Using Chain-of-Thought Reasoning
Shayan Ali Akbar, Erwin Cornejo (Amazon.com)
ClassificationTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: We propose HallMeasure, which uses LLMs to decompose answers into atomic claims, determines support for each claim through chain-of-thought reasoning, and outputs 10 fine-grained hallucination subtypes, enabling automated measurement of hallucinations in LLM responses.
Hate Personified: Investigating the role of LLMs in content moderation
Sarah Masud (IIIT Delhi), Tanmoy Chakraborty (Technische Universität München)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Investigate the sensitivity of LLMs to geographical location, identity attributes, and numerical anchoring contexts in content moderation, and explore their impact on the consistency of subjective annotations.
Hateful Word in Context Classification
Sanne Hoeken (Bielefeld University), Özge Alacam (Bielefeld University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the Hateful Word in Context Classification (HateWiC) task and construct a dataset with approximately 4,000 word-context instances to detect whether a word's meaning in a specific context carries hateful connotations.
HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs
Jocelyn Shen (Massachusetts Institute of Technology), Maarten Sap (Carnegie Mellon University)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose HEART, a theory-based narrative style taxonomy, and automatically extract features using LLMs, quantifying the relationship between narrative style and empathy through empathy assessments from 2624 participants.
HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding
Fan Yuan (Nanjing University of Aeronautics and Astronautics), Piji Li (Nanjing University of Aeronautics and Astronautics)
GenerationExplainability and InterpretabilityLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Propose the HELPD framework, combining hierarchical feedback learning (object-level and sentence-level) and visual-enhanced penalty decoding, significantly reducing hallucinations in large vision-language models and improving text quality.
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models
Yae Jee Cho (Carnegie Mellon University), Gauri Joshi (Carnegie Mellon University)
Federated LearningComputational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: This paper investigates federated fine-tuning on a small base model (ODFM) for resource-constrained edge devices.
Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters
Yujin Potter (University of California, Berkeley), Dawn Song (University of California, Berkeley)
TransformerLarge Language ModelTextTabular
🎯 What it does: Study the political bias of LLMs in the 2024 U.S. presidential election and their impact on voters using voting simulations, Q&A analysis, and user experiments.
Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization
Miyoung Ko (KAIST), Minjoon Seo (KAIST)
Explainability and InterpretabilityGraph Neural NetworkLarge Language ModelPrompt EngineeringText
🎯 What it does: Built a hierarchical graph-based framework that decomposes complex real-world problems into subproblems at D1, D2, and D3 levels to analyze the reasoning process of large language models (LLMs).
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy
YongKang Liu, Hinrich Schuetze
OptimizationComputational EfficiencyLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a hierarchical full-parameter fine-tuning strategy named HiFT, which updates only part of the model's layers at each step, significantly reducing GPU memory usage and enabling full-parameter fine-tuning of 7B models on 24G GPUs.
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction
Jinchuan Zhang (Chinese Academy of Sciences), Songlin Hu (Chinese Academy of Sciences)
Safty and PrivacyAdversarial AttackReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AIPrompt EngineeringText
🎯 What it does: Propose the HARM framework to achieve top-down test case generation based on fine-grained risk classification and multi-round dialogue red team attacks, and enhance LLM safety through post-detection alignment.
Holistic Evaluation for Interleaved Text-and-Image Generation
Minqian Liu (Virginia Tech), Lifu Huang (Virginia Tech)
GenerationLarge Language ModelVision Language ModelDiffusion modelMultimodalityBenchmark
🎯 What it does: Created the first evaluation benchmark INTERLEAVEDBENCH and a reference-free multidimensional metric INTERLEAVEDEVAL for comprehensive evaluation of interleaved text and image generation.
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries
Eden Biran (Tel Aviv University), Amir Globerson (Tel Aviv University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Investigate the potential reasoning process of large language models in multi-hop queries, revealing that models resolve the first hop in early layers and complete the second hop in later layers; propose back-patching techniques to verify and improve the accuracy of error cases.
Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspective
Van-Cuong Pham (VinAI Research), Thien Huu Nguyen (University of Oregon)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes an activation editing method based on Householder pseudo-rotation (HPR), achieving behavior adjustment by rotating LLM internal activations in the direction-magnitude perspective of vectors.
How Do Humans Write Code? Large Models Do It the Same Way Too
Long Li (East China Normal University), Liang He (East China Normal University)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Propose Human-Think Language (HTL), which improves the accuracy of mathematical reasoning by guiding Program-of-Thought (PoT) with complete Chain-of-Thought (CoT) logic, using Focus Attention and reinforcement learning.
How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning
Zeping Yu (University of Manchester), Sophia Ananiadou (University of Manchester)
ClassificationTransformerLarge Language ModelTextFinance Related
🎯 What it does: Investigated the context learning mechanism of large language models in sentence classification tasks, identifying and analyzing 1% attention heads critical to learning effectiveness.
How Do Your Code LLMs perform? Empowering Code Instruction Tuning with Really Good Data
Yejie Wang (Beijing University of Posts and Telecommunications), Weiran Xu (Meituan)
Data-Centric LearningAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed data cleaning and selection strategies tailored for code instruction tuning, and trained the XCoder model based on these strategies, demonstrating its outstanding performance on multiple code evaluation benchmarks.
How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?
Zhuoyan Li (Purdue University), Ming Yin (University of Connecticut)
TransformerLarge Language ModelText
🎯 What it does: Studied the impact of disclosing AI assistance information in writing on readers' evaluation of text quality and rankings.
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning?
Yang Luo (National University of Singapore), Yang You (National University of Singapore)
RetrievalTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Studied retrieval strategies of multimodal large language models (MLLM) in multimodal in-context learning (M-ICL), explored the impact of textual information on retrieval effectiveness, and proposed a supervised retrieval method MSIER based on MLLM self-assessment;
How Far Can We Extract Diverse Perspectives from Large Language Models?
Shirley Anugrah Hayati (University of Minnesota), Dongyeop Kang (University of Minnesota)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper studies how to reverse-engineer diverse perspectives from training data using large language models (LLMs), and explores the maximum achievable diversity coverage.
How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics
Adrian Cosma (National University of Science and Technology POLITEHNICA Bucharest), Cornelia Caragea (University of Illinois at Chicago)
ClassificationData-Centric LearningTransformerTextBenchmark
🎯 What it does: By analyzing the training dynamics of NLI samples, automatically divide the test set into three categories: easy, ambiguous, and hard, thereby constructing a more challenging unbiased test set, and using the same method to filter the training set to improve data quality.
How Susceptible are Large Language Models to Ideological Manipulation?
Kai Chen (University of Southern California), Kristina Lerman (University of Southern California)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Investigated the susceptibility of large language models (LLMs) to ideological bias during instruction tuning, and systematically evaluated the impact of a small number of biased data points on the model's ideology and cross-topic generalization.
How to Compute the Probability of a Word
Tiago Pimentel (ETH Zurich), Clara Meister (ETH Zurich)
TransformerLarge Language ModelText
🎯 What it does: This paper derives a method to correctly calculate word-level context probabilities from subword-level language models and corrects the erroneous practices in previous studies regarding probability calculations.
How to Leverage Demonstration Data in Alignment for Large Language Model? A Self-Imitation Learning Perspective
Teng Xiao (Pennsylvania State University), Vasant G Honavar (Pennsylvania State University)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Proposes a generic self-imitation learning (GSIL) framework that aligns large language models using offline demonstration data, eliminating adversarial training in traditional RLHF.
Human-LLM Hybrid Text Answer Aggregation for Crowd Annotations
Jiyi Li (University of Yamanashi)
Data-Centric LearningTransformerLarge Language ModelText
🎯 What it does: This paper proposes a human-AI collaborative multi-stage text answer aggregation framework called CAMS, and verifies that LLMs can serve as aggregators to improve the quality of crowd-sourced text answers.
Humans or LLMs as the Judge? A Study on Judgement Bias
Guiming Hardy Chen (Chinese University of Hong Kong Shenzhen), Benyou Wang (Chinese University of Hong Kong Shenzhen)
Explainability and InterpretabilityAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper explores four types of biases (misinformation neglect bias, gender bias, authority bias, and beauty bias) in human and large language models (LLMs) when evaluating open-ended generation tasks, and proposes a reference-free evaluation framework that does not rely on reference answers;
I Could’ve Asked That: Reformulating Unanswerable Questions
Wenting Zhao (Cornell University), Alexander M Rush (Cornell University)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Constructed the COULDASK benchmark to evaluate LLMs' ability to detect and rewrite unanswerable questions.
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses
Xuan Ren (University of Adelaide), Lingqiao Liu (University of Adelaide)
Data SynthesisData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigate why using LLM-generated responses instead of human-annotated answers can achieve better performance during fine-tuning of the target LLM, and verify the impact of 'familiarity' on learning effectiveness.
I love pineapple on pizza != I hate pineapple on pizza: Stance-Aware Sentence Transformers for Opinion Mining
Vahid Ghafouri (IMDEA Networks Institute), Guillermo Suarez-Tangil (IMDEA Networks Institute)
ClassificationRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: Fine-tuning the all-mpnet-base-v2 sentence transformer using stance alignment and triplet data to enhance its stance-aware capabilities, thereby improving the effectiveness of opinion mining and stance detection in controversial texts.
I Need Help! Evaluating LLM’s Ability to Ask for Users’ Support: A Case Study on Text-to-SQL Generation
Cheng-Kuang Wu (Appier AI Research), Yun-Nung Chen (National Taiwan University)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Studied the ability of LLMs to actively request user support in text-to-SQL generation tasks, and evaluated the trade-off between support requests and model performance versus user burden;
I-AM-G: Interest Augmented Multimodal Generator for Item Personalization
Xianquan Wang (University of Science and Technology of China), Qi Liu (University of Science and Technology of China)
GenerationRecommendation SystemTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelMultimodalityRetrieval-Augmented Generation
🎯 What it does: Designed and implemented a user-interest-oriented multimodal personalized generation framework called I-AM-G, which automatically extracts interest tags from user historical interactions and rewrites target item descriptions. Subsequently, it retrieves similar text/images and fuses them through cross-modal attention to guide diffusion models in generating text or images aligned with individual preferences;
IDEAW: Robust Neural Audio Watermarking with Invertible Dual-Embedding
Pengcheng Li (Ping An Technology Co., Ltd.), Jianzong Wang (Ping An Technology Co., Ltd.)
Flow-based ModelGenerative Adversarial NetworkAudio
🎯 What it does: Designed and implemented an audio watermark model based on a two-stage reversible neural network, separately embedding location codes and information codes to achieve reversible embedding and extraction.
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions
Reza Esfandiarpoor (Brown University), Stephen Bach
ClassificationRepresentation LearningTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Proposed the EX2 method, which uses reinforcement learning to align the preferences of large language models (LLM) with vision-language models (VLM), generating and analyzing the best conceptual descriptions according to VLM, thereby revealing the text features that VLM relies on when representing concepts.
IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning
Soeun Lee (Hanyang University), Dong-Jin Kim (Hanyang University)
GenerationRetrievalTransformerPrompt EngineeringVision Language ModelContrastive LearningImageVideoTextRetrieval-Augmented Generation
🎯 What it does: Propose a zero-shot image and video captioning method trained using only text data, bridging the gap between text and image modalities through image-like retrieval, fusion module, and frequency-based entity filtering.
IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method
MiHyeon Kim, YoungBin Kim
Adversarial AttackTransformerSupervised Fine-TuningTextOrdinary Differential Equation
🎯 What it does: View BERT as an ordinary differential equation (ODE) solver, introducing implicit Euler (IM-connection) to realize IM-BERT, enhancing the model's adversarial robustness.
ImageInWords: Unlocking Hyper-Detailed Image Descriptions
Roopal Garg (Google DeepMind), Radu Soricut (Google DeepMind)
GenerationTransformerSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: This paper proposes the ImageInWords framework, generating high-detail, accurate, and hallucination-free image descriptions through human-machine collaborative multi-round annotation;
Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective
Zhaotian Weng (University of Southern California), Jieyu Zhao (University of Southern California)
Object DetectionExplainability and InterpretabilityVision Language ModelImageMultimodality
🎯 What it does: Quantify and explain the generation and propagation paths of gender bias in vision-language models through causal mediation analysis, verified on the GLIP object detection model, followed by mitigating the bias by erasing gender information in the image encoder.
Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation
Saiful Islam Salim (University of Arizona), Sazzadur Rahaman (University of Arizona)
Adversarial AttackAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: The study prevents students from using large language models (LLMs) to complete assignments by adding adversarial perturbations to programming assignment instructions, reducing LLM cheating.
Improve Dense Passage Retrieval with Entailment Tuning
Lu Dai (Hong Kong University of Science and Technology), Hui Xiong (Hong Kong University of Science and Technology)
RetrievalSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed and implemented a training method called Entailment Tuning, which improves dense retrieval through NLI reasoning. This method utilizes a unified prompt and a masked language modeling (MLM) task with extensive hypothesis masking, making the retriever focus more on inferable information.
Improve Student’s Reasoning Generalizability through Cascading Decomposed CoTs Distillation
Chengwei Dai (Institute of Information Engineering, Chinese Academy of Sciences), Songlin Hu (Institute of Information Engineering, Chinese Academy of Sciences)
Knowledge DistillationTransformerSupervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: By splitting traditional single-step CoT distillation into two-step cascading learning (first learning the reasoning steps, then learning the answers), answer interference is removed, enhancing the student model's reasoning generalization ability on both in-domain and out-of-domain tasks.
Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning
Lu Chen (Fudan University), Xuanjing Huang (Fudan University)
Reinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningContrastive LearningTextBenchmark
🎯 What it does: In RLHF, the authors incorporate contrastive learning into the training of the reward model, combining the original ranking loss with an unsupervised contrastive loss to enhance the discriminative ability and generalization of the reward model.
Improving Knowledge Graph Completion with Structure-Aware Supervised Contrastive Learning
Jiashi Lin (Northwestern Polytechnical University), Wenxuan Lu (Northwestern Polytechnical University)
Representation LearningTransformerContrastive LearningTextGraphBenchmark
🎯 What it does: Propose a structure-aware contrastive learning framework, StructKGC, which leverages pre-trained language models (PLM) to fuse multiple subgraph structures for supervised contrastive learning in knowledge graph completion (KGC) tasks.
Improving Minimum Bayes Risk Decoding with Multi-Prompt
David Heineman (Georgia Institute of Technology), Wei Xu (Georgia Institute of Technology)
GenerationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a Multi-Prompt MBR decoding scheme, generating candidate sets using multiple prompts and selecting the optimal output by minimizing Bayesian risk (MBR);
Improving Multi-party Dialogue Generation via Topic and Rhetorical Coherence
Yaxin Fan (Soochow University), Qiaoming Zhu (Soochow University)
GenerationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposes a reinforcement learning framework (RL-TRC) based on topic and rhetoric consistency for multi-party dialogue generation. It enhances the model's perception of target utterances through topic consistency tasks and rhetoric consistency tasks, and guides the generation of more target-aligned responses using three discourse-aware rewards.
Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
Zhili Shen (Huawei Technologies), Jeff Z. Pan (Huawei Technologies)
RetrievalAI Code AssistantLarge Language ModelPrompt EngineeringTextTabularRetrieval-Augmented Generation
🎯 What it does: This paper proposes the ASTRES framework, which improves the performance of LLMs in Text-to-SQL tasks by dynamically retrieving database information, re-ranking examples using AST (Abstract Syntax Tree), and pruning the database schema along with numerical selection.
Improving Spoken Language Modeling with Phoneme Classification: A Simple Fine-tuning Approach
Maxime Poli (ENS PSL), Emmanuel Dupoux (ENS PSL)
ClassificationRepresentation LearningRecurrent Neural NetworkTransformerSupervised Fine-TuningAudio
🎯 What it does: This paper significantly improves the context independence of phoneme representations by fine-tuning frame-wise phoneme classification on HuBERT, and uses these improved discrete units to train language models.
Improving Zero-shot LLM Re-Ranker with Risk Minimization
Xiaowei Yuan (Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences), Kang Liu (Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences)
RetrievalTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed an unsupervised risk minimization re-ranking framework UR3 based on Bayesian decision theory, which compensates for the bias in LLM's QLM estimation by simultaneously maximizing the query generation probability and document generation probability, thereby improving the retrieval incremental ranking effect.
In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search
Huihan Li (University of Southern California), Xiang Ren (University of Southern California)
GenerationData SynthesisLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes the Logic-Induced Knowledge Search framework (LINK) for systematically generating long-tail reasoning knowledge and constructs the LINT dataset based on this.
In-Context Compositional Generalization for Large Vision-Language Models
Chuanhao Li (Beijing Institute Of Technology), Yunde Jia (Shenzhen Msu Bit University)
Representation LearningMeta LearningVision Language ModelMultimodality
🎯 What it does: Study how example selection can enhance the performance of large-scale vision-language models in context composition generalization
In-context Contrastive Learning for Event Causality Identification
Liang Chao (Huazhong University of Science and Technology), Bang Wang (Huazhong University of Science and Technology)
ClassificationTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Propose an ICCL model based on prompt learning and contrastive learning to identify causal relationships between event pairs.
Incomplete Utterance Rewriting with Editing Operation Guidance and Utterance Augmentation
Zhiyu Cao (Soochow University), Qiaoming Zhu (Soochow University)
GenerationGraph Neural NetworkTransformerLarge Language ModelText
🎯 What it does: This paper proposes a complete sentence rewriting model EO-IUR based on edit operation guidance, significantly improving the effectiveness of incomplete sentence rewriting through multi-task learning that combines sequence labeling and generation tasks.
Incubating Text Classifiers Following User Instruction with Nothing but LLM
Letian Peng (University of California San Diego), Jingbo Shang (University of California San Diego)
ClassificationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: This paper proposes the Incubator framework, which automatically generates text classification training data based on complex and interdependent category definitions provided by users through instruction fine-tuning of large language models, thus enabling the training of text classifiers without the need for human annotation;
Induct-Learn: Short Phrase Prompting with Instruction Induction
Po-Chun Chen (National Taiwan University), Hsin-Hsi Chen (National Taiwan University)
Computational EfficiencyLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes the low-cost INDUCT-LEARN framework, which significantly improves the performance of large language models on new tasks by using a small number of input-output examples and a short phrase to induce pseudo-instructions and pseudo-chain-of-thought (CoT).
Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues
Jiao Ou (Kuaishou), Kun Gai (Kuaishou)
GenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the IDEAS method, which first inductively extracts instruction strategies from real human-computer dialogues, then uses these strategies to deductively generate diverse and in-depth instructions in new dialogues, constructing multi-round instruction dialogues and using these dialogues to fine-tune chat models.
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
Pengyu Wang (Fudan University), Xipeng Qiu (Fudan University)
Safty and PrivacyTextBenchmarkFinance Related
🎯 What it does: Proposed InferAligner, a simple method that achieves harmless alignment during the inference phase, significantly reducing the model's harmful responses without compromising the performance of downstream tasks.
Inference Helps PLMs’ Conceptual Understanding: Improving the Abstract Inference Ability with Hierarchical Conceptual Entailment Graphs
Juncai Li, Jeff Z. Pan (Shanxi University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Construct a hierarchical concept entailment graph (HiCon-EG) by simplifying complex sentences, pyramidizing concepts, and selecting concepts based on entropy to mine multi-level predicate and noun entailment relationships, thereby enhancing the abstract reasoning and common-sense inference capabilities of pre-trained language models (PLMs).
InfiniPot: Infinite Context Processing on Memory-Constrained LLMs
Minsoo Kim (Hanyang University), Simyung Chang (Qualcomm AI Research)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose InfiniPot, a KV cache control framework that allows pre-trained LLMs to handle arbitrarily long input contexts under a fixed memory budget.
Information Flow Routes: Automatically Interpreting Language Models at Scale
Javier Ferrando (Universitat Politecnica De Catalunya), Elena Voita (Meta Ai)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposed an explanation method based on Information Flow Routes, which automatically identifies important subgraphs in language model predictions through top-down graph extraction techniques, and evaluates edge importance using attribution rather than patches.
Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes
Kosuke Nishida (NTT Human Informatics Laboratories, NTT Corporation), Kuniko Saito (NTT Human Informatics Laboratories, NTT Corporation)
TransformerLarge Language ModelText
🎯 What it does: Proposes the WeSaR method, which introduces a trainable gating parameter α for each parameter matrix to achieve reparameterization with a unified standard deviation, thereby eliminating loss spikes during LLM pre-training and enhancing training stability and speed.
Instruction Fine-Tuning: Does Prompt Loss Matter?
Mathew Huerta-Enochian (University of Oregon), Seung Yong Ko (EQ4ALL)
GenerationOptimizationTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: The study reveals the impact of Policy Loss Weight (PLW) on supervised instruction fine-tuning (SIFT), and experimentally demonstrates its quadratic relationship with model performance.
Instruction Matters: A Simple yet Effective Task Selection for Optimized Instruction Tuning of Specific Tasks
Changho Lee (LG AI Research), Kyunghoon Bae (LG AI Research)
OptimizationTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a task selection method (INSTA) that uses only task instruction text to optimize instruction tuning, enhancing zero-shot performance on specific tasks
Instruction Pre-Training: Language Models are Supervised Multitask Learners
Daixuan Cheng (Microsoft Research), Furu Wei (Microsoft Research)
Data SynthesisRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextFinance Related
🎯 What it does: Explored supervised multi-task learning using instruction-response pairs during pre-training, proposing the Instruction Pre-Training framework
IntCoOp: Interpretability-Aware Vision-Language Prompt Tuning
Soumya Suvra Ghosal (University of Maryland), Dinesh Manocha (University of Maryland)
Explainability and InterpretabilityRepresentation LearningTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Propose the Int CoOp method, introducing attribute-level interpretability in CLIP's prompt tuning, enabling prompts to incorporate and learn image attributes.
Integrating Argumentation and Hate-Speech-based Techniques for Countering Misinformation
Sougata Saha (State University of New York at Buffalo), Rohini Srihari (State University of New York at Buffalo)
Graph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a persuasive dialogue system based on argumentation graphs, which generates persuasive responses to online misinformation by first planning response strategies and then generating response texts.
Integrating Plutchik’s Theory with Mixture of Experts for Enhancing Emotion Classification
Dongjun Lim (Sungkyunkwan University), Yun-Gyung Cheong (Sungkyunkwan University)
ClassificationTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: This paper proposes a composite emotion splitting and re-annotation method based on Plutchik's emotion theory, combining it with a Mixture of Experts (MoE) architecture for multi-label emotion classification.
Integrating Structural Semantic Knowledge for Enhanced Information Extraction Pre-training
Xiaoyang Yi (Nankai University), Faxin Lin (Nankai University)
RecognitionGraph Neural NetworkTransformerContrastive LearningTextGraph
🎯 What it does: Propose the SKIE pre-training framework, leveraging AMR graphs and their generable subgraphs for contrastive learning to enhance the structural semantic capabilities of information extraction models.
InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context
Ziyi Liu (University of Southern California), Jieyu Zhao (University of Southern California)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Developed the INTERINTENT framework, systematically evaluating the social intelligence of LLMs in the Avalon social reasoning game through four dimensions of intention understanding (situational awareness, self-regulation, self-awareness, theory of mind).
Interpretability-based Tailored Knowledge Editing in Transformers
Yihuai Hong (University College London), Aldo Lipani (University College London)
Explainability and InterpretabilityRepresentation LearningTransformerText
🎯 What it does: Propose a customized knowledge editing method called TailoredKE based on the interpretability of the Transformer's internal mechanisms
Interpretable Composition Attribution Enhancement for Visio-linguistic Compositional Understanding
Wei Li (University of Science and Technology of China), Jieping Ye (Alibaba Cloud)
Explainability and InterpretabilityTransformerSupervised Fine-TuningContrastive LearningImageTextMultimodality
🎯 What it does: Investigated the shortcomings of contrastive vision-language models like CLIP in understanding attributes and relationships, and introduced an attribution loss during training to encourage the model to pay more attention to relational and attribute words in the text, thereby enhancing compositional reasoning capabilities.
Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis
Zeping Yu (University of Manchester), Sophia Ananiadou (University of Manchester)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Investigate the internal mechanisms of arithmetic capabilities in large language models (LLMs), proposing the Comparative Neuron Analysis (CNA) method to identify key attention heads and FFN neurons, and constructing a four-stage internal logic chain from input to prediction.
Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
Clement Neo (Apart Research), Fazl Barez (Apart Research)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Studied the interaction between attention heads in Transformers and next-word prediction MLP neurons, revealing that attention heads activate corresponding prediction neurons in specific contexts, and utilized GPT-4 to automatically generate and evaluate explanations for these interactions.
Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding
YeonJoon Jung, Seung-won Hwang (Seoul National University)
RecognitionGenerationData SynthesisDomain AdaptationTransformerTextAudio
🎯 What it does: Propose a speech noise injection method based on causal intervention (ISNI) to generate pseudo transcriptions that are generalizable across any ASR system, thereby enhancing the robustness of spoken language understanding models against ASR errors.
Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations
Yucheng Jiang (Stanford University), Monica Lam (Stanford University)
TransformerLarge Language ModelAgentic AIPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Built Co-STORM, a system that simulates collaborative discussions, allowing users to observe and occasionally participate in dialogues between multilingual model agents, track the discussion process through dynamic mind maps, and ultimately generate long-form reports with citations.
Intrinsic Self-correction for Enhanced Morality: An Analysis of Internal Mechanisms and the Superficial Hypothesis
Guangliang Liu (Michigan State University), Kristen Johnson
Safty and PrivacyExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Conduct an in-depth analysis of the moral self-correction mechanisms in large language models (LLMs) and verify that they are primarily superficial.
Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models
Yufang Liu (East China Normal University), Aimin Zhou (East China Normal University)
RecognitionAnomaly DetectionLarge Language ModelSupervised Fine-TuningContrastive LearningImageTextBenchmark
🎯 What it does: This paper systematically investigates the object hallucination problem of the CLIP model in vision-language tasks, constructs the OHD-Caps benchmark, and fine-tunes CLIP through generating negative samples and fine-grained contrastive loss, significantly reducing the hallucination rate.
Investigating Large Language Models for Complex Word Identification in Multilingual and Multidomain Setups
Răzvan-Alexandru Smădu, Mihaela-Claudia Cercel (Paris 1 Panthéon-Sorbonne University)
ClassificationRecognitionMeta LearningTransformerSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: This study explores the performance of large language models (LLMs) in complex word identification (CWI) and lexical complexity prediction (LCP) tasks across multiple languages and domains, combining zero-shot, few-shot, and fine-tuning settings;
Investigating LLMs as Voting Assistants via Contextual Augmentation: A Case Study on the European Parliament Elections 2024
Ilias Chalkidis (University of Copenhagen)
Recommendation SystemTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper evaluates the feasibility of the latest open-weight LLMs (Mistral 7B and Mistral 8×7B Mixtral) in a European Parliament election voting advice application (VAA), focusing on testing their stance prediction capabilities on the EUANDI-2024 questionnaire.
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?
Alexander Arno Weber (Lamarr Institute), Mehdi Ali (Lamarr Institute)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This study systematically evaluates the impact of different language combinations, data scales, and generation methods on the instruction-following performance of multilingual large language models (medium 7B and large Mixtral-8x7B) by constructing a high-quality parallel multilingual instruction fine-tuning dataset (Lima-X) and an evaluation benchmark (MT-Bench-X).
Investigating Mysteries of CoT-Augmented Distillation
Somin Wadhwa (Northeastern University), Byron C Wallace (Northeastern University)
Knowledge DistillationTransformerTextChain-of-Thought
🎯 What it does: Studied the effect of using chain-of-thought (CoT) reasoning chains during model distillation on small models, and analyzed the reasons through various ablation experiments.
Investigating the Role of Instruction Variety and Task Difficulty in Robotic Manipulation Tasks
Amit Parekh (Heriot Watt University), Ioannis Konstas (Heriot Watt University)
Robotic IntelligenceTransformerVision-Language-Action ModelMultimodalityBenchmark
🎯 What it does: Proposed a systematic evaluation framework that comprehensively assesses the robustness and generalization ability of multimodal instructions in robotic manipulation tasks using the VIMA-BENCH benchmark.
Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
Abhinav Bandari (University of Washington), Shiwei Liu (University of Washington)
Computational EfficiencyTransformerTextChain-of-Thought
🎯 What it does: Systematically evaluated the effects of different calibration data (pre-training sets and downstream task sets, combined with In-Context Learning and Chain-of-Thought) on pruning large language models.