🎯 What it does: This paper proposes a complete workflow for pretraining low-resource languages using synthetic data generated from machine translation, known as 'translationese,' and verifies the effectiveness of this method on various downstream tasks.
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Seungone Kim (KAIST AI), Minjoon Seo (KAIST AI)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose PROMETHEUS 2, a unified evaluation language model that merges model weights trained separately on direct assessment and pairwise ranking tasks, enabling simultaneous support for both evaluation formats.
PSC: Extending Context Window of Large Language Models via Phase Shift Calibration
Wenqiao Zhu (HiThink Research), Jun Wu (HiThink Research)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a Phase Shift Calibration (PSC) module to fine-tune the frequencies of existing RoPE extensions, thereby enhancing the performance of large language models in long-context scenarios.
🎯 What it does: Propose PTD-SQL, which first divides queries into four categories—multi-set, combination, filtering, and simple—based on SQL keywords, then constructs corresponding target practice libraries for each category, using a small number of examples to guide LLMs in generating SQL.
QuBE: Question-based Belief Enhancement for Agentic LLM Reasoning
Minsoo Kim (Seoul National University), Seung-won Hwang (Seoul National University)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AITextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes the QuBE method, which constructs belief states through question-answering and generates belief-based reasoning to address the reasoning drift problem of LLM agents in partially observable environments.
James Liyuan Wang (Columbia University), Chengzhi Mao (Columbia University)
CodeAdversarial AttackLarge Language ModelText
🎯 What it does: Proposes RAFT, a zero-shot black-box attack framework that misleads text detectors by replacing key words in LLM-generated text with grammatically correct substitutions.
Rujun Han (Google), Vittorio Castelli (AWS AI Labs)
CodeDomain AdaptationTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed the LFRQA dataset, which includes multi-domain, long-form, and coherent human-annotated answers for cross-domain robustness evaluation of retrieval-augmented generative QA (RAG-QA), and built the RAG-QA ARENA evaluation framework based on this dataset;
🎯 What it does: For low-resource programming languages, a two-step retrieval (RAR) mechanism is proposed, first retrieving the driving context through examples or documents, and then filtering relevant information from another resource (document or example) via affected retrieval for GPT-4 code generation.
REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering
Yuhao Wang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeRetrievalLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes the REAR framework, which introduces an explicit relevance assessment module into retrieval-augmented generation (RAG) systems to improve the LLM's ability to judge and utilize the credibility of retrieved documents;
Red Teaming Language Models for Processing Contradictory Dialogues
Xiaofei Wen (University of California, Davis), Muhao Chen (University of California, Davis)
CodeGenerationData SynthesisExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a new task of detecting and modifying dialog contradictions, and constructs a dataset containing 12,387 dialogues (6,130 containing contradictions), accompanied by annotations on contradiction positions and reasons;
RepEval: Effective Text Evaluation with LLM Representation
Shuqian Sheng (Shanghai Jiao Tong University), Chenghu Zhou (Chinese Academy Of Sciences)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes RepEval, a metric that evaluates text quality by leveraging the projection of hidden representations from large language models, supporting both absolute evaluation and comparative evaluation;
Representational Analysis of Binding in Language Models
Qin Dai (Tohoku University), Kentaro Inui (MBZUAI)
CodeExplainability and InterpretabilityRepresentation LearningTransformerText
🎯 What it does: Study the internal binding mechanisms in language models, identify the low-rank subspace (OI subspace), and demonstrate its causal impact on entity-attribute binding behaviors.
Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation
Yuanjie Lyu, Enhong Chen (Anhui Conch Information Technology Engineering Co Ltd)
CodeGenerationRetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposes the Retrieve-Plan-Generation (RPG) framework, integrating iterative planning and retrieval into LLM generation to enhance relevance and accuracy for knowledge-intensive generation tasks.
CodeRecurrent Neural NetworkTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Construct an LLM agent framework called Retrospex, which rescores past experiences using an offline RL critic to improve decision quality without expanding the LLM context.
Revealing Personality Traits: A New Benchmark Dataset for Explainable Personality Recognition on Dialogues
Lei Sun (Renmin University of China), Qin Jin (Renmin University of China)
CodeRecognitionExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: Propose the explainable personality recognition task, constructing a dialogue-based explainable personality dataset named PersonalityEvd, and defining two subtasks: EPR-S (state recognition) and EPR-T (trait recognition);
Revealing the Parallel Multilingual Learning within Large Language Models
Yongyu Mu (Northeastern University), JingBo Zhu
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: The paper proposes and verifies the use of parallel multilingual input (PMI) for cross-lingual context learning in large language models (LLMs), exploring its impact on model performance and neural activation patterns.
Revisiting Automated Evaluation for Long-form Table Question Answering
Yuqi Wang (Independent Researcher), Yilun Zhao (Yale University)
CodeTransformerLarge Language ModelTabularBenchmark
🎯 What it does: Constructed a meta-evaluation dataset called LFTQA-Eval containing 2988 human-annotated examples, and systematically evaluated the reliability of existing automatic evaluation metrics for long-table question answering.
Olga Zamaraeva (Universidade da Coruña), Carlos Gómez-Rodríguez (Universidade da Coruña)
CodeComputational EfficiencyRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Multiple HPSG super taggers are constructed using SVM, LSTM+CRF, and BERT, with the best model integrated into the ACE parser to enhance parsing speed and accuracy.
🎯 What it does: In the Weibo text classification task, the authors fine-tune RoBERTa-base and incorporate a linear combination of supervised contrastive learning (SCL) and cross-entropy loss in the loss function.
Revisiting the Robustness of Watermarking to Paraphrasing Attacks
Saksham Rastogi (Indian Institute of Science), Danish Pruthi (Indian Institute of Science)
CodeAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This paper investigates the robustness of text watermarks against rewriting attacks and successfully extracts a green list through reverse engineering methods, further enhancing the effectiveness of rewriting attacks.
Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering
Armin Toroghi (University of Toronto), Scott Sanner (University of Toronto)
CodeExplainability and InterpretabilityTransformerLarge Language ModelTextGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose a verifiable common-sense knowledge graph question-answering framework, R³, which leverages the inherent common-sense knowledge of large language models (LLMs) and strictly grounds reasoning steps on knowledge graph triplets, forming a tree-structured search process;
RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework
Yifan Wang (Saarland University), Vera Demberg (Saarland University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose the RSA-Control framework, which utilizes speaker-listener recursive reasoning to achieve zero-training controllable text generation, and evaluates it on tasks such as toxicity reduction, bias mitigation, and readability summarization.
Safely Learning with Private Data: A Federated Learning Framework for Large Language Model
Jia-Ying Zheng, Zhi-Ming Zheng
CodeFederated LearningSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: Designed the FL-GLM framework, which splits the LLM into input/output blocks on the client side and large blocks on the server side, achieving secure and efficient federated learning through encrypted communication;
Satyrn: A Platform for Analytics Augmented Generation
Marko Sterbentz (Northwestern University), Kristian J Hammond (Northwestern University)
CodeGenerationTransformerLarge Language ModelTextTabularElectronic Health Records
🎯 What it does: Propose the SATYRN platform to achieve analysis-enhanced generation (AAG), generating fact sets by analyzing structured databases and guiding LLMs to produce accurate and coherent reports.
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: Train LLM to provide fine-grained confidence and self-reflective reasoning justifications.
Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
Clara Na (Allen Institute for AI), Pradeep Dasigi (Allen Institute for AI)
CodeComputational EfficiencyData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Propose a method to approximate large-scale language model data ablation through modular training and parameter averaging, significantly reducing experimental costs.
Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention
Xingtai Lv, Bowen Zhou (Tsinghua University)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed a low-dimensional projection attention (LPA) structure that applies low-rank modules only to the attention layer, significantly reducing the number of parameters and computational overhead while maintaining or even improving model performance.
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Conduct large-scale pretraining of three linear-complexity language models (TNL, HGRN2, cosFormer2) across scales from 70M to 7B, establishing power-law scaling patterns between training loss, computational budget, and model/data size, and evaluating them on multiple downstream tasks (validation perplexity, common sense reasoning, retrieval generation) with traditional Transformers (LLaMA).
Santiago Cuervo (Université de Toulon), Ricard Marxer (Université de Toulon)
CodeData SynthesisComputational EfficiencyData-Centric LearningTransformerLarge Language ModelTextAudio
🎯 What it does: Trained over 50 Speech Language Models (SLMs) under different scales and data budgets, evaluating their upstream loss and downstream syntactic and semantic performance, and proposed a new synthetic corpus STINYSTORIES to assess the impact of coarse-grained speech segmentation on performance.
SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers
Shruti Singh (Iit Gandhinagar), Arman Cohan
CodeLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed the SCIDQA dataset, consisting of 2,937 question-answer pairs extracted from OpenReview peer review discussions. The dataset underwent multi-stage refinement, including LLM extraction, human screening, and decontextualization, aiming to test models' deep understanding of scientific papers.
SciEx: Benchmarking Large Language Models on Scientific Exams with Human Expert Grading and Automatic Grading
Tu Anh Dinh (Karlsruhe Institute of Technology), Jan Niehues (Karlsruhe Institute of Technology)
CodeTransformerLarge Language ModelMultimodalityBenchmark
🎯 What it does: Introduce the SciEx benchmark, which evaluates the capabilities of large language models on scientific tasks using university-level computer science exam questions, and provides both expert scoring and automatic scoring.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed a context example method SCOI based on alternating selection of syntactic and lexical coverage to enhance the ICL performance of large language models in machine translation.
🎯 What it does: Built the SEACrowd resource center, integrating approximately 500 datasets, 399 standardized data loaders, covering nearly 1000 Southeast Asian languages, and created a multimodal benchmark based on 13 tasks (covering 36 languages); simultaneously conducted zero-shot evaluation of existing LLM, VLM, and ASR models.
Searching for Best Practices in Retrieval-Augmented Generation
Xiaohua Wang (Fudan University), Xuanjing Huang (Fudan University)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper explores the optimal implementation methods for each module in the Retrieval-Augmented Generation (RAG) framework through systematic experiments, and proposes two practical best practices (maximum performance and balanced efficiency) along with multimodal retrieval expansion;
Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?
Neeladri Bhuiya (National University of Singapore), Stefan Winkler (ASUS Intelligent Cloud Services (AICS))
CodeAdversarial AttackTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: The study investigates how large language models are easily misled by 'plausible but incorrect' distractor paths in multi-hop reasoning tasks, and proposes an evaluation method based on generable feasible distractor paragraphs.
🎯 What it does: Designed and implemented the SEER framework for self-aligned evidence extraction in retrieval-augmented generation, reducing computational costs and improving answer quality.
CodeRecognitionObject DetectionLarge Language ModelVision Language ModelImageMultimodalityBenchmark
🎯 What it does: Constructed the VisArgs dataset and proposed three evaluation tasks for visual argumentation (localization, identification, deduction).
CodeExplainability and InterpretabilityTransformerPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes Self-AMPLIFY, a framework that enhances the performance of small autoregressive language models (SLM) in in-context learning (ICL) by automatically generating reasoning steps using self-model post-hoc explanations.
Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
Christopher Schröder (Center for Scalable Data Analytics and Artificial Intelligence), Gerhard Heyer (Center for Scalable Data Analytics and Artificial Intelligence)
🎯 What it does: The study combines self-training with active learning to improve sample efficiency in text classification using pre-trained language models.
Semantics and Sentiment: Cross-lingual Variations in Emoji Use
Giulio Zhou (University of Edinburgh), Sumin Zhao (University of Edinburgh)
CodeTransformerLarge Language ModelText
🎯 What it does: This study conducted two experiments to collect the literal meanings of emojis and their literal/metaphorical uses in sentences across three languages: English, Portuguese, and Chinese, and explored their relationship with emotions.
CodeRepresentation LearningTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Propose the Semformer model, which introduces learnable planning tokens into Transformer language models and enhances the model's prospective reasoning by predicting the latent semantic representations of future text through an autoencoder;
Simultaneous Masking, Not Prompting Optimization: A Paradigm Shift in Fine-tuning LLMs for Simultaneous Translation
Matthew Raffel (Oregon State University), Lizhong Chen (Oregon State University)
CodeGenerationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed a new Fine-tuning paradigm called SimulMask, which simulates parallel translation directly within LLMs through attention masks, avoiding issues such as training-inference mismatch, position confusion, and high computational costs caused by traditional prompting optimization methods.
Small LLMs Are Weak Tool Learners: A Multi-LLM Agent
Weizhou Shen (Sun Yat Sen University), Fei Huang (Alibaba Group)
CodeTransformerLarge Language ModelSupervised Fine-TuningAgentic AITextBenchmark
🎯 What it does: Propose the α-UMi multi-LLM framework, decomposing tool learning tasks into three small models as planner, caller, and summarizer, significantly enhancing the tool usage capability of small models.
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
Jinghan Jia (Michigan State University), Sijia Liu (MIT-IBM Watson AI Lab, IBM Research)
CodeOptimizationTransformerLarge Language ModelText
🎯 What it does: Proposed a second-order optimization-based LLM forgetting framework called SOUL to enhance the forgetting effectiveness of large language models while maintaining their original utility.
🎯 What it does: This paper proposes a sparse gradient fine-tuning method called SparseGrad for the MLP block of Transformer models, achieving parameter-efficient fine-tuning by leveraging the sparsity in the gradient space;
SpeechQE: Estimating the Quality of Direct Speech Translation
HyoJung Han (University of Maryland), Marine Carpuat (University of Maryland)
CodeConvolutional Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkAudio
🎯 What it does: For the speech translation quality estimation (SpeechQE) task, a benchmark dataset was constructed and two systems were implemented: end-to-end (E2E) and cascaded (ASR + text-QE). The systems can provide sentence-level quality scores for direct speech translation outputs and detect error segments.
Split and Merge: Aligning Position Biases in LLM-based Evaluators
Zongjie Li (Hong Kong University of Science and Technology), Yang Liu (Nanyang Technological University)
CodeLarge Language ModelTextBenchmark
🎯 What it does: Proposed the PORTIA system, which eliminates position bias in LLM evaluation and improves consistency by segmenting answers, aligning lengths, and aligning semantics.
Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosis
Amey Hengle (Indian Institute of Technology Bombay), Rashmi Gupta (Sophia College for Women)
CodeClassificationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This study proposes ANGST, a new benchmark for the classification of comorbid depression and anxiety based on social media posts. Unlike existing datasets, ANGST supports multi-label classification, allowing each post to be labeled as both/depression and/or anxiety simultaneously.
Story Embeddings — Narrative-Focused Representations of Fictional Stories
Hans Ole Hatzel (Universitat Hamburg), Chris Biemann (Universitat Hamburg)
CodeRetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Proposed the StoryEmb model, which trains story embeddings using contrastive learning to make summaries of similar narratives have close vectors.
Story Morals: Surfacing value-driven narrative schemas using large language models
David G Hobson (McGill University), Andrew Piper (McGill University)
CodeTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This paper proposes a narrative structure annotation task based on the concept of 'story morality,' utilizing large language models to automatically extract and verify values and lessons across various text types (fairy tales, novels, movies/tv shows, social media personal stories, news);
CodeGenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed the STORYSUMM dataset for evaluating the faithfulness of narrative text summarization, constructed with extended golden labels through multiple human annotation protocols.
Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations
Matthias Lindemann (University of Edinburgh), Ivan Titov (University of Edinburgh)
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose an intermediate pre-training method on Transformer, enabling the model to learn syntactic transformations based on dependency trees, thereby reinforcing structural inductive bias;
Style-Specific Neurons for Steering LLMs in Text Style Transfer
Wen Lai (Technical University of Munich), Alexander Fraser (Technical University of Munich)
CodeGenerationDomain AdaptationTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose the sNeuron-TST framework, which first identifies and removes source-style-specific neurons, then combines contrastive decoding (Dola) to improve the generation performance of LLMs in text style transfer.
Subword Segmentation in LLMs: Looking at Inflection and Consistency
Marion Di Marco (Technische Universität München), Alexander Fraser (Technische Universität München)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Analyze the subword segmentation strategies of large language models (e.g., GPT-4o), and evaluate the segmentation quality through morphological analysis across ten languages, investigating the impact of segmentation quality on semantic capture and morphological generation.
SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories
Ben Bogin (Allen Institute for AI), Tushar Khot (Allen Institute for AI)
CodeAI Code AssistantTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: Proposes the SUPER benchmark to evaluate the ability of large language models to set up, configure, and execute complete experiments in low-attention research repositories.
SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information
Jiashuo Sun (Xiamen University), Yu Cheng (Chinese University of Hong Kong)
CodeClassificationRecognitionGenerationRetrievalSupervised Fine-TuningPrompt EngineeringVision Language ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes the self-improving SURf framework, which trains large vision-language models (LVLMs) to selectively utilize retrieved information and suppress irrelevant or misleading retrieval content, enhancing the robustness and performance of RAG tasks.
Surprise! Uniform Information Density Isn’t the Whole Story: Predicting Surprisal Contours in Long-form Discourse
Eleftheria Tsipidi (ETH Zürich), Alex Warstadt (ETH Zürich)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the Structured Context Hypothesis, predicting surprisal contours in text through hierarchical discourse structures (RST and ordinary prose structures), and quantitatively evaluate its effectiveness using Bayesian linear regression.
Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems
Vishal Vivek Saley (Indian Institute of Technology), Mausam .
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: By combining the in-context learning of large language models with task prompts (entity type, response length, dialogue closure), an end-to-end task-oriented dialogue system named SyncTOD was constructed to enhance response alignment and performance in low-data scenarios.
🎯 What it does: This paper proposes SYNTHESIZRR, a text dataset synthesis method based on retrieval enhancement. It uses a teacher LLM to perform task inversion on retrieved documents, generating diverse synthetic samples, which are then used for distillation training of the student model.
T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings
Björn Deiseroth (Aleph Alpha), Samuel Weinbach (Aleph Alpha)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Proposed T-FREE, a tokenizer-free sparse representation method that directly activates word embeddings using hashed character triplets, enabling text encoding and decoding without requiring a reference corpus.
Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition
Hsuan Su (National Taiwan University), Hung-yi Lee (National Taiwan University)
CodeRecognitionDomain AdaptationTransformerAudio
🎯 What it does: This paper proposes SYN2REAL, a task vector that encodes the differences between synthetic and real speech into model parameters, enabling ASR adaptation to unseen domains.
Teaching LLMs to Abstain across Languages via Multilingual Feedback
Shangbin Feng (University of Washington), Yulia Tsvetkov (University of Washington)
CodeRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes using multilingual feedback to enable large language models to self-reflect in multi-lingual question answering, thereby enhancing their ability to refuse inappropriate queries.
TempoFormer: A Transformer for Temporally-aware Representations in Change Detection
Talia Tseriotou (Queen Mary University of London), Maria Liakata (Queen Mary University of London)
CodeClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextTime Series
🎯 What it does: Propose TempoFormer, a Transformer model for time-aware change detection, which can simultaneously model word-level, post-level, and time-series-level representations of text units without using recursion.
🎯 What it does: This paper proposes the RAPID framework, which repairs the semantics of text adversarial samples by integrating adversarial detectors and perturbation focusing technology within a pre-trained language model.
The Greatest Good Benchmark: Measuring LLMs’ Alignment with Utilitarian Moral Dilemmas
Giovanni Franco Gabriel Marraffini (Universidad De Buenos Aires), Luciano Del Corro (Universidad De Buenos Aires)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: By constructing the 'Greatest Good Benchmark (GGB)', the Oxford Utilitarianism Scale (OUS) was expanded and adapted for large language models (LLMs), evaluating and comparing the judgments of 15 LLMs of different scales and origins in moral dilemmas.
The Illusion of Competence: Evaluating the Effect of Explanations on Users’ Mental Models of Visual Question Answering Systems
Judith Sieker (Bielefeld University), Sina Zarrieß (Honda Research Institute Europe)
CodeExplainability and InterpretabilityVision Language ModelMultimodality
🎯 What it does: Conduct experimental research on whether users can more accurately build mental models of a system's capabilities and limitations when receiving natural language explanations in visual question-answering systems. The experiment controls system inputs (color vs. grayscale images) to artificially induce model defects, comparing two experimental settings with and without explanations.
The Instinctive Bias: Spurious Images lead to Illusion in MLLMs
Tianyang Han (Hong Kong Polytechnic University), Tong Zhang (University of Illinois at Urbana-Champaign)
CodeData SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Study and quantify the 'instinctive bias' (visual hallucination) in multimodal large language models (MLLMs) when confronted with pseudo-relevant images, and construct the first such benchmark—CorrelationQA.
The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?
Alexander S. Choi (George Mason University), Antonios Anastasopoulos (George Mason University)
CodeTransformerLarge Language ModelText
🎯 What it does: Conduct topic modeling experiments on AI policy documents to evaluate the interactive effects of efficiency gains and cognitive biases when experts collaborate with large language models (LLMs).
The Lou Dataset - Exploring the Impact of Gender-Fair Language in German Text Classification
Andreas Waldis (Technical University of Darmstadt), Iryna Gurevych (Lucerne University of Applied Sciences and Arts)
CodeClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper constructs the Lou dataset to systematically evaluate the impact of gender-fair language (Gender-Fair Language) on German text classification tasks.
🎯 What it does: Conducted a qualitative analysis of 150 papers involving 'low-resource languages,' extracting four dimensions defining 'low-resource languages' and proposing corresponding terminology recommendations.
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability
Xinyu Hu (Peking University), Xiaojun Wan (Peking University)
CodeExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose a large-scale no-reference NLG evaluation corpus, NLG-Eval, and train a specialized evaluation LLM, Themis, to achieve flexible and interpretable assessments.
Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting
Stephen Meisenbacher (Technical University of Munich), Florian Matthes (Technical University of Munich)
CodeSafty and PrivacyData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper conducts a case study on DP-PROMPT, comparing strict differential privacy (DP), loose DP (Quasi-DP), and non-DP text rewriting methods, evaluating their semantic similarity, readability, and privacy protection effects.
Threshold-driven Pruning with Segmented Maximum Term Weights for Approximate Cluster-based Sparse Retrieval
Yifan Qiao (University of California at Santa Barbara), Tao Yang (University of California at Santa Barbara)
CodeRetrievalComputational EfficiencyText
🎯 What it does: This paper proposes ASC ((µ,η)-approximate control scheme), a cluster-level pruning method based on threshold-driven, segmented maximum term weight, which can significantly accelerate retrieval in sparse retrieval while maintaining high relevance.
🎯 What it does: Designed and implemented a time-aware retrieval-rewrite-retrieval-re-ranking (TimeR4) framework, which injects temporal constraints from time knowledge graphs (TKG) into large language models through retrieval and rewriting strategies to enhance their reasoning capabilities in time knowledge graph question answering (TKGQA).
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextTabularBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper redefines the text-to-table task in the long-text domain, proposing a two-stage TKGT pipeline: first generating domain-specific knowledge graph classes using hybrid IE (rule-based, statistical, and deep learning), then utilizing Hybrid-RAG with dynamic prompting to guide LLMs to populate tables based on KG classes; additionally, it constructs a challenging CPL dataset with legal judgment texts.
TL-CL: Task And Language Incremental Continual Learning
Shrey Satapara (Indian Institute of Technology Hyderabad), P. K. Srijith (Indian Institute of Technology Hyderabad)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the task and language incremental continual learning (TLCL) problem, proposed Task and Language-Specific Adapters (TLSA) adapter solution, and compared various continual learning methods under a multi-task and multi-language setting.
Craig W Schmidt, Chris Tanner (Kensho Technologies)
CodeCompressionRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: This paper introduces the PATHPIECE tokenizer and systematically evaluates the impact of each stage of tokenization (pre-tokenization, vocabulary construction, segmentation) on LLM training and downstream task performance, verifying that compression rate is not a decisive factor;
ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback
Qinzhuo Wu (XiaoMi AI Lab), Bin Wang (XiaoMi AI Lab)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITextBenchmark
🎯 What it does: Construct a multi-granularity instruction dataset MGToolBench and propose a two-stage reinforcement learning framework ToolPlanner to enhance task completion and instruction following capabilities of tool-augmented LLMs under multi-granularity instructions.
TopViewRS: Vision-Language Models as Top-View Spatial Reasoners
Chengzu Li (University of Cambridge), Ivan Vulić (University of Cambridge)
CodeClassificationRecognitionPrompt EngineeringVision Language ModelImageTextBenchmarkChain-of-Thought
🎯 What it does: Propose the TOPVIEWRS dataset to evaluate the spatial reasoning capabilities of vision-language models (VLMs) from top-down views, and conduct fine-grained assessment through four progressively complex tasks (identification, localization, static reasoning, dynamic reasoning).
Towards a Greek Proverb Atlas: Computational Spatial Exploration and Attribution of Greek Proverbs
John Pavlopoulos (Athens University of Economics and Business), Panagiotis Filos (Athens University of Economics and Business)
CodeClassificationTransformerTextBenchmark
🎯 What it does: Constructed the first public, machine-processable large-scale dataset of Greek proverbs, and performed computational spatial exploration of their geographic distribution; achieved prediction of proverb location attribution through text classification and regression algorithms, and localized unattributed proverbs;
Clara Meister (ETH Zürich), Tiago Pimentel (ETH Zürich)
CodeTransformerLarge Language ModelText
🎯 What it does: Proposed 'similarity-regulated surprise'—incorporating word similarity into traditional surprise to measure predictability in context, thereby enabling a more refined quantification of sentence comprehension cost.
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the ALT (ALignment with Textual feedback) method, which aligns language models using textual feedback, directly achieved through conditional language model training
🎯 What it does: This paper proposes evaluation and improvement methods for cross-cultural machine translation, constructing a benchmark dataset XC-Translate specifically for texts containing culturally diverse entity names, and introduces the KG-MT translation framework that utilizes multilingual knowledge graphs for retrieval-augmented generation;
Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs
Mihir Parmar (Arizona State University), Trung Bui (Adobe Research)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed an extractive summary dataset containing natural language user intent feedback, and used this dataset to fine-tune large language models to improve the coherence of summaries.
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
Euiin Yi (KAIST AI), Se-Young Yun (KAIST AI)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a voting-based reasoning method that combines small, language-specific 'drafter' models trained for each language with large models to accelerate the inference speed of multilingual LLMs.
🎯 What it does: Proposed an end-to-end online continuous sign language recognition (CSLR) framework. First, a pre-trained CTC model is used for sign segmentation to build a dictionary. Then, an improved ISLR model is trained on the dictionary, and real-time recognition is achieved through sliding windows and post-processing. The framework can also integrate with a gloss-to-text network to realize online sign language translation (SLT), and performance of offline CSLR models can be enhanced through lightweight adapters.
🎯 What it does: Developed the XEUS cross-lingual speech representation learning model, pre-trained on a large-scale dataset with over 1 million hours of speech across 4,057 languages.
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
Yuping Lin (Michigan State University), Jiliang Tang (Michigan State University)
CodeRepresentation LearningAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Conduct a representation space analysis of jailbreak attacks in large language models (LLMs), introduce the concept of 'acceptance direction,' and incorporate a new optimization objective into existing white-box jailbreak methods (GCG, AutoDAN) to improve attack success rates.
Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderators through a User-Centric Method
Yang Trista Cao (University of Maryland College Park), Hal Daumé III (University of Maryland College Park)
CodeTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: A systematic evaluation of the applicability of existing models across various community rules was conducted, with experiments on rule violation detection in the AskHistorians subreddit using GPT-4 and Llama-2, alongside collecting user feedback from volunteer moderators through questionnaires.
ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations
Yunze Xiao (Carnegie Mellon University Qatar), Roy Ka-Wei Lee (Singapore University Of Technology And Design)
CodeClassificationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Investigated the robustness of Chinese hate speech detection models against homonym and emoji obfuscation attacks, proposed the ToxiCloakCN dataset, and conducted evaluations.
Matanel Oren (Hebrew University of Jerusalem), Roy Schwartz (Hebrew University of Jerusalem)
CodeCompressionRecurrent Neural NetworkTransformerLarge Language ModelText
🎯 What it does: This paper reconsiders decoder-only Transformer as a multi-state RNN (MSRNN) and proves that they can be transformed into bounded MSRNN through compression strategies.
TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering
Chuyi Shang (University of California Berkeley), Roei Herzig (University of California Berkeley)
CodeRetrievalLarge Language ModelAgentic AIVision Language ModelVideoTextMultimodality
🎯 What it does: Propose a modular framework called TraveLER based on multi-agent systems to achieve video question answering without fine-tuning; the framework iteratively constructs plans, locates key frames, asks questions to obtain details, evaluates answers, and re-plans through four stages: Planner, Retriever, Extractor, and Evaluator;
TroL: Traversal of Layers for Large Language and Vision Models
Byung-Kwan Lee (Korea Advanced Institute of Science and Technology), Yong Man Ro (Korea Advanced Institute of Science and Technology)
CodeTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Proposes TroL, a 1.8B/3.8B/7B-scale LLVM family, which enhances learning capabilities by reusing layers through layer traversing technology without increasing parameters.
Giuseppe Attanasio (Instituto de Telecomunicações), Dirk Hovy (Bocconi University)
CodeRecognitionExplainability and InterpretabilityTransformerAudio
🎯 What it does: Systematic evaluation of Whisper and SeamlessM4T multilingual ASR models across 19 languages and 3 datasets, quantifying the recognition error gap between males and females;
UNICORN: A Unified Causal Video-Oriented Language-Modeling Framework for Temporal Video-Language Tasks
Yuanhao Xiong (UCLA), Jie Lei (Meta)
CodeGenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelVideoText
🎯 What it does: Proposes the Unified Causal Video-Language Model Framework UNICORN, achieving temporal video-language tasks through instruction tuning.