Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs
Danni Liu (Karlsruhe Institute of Technology), Jan Niehues (Karlsruhe Institute of Technology)
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Analyze the internal representations of large language models (LLMs), finding that intermediate layers have the greatest potential for cross-lingual alignment. Subsequently, introduce an alignment objective during task-specific fine-tuning, using alignment loss to align sentence representations across languages in the intermediate layers. Validate the effectiveness on multiple tasks (slot filling, machine translation, structured text generation).
Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures
Akhila Yerukola (Carnegie Mellon University), Maarten Sap (Carnegie Mellon University)
CodeSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextBenchmark
🎯 What it does: Constructed the MC-SIGNS dataset and evaluated the detection and interpretation capabilities of cross-cultural gestures using text-to-image models, language models, and vision-language models.
Igor Sterner (University of Cambridge), Simone Teufel (University of Cambridge)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: By constructing an evaluation benchmark based on minimal pairs, systematically assess the acceptance of large language models (LLMs) in code-switching (CS) texts, verifying whether LLMs handle CS in a manner consistent with bilingual speakers.
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningTextGraph
🎯 What it does: Proposes the 'Methodology Inspiration Retrieval (MIR)' task aimed at retrieving papers from literature that can provide methodological inspiration for research questions, and constructs a specialized dataset and retrieval method.
MISP-Meeting: A Real-World Dataset with Multimodal Cues for Long-form Meeting Transcription and Summarization
HangChen HangChen, Jun Du (NERC-SLIP University of Science and Technology of China)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningVideoTextMultimodalityBenchmarkAudio
🎯 What it does: Introduces the MISP-Meeting dataset and benchmarks automatic meeting transcription and summarization on this dataset, exploring the performance improvements brought by multimodal cues.
🎯 What it does: Researchers propose two bias mitigation methods based on weight masking (Extended Confounding Filter and Dual Filter) to eliminate gender-related spurious correlations in speech text, thereby improving fairness in detecting cognitive decline (Alzheimer's disease).
Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering
Rongzhi Zhu (Nanjing University), Wei Hu (Nanjing University)
CodeRetrievalLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes a progressive retrieval framework called ChainRAG, addressing retrieval errors in multi-hop QA by using sub-question rewriting and sentence graphs to complete missing entities, thereby improving retrieval quality.
Mixture of Small and Large Models for Chinese Spelling Check
Ziheng Qiao (Soochow University), Zhenghua Li (Soochow University)
CodeTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: Propose a Chinese spelling correction method that dynamically hybridizes a small model (BERT) with a large language model (LLM) during the beam search phase, leveraging the high precision of the small model and the fluency of the LLM for complementary enhancement.
MMDEND: Dendrite-Inspired Multi-Branch Multi-Compartment Parallel Spiking Neuron for Sequence Modeling
Kexin Wang (Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences), Guoqi Li (Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences)
🎯 What it does: Proposed the multi-branch, multi-compartment parallel spiking neuron MMDEND based on dendritic structure, and addressed the long-tailed membrane potential distribution and binarization errors through the Scaling-Shifting Integer Firing (SSF) mechanism.
🎯 What it does: Built MMRC — an open-source evaluation benchmark containing 5,120 real multimodal dialogues and 28,720 manually annotated question-answer pairs;
MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System
Jihao Zhao (Renmin University of China), Zhiyu Li (Institute for Advanced Algorithms Research)
CodeRetrievalTransformerLarge Language ModelMixture of ExpertsTextRetrieval-Augmented Generation
🎯 What it does: Proposed two new metrics for evaluating text chunking quality—Boundary Clarity (BC) and Chunk Stickiness (CS)—and designed a multi-granularity hybrid chunking framework (MoC) based on these metrics to enhance the performance of retrieval-augmented generation (RAG) systems.
MockConf: A Student Interpretation Dataset: Analysis, Word- and Span-level Alignment and Baselines
Dávid Javorský (Charles University), François Yvon (Sorbonne Université)
CodeData-Centric LearningTransformerLarge Language ModelTextMultimodalityBenchmarkAudio
🎯 What it does: This paper constructs and releases the MockConf dataset, collecting students' simultaneous interpretation records in simulated conferences, and provides word-level and segment-level alignment annotations; meanwhile, it introduces a web tool called InterAlign for this alignment, and presents a baseline alignment system along with evaluation metrics;
CodeExplainability and InterpretabilityDrug DiscoveryTransformerLarge Language ModelPrompt EngineeringBiomedical DataRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed the MolRAG framework, utilizing retrieval-augmented generation (RAG) and chain-of-thought (CoT) to achieve molecular property prediction without additional training.
More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives
Xiaoqing Zhang (Renmin University of China), Rui Yan (Renmin University of China)
CodeOptimizationMeta LearningTextBenchmark
🎯 What it does: Proposed a multi-example context learning optimization framework named DrICL, addressing performance degradation caused by improper NLL objectives and demonstration noise.
MPO: Multilingual Safety Alignment via Reward Gap Optimization
Weixiang Zhao (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
CodeOptimizationSafty and PrivacyTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposes a multilingual reward gap optimization (MPO) method that leverages the reward gap of the main language to guide safe alignment in low-resource languages.
MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming
Weiyang Guo (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeSafty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Propose the MTSA (Multi-Turn Safety Alignment) framework, combining thinking-guided multi-turn attack learning with multi-turn reinforcement learning to form a red-blue adversarial iterative training process, enhancing the safety and attack robustness of LLMs in multi-turn dialogues.
Multi-Facet Blending for Faceted Query-by-Example Retrieval
Heejin Do (POSTECH), Gary Lee (POSTECH)
CodeRetrievalTransformerLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: Proposed a multi-facet fusion (FaBle) augmentation method that decomposes and recombines documents using LLM to generate facet-specific positive and negative samples, enabling unlabeled facet-oriented QBE training;
Multi-Level Explanations for Generative Language Models
Lucas Monteiro Paes (Harvard University), Soumya Ghosh (IBM Research)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Proposes the MExGen framework, which generates importance scores for input text to explain model outputs in context-driven generative language models (e.g., summarization, QA), through multi-level chunking, linear-complexity variants of LIME/SHAP, and text-to-real scalarizers.
MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents
Kunlun Zhu (University of Illinois Urbana Champaign), Jiaxuan You (University of Illinois Urbana Champaign)
CodeTransformerLarge Language ModelAgentic AITextBenchmarkChain-of-Thought
🎯 What it does: Proposed the MultiAgentBench benchmark and the MARBLE framework to evaluate the performance of large language models in multi-agent collaborative and competitive scenarios
Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages
Zeli Su (Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE), Yushuang Dong (Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE)
CodeGenerationTransformerAuto EncoderText
🎯 What it does: For extremely low-resource languages, a shared-weight encoder-decoder framework named XLM-SWCM is constructed to efficiently generate text.
MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts
Dominik Macko (Kempelen Institute of Intelligent Technologies), Ivan Srba (Kempelen Institute of Intelligent Technologies)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Created and released a large-scale benchmark dataset for detecting machine-generated text on social media, MultiSocial, which is multi-platform, multi-language (22 languages), and multi-generator (7 LLMs). This dataset was used to evaluate the performance of 17 state-of-the-art detection methods (statistical zero-shot, pre-trained, and fine-tuned categories) across languages, platforms, and generators.
Tharindu Ranasinghe (Lancaster University), Ruslan Mitkov (Lancaster University)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes a multilingual semantic text similarity benchmark called MUSTS, covering 13 languages (including low-resource languages), and evaluates over 25 unsupervised and supervised methods on this benchmark.
My Words Imply Your Opinion: Reader Agent-Based Propagation Enhancement for Personalized Implicit Emotion Analysis
Jian Liao (Shanxi University), JianXing Zheng
CodeRecognitionRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelAgentic AITextGraphTabular
🎯 What it does: Propose the RAPPIE model to address the personalized implicit emotion analysis (PIEA) task, generating reader agents via LLM to simulate reader feedback and enhancing emotion recognition by combining role-aware multi-view graph learning.
Navigating Rifts in Human-LLM Grounding: Study and Benchmark
Omar Shaikh (Stanford University), Eric Horvitz (Microsoft Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper systematically studies grounding behavior in human-large language model (LLM) dialogues, constructs a classification of grounding behavior, uses GPT for automatic annotation and trains a prediction model, and finally creates the RIFTS benchmark based on prediction results to evaluate LLM grounding capabilities, while proposing a simple intervention method based on the predictor.
Neural Topic Modeling with Large Language Models in the Loop
Xiaohao Yang (Monash University), Lan Du (Monash University)
CodeExplainability and InterpretabilityKnowledge DistillationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes the LLM-ITL framework, integrating large language models (LLM) with neural topic models (NTM). The LLM refines the topic words learned by NTM and aligns them using Optimal Transport, enhancing the interpretability of topics;
Neuron-Level Sequential Editing for Large Language Models
Houcheng Jiang (University of Science and Technology of China), Xiang Wang (University of Science and Technology of China)
CodeRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Proposed a neuron-level based sequential editing method (NSE) for continuously modifying the internal knowledge of a model without retraining large language models.
Nudging: Inference-time Alignment of LLMs via Guided Decoding
Yu Fei (University of California Irvine), Sameer Singh (University of California Irvine)
CodeTransformerLarge Language ModelText
🎯 What it does: Propose a no-training method called NUDGING for aligning any large language model during inference, which improves the output of large models by inserting guiding tokens into the generation process using a small alignment model
OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval
Wei Yang (Microsoft Research Asia), Jiang Bian (Microsoft Research Asia)
CodeRetrievalComputational EfficiencyTransformerVision Language ModelContrastive LearningMultimodalityRetrieval-Augmented Generation
🎯 What it does: Built a multi-step, coarse-to-fine hierarchical multimodal retrieval + RAG framework to address knowledge-driven visual question answering (KB-VQA) problems.
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
Qinglin Zhang (Tongyi Lab), ShiLiang Zhang
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityAudio
🎯 What it does: Propose OmniFlatten, a GPT-based end-to-end full-duplex speech dialogue model that enables simultaneous bidirectional speech interaction.
On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs
Herun Wan (School of Computer Science and Technology, Xi'an Jiaotong University, China), Xiang Zhao (National University of Defense Technology)
CodeAnomaly DetectionData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsTextBenchmark
🎯 What it does: Systematically studied the evidence contamination risks of large language models in malicious social text detection, designed 13 LLM-based contamination methods, evaluated their impact on four types of detection tasks, and proposed and experimentally tested three defense strategies (machine-generated text detection, expert mixing, parameter updates).
Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization
Meng Li (Renmin University of China), Anxiang Zeng (Shopee Pte Ltd)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: Propose an unbiased optimal transport-based token weighting scheme (OTPO) that dynamically assigns token weights in direct preference optimization (DPO) to enhance the alignment of large language models (LLMs) with human preferences.
Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs
Fakhraddin Alwajih (University of British Columbia), Muhammad Abdul-Mageed (University of British Columbia)
CodeData SynthesisLarge Language ModelTextBenchmark
🎯 What it does: Constructed a fully human-generated instruction dataset called PALM, covering 22 Arab countries, including Modern Standard Arabic and local dialects, across 20 cultural themes.
Pandora’s Box or Aladdin’s Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models
Jinyang Wu (Tsinghua University), Jianhua Tao (Tsinghua University)
CodeTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed a seven-category system for RAG retrieval noise and proposed the NoiserBench benchmark to evaluate LLM performance under various noise conditions, discovering that certain noises (e.g., illegal sentence noise) can enhance model performance.
PaSa: An LLM Agent for Comprehensive Academic Paper Search
Yichen He (ByteDance Seed), Weinan E (Peking University)
CodeRetrievalReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AIText
🎯 What it does: Proposes PaSa, which can mimic researchers' behavior to search for papers, read, traverse citation networks, and automatically obtain comprehensive and accurate academic search results.
People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text
Jenna Russell (University of Maryland College Park), Mohit Iyyer (University of Maryland College Park)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper investigates the ability of commercial LLMs (GPT-4O, Claude 3.5, O1-PRO) to generate text, employing expert-level annotators to label and explain 300 non-fiction English articles, and comparing them with multiple automatic detectors.
PerSphere: A Comprehensive Framework for Multi-Faceted Perspective Retrieval and Summarization
Yun Luo (Westlake University), Yue Zhang (Westlake University)
CodeRetrievalLarge Language ModelAgentic AITextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the PerSphere framework for multi-perspective retrieval and summarization, helping people break out of information silos.
PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization
Yidan Wang (Chinese Academy of Sciences), Binxing Fang (Guangzhou University)
CodeOptimizationSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed a privacy jailbreak attack framework called PIG for large language models, which can effectively extract personal identifying information (PII).
Planning with Diffusion Models for Target-Oriented Dialogue Systems
Hanwen Du (Ohio State University), Xia Ning (Ohio State University)
CodeLarge Language ModelDiffusion modelText
🎯 What it does: Proposed a non-sequential planning framework for goal-oriented dialogue systems called DiffTOD, which generates global dialogue trajectories using diffusion models.
Planning-Driven Programming: A Large Language Model Programming Workflow
Chao Lei (University of Melbourne), Krista A. Ehinger (University of Melbourne)
CodeOptimizationAI Code AssistantLarge Language ModelTextBenchmark
🎯 What it does: Propose a two-stage large language model programming workflow named LPW, integrating plan formulation, plan verification, and visible testing to enhance the accuracy of text-to-code generation.
Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption
Dongjin Park (Chung-Ang University), Joon-Woo Lee (Sejong University)
CodeSafty and PrivacyComputational EfficiencyKnowledge DistillationTransformerText
🎯 What it does: Proposes Powerformer, a homomorphic encryption-based privacy-preserving language model that significantly reduces inference time through multiple technological optimizations while maintaining the same accuracy as the original model.
Pre-Training Curriculum for Multi-Token Prediction in Language Models
Ansar Aynetdinov (Humboldt Universität zu Berlin), Alan Akbik (Humboldt Universität zu Berlin)
CodeTransformerLarge Language ModelText
🎯 What it does: Proposed a pre-training curriculum learning strategy for multi-token prediction (MTP) to help small language models (SLM) better leverage MTP.
🎯 What it does: Developed a structured generation decoding method called Pre3 based on deterministic pushdown automata (DPDA), which directly converts LR(1) grammar into DPDA to achieve efficient parallel transitions;
Predicting Implicit Arguments in Procedural Video Instructions
Anil Batra (University of Edinburgh), Frank Keller (University of Edinburgh)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodalityChain-of-Thought
🎯 What it does: Designed and constructed the Implicit-VidSRL dataset for semantic role labeling of multi-step cooking videos, and proposed the iSRL-Qwen2-VL model for implicit argument prediction and next-step action prediction.
Predicting Turn-Taking and Backchannel in Human-Machine Conversations Using Linguistic, Acoustic, and Visual Signals
Yuxin Lin (Xiamen University), Wangzheng Shi (Xiamen University)
CodeClassificationTransformerLarge Language ModelVideoTextMultimodalityAudio
🎯 What it does: Constructed the MM-F2F human-machine dialogue multimodal dataset and proposed an end-to-end multimodal framework to predict alternation and background speech in conversations.
PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy
Shuhao Guan (University College Dublin), Derek Greene (Trinity College Dublin)
CodeRecognitionRestorationData SynthesisTransformerLarge Language ModelImageText
🎯 What it does: Built a two-stage complete pipeline, first training an image restoration model using synthetic data to enhance the quality of historical document images, then using a ByT5-based semantic correction module to correct OCR errors.
Principled Understanding of Generalization for Generative Transformer Models in Arithmetic Reasoning Tasks
Xingcheng Xu (Shanghai Artificial Intelligence Laboratory), Yanqing Yang (Fudan University)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper proposes a unified theoretical framework to explain the length generalization (upstream/downstream OOD) behavior of Transformers in arithmetic tasks (addition, multiplication, modular addition, modular multiplication).
Private Memorization Editing: Turning Memorization into a Defense to Strengthen Data Privacy in Large Language Models
Elena Sofia Ruzzetti (University of Rome Tor Vergata), Fabio Massimo Zanzotto (University of Rome Tor Vergata)
CodeSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: Developed a model editing method called Private Memorization Editing (PME), which defends against privacy leakage by identifying and modifying personally identifiable information (PII) already memorized in large language models (LLMs).
Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set
Florian Eichin (LMU Munich), Michael A. Hedderich (LMU Munich)
CodeClassificationRecognitionExplainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes a cross-framework and cross-lingual unified discourse relation label set, and uses this label set to conduct probe experiments on large language models (LLMs) on the multilingual, multi-framework DISRPT dataset to evaluate their generalization ability in discourse relation extraction.
🎯 What it does: Proposed the RICEA framework, utilizing relative interaction and dynamic calibration techniques to achieve multi-modal entity alignment.
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed and released the PROCESSBENCH dataset to evaluate the ability of language models to identify erroneous steps in mathematical reasoning processes.
ProgCo: Program Helps Self-Correction of Large Language Models
Xiaoshuai Song (Taobao & Tmall Group of Alibaba), Bo Zheng (Taobao & Tmall Group of Alibaba)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Designed a program-driven self-correction framework named ProgCo, which first allows LLM to self-generate and self-execute pseudo-programs for verification (ProgVe), and then performs dual reflection and program refinement based on feedback to improve answers (ProgRe)
Programming by Example meets Historical Linguistics: A Large Language Model Based Approach to Sound Law Induction
Atharva Naik (Carnegie Mellon University), David R. Mortensen
CodeData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Convert sound change induction tasks into Programming by Example (PBE) problems, construct a low-resource evaluation benchmark, design multiple synthetic data generation methods with adjustable structural/substantial bias, and train the PySLICoder model using LLM fine-tuning.
PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation
Linhai Zhang (King's College London), Yulan He (King's College London)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: Propose the PROPER framework, achieving LLM personalization through hierarchical fine-tuning using LoRA and MoE at multiple levels (population-level, group-level, user-level);
ProvBench: A Benchmark of Legal Provision Recommendation for Contract Auto-Reviewing
Xiuxuan Shen (Xidian University), Philip S. Yu (University of Illinois Chicago)
CodeRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose PROVBENCH as a benchmark for legal clause recommendation and conflict detection in automated contract review, and construct the PROVDATA dataset
ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clustering
Alexander Hoyle (ETH Zürich), Philip Resnik (University of Maryland)
CodeLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Designed a human-assessment-based topic model and document clustering evaluation protocol, and proposed an LLM proxy named PROXANN that can replace humans.
PsyAdvisor: A Plug-and-Play Strategy Advice Planner with Proactive Questioning in Psychological Conversations
Yuxin Hu (Southeast University), Yan Liu (Nanjing Derong Wisdom Information Technology Co., Ltd.)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: Proposed a plug-and-play plugin called PsyAdvisor, enabling psychological LLMs to proactively determine the timing of questions and provide strategic advice during conversations, thereby enhancing the depth and quality of psychological counseling dialogues.
PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling
Haojie Xie (South China University of Technology), Xiangmin Xu (South China University of Technology)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Built a framework based on LLM (PsyDT) that can quickly generate personalized counseling-style digital twin models of psychological counselors.
PVP: An Image Dataset for Personalized Visual Persuasion with Persuasion Strategies, Viewer Characteristics, and Persuasiveness Ratings
Junseo Kim (Seoul National University), Yohan Jo (Seoul National University)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningImageTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed the PVP (Personalized Visual Persuasion) dataset, containing 28,454 visual persuasion images based on 596 informational messages (e.g., 'Don't smoke'), covering 9 persuasion strategies; simultaneously collected 2,521 raters' persuasiveness scores, demographic characteristics, and psychological traits (Big Five personality, values, moral foundations). On this basis, two tasks were proposed and implemented: ① Using an estimator to predict image persuasiveness scores; ② Generating personalized persuasive images tailored to individual psychological traits.
PwnGPT: Automatic Exploit Generation Based on Large Language Models
Wanzong Peng (Harbin Institute of Technology), Chen Zhang (China Mobile Group Design Institute Company Ltd)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Proposed PwnGPT, an LLM-based automated exploit framework that improves the efficiency of automatically exploiting binary vulnerabilities (CTF challenges) through a modular workflow (analysis, generation, verification).
QAEval: Mixture of Evaluators for Question-Answering Task Evaluation
Tan Yue (Peking University), Dongyan Zhao (Peking University)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsTextBenchmark
🎯 What it does: Propose the QAEval hybrid evaluation framework, combining answer extraction, rule-based rapid scoring, and MOE evaluation to efficiently and accurately assess QA tasks.
QG-SMS: Enhancing Test Item Analysis via Student Modeling and Simulation
Bang Nguyen (University of Notre Dame), Meng Jiang (University of Notre Dame)
CodeTransformerLarge Language ModelText
🎯 What it does: Constructing test question pairs of varying quality, utilizing LLM for student modeling and simulation to assess the educational value of generated questions.
Quantification of Large Language Model Distillation
Sunbowen Lee (Shenzhen Institutes of Advanced Technology), Shiwen Ni (Shenzhen Institutes of Advanced Technology)
CodeExplainability and InterpretabilityKnowledge DistillationLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed a framework to quantify the distillation degree of large language models (LLMs), primarily through two metrics: Identity Consistency Evaluation (ICE) and Response Similarity Evaluation (RSE);
Qwen2.5-xCoder: Multi-Agent Collaboration for Multilingual Code Instruction Tuning
Jian Yang (Beihang University), Junyang Lin (Alibaba Group)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningAgentic AIText
🎯 What it does: Proposed a multilingual multi-agent collaborative framework to generate multilingual code instruction data for instruction fine-tuning of Qwen2.5-xCoder, significantly enhancing cross-language code generation capabilities.
R-Fairness: Assessing Fairness of Ranking in Subjective Data
Lorenzo Balzotti (Sapienza Università di Roma), Sihem Amer-Yahia (CNRS, University Grenoble Alpes)
CodeRecommendation SystemText
🎯 What it does: Designed and evaluated a framework (R-Fairness) to assess ranking fairness across different reviewer groups in collaborative rating platforms, with experiments conducted on real-world Yelp and Amazon datasets.
R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory
Tenghao Huang (University of Southern California), Muhao Chen (University of California Davis)
CodeTransformerLarge Language ModelAgentic AITextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the R2D2 framework, combining two paradigms: Remember (replay buffer + A* search) and Reflect (error analysis and reflective memory), to enhance the navigation and execution capabilities of web agents.
RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information
Zhiwei Liu (University of Manchester), Eduard Hovy (University of Melbourne)
CodeClassificationDomain AdaptationAnomaly DetectionTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose the RAEmoLLM framework, which utilizes retrieval-augmented LLM (RAG) for cross-domain misinformation detection by using sentiment information as the retrieval basis, constructing unsupervised few-shot examples, and completing judgment through in-context learning;
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework
Kunlun Zhu (Tsinghua University), Maosong Sun (Tsinghua University)
CodeGenerationData SynthesisRetrievalLarge Language ModelTextBiomedical DataBenchmarkFinance RelatedRetrieval-Augmented Generation
🎯 What it does: This paper proposes the RAGEval framework, which can automatically generate scenario-based RAG evaluation datasets, including structured documents, question-answer pairs, and reference snippets, and evaluates answers through key point extraction.
🎯 What it does: Propose the RankCoT method, integrating reranking signals into Chain-of-Thought (CoT) generation to improve the knowledge refinement process of Retrieval-Augmented Generation (RAG).
Re^{3}Syn: A Dependency-Based Data Synthesis Framework for Long-Context Post-training
Zhiyang Zhang (Central South University), De Wen Soh (Singapore University of Technology and Design)
CodeData SynthesisTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose a long-text synthesis framework RE SYN 3 based on semantic similarity retrieval, dependency identification, and re-ranking for generating high-quality long-context training data.
Read it in Two Steps: Translating Extremely Low-Resource Languages with Code-Augmented Grammar Books
Chen Zhang (Wangxuan Institute of Computer Technology, Peking University), Yansong Feng (Wangxuan Institute of Computer Technology, Peking University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Explored how to utilize grammar books in machine translation for extremely low-resource languages by splitting the process into two steps: rule retrieval and rule application, and proposed using code-based rules and a per-item retrieval strategy to improve translation effectiveness.
Yangqin Jiang (University of Hong Kong), Chao Huang (University of Hong Kong)
CodeRecommendation SystemKnowledge DistillationGraph Neural NetworkTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: This paper proposes a model-agnostic instruction tuning framework called RecLM, which combines large language models (LLMs) with collaborative filtering. It leverages multi-round dialogues and reinforcement learning to generate high-quality user/item features, achieving significant improvements in cold start and sparse data scenarios.
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning
Yujie Feng (Hong Kong Polytechnic University), Xiao-Ming Wu (Hong Kong Polytechnic University)
CodeOptimizationSafty and PrivacyRepresentation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the Recurrent-KIF framework to achieve dynamic parameter importance estimation for continual learning and multi-round knowledge fusion
Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
Xinxin Liu (Southern University of Science and Technology), Xitong Gao (Shenzhen Institutes of Advanced Technology, CAS)
CodeComputational EfficiencyRepresentation LearningNeural Architecture SearchTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Studied a sparse parameter efficient fine-tuning (SPEFT) method, systematically evaluated multiple importance metrics to construct sparse masks, and compared the effects of static and dynamic masks.
RefreshKV: Updating Small KV Cache During Long-form Generation
Fangyuan Xu (New York University), Eunsol Choi (New York University)
CodeComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose the RefreshKV inference method, which utilizes dynamic refreshing of small KV caches and alternates between full KV attention and small KV attention in long text generation to improve inference speed while reducing performance loss.
Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation
Zhi Qu (Nara Institute of Science and Technology), Taro Watanabe (National Institute of Information and Communications Technology)
CodeGenerationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied a method called Registering to address the off-target problem in multilingual neural machine translation, and trained the MITRE series of models based on this method.
Rethinking Evaluation Metrics for Grammatical Error Correction: Why Use a Different Evaluation Process than Human?
Takumi Goto (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeData-Centric LearningText
🎯 What it does: Researchers propose a new automatic evaluation process that converts sentence-level scores of existing evaluation metrics into pairwise comparisons, and then uses the same TrueSkill algorithm as human assessments to generate system rankings, thereby narrowing the gap between automatic evaluation and human evaluation.
Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up
Jiahao Yuan (East China Normal University), Usman Naseem (Macquarie University)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Proposed the RoT framework, enhancing LLM's logical reasoning ability through reverse reasoning warm-up, preference-guided reverse reasoning, and a cognitive preference manager.
Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability
Yusuke Sakai (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose the Ordered CommonGen benchmark, which utilizes full permutations of all concept sets and incorporates the phrase 'in the specified order' in prompts to evaluate LLMs' instruction following and combinatorial generation capabilities.
Revisiting LLMs as Zero-Shot Time Series Forecasters: Small Noise Can Break Large Models
Junwoo Park (KAIST AI), Jaegul Choo (KAIST AI)
CodeTransformerLarge Language ModelPrompt EngineeringTime SeriesBenchmark
🎯 What it does: Evaluate the performance of large language models (LLMs) in zero-shot time series forecasting, and conduct a comprehensive comparison with state-of-the-art domain-specific prediction models and single-shot linear models.
Reviving Cultural Heritage: A Novel Approach for Comprehensive Historical Document Restoration
Yuyi Zhang (South China University of Technology), Lianwen Jin (South China University of Technology)
CodeRecognitionRestorationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Proposed a complete page historical document restoration system named AutoHDR, consisting of a three-stage pipeline: OCR-assisted damage localization, visual-language context text prediction, and Patch-Autoregressive visual restoration; simultaneously constructed a full-page HDR public dataset called FPHDR.
CodeLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes an automatic prompt optimization framework based on residual optimization tree (RIOT), which iteratively improves prompts using text gradients and achieves diverse exploration through multi-candidate prompt generation combined with perplexity selection;
Robust Utility-Preserving Text Anonymization Based on Large Language Models
Tianyu Yang (Technical University of Darmstadt), Iryna Gurevych (Technical University of Darmstadt)
CodeOptimizationSafty and PrivacyKnowledge DistillationData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed a robust text anonymization framework RUPTA based on large language models (LLMs), which jointly iteratively rewrites text using a privacy evaluator, utility evaluator, and optimizer to defend against LLM re-identification attacks while preserving the utility for downstream tasks.
RoCoFT: Efficient Finetuning of Large Language Models with Row-Column Updates
Md Kowsher (Nokia Bell Labs), Niloofar Yousefi (University of Central Florida)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study proposes the RoCoFT method, which achieves parameter-efficient fine-tuning by updating only a small number of rows or columns in the Transformer weight matrix;
RoToR: Towards More Reliable Responses for Order-Invariant Inputs
Soyoung Yoon (Seoul National University), Seung-won Hwang (Seoul National University)
CodeLarge Language ModelTextBenchmark
🎯 What it does: Investigated the position bias problem in language models with list inputs, proposing the RoToR method and Selective Routing to adapt to mixed order-sensitive and order-insensitive inputs.
RUBY: An Effective Framework for Multi-Constraint Multi-Hop Question Generation
Wenzhuo Zhao (South China Normal University), Shuangyin Li (South China Normal University)
CodeGenerationTransformerLarge Language ModelText
🎯 What it does: Developed the RUBY framework for multi-constraint multi-hop question generation tasks. It first compresses high-dimensional constraints such as multi-hop types and intentions into low-dimensional multi-hop skeletons via a High-Dimensional Semantic Constraint Reduction (HDR) module. Then, it employs a divide-and-conquer strategy with Sub-Question Answer Pair Generation (SQAG) and Multi-Hop Question Generation (MHQG) processes. During decoding, it introduces Reasoning Dynamic Projection (RD-Projection) technology to enhance generation accuracy and constraint consistency.
S-RAG: A Novel Audit Framework for Detecting Unauthorized Use of Personal Data in RAG Systems
Zhirui Zeng (University of Auckland), Zijian Zhang (Beijing Institute of Technology)
CodeAnomaly DetectionSafty and PrivacyTransformerLarge Language ModelTextBiomedical DataRetrieval-Augmented Generation
🎯 What it does: Propose a black-box auditing framework named S-RAG, which can detect whether user text data is used by the external database of a retrieval-augmented generation (RAG) system, thereby achieving traceability of personal data usage.
SCALE: Towards Collaborative Content Analysis in Social Science with Large Language Model Agents and Human Intervention
Chengshuai Zhao (Arizona State University), Huan Liu (Arizona State University)
CodeClassificationTransformerLarge Language ModelAgentic AIPrompt EngineeringTextChain-of-Thought
🎯 What it does: Developed the SCALE framework, which uses multi-agent systems to simulate the content analysis process in social sciences and combines human expert intervention to achieve automated annotation.