ESQA: Event Sequences Question Answering
Irina Abdullaeva, Andrei Filatov, Mikhail Orlov, Ivan Karpukhin,, Viacheslav Vasilev, Denis Dimitrov, Andrey Kuznetsov, Ivan Kireev, Andrey, Savchenko
TL;DR
ESQA introduces a novel approach leveraging large language models to effectively process and analyze event sequences across various domains, addressing challenges of sequence length and feature processing, and achieving state-of-the-art results.
Contribution
The paper presents ESQA, a new method that adapts large language models for event sequence analysis with minimal fine-tuning, handling long sequences and complex features.
Findings
Achieves state-of-the-art results in event sequence tasks
Effectively processes long sequences and numeric features
Requires little or no fine-tuning of LLMs
Abstract
Event sequences (ESs) arise in many practical domains including finance, retail, social networks, and healthcare. In the context of machine learning, event sequences can be seen as a special type of tabular data with annotated timestamps. Despite the importance of ESs modeling and analysis, little effort was made in adapting large language models (LLMs) to the ESs domain. In this paper, we highlight the common difficulties of ESs processing and propose a novel solution capable of solving multiple downstream tasks with little or no finetuning. In particular, we solve the problem of working with long sequences and improve time and numeric features processing. The resulting method, called ESQA, effectively utilizes the power of LLMs and, according to extensive experiments, achieves state-of-the-art results in the ESs domain.
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
1. The authors take advantage of categorical and numeric features from tabular neural networks at the same time in Section 3.2. 2. In Section 3.3, the choice of Whisper as an Encoder after extensive comparative experiments is convincing.
1. I think the biggest problem with this paper is the lack of sensible baselines. The authors ignore some recent applications of large models for event prediction in the last year. For example [1], [2]. The authors use earlier baselines and therefore the results obtained are not convincing. In particular, according to Table 3, ESQA does not achieve a consistent lead compared to NPPR either. [1] Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Conte
S1: Experimental Design: The paper supports its proposed approach with a series of experimental setup, utilizing multiple datasets that span various event sequence types and domains, enhancing the robustness and applicability of the findings. S2: Each component of the ESQA framework is backed by experimental results, validating the selection and effectiveness of different modules. S3: ESQA achieves state-of-the-art or competitive results on a range of tasks, highlighting its versatility and
Overall, the paper lacks novelty beyond the designed QA framework with backbone encoder and LLM. Moreover, the organization and experiment settings in paper can be further optimized. W1: Inadequate Background Explanation: The paper contains some strongly worded statements in the background, such as “no effort was made to adapt LLMs to event sequences.” However, event sequence tasks share similarities with event prediction tasks, making it essential for the authors to clarify the differences be
1. The paper proposes a method to encode features of different formats and integrate the embedded features with questions as input to LLMs. The motivation for using LLMs is clearly presented. 2. By leveraging LLMs, the proposed method can generalise to unseen tasks. 3. The proposed method outperforms the baseline methods on some of the datasets. 4. In general, the paper is well-written and easy to follow.
**Novelty**: 1. The paper claimed it is the first to apply LLMs to event sequences. This claim does not seem to be correct. There have studies [1,2] using LLMs for event sequences. 2. The technical novelty of this paper is weak. The event encoder adopts several well-known machine learning techniques, such as binning, to convert the raw features to learnable embeddings. The connector is a direct application of an existing technique Q-former. The training protocol applies LORA, a well-known fine-t
- Originality: The paper introduces a creative adaptation of LLMs to the ES domain that has not been extensively explored with such models. While prior work has adapted LLMs to other non-language data (e.g., time series), the specific application to ESs with the question-answering format is novel. - Quality: The paper offers extensive experiments on relevant datasets from real-world domains. The use of LLMs for ES modeling and the integration of Q-Former as a connector between the LLM and event
- Quality: The proposed method has certain limitations, particularly in handling imbalanced classes and discretization errors in numeric features, as noted by the authors. - Clarity: There are several typos and styling issues, for example: - Numerous citations should be in parenthesis using \citep command. - The authors have mentioned that "the second best results are underlined" in Table 4's caption, but the table contains no underlines. - In the captions of Table 3-5, "baseline approache
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Quality and Management · Semantic Web and Ontologies · Advanced Text Analysis Techniques