Investigating LLM Applications in E-Commerce

TL;DR

This paper evaluates the performance of large language models in e-commerce tasks, comparing instruction-tuned LLMs with traditional models, and analyzes training methodologies to optimize NLP applications in the industry.

Contribution

It provides a comprehensive comparison of LLMs and traditional models in e-commerce tasks and explores various training strategies for improved performance.

Findings

Few-shot inference with large LLMs often underperforms fine-tuned smaller models.

Task-specific fine-tuning is crucial for optimal performance.

Different training methodologies significantly impact model effectiveness.

Abstract

The emergence of Large Language Models (LLMs) has revolutionized natural language processing in various applications especially in e-commerce. One crucial step before the application of such LLMs in these fields is to understand and compare the performance in different use cases in such tasks. This paper explored the efficacy of LLMs in the e-commerce domain, focusing on instruction-tuning an open source LLM model with public e-commerce datasets of varying sizes and comparing the performance with the conventional models prevalent in industrial applications. We conducted a comprehensive comparison between LLMs and traditional pre-trained language models across specific tasks intrinsic to the e-commerce domain, namely classification, generation, summarization, and named entity recognition (NER). Furthermore, we examined the effectiveness of the current niche industrial application of very large LLM, using in-context learning, in e-commerce specific tasks. Our findings indicate that few-shot inference with very large LLMs often does not outperform fine-tuning smaller pre-trained models, underscoring the importance of task-specific model optimization.Additionally, we investigated different training methodologies such as single-task training, mixed-task training, and LoRA merging both within domain/tasks and between different tasks. Through rigorous experimentation and analysis, this paper offers valuable insights into the potential effectiveness of LLMs to advance natural language processing capabilities within the e-commerce industry.