EEO-TFV: Escape-Explore Optimizer for Web-Scale Time-Series Forecasting and Vision Analysis

TL;DR

This paper introduces EEO-TFV, a lightweight Transformer with a novel optimizer that improves long-sequence prediction and out-of-distribution robustness in large-scale Web data analysis, achieving competitive results across diverse tasks.

Contribution

It presents a new lightweight Transformer architecture combined with the Escape-Explore Optimizer, enhancing exploration, generalization, and robustness in Web-scale time-series and vision tasks.

Findings

Achieves state-of-the-art performance on 11 time-series datasets

Demonstrates superior generalization and stability in Web data scenarios

Validates effectiveness across both forecasting and medical image segmentation

Abstract

Transformer-based foundation models have achieved remarkable progress in tasks such as time-series forecasting and image segmentation. However, they frequently suffer from error accumulation in multivariate long-sequence prediction and exhibit vulnerability to out-of-distribution samples in image-related tasks. Furthermore, these challenges become particularly pronounced in large-scale Web data analysis tasks, which typically involve complex temporal patterns and multimodal features. This complexity substantially increases optimization difficulty, rendering models prone to stagnation at saddle points within high-dimensional parameter spaces. To address these issues, we propose a lightweight Transformer architecture in conjunction with a novel Escape-Explore Optimizer (EEO). The optimizer enhances both exploration and generalization while effectively avoiding sharp minima and saddle-point traps. Experimental results show that, in representative Web data scenarios, our method achieves performance on par with state-of-the-art models across 11 time-series benchmark datasets and the Synapse medical image segmentation task. Moreover, it demonstrates superior generalization and stability, thereby validating its potential as a versatile cross-task foundation model for Web-scale data mining and analysis.