Rethinking Time Encoding via Learnable Transformation Functions

TL;DR

This paper introduces LeTE, a learnable time encoding method using deep function learning to better model complex and diverse temporal patterns in real-world applications.

Contribution

LeTE is a novel, learnable time encoding approach that generalizes existing methods by modeling complex time patterns with deep non-linear transformations.

Findings

LeTE outperforms traditional fixed-pattern encodings in various tasks.

LeTE effectively captures diverse and complex temporal dynamics.

Experimental results validate the versatility of LeTE across domains.

Abstract

Effectively modeling time information and incorporating it into applications or models involving chronologically occurring events is crucial. Real-world scenarios often involve diverse and complex time patterns, which pose significant challenges for time encoding methods. While previous methods focus on capturing time patterns, many rely on specific inductive biases, such as using trigonometric functions to model periodicity. This narrow focus on single-pattern modeling makes them less effective in handling the diversity and complexities of real-world time patterns. In this paper, we investigate to improve the existing commonly used time encoding methods and introduce Learnable Transformation-based Generalized Time Encoding (LeTE). We propose using deep function learning techniques to parameterize non-linear transformations in time encoding, making them learnable and capable of modeling generalized time patterns, including diverse and complex temporal dynamics. By enabling learnable transformations, LeTE encompasses previous methods as specific cases and allows seamless integration into a wide range of tasks. Through extensive experiments across diverse domains, we demonstrate the versatility and effectiveness of LeTE.