Predicting Sparse Clients' Actions with CPOPT-Net in the Banking Environment

TL;DR

This paper introduces CPOPT-Net, a novel algorithm combining tensor decomposition and neural networks to accurately predict sparse client actions in banking, enhancing personalized recommendations and handling high sparsity effectively.

Contribution

It presents the first use of non-linear conjugate gradient tensor resolution combined with neural networks for predicting financial activities in banking.

Findings

CPOPT-Net achieves high prediction accuracy on a public dataset.

The method effectively handles sparse client activity data.

It improves personalized banking recommendations.

Abstract

The digital revolution of the banking system with evolving European regulations have pushed the major banking actors to innovate by a newly use of their clients' digital information. Given highly sparse client activities, we propose CPOPT-Net, an algorithm that combines the CP canonical tensor decomposition, a multidimensional matrix decomposition that factorizes a tensor as the sum of rank-one tensors, and neural networks. CPOPT-Net removes efficiently sparse information with a gradient-based resolution while relying on neural networks for time series predictions. Our experiments show that CPOPT-Net is capable to perform accurate predictions of the clients' actions in the context of personalized recommendation. CPOPT-Net is the first algorithm to use non-linear conjugate gradient tensor resolution with neural networks to propose predictions of financial activities on a public data set.

Figures

Tables

Table 1. Table 1: Residual errors of the objective function W c subscript 𝑊 𝑐 W_{c} between CPOPT-Net resolution and ALS resolution at convergence (the smaller, the better). Both methods have similar computation time.

	CPOPT-Net	CP-ALS
$W_c$ Error	10.099	15.896

Table 2. Table 2: Latent predictions errors on personal savings. LSTM achieves superior performance.

Error Measure	DT	MLP	CNN	LSTM
MAE	0.044	0.004	0.282	0.002
Jaccard dist.	0.053	0.027	0.348	0.003
cosine sim.	0.967	0.953	0.966	0.969
RMSE	0.047	0.031	0.354	0.003

Equations

$$\mathcal{X}=\sum _{r=1}^R{\text{a}}_r^{(1)}\circ {\text{a}}_r^{(2)}\circ {\text{a}}_r^{(3)}\circ ...\circ {\text{a}}_r^{(N)}$$

$${\mathcal{X}}_{true}\approx {\mathcal{X}}_{target}=\sum _{r=1}^R{\text{a}}_r^{(1)}\circ ...\circ {\text{a}}_r^{(N)}$$

$$W_c({\mathcal{X}}_{true},{\mathcal{X}}_{target})=\min f({\mathcal{X}}_{true},{\mathcal{X}}_{target})=\frac{1}{2}||{\mathcal{X}}_{true}-{\mathcal{X}}_{target}|{|}^2$$

$$\text{x}=[{\text{a}}_1^{(1)}\cdots {\text{a}}_R^{(1)}\cdots {\text{a}}_1^{(N)}\cdots {\text{a}}_R^{(N)}{]}^T$$

$$\begin{array}{rl}{W}_c(\text{x})& =W_c({\mathcal{X}}_{true},{\mathcal{X}}_{target})=\frac{1}{2}||{\mathcal{X}}_{true}|{|}^2-⟨{\mathcal{X}}_{true},{\mathcal{X}}_{target}⟩+\frac{1}{2}||{\mathcal{X}}_{target}|{|}^2\end{array}$$

Full text

1 Motivation

The modern banking environment is experiencing its own digital revolution. Strong regulatory directives are now applicable, especially in Europe with the Revised Payment Directive, PSD2, or with the General Data Protection Regulation, GDPR. Consequently, financial actors are now exploring the latest progress in data analytics and machine learning to leverage their clients’ information in the context of personalized financial recommendation and client’s action predictions. Recommender engines usually rely on second order matrix factorization since their accuracy has been proved in various publications [1, 2, 3]. However, matrix factorization are limited to the unique modeling of clients $\times$ products. Therefore, tensor factorization have skyrocketed for the past few years [4, 5, 6]. Various tensor factorization, or tensor decomposition, exist for different applications [7, 8]. However, the CP decomposition [9, 10] is the most frequently used. Two of the most popular resolution algorithms, the Alternating Least Square (ALS) [9, 10] and the non-negative ALS [11], offer a relatively simple mathematical framework explaining its success for the new generation of recommender engines [12, 13, 14]. In this paper, we use the gradient-based resolution for the CP decomposition [15] to address the predictions of clients’ financial activities based on time, clients’ ID and transactions type. The method, illustrated in figure 1, reduces the sparsity of the information while a neural network performs the predictions of events. We outline three contributions of our paper:

• •

We use the CP decomposition for separate modeling of each order of the data set. Since one client can have several financial activities simultaneously, we include the independent modeling of clients and financial transactions.

• •

We build upon non-linear conjugate gradient resolution for the CP decomposition, CPOPT [15]. We show CPOPT applied on a financial data set leads to small numerical errors while achieving reasonable computational time.

• •

Finally, we combine CPOPT with neural network leading to CPOPT-Net. A compressed dense data set, inherited from CP, is used as an optimized input for the neural network to predict the financial activities of the clients.

The remaining of the paper is organized as follows. Section 2 describes the CP tensor decomposition with its gradient-based resolution applied to third order financial predictions with neural network. Then, we highlight the experimental results in section 3 and we conclude by emphasizing pointers to future work.

2 CPOPT-Net and third order financial predictions

In the CP tensor decomposition [9, 10], the tensor $\mathscr{X}\in\mathbb{R}^{I_{1}\times I_{2}\times I_{3}\times...\times I_{N}}$ is described as the sum of the rank-one tensors

[TABLE]

where $\textbf{a}_{r}^{(1)},\textbf{a}_{r}^{(2)},\textbf{a}_{r}^{(3)},...,\textbf{a}_{r}^{(N)}$ are vectors of size $\mathbb{R}^{I_{1}},\mathbb{R}^{I_{2}},\mathbb{R}^{I_{3}},...,\mathbb{R}^{I_{N}}$. Each vector $a_{r}^{(n)}$ with $n\in\left\{1,2,...,N\right\}$ refers to one order and one rank of the tensor $\mathscr{X}$. We point out to [7] for further information. We use the Nonlinear Conjugate Gradient (NCG) method proposed in [15], CPOPT, with the strong Wolfe line search as it appears to be more stable in our case. Let $\mathcal{X}_{true}$ a real-valued N-order tensor of size $I_{1}\times I_{2}\times...\times I_{N}$. Given $R$, the objective is to find a factorization

[TABLE]

with the factors $\textbf{a}_{r}^{(1)},...,\textbf{a}_{r}^{(N)}$ initially randomized. Therefore, we denote by $\mathcal{X}_{target}$ the target tensor composed of the factor vectors $\textbf{a}_{r}^{(1)},...,\textbf{a}_{r}^{(N)}$.

The objective minimization function is denoted by $W_{c}(\mathcal{X}_{true},\mathcal{X}_{target})$.

[TABLE]

The values of the factor vectors can be stacked in a parameter vector x.

[TABLE]

Therefore, we can rewrite the objective function (3) as three summands.

[TABLE]

From (5), we deduce the gradient function of the CP decomposition involved in the minimization process according to the factor vectors $\textbf{a}_{1}^{(1)},...,\textbf{a}_{R}^{(N)}$. We refer to [15] for more details about the gradient computation. Therefore, CPOPT-Net achieves a NCG resolution of the objective function $W_{c}(\mathcal{X}_{true},\mathcal{X}_{target})$. Sparse information contained in $\mathcal{X}_{true}$ are removed in the factor vectors $\textbf{a}^{(1)},...,\textbf{a}^{(N)}$ of $\mathcal{X}_{target}$. Then, the factor vectors are sent as optimized inputs to the neural network. Through the training of the data set to learn the function $g(.):\mathbb{R}^{3}\rightarrow\mathbb{R}^{1}$, the neural network is able to predict the financial activities of the bank’s clients. The implementation of CPOPT-Net is summed up in algorithm 1.

3 Predictions of clients’ actions for banking recommendation

Data Availability and Experimental Setup In 2016, the Santander bank released an anonymized public dataset containing financial activities from its clients111The data set is available at https://www.kaggle.com/c/santander-product-recommendation. The file contains activities of 2.5 millions of clients classified in 22 transactions labels for a 16 months period between 28 January 2015 and 28 April 2016. We choose the 200 clients having the most frequent financial activities since regular activities are more interesting for the prediction modeling. All the information is gathered in the tensor $\mathcal{X}_{true}$ of size 200$\times$22$\times$16. We define the tensor rank equal to 25. We use the Adam solver with the default parameters $\beta_{1}=0.5,\beta_{2}=0.999$ for the training of the neural network222The code is available at https://github.com/dagrate/cpoptnet..

Results and Discussions on CPOPT-Net We test CPOPT-Net using three different type of neural networks: Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) network. Additionally, we cross-validate the performance of the neural networks with a Decision Tree (DT). The models have been trained on one year period from 28 January 2015 until 28 January 2016. Then, the activities for the next three months are predicted with a rolling time window of one month. First, the table 1 highlight the lower numerical error obtained with the CPOPT resolution in comparison to the ALS resolution. Then, the figure 1 shows that the LSTM models the most accurately the future personal savings activities followed by the MLP, the DT, and finally the CNN. The CNN fails visually to predict accurately the savings activity in comparison to the other three methods, while the LSTM seems to achieve the most accurate predictions. We highlight this preliminary conclusion for figure 1 in table 3 by reporting four metrics: the Mean Absolute Error (MAE), the Jaccard distance, the cosine similarity and the Root Mean Square Error (RMSE). In table 3, we show the aggregated metrics among all transaction predictions. In all the experiments, the LSTM network predicts the activities the most accurately, followed by the MLP, the DT and the CNN.

4 Conclusion

Building upon the CP tensor decomposition, the non-linear conjugate gradient resolution and the neural networks, we propose CPOPT-Net, a predictive method for the banking industry in which the sparsity of the financial transactions is removed before performing the predictions on future clients’ transactions. We conducted experiments on a public data set highlighting the prediction differences depending on the neural network involved in CPOPT-Net. Due to the recurrent activities of most of the financial transactions, we underlined the best results were found when CPOPT-Net was used with LSTM. Future work will concentrate on a limited memory resolution for a usage on very large data sets. Furthermore, the personal financial recommendation will be assessed on smaller time frame discretization, weekly or daily, with other financial transactions. It will offer a larger choice of financial product recommendations depending on the clients’ mid-term and long-term interests.