Prediction defaults for networked-guarantee loans

TL;DR

This paper introduces a novel risk diffusion model for predicting enterprise loan defaults in networked-guarantee systems, addressing the challenge of imbalanced data and contagion effects, with promising results on real-world bank data.

Contribution

It develops an imbalanced network risk diffusion model and a positive weighted k-nearest neighbors algorithm to improve default prediction accuracy in guarantee networks.

Findings

Outperforms conventional credit risk methods in AUC.

Effective in short-term default risk forecasting.

Demonstrates utility on real-world bank data.

Abstract

Networked-guarantee loans may cause the systemic risk related concern of the government and banks in China. The prediction of default of enterprise loans is a typical extremely imbalanced prediction problem, and the networked-guarantee make this problem more difficult to solve. Since the guaranteed loan is a debt obligation promise, if one enterprise in the guarantee network falls into a financial crisis, the debt risk may spread like a virus across the guarantee network, even lead to a systemic financial crisis. In this paper, we propose an imbalanced network risk diffusion model to forecast the enterprise default risk in a short future. Positive weighted k-nearest neighbors (p-wkNN) algorithm is developed for the stand-alone case -- when there is no default contagious; then a data-driven default diffusion model is integrated to further improve the prediction accuracy. We perform the empirical study on a real-world three-years loan record from a major commercial bank. The results show that our proposed method outperforms conventional credit risk methods in terms of AUC. In summary, our quantitative risk evaluation model shows promising prediction performance on real-world data, which could be useful to both regulators and stakeholders.

Tables

Table 1. TABLE I: Results for default prediction.

	Imbalance	Diffusion	Precision	Recall	F1-Score
INDDP	$\surd$	$\surd$	0.81	0.75	0.78
P-wkNN	$\surd$	-	0.72	0.75	0.73
wkNN	-	$\surd$	0.64	0.76	0.69
RF	-	-	0.59	0.67	0.63
SVM	-	-	0.59	0.65	0.62
LR	-	-	0.54	0.63	0.58

Equations

$$P(A)=P_s(A)+(1-P_s(A))\cdot P_g(A)$$

$$P_g(A)=1-\prod _{i=1}^m(1-P_s(x_i)P_g(A|x_i))\text{vspace}-5pt$$

$$P_g(A)=g\left(\prod _{x_i\in {\text{O}}_d(A)}g\left(P_s(x_i)P_g(A|x_i)\right)\right)\text{vspace}-5pt$$

$$P(A)$$

$$g(P_s(A))\cdot g\left(\prod _{x_i\in {\text{O}}_d(A)}g\left(P_s(x_i)P_g(A|x_i)\right)\right)$$

$$P(A)$$

$$\text{FUN}\left(\text{FUN}(x|x\in {\text{O}}_d(A))\right)$$

Full text

I Introduction.

One of the most important drivers of macroeconomic conditions and systemic risk is consumer spending, which accounted for over two thirds of American gross domestic product[13]. The opportunities and risk exposures in consumer lending are equally outsized, which makes it necessary to rely on models and algorithms rather than human discretion. Consumer credit risk evaluation is often technically addressed in a data-driven fashion and has been extensively investigated [2, 9].

As one of major part in consumer lending business, the economic and banking importance of the small and medium enterprise (SME) loans is well recognized in academic and policy literature [4, 24]. However, the existing mechanism for loan decision-making falls behind business demand. Most of the criteria are designed for major independent players, while the less formal small and medium enterprises may provide inaccurate or manipulated information.

In order to meet the criteria of financial security by the banks, groups of small and medium enterprises back each other. When more and more enterprises are involved, they form networks with complex structures. Figure 1 gives a guarantee network with more than 600 enterprises bound together in a connected network. Such a complex network is a double-edged sword to the national economy. On the one hand, these secured loans can help an enterprise to source financing rapidly and promote development during a period of economic growth. On the other hand, although a complex network can slow down the risk of corporate default during a period of economic downturn, it may also lead to large-scale defaults and spread the infection.

Abundant machine learning approaches have been studied for credit risk on SMEs [6, 21, 26]. As features in financial aspects are not as high as tasks in image and speech domains, learning algorithms with high interpretability and low complexity is required [14]. The K-Nearest Neighbor(KNN) classifier is one of the most fundamental and simple classification methods and should be one of the first choices for a classification study when there is little or no prior knowledge about the distribution of the data [23]. It is one of the popularest in financial domain because of the high performance on low dimensional data, simple to implement and explanatory, compared with logistic regression, SVM, random forest and neural network.

However, all above approaches are designed with the assumption that the number of samples of each class is roughly the same. This is not the case in financial scenarios, loan default records is much smaller than normal repayments. Our statistics show that only about 6% of borrowers have default records. Such highly skewed datasets need tailored algorithms for data mining. Sampling-based approach is widely used in industry and academic research [16, 8, 10]. But it inevitably change the distribution of the overall dataset. Other approaches including cost sensitive learning [31] and ensemble learning [29] are suffer from high model complexity and low interpretability.

Besides, there are some studies on networked guarantee[17, 20], but most of them focus on statistical analysis [19] or revel some structure feature of the network [7]. Little of them research on predicting credit risk as well as the risk diffusion over the network. In the social network literature, some studies address the problem of information diffsion over social network [25, 11, 27], but as a financial behavior, risk diffusion from economic network is greatly different from social network.

In this paper, in order to tackle above challenges, we propose INDDP (Imbalance Network Diffusion Default Prediction). Rich financial features, including network related measures, are first feed to P-wKNN(positive weighted k-Nearest-Neighbor Classification and Regression method), which is easy to implement and naturally designed for imbalanced datasets without biased sampling. Then, a probabilistic graph model is proposed to model the risk diffusion over networks.

In a nutshell, the main contributions of this paper are:

To the best of our knowledge, this is the first study to use probabilistic graph model to predict credit risks diffusion on guarantee network. 2. 2.

We propose a novel positive weighted k-Nearest-Neighbor Classification and Regression method for probability estimation, which is easy to implement, naturally designed for imbalanced data and with high interpretability. 3. 3.

We propose a practical solution to evaluate default risks for networked-guarantee loans. which is much simpler than the classic physical diffusion model and more effective for predicting defaults in networked-guarantee loan data.

II Preliminaries.

We briefly introduce the related financial business procedure and our feature description.

II-A Business procedure.

In order to obtain a loan, a borrower needs to open an account and provide detailed information to the bank. Banks may be reluctant to issue the loan, as it is difficult for SMEs to meet the bank’s lending criteria, which are designed for big companies. There is something of a blank area for setting the criteria for SMEs due to their lack of security. However, they are permitted to offer other corporations as endorsement. Usually, banks need to collect as much fine-grained information as possible, including transaction information, customer information, asset information such as mortgage status, and history of loan approval.

As Figure 2 shows, there is often more than one guarantor per loan transaction, and a single guarantor may make several loan transactions in a period. Upon approval of the loan, the SMEs usually obtain the full loan amount immediately, and start making repayments by regular installments until the end of the loan contract. We reconstruct the guaranteed network based on nine data tables: customer profile, loan account information, repayment status, guarantee profile, customer credit, loan contract, guarantee relationship, guarantee contract and default status.

II-B Data Description.

We collect loan records spanning three years from a major commercial bank in China. The names of the customers in the records are encrypted and replaced by an ID; we can access basic profile information such as the enterprise scale, and loan information such as the guarantee ID and the loan credit.

In order to build a feature that can reliably represent the statistical relationships between the customer information and their repayment ability, we cleaned the data and constructed the features including the following factors:

Registration information: This includes the character, capital, collateral, capability, condition and stability. We use business nature, registered capital, enterprise scale, employee number and other features as a corporation’s basic profile. Most banks require the company to update this basic information when the enterprise makes a loan application; we use the latest information.

Historical behavior: This includes credit history, default records, default amount, total loan amount, loan count, total loan frequency, and total default rates. They are calculated with all the loan records before the current one.

Network related measures: For example, the centralities are extracted. As discussed above, the basic profile may not be completely trustworthy, as the small business might provide out of date or even fake information to the bank. However, the guaranteed network offers trustworthy information, as the bank can build this from its own records.

III Algorithm.

As loan and repayment behaviors are temporal events, we collect all these events in the current time windows as training datasets, then predict the probability of defaults in the next time window. We define a default event as the record that the borrower did not refund the repayment in time according to loan contracts. We address the following problems:

Given: a subsequence of event stream $E_{-\omega}$ , where $\omega$ is a user-defined time window.

Predict: the probability $e^{*}$ of a default occurrence in the next time window $\omega$.

Predicting default can be defined as estimating the conditional probability $P(\exists e^{*}\in E_{+\omega}|E_{-\omega})$. Due to the unbalanced data, there is insufficient data to estimate an accurate conditional distribution. We first model the conditional distribution by our tailored nearest-neighbor-based algorithm. Then, we model the diffusion distribution using a generative model.

III-A Default modeling.

As explained before, a loan guarantee is a promise by the guarantor to assume the debt obligation if the borrower defaults, and the default may spread across the network in the manner of a viral infection. The probability that the business will default (not repay the bank in time) is noted as $P(A)$, where $A$ is a small business.

[TABLE]

Where $P_{s}(A)$ is its static default possibility, which means the default probability is caused by the self state of the business. $P_{g}(A)$ is the diffusion probability of how likely $A$ will be influenced when the guarantee of $A$ has defaulted. As $A$ provides a guarantee for other borrowers, $A$ is responsible for repaying the debt if the borrowers default; this may also lead to $A$ defaulting. $P_{g}(A)$ is designed to represent this type of default probability. $P_{g}(A|x)$ denotes the conditional probability when a specific guarantee $x$ defaults. $P_{g}(A)$ can be calculated by estimating $P_{g}(A|x)$ and $P_{s}(x)$ for all of its warrantees.

[TABLE]

Then, we bring function $g(x)=1-x$ for simplified representation and define the $d-th$ order $i-th$ neighbors of $A$ in out-degree as $N_{A,d,i}$. Therefore, all neighbors of $A$ whose distances are smaller than $d-th$ order are notated as $\textrm{O}_{d}(A)=\{x_{i}|N_{A,d,i},i\in 1:m\}$. Equation 2 can be reformulated as:

[TABLE]

Based on the equation 1 and 3, we approximate the desired conditional distribution with the following joint distribution. The joint distribution of $A$ is:

[TABLE]

III-B Sliding window scheme.

Prediction shall be adapted to a dynamic setting with regularly updated forecasting results. Using a sliding window is a typical way for rolling prediction and is commonly adopted in event prediction practices. The business often runs on a quarterly basis; thus it would be helpful to know which borrowers might default on a quarterly basis from a business demand perspective.

In this work, we set a time window of $\omega$ days. Features are extracted in the training window; the corresponding outcome default label is collected in the observation window. The features and default outcome are used to train the model. The trained model is used by collecting the input features during the prediction window and verifying their performance when we reach the end of the evaluation window.

Specifically, in this paper we use a three-month window for training, observation, prediction, and evaluation. As Figure 3 shows, in the training stage, for all customers who obtain bank loans from 2013 Q1 (first quarter of 2013), the features are extracted in that period, and the repayment status between 2013 Q2 are the labels to train the model. In the testing stage, we use the trained model to predict the customers who obtain loans between 2013 Q2 and use the real repayment status from 2013 Q3 to evaluate the performance when reaching the end of September 2013.

There are two reasons for such a sliding window setting:

Prediction shall be adapted to a dynamic setting with regularly updated forecasting results. In fact, using a sliding window is a typical way for rolling prediction and is commonly adopted in event prediction practices such as [28]. 2. 2.

After observing from our datasets and acquiring feedback from financial experts, the business often runs on a quarterly basis. Thus, from a business demand perspective, it would be helpful to know which borrowers may default on a quarterly basis.

III-C Estimating parameters.

In order to obtain $P(A)$, the main task is to estimate $P_{s}(A)$ and $P_{g}(A|x)$ as present in Equation 4.

Estimating $P_{s}(A)$: Estimating this probability requires predicting how the current event sequence will evolve over the next ω days, as this influences whether defaults occur or not. Given the current subsequence $E_{-\omega}$, we designed a positive weighted k-nearest-neighbor classification and regression (P-wkNN) approach.

Naive kNN, the non-parametric lazy learning algorithm, updates a target sample label by its top k nearest neighbors’ label. For extremely imbalanced sample cases, negative samples (majority class) are much more frequent than positive samples (minority class). How to set the hyper parameter value of k is a tradeoff between performance and efficiency, for example, if k is too small, as the negative samples may overwhelm its near positive samples; if k is too large, the computation may be too complex. Hence, if we get top k positive samples, we could update a precise value for the sample. Figure 3 illustrates the algorithm if the target value is more likely to be a positive sample. P-wkNN could predict its value to 0.5 when k is set to 3 and weight set to equal, while kNN predicts 0. In situation b, the predicted instances are more likely to be a negative sample. If we set k to 3 and weights to equal, p-wkNN predicts 0.15 and kNN predicts 0. P-wkNN would not be influenced by outliers.

The detail of the algorithm is described below:

Let $S=\{(x_{i},y_{i}),i=1,2,\ldots,n\}$ be the training set and $x_{i}$ be the observations with a class label $y_{i}$. Let $x$ be a new observation; we need to predict its label $y$. 2. 2.

Find the $k$ nearest positive neighbors to $x$ by a distance function $d(x,x_{i})$ . 3. 3.

Choose the $k-th$ positive neighbor’s distance $\sigma=d(x,x_{dk})$ as the threshold. So the defined neighbors of $x$ are $N=\{x_{i}|d(x,x_{i})\leq\sigma,i=1,2,\ldots,n\}$. 4. 4.

Normalize the distance by function $D_{i}=D(x,x_{i})=\frac{d(x,x_{i})}{\sigma}\in[0,1]$. Then transform $D_{i}$ with a kernel function $K(\cdot)$ into a weight $w_{i}=K(D_{i})$ . 5. 5.

Calculate the total weighted class label. Let be the predicted class label; we have two methods to give its estimations. For the classification method, we have $y=\max_{r}\left(\sum_{x_{i}\in N}{w_{i}I(y+i=r)}\right)$ and for the regression method $y=\frac{\sum_{x_{i}\in N}{w_{i}y_{i}}}{\sum_{x_{i}\in N}{w_{i}}}$.

Estimation $P_{g}(A|x)$: the likelihood of defaults diffusion from $x$ to $A$. Once the warrantee has defaulted, a label column is added to the filtered records table $X$ as shown in Figure 5 (b). The label’s value is decided by its guarantor’s situation in a given time window $\omega$. If the guarantor defaults, the label of this warrantee record will be set to 1, otherwise 0.

We extract the features of the edges from the guarantors to the warrantees, including guaranteed amount, loan amount, warrantees’ degrees, guarantors’ degrees and essential company registration information from both sides. We apply P-wkNN to train the diffusion probability. For a new guarantee relationship, we find its k nearest positive neighborhoods, then estimate the value as described in P-wkNN algorithm.

III-D Make predictions.

In practice, a borrower may have one or more guarantor, making a set of directed acyclic graphs, an example of which is shown in Figure 5 (a). The cyclic graph in a guarantee network is forbidden by administration of regulation departments. Default risk probability diffuses directly from the borrower to the guarantor, which means the probability update needs to follow the signal directed dependency chain. As shown in Figure 5, node B can be both a borrower and a guarantor. If we want to obtain the default probability of node B, we need to calculate node E and node D in advance. Node D is dependent on node E at the same time. All this makes the update path much more complex.

Therefore, we designed a d-order risk diffusion algorithm to predict default risks. This is a BFS (breadth-first-search) based recursion algorithm. First, given historical event streams, we estimate $P_{s}(A)$ and $P_{g}(A|x)$ as described in section 4.3, setting the hyper-parameter $d$, the diffusion order of networked-guarantee loans. Second, we construct table $X$ as the format of $\left(x,A,P_{s}(A),P_{g}(A|x)\right)$ and initialize the output probability $P(A)$ to 0. Third, we set the recursion stop conditions and main procedure. If diffusion order $d$ is equal to 0 or $A$ does not provide any guarantees for others, we return $P(A)$ by $P_{s}(A)$. Meanwhile, if all the $P_{g}(A|x_{i})$ is 0, $P(A)$ is also equal to $P_{s}(A)$. In other conditions, for each node $x\in\textrm{O}_{d}(A)=\{x_{i}|N_{A,d,i},i\in 1:m\}$, calculate $P(x)$ as given $P_{s}(x)$, $P_{g}(x|N_{x,d-1,i})$, which is estimated by section 4.3. For a simple illustration, we reformulate equation 4 as function $\textrm{FUN}(A)$, then we obtain the formulation below:

[TABLE]

The pseudocode is given in algorithm 1.

III-E Complexity analysis.

Assuming there are total $n$ borrowers, each borrower $A$ has $m$ guarantors. As the number of recursions is different for each guarantor, we approximate each vertex out degree as $v_{out}$ and recursive depth as $Depth$. If the diffusion order $d$ is smaller than $Depth$, the function will be called $v_{out}^{d}$ times; otherwise, for each guarantor, the function will be called $v_{out}^{Depth}$. Therefore, the times of function call is the minimal of $n\cdot m\cdot v_{out}^{Depth}$ and $n\cdot m\cdot v_{out}^{d}$. Finally, the time complexity is $\textrm{O}\left(n\cdot v_{out}^{d}\right)$ and space complexity is $\textrm{O}(n)$ .

IV Experiments.

Empirical study is performed to evaluate the prediction performance. Next, we briefly review the dataset and then report the results.

IV-A Data description

The dataset covers the credit behavior of 23096 enterprises in the past three years. There are 3218428 bank loan records in total, 163665 of which have a default history, and 251743 guarantee records, with a total amount of 1147 billion dollars.

We predict default probability as explained earlier. For each small business, we use 3 sequences as a feature. We set the label to 1 if it defaults in the next 3 sequences and 0 otherwise. Then, we slide the window by 3 to generate features as predicting window as well as validation window for the next 3 sequences.

We measure the results in the validation window. All the small businesses with credit activities will be assigned a predicting probability of defaults. Then, we validate the probability with real repayment records and perform an AUC score for different approaches. Based on our experiment and domain knowledge, we set the number of nearest neighbor’s $k$ from 1 to 10 and the diffusion degree $d$ from 0 to 6. The choice of $d$ is based on the analysis of the training data.

IV-B Results.

As mentioned earlier, the main challenges are the imbalanced data and the diffusion risk. We first compare our proposed P-wkNN with start-of-the-art imbalanced learning approaches, such as SMOTE [5], over-sampling, under-sampling, One Sided Selection (OSS) [15], Edited Nearest Neighbor (ENN) [22] and Neighborhood Cleaning Rule (NCL) [3]. Figure 6 gives the AUC curves. It is clear that our proposed pwkNN outperforms classic approaches with a much larger AUC value. In addition, we increase the value of $k$ to see its impact on performance. We can see that our approach always performs better than the other methods. All of them have a rapid lifting when the value of $k$ is smaller than 5, and the AUC gradually stabilizes when the $k$ becomes larger.

The second experiment is to explore the impact of diffusion. We alter the value of $k$ and d of our diffusion model and obtain the prediction results. We can see that a larger value of $k$ leads to better performance. Once the $k$ is greater than 5, the AUC becomes rather stable. For various $k$ values, taking the default diffusion into consideration ($d=1$) is much better than the assumption of no diffusion ($d=0$). This indicates that default diffusion does exist between the neighborhood borrower and the warrantee. However, the default does not diffuse across multiple nodes.

Finally, we perform experiments to validate the effectiveness. We devise strong baseline algorithms and compare the precision, recall and F1 score with the following approach:

INDDP: Imbalance Network Diffusion Default Prediction, our proposed approach for networked-guarantee loan default prediction. 2. 2.

P-wkNN: our proposed classification model for imbalanced learning. 3. 3.

wkNN: Distance weighted k-NN algorithm, a refinement of the k-NN classification algorithm by weighting the contribution of each of the k neighbors according to their distance to the query point, giving greater weight to closer neighbors. Our P-wkNN is an extension of the original wkNN [18]. 4. 4.

Random Forest (RF): an ensemble learning method for classification, regression and other tasks, which has proved effective in credit risk rating procedures [12]. 5. 5.

Support Vector Machines (SVM): a classical supervised learning model proven to be one of the most effective credit risk evaluation tools [30]. 6. 6.

Logistic regression (LR): the classic credit evaluation model, widely used in the financial industry [1].

We set $k$ to be 5 and $d$ to be 1 according to previous experiments. Our approach works best with such parameter setting. Table I summarizes the results: IDNNP outperforms all the baselines in terms of Precision, Recall and F1 score, indicating that considering the full scope of imbalanced and network diffusion is essential.

V Conclusion.

This paper presents a specifically tailored algorithm for automatically predicting default risks for networked-guarantee loans. As in many other financial tasks, the data is extremely imbalanced. The existing network and the potential default diffusion make risk assessment even more difficult. We adapt the classic KNN to handle the imbalanced datasets, using it to estimate network diffusion risks with a data-driven approach instead of forward modeling the diffusion physical model. We perform empirical experiments, give the best parameter settings for the networked-guarantee loan record, and compare it with state-of-the-art credit rating approaches. Future work will include refining the approach and experimenting for more sophisticated and imbalanced learning tasks.

VI Acknowledgment

The work was supported by the Key Basic Research Program of Shanghai Science and Technology Commission, China (15JC1400103, 16JC1402800) and the National Basic Research Program of China (2015CB856004).