Gradient Boosting Survival Tree with Applications in Credit Scoring

TL;DR

This paper introduces a novel gradient boosting survival tree model tailored for credit scoring, effectively handling heterogeneous data and improving risk prediction accuracy in consumer finance applications.

Contribution

The paper presents a nonparametric ensemble tree model called GBST that extends survival trees with gradient boosting, optimizing survival probabilities across time periods.

Findings

GBST outperforms existing survival models in C-index, KS index, and AUC.

The model effectively handles large-scale, heterogeneous financial data.

Application demonstrates improved credit risk quantification.

Abstract

Credit scoring plays a vital role in the field of consumer finance. Survival analysis provides an advanced solution to the credit-scoring problem by quantifying the probability of survival time. In order to deal with highly heterogeneous industrial data collected in Chinese market of consumer finance, we propose a nonparametric ensemble tree model called gradient boosting survival tree (GBST) that extends the survival tree models with a gradient boosting algorithm. The survival tree ensemble is learned by minimizing the negative log-likelihood in an additive manner. The proposed model optimizes the survival probability simultaneously for each time period, which can reduce the overall error significantly. Finally, as a test of the applicability, we apply the GBST model to quantify the credit risk with large-scale real market datasets. The results show that the GBST model outperforms the existing survival models measured by the concordance index (C-index), Kolmogorov-Smirnov (KS) index, as well as by the area under the receiver operating characteristic curve (AUC) of each time period.

Tables

Table 1. Table 1 : Summary of datasets.

dataset	issued month	sample size
training set	January 2007 to April 2013	124,219
test set	May 2013 to December 2013	100,188

Table 2. Table 2 : 10 datasets of Lending Club data by issued months.

index	issued month	sample size
1	June 2007 to March 2011	23,726
2	April 2011 to February 2012	24,013
3	March 2012 to July 2012	20,399
4	August 2012 to November 2012	24,591
5	December 2012 to February 2013	22,278
6	March 2013 to May 2013	29,996
7	June 2013 to July 2013	24,046
8	August 2013	12,694
9	September 2013 to October 2013	28,055
10	November 2013 to December 2013	14,609

Table 3. Table 3 : C-indices of 10 leave-one-out cross validatiton experiments (in terms of mean values and 95% confidence intervals).

Algorithm	GBST	COX	RSF	XGB
C-index	0.6867 (0.6735-0.6999)	${0.6799}^{+}$ (0.6624-0.6934)	${0.6666}^{\ast }$ (0.6455-0.6877)	${0.6809}^{+}$ (0.6665-0.6953)
Algorithm	CoxBoost	GBMCI	CIF	DeepHit
C-index	${0.6779}^{++}$ (0.6670-0.6888)	${0.6742}^{\ast }$ (0.6561-0.6923)	${0.6694}^{\ast }$ (0.6455-0.6933)	${0.6745}^{\ast }$ (0.6587-0.6903)

Table 4. Table 4 : AUC and KS indices of all candidate models (in terms of mean values and 95% confidence intervals) for every 2 repayment months. The highest score of each row is highlighted in boldface type.

Month	Index	GBST	XGB	COX	RSF	CoxBoost	CIF	DeepHit
1	AUC	0.7702       (0.7544-0.7861)	${0.7389}^{\ast }$       (0.72169-0.7562)	${0.7132}^{\ast }$       (0.6946-0.7318)	${0.6639}^{\ast }$       (0.6386-0.6892)	${0.7393}^{++}$       (0.7262-0.7524)	${0.5972}^{\ast }$       (0.5685-0.6259)	${0.6923}^{\ast }$       (0.6734-0.7113)
	KS	0.4245       (0.4087-0.4404)	${0.4010}^{\ast }$       (0.3837-0.4183)	${0.3238}^{\ast }$       (0.3052-0.3424)	${0.2911}^{\ast }$       (0.2658-0.3164)	${0.3895}^{++}$       (0.3764-0.4026)	${0.2594}^{\ast }$       (0.2307-0.2881)	${0.2586}^{\ast }$       (0.2396-0.2775)
3	AUC	0.7101       (0.6915-0.7288)	0.7159       (0.6955-0.7363)	${0.7004}^{+}$       (0.6785-0.7224)	${0.6631}^{\ast }$       (0.6333-0.6930)	0.7185       (0.7030-0.7339)	${0.6726}^{+}$       (0.6388-0.7065)	${0.6875}^{++}$       (0.6651-0.7099)
	KS	0.3002       (0.2819-0.3185)	0.3107       (0.2907-0.3306)	${0.3008}^{+}$       (0.2793-0.3223)	${0.2392}^{\ast }$       (0.2099-0.2685)	0.3201       (0.3050-0.3352)	${0.3026}^{+}$       (0.2694-0.3358)	${0.2430}^{++}$       (0.2210-0.2649)
5	AUC	0.7267       (0.7055-0.7480)	${0.7252}^{\ast }$       (0.7021-0.7484)	${0.7124}^{\ast }$       (0.6875-0.7374)	${0.6942}^{+}$       (0.6602-0.7282)	${0.7260}^{\ast }$       (0.7084-0.7435)	${0.7131}^{\ast }$       (0.6746-0.7516)	${0.6862}^{++}$       (0.6608-0.7116)
	KS	0.3376       (0.3176-0.3576)	${0.3343}^{\ast }$       (0.3125-0.35612)	${0.3075}^{\ast }$       (0.2841-0.3310)	${0.2834}^{+}$       (0.2515-0.3153)	${0.3320}^{\ast }$       (0.3155-0.3485)	${0.3192}^{\ast }$       (0.2830-0.3553)	${0.2393}^{++}$       (0.2154-0.2632)
7	AUC	0.7228       (0.6999-0.7457)	${0.7207}^{+}$       (0.6956-0.7457)	${0.7111}^{\ast }$       (0.6842-0.7381)	${0.6925}^{\ast }$       (0.6557-0.7292)	0.7206       (0.7017-0.7396)	${0.6938}^{\ast }$       (0.6522-0.7353)	${0.6806}^{\ast }$       (0.6531-0.7081)
	KS	0.3167       (0.2956-0.3378)	${0.3162}^{+}$       (0.2932-0.3393)	${0.3015}^{\ast }$       (0.2767-0.3263)	${0.2807}^{\ast }$       (0.2469-0.3145)	0.3137       (0.2962-0.3311)	${0.2987}^{\ast }$       (0.2604-0.3370)	${0.2308}^{\ast }$       (0.2055-0.2561)
9	AUC	0.7154       (0.6918-0.7391)	${0.7144}^{+}$       (0.6886-0.7403)	${0.7069}^{+}$       (0.6791-0.7347)	${0.6926}^{\ast }$       (0.6548-0.7305)	${0.7130}^{++}$       (0.6935-0.7325)	${0.6915}^{\ast }$       (0.6487-0.7344)	${0.6746}^{\ast }$       (0.6463-0.7029)
	KS	0.3048       (0.2835-0.3261)	${0.3044}^{+}$       (0.2812-0.3276)	${0.2893}^{+}$       (0.2643-0.3143)	${0.2719}^{\ast }$       (0.2378-0.3059)	${0.2988}^{++}$       (0.2813-0.3164)	${0.2974}^{\ast }$       (0.2588-0.3359)	${0.2256}^{\ast }$       (0.2001-0.2511)
11	AUC	0.7161       (0.6927-0.7395)	0.7146       (0.6891-0.7402)	${0.7079}^{+}$       (0.6804-0.7354)	${0.6934}^{\ast }$       (0.6559-0.7309)	0.7122       (0.6928-0.7315)	${0.6954}^{\ast }$       (0.6530-0.7378)	${0.6736}^{\ast }$       (0.6455-0.7016)
	KS	0.3052       (0.2846-0.3258)	0.3031       (0.2807-0.3256)	${0.2913}^{+}$       (0.2671-0.3155)	${0.2771}^{\ast }$       (0.2441-0.3101)	0.3009       (0.2839-0.3179)	${0.2935}^{\ast }$       (0.2562-0.3308)	${0.2240}^{\ast }$       (0.1993-0.2487)
13	AUC	0.7139       (0.6920-0.7357)	${0.7123}^{\ast }$       (0.6885-0.7362)	${0.7063}^{\ast }$       (0.6806-0.7320)	${0.6907}^{\ast }$       (0.6558-0.7256)	${0.7096}^{\ast }$       (0.6915-0.72766)	${0.6913}^{\ast }$       (0.6517-0.7309)	${0.6721}^{\ast }$       (0.6460-0.6983)
	KS	0.3009       (0.2821-0.3196)	${0.2984}^{\ast }$       (0.2779-0.3189)	${0.2888}^{\ast }$       (0.2666-0.3109)	${0.2664}^{\ast }$       (0.2363-0.2964)	${0.2980}^{\ast }$       (0.2824-0.3135)	${0.2823}^{\ast }$       (0.2482-0.3163)	${0.2220}^{\ast }$       (0.1995-0.2445)
15	AUC	0.7081       (0.6889-0.7273)	0.7048       (0.6839-0.7257)	${0.6988}^{+}$       (0.6763-0.7213)	${0.6859}^{\ast }$       (0.6553-0.7166)	${0.7018}^{+}$       (0.6860-0.7176)	${0.6855}^{\ast }$       (0.6508-0.7202)	${0.6692}^{\ast }$       (0.6462-0.6921)
	KS	0.2937       (0.2775-0.3100)	${0.2908}^{++}$       (0.2730-0.3086)	${0.2798}^{+}$       (0.2606-0.2989)	${0.2583}^{\ast }$       (0.2323-0.2844)	${0.2871}^{+}$       (0.2736-0.3005)	${0.2794}^{\ast }$       (0.2499-0.3090)	${0.2196}^{\ast }$       (0.2001-0.2391)
17	AUC	0.7029       (0.6867-0.7192)	${0.7001}^{+}$       (0.6823-0.7179)	${0.6948}^{\ast }$       (0.6756-0.7139)	${0.6817}^{\ast }$       (0.6557-0.7078)	${0.6972}^{\ast }$       (0.6837-0.7106)	${0.6828}^{\ast }$       (0.6533-0.7124)	${0.6914}^{\ast }$       (0.6719-0.7109)
	KS	0.2857       (0.2721-0.2994)	${0.2851}^{+}$       (0.2702-0.3001)	${0.2758}^{\ast }$       (0.2598-0.2919)	${0.2552}^{\ast }$       (0.2334-0.2771)	${0.2835}^{\ast }$       (0.2722-0.2948)	${0.2808}^{\ast }$       (0.2561-0.3056)	${0.2559}^{\ast }$       (0.2395-0.2723)
19	AUC	0.6966       (0.6836-0.7096)	0.6930       (0.6788-0.7072)	${0.6885}^{+}$       (0.6732-0.7038)	${0.6756}^{\ast }$       (0.6548-0.6964)	${0.6898}^{\ast }$       (0.67901-0.7006)	${0.6772}^{\ast }$       (0.6536-0.7007)	${0.6823}^{+}$       (0.6667-0.6979)
	KS	0.2771       (0.2666-0.2876)	0.2752       (0.2637-0.2867)	${0.2681}^{+}$       (0.2557-0.2804)	${0.2467}^{\ast }$       (0.2299-0.2636)	${0.2753}^{\ast }$       (0.2666-0.2840)	${0.2656}^{\ast }$       (0.2465-0.2847)	${0.2471}^{+}$       (0.2345-0.2597)
21	AUC	0.6911       (0.6814-0.7008)	${0.6868}^{+}$       (0.6763-0.6974)	${0.6835}^{+}$       (0.6721-0.6949)	${0.6715}^{\ast }$       (0.6560-0.6870)	0.6839       (0.6759-0.6919)	${0.6747}^{\ast }$       (0.6572-0.6923)	${0.6792}^{\ast }$       (0.6676-0.6908)
	KS	0.2716       (0.2639-0.2792)	${0.2691}^{+}$       (0.2607-0.2774)	${0.2635}^{+}$       (0.2545-0.2725)	${0.2413}^{\ast }$       (0.2290-0.2535)	0.2687       (0.2624-0.2750)	${0.2622}^{\ast }$       (0.2483-0.2760)	${0.2439}^{\ast }$       (0.2347-0.2530)
23	AUC	0.6875       (0.6806-0.6944)	${0.6829}^{++}$       (0.6753-0.6904)	${0.6797}^{+}$       (0.6716-0.6878)	${0.6671}^{\ast }$       (0.6561-0.6781)	${0.6796}^{++}$       (0.6739-0.6853)	${0.6744}^{\ast }$       (0.6619-0.6869)	${0.6576}^{\ast }$       (0.6493-0.6658)
	KS	0.2668       (0.26159-0.2721)	${0.2662}^{++}$       (0.2605-0.2719)	${0.2599}^{+}$       (0.2538-0.2661)	${0.2364}^{\ast }$       (0.2281-0.2448)	${0.2640}^{++}$       (0.2597-0.2683)	${0.2660}^{\ast }$       (0.2565-0.2755)	${0.2173}^{\ast }$       (0.2110-0.2235)

Table 5. Table 5 : Summary of datasets.

dataset	transaction date	sample size
training set	January 2018	200,000
test set	March 2018	120,000

Table 6. Table 6 : List of important features, which are selected according to the feature importance generated by an XGB model trained with the training dataset.

source	feature
PBC report	income score
	credit score
	overdue information of credit cards
personal information	age
	sex
	education level
third-party rate agency	no. of loans in other lending platforms
	travel intensity
other information	whether possessing a car
	application channel

Table 7. Table 7 : 10 datasets of 360 Finance data by date of first transaction (the date when loans are borrowed).

index	first transaction date	sample size
1	January 1, 2018 to January 4, 2018	34,904
2	January 5, 2018 to January 8, 2018	30,355
3	January 9, 2018 to January 13, 2018	34,868
4	January 14, 2018 to January 17, 2018	27,143
5	January 18, 2018 to January 23, 2018	35,978
6	January 24, 2018 to January 28, 2018	27,309
7	January 29, 2018 to March 3, 2018	35,273
8	March 4, 2018 to March 8, 2018	34,163
9	March 9, 2018 to March 13, 2018	30,273
10	March 14, 2018 to March 20, 2018	29,734

Table 8. Table 8 : C-indices of 10 LOOCV experiments (in terms of mean values and 95% confidence intervals).

Algorithm	GBST	COX	RSF	XGB
C-index	0.7997 (0.7826-0.8167)	${0.7797}^{+}$ (0.7652-0.7941)	${0.7765}^{\ast }$ (0.7576-0.7955)	${0.7742}^{\ast }$ (0.7537-0.7947)
Algorithm	CoxBoost	GBMCI	CIF	DeepHit
C-index	${0.7691}^{+}$ (0.7529-0.7853)	${0.6905}^{\ast }$ (0.6767-0.7043)	${0.7395}^{\ast }$ (0.7189-0.7601)	${0.7808}^{++}$ (0.7659-0.7957)

Table 9. Table 9 : AUC and KS indices of all candidate models (in terms of mean values and 95% confidence intervals) for every repayment months. The highest score of each row is highlighted in boldface type.

Month	Index	GBST	XGB	COX	RSF	CoxBoost	CIF	DeepHit
1	AUC	0.8251       (0.8046-0.8456)	${0.8068}^{+}$       (0.78945-0.8241)	${0.8124}^{\ast }$       (0.7897-0.8352)	${0.8180}^{+}$       (0.7934-0.8426)	${0.7976}^{\ast }$       (0.7781-0.8170)	${0.6959}^{\ast }$       (0.6711-0.7206)	${0.7925}^{\ast }$       (0.7746-0.8104)
	KS	0.5164       (0.4959-0.5369)	${0.4687}^{+}$       (0.4513-0.4860)	${0.4766}^{\ast }$       (0.4539-0.4994)	${0.4999}^{+}$       (0.4753-0.5245)	${0.4561}^{\ast }$       (0.4366-0.4755)	${0.3377}^{\ast }$       (0.3129-0.3624)	${0.4608}^{\ast }$       (0.4429-0.4786)
2	AUC	0.8225       (0.7984-0.8467)	${0.8060}^{\ast }$       (0.7855-0.8265)	${0.8149}^{+}$       (0.7881-0.8417)	${0.8149}^{\ast }$       (0.7859-0.8439)	${0.7952}^{++}$       (0.7723-0.8181)	${0.7609}^{\ast }$       (0.7317-0.7900)	${0.7906}^{\ast }$       (0.7695-0.8117)
	KS	0.5135       (0.4899-0.5372)	${0.4616}^{\ast }$       (0.4416-0.4817)	${0.4775}^{+}$       (0.4512-0.5038)	${0.4944}^{\ast }$       (0.4660-0.5229)	${0.4480}^{++}$       (0.4255-0.4705)	${0.4043}^{\ast }$       (0.3757-0.4329)	${0.4542}^{+}$       (0.4335-0.4749)
3	AUC	0.8224       (0.7950-0.8498)	${0.8048}^{+}$       (0.7815-0.8280)	${0.8120}^{\ast }$       (0.7815-0.8425)	${0.8158}^{++}$       (0.7828-0.8488)	${0.7975}^{\ast }$       (0.7714-0.8236)	${0.7650}^{\ast }$       (0.7319-0.7982)	${0.7923}^{\ast }$       (0.7683-0.8163)
	KS	0.5145       (0.4887-0.5403)	${0.4556}^{+}$       (0.4338-0.4775)	${0.4707}^{\ast }$       (0.4420-0.4993)	${0.4943}^{++}$       (0.4632-0.52538)	${0.4495}^{\ast }$       (0.4250-0.4740)	${0.4060}^{\ast }$       (0.3749-0.4372)	${0.4551}^{\ast }$       (0.4325-0.4776)
4	AUC	0.8199       (0.7902-0.8496)	${0.8051}^{\ast }$       (0.7800-0.8303)	0.8105       (0.7775-0.8434)	0.8133       (0.7776-0.8489)	${0.7973}^{+}$       (0.7692-0.8255)	${0.7488}^{\ast }$       (0.7130-0.7847)	${0.7921}^{\ast }$       (0.7662-0.8180)
	KS	0.5098       (0.4825-0.5371)	${0.4571}^{\ast }$       (0.4340-0.4802)	0.4653       (0.4350-0.4957)	0.4954       (0.4626-0.5283)	${0.4467}^{+}$       (0.4207-0.4726)	${0.379}^{\ast }6$       (0.3467-0.4126)	${0.4476}^{\ast }$       (0.4238-0.4715)
5	AUC	0.8210       (0.7904-0.8515)	${0.8041}^{++}$       (0.7782-0.8300)	${0.8088}^{+}$       (0.7749-0.8428)	${0.8051}^{+}$       (0.7684-0.8419)	${0.7967}^{\ast }$       (0.7677-0.8258)	${0.7420}^{\ast }$       (0.7050-0.7789)	${0.7903}^{\ast }$       (0.7636-0.8170)
	KS	0.5107       (0.4832-0.5383)	${0.4557}^{++}$       (0.4324-0.4790)	${0.4591}^{+}$       (0.4285-0.4896)	${0.4865}^{+}$       (0.4535-0.5196)	${0.4426}^{\ast }$       (0.4165-0.4688)	${0.3703}^{\ast }$       (0.3370-0.4035)	${0.4403}^{\ast }$       (0.4162-0.4643)
6	AUC	0.8186       (0.7883-0.8488)	${0.7999}^{\ast }$       (0.7742-0.8255)	0.8040       (0.7704-0.8377)	${0.8021}^{+}$       (0.7657-0.8385)	${0.7970}^{\ast }$       (0.7683-0.8258)	${0.7538}^{\ast }$       (0.7172-0.7904)	${0.7882}^{\ast }$       (0.7618-0.8147)
	KS	0.5113       (0.4847-0.5379)	${0.4508}^{\ast }$       (0.4282-0.4734)	0.4525       (0.4229-0.4821)	${0.4862}^{+}$       (0.4542-0.5182)	${0.4421}^{\ast }$       (0.4168-0.4674)	${0.3851}^{\ast }$       (0.3530-0.4173)	${0.4366}^{\ast }$       (0.4133-0.4599)
7	AUC	0.8156       (0.7874-0.8438)	${0.7944}^{\ast }$       (0.7705-0.8184)	${0.7980}^{\ast }$       (0.7667-0.8294)	${0.7941}^{\ast }$       (0.7602-0.8281)	${0.7921}^{\ast }$       (0.7653-0.8189)	${0.7660}^{\ast }$       (0.7319-0.8001)	${0.7883}^{\ast }$       (0.7637-0.8130)
	KS	0.5116       (0.4873-0.5359)	${0.4411}^{\ast }$       (0.4205-0.4616)	${0.4418}^{\ast }$       (0.4148-0.4688)	${0.4708}^{\ast }$       (0.4416-0.5000)	${0.4317}^{\ast }$       (0.4086-0.4548)	${0.4111}^{\ast }$       (0.3818-0.4405)	${0.4363}^{\ast }$       (0.4151-0.4576)
8	AUC	0.8120       (0.7872-0.8368)	0.7900       (0.7691-0.8110)	${0.7932}^{\ast }$       (0.7657-0.8207)	${0.7904}^{\ast }$       (0.7606-0.8202)	${0.7851}^{\ast }$       (0.7616-0.8086)	${0.7665}^{\ast }$       (0.7366-0.7964)	${0.7840}^{\ast }$       (0.7624-0.8057)
	KS	0.5103       (0.4892-0.5313)	0.4349       (0.4171-0.4528)	${0.4350}^{\ast }$       (0.4116-0.4584)	${0.4686}^{\ast }$       (0.4433-0.4939)	${0.4224}^{\ast }$       (0.4024-0.4424)	${0.4067}^{\ast }$       (0.3813-0.4322)	${0.4298}^{\ast }$       (0.4114-0.4482)
9	AUC	0.8095       (0.7884-0.8305)	${0.7863}^{++}$       (0.7685-0.8042)	${0.7878}^{+}$       (0.7644-0.8112)	${0.7871}^{\ast }$       (0.7618-0.8124)	${0.7806}^{\ast }$       (0.7606-0.8006)	${0.7639}^{\ast }$       (0.7385-0.7894)	${0.7804}^{+}$       (0.7620-0.7988)
	KS	0.5093       (0.4917-0.5270)	${0.4301}^{++}$       (0.4151-0.4451)	${0.4285}^{+}$       (0.4089-0.4482)	${0.4683}^{\ast }$       (0.4471-0.4896)	${0.4177}^{\ast }$       (0.4009-0.4345)	${0.4078}^{+}$       (0.3865-0.4292)	${0.4274}^{\ast }$       (0.4119-0.4428)
10	AUC	0.8061       (0.7893-0.8229)	${0.7808}^{+}$       (0.7666-0.7950)	${0.7809}^{++}$       (0.7623-0.7996)	0.7820       (0.7618-0.8022)	${0.7749}^{\ast }$       (0.7589-0.7908)	${0.7587}^{\ast }$       (0.7384-0.7790)	${0.7772}^{\ast }$       (0.7625-0.7918)
	KS	0.5087       (0.4951-0.5223)	${0.4231}^{+}$       (0.4115-0.4346)	${0.4206}^{++}$       (0.4055-0.4357)	0.4630       (0.4466-0.4794)	${0.4100}^{\ast }$       (0.3970-0.4229)	${0.3785}^{\ast }$       (0.3621-0.3949)	${0.4225}^{\ast }$       (0.4107-0.4344)
11	AUC	0.8022       (0.7897-0.8147)	0.7765       (0.7658-0.7871)	${0.7767}^{\ast }$       (0.7628-0.7906)	${0.7784}^{+}$       (0.7634-0.7935)	${0.7679}^{\ast }$       (0.7561-0.7798)	${0.7582}^{\ast }$       (0.7431-0.7734)	${0.7736}^{\ast }$       (0.7626-0.7845)
	KS	0.5082       (0.4983-0.5181)	0.4175       (0.4092-0.4259)	${0.4145}^{\ast }$       (0.4035-0.4255)	${0.4617}^{+}$       (0.4498-0.4735)	${0.4014}^{\ast }$       (0.3920-0.4108)	${0.3963}^{\ast }$       (0.3843-0.4082)	${0.4183}^{\ast }$       (0.4097-0.4270)
12	AUC	0.7988       (0.7899-0.8077)	${0.7716}^{\ast }$       (0.7640-0.7791)	${0.7720}^{\ast }$       (0.7622-0.7819)	${0.7750}^{\ast }$       (0.7643-0.7857)	${0.7637}^{\ast }$       (0.7553-0.7722)	${0.7506}^{\ast }$       (0.7398-0.7613)	${0.7693}^{\ast }$       (0.7615-0.7771)
	KS	0.5053       (0.4985-0.5120)	${0.4121}^{\ast }$       (0.4063-0.4178)	${0.4092}^{\ast }$       (0.4017-0.4167)	${0.4597}^{\ast }$       (0.4516-0.4679)	${0.3961}^{\ast }$       (0.3896-0.4025)	${0.3870}^{\ast }$       (0.3789-0.3952)	${0.4126}^{\ast }$       (0.4067-0.4186)

Model	Link Address
GBST	https://github.com/360jinrong/GBST
COX	https://lifelines.readthedocs.io/en/latest/fitters/regression/CoxPHFitter.html
RSF	https://scikit-survival.readthedocs.io/en/latest/api/generated/sksurv.ensemble.RandomSurvivalForest.html
CoxBoost	https://cran.r-project.org/web/packages/CoxBoost/index.html
CIF	https://cran.r-project.org/web/packages/party/party.pdf
GBMCI	https://github.com/uci-cbcl/GBMCI
DeepHit	https://github.com/havakv/pycox

Equations

$$S(t):=\mathbb{P}(T>t),$$

$$h({\tau }_j):=\mathbb{P}({\tau }_{j-1}<T\le {\tau }_j|T>{\tau }_{j-1}),j=1,2,\dots ,$$

$$S({\tau }_j)=$$

$$=$$

$$\mathbb{P}({\tau }_{j-1}<T\le {\tau }_j)=h({\tau }_j)S({\tau }_{j-1})=h({\tau }_j)\prod _{l=1}^{j-1}\left(1-h({\tau }_l)\right).$$

$$f(t):=$$

$$h(t)=$$

$$\displaystyle y_{j}(t):=\left\{\begin{array}[]{ll}-1,&\textrm{ if }\delta=0\ or\ (\delta=1\ and\ t>\tau_{j}),\\ 1,&\textrm{ if }\delta=1\ and\ t\leq\tau_{j},\end{array}\right.,\textrm{ and }$$

$$\displaystyle J(t):=\left\{\begin{array}[]{ll}j,&\textrm{if }t\in(\tau_{j-1},\tau_{j}],\\ J+1,&\textrm{if }t>\tau_{J}.\end{array}\right.$$

$$\mathbb{P}(T=t)=\prod _{j=1}^{J(t)}\frac{1}{1+e^{-y_j(t)f({\tau }_j)}},\forall t\le {\tau }_J.$$

$$\mathbb{P}(T=t>{\tau }_J)=\prod _{j=1}^J\frac{1}{1+e^{f({\tau }_j)}}=\prod _{j=1}^J\frac{1}{1+e^{-y_j(t)f({\tau }_j)}}.$$

$$\mathbb{P}(T=t)=\prod _{j=1}^{J(t)\wedge J}\frac{1}{1+e^{-y_j(t)f({\tau }_j)}},$$

$$f(t;\mathbf{x})\cong \hat{f}(t;\mathbf{x}):=\sum _{k=1}^K{f}_k(t;\mathbf{x}),f_k\in \mathcal{F},$$

$$\mathcal{F}:=\left\{f(t;\mathbf{x})=w(t,q(\mathbf{x}))|q:{\mathbb{R}}^n\to L,w:[0,\infty )\times L\to \mathbb{R}\right\}$$

$$\mathcal{L}(\varphi )=$$

$$\cong$$

$$=$$

$$N_j:=\{i\in \{1,2,\dots ,N\}|J(t_i)\ge j\}.$$

$$\mathcal{L}(\varphi )=$$

$$f_m=\underset{f\in \mathcal{F}}{\mathrm{argmin}}{\mathcal{L}}^{(m)}:=$$

$$+\frac{\lambda }{2}\Vert w{\Vert }^2$$

$${\hat{f}}^{(m)}(t;\mathbf{x})={\hat{f}}^{(m-1)}(t;\mathbf{x})+f_m(t;\mathbf{x}).$$

$$\varrho (y,f):=\log (1+\exp (-y\cdot f)).$$

$${\mathcal{L}}^{(m)}(f)\cong$$

$$+\frac{\lambda }{2}\Vert w{\Vert }^2,$$

$$r_{i,j}^{(m-1)}:=\frac{\partial \varrho }{\partial f}\left(y_j(t_i),{\hat{f}}^{(m-1)}({\tau }_j;{\mathbf{x}}_i)\right),$$

$${\sigma }_{i,j}^{(m-1)}:=\frac{{\partial }^2\varrho }{\partial f^2}\left(y_j(t_i),{\hat{f}}^{(m-1)}({\tau }_j;{\mathbf{x}}_i)\right).$$

$${\tilde{\mathcal{L}}}^{(m)}(f):=\sum _{j=1}^J\sum _{i\in N_j}\left(r_{i,j}^{(m-1)}f({\tau }_j;{\mathbf{x}}_i)+\frac{1}{2}{\sigma }_{i,j}^{(m-1)}{f}^2({\tau }_j;{\mathbf{x}}_i)\right)+\frac{\lambda }{2}\Vert w{\Vert }^2,$$

$$\frac{\partial \varrho }{\partial f}(y,f)=$$

$$\frac{{\partial }^2\varrho }{\partial f^2}(y,f)=$$

$$f({\tau }_j;{\mathbf{x}}_i)=\sum _{l=1}^L{w}_l({\tau }_j)1(i\in I_l)$$

$${\tilde{\mathcal{L}}}^{(m)}(w):=\sum _{j=1}^J\sum _{l=1}^L\left\{\sum _{i\in N_j\cap I_l}\left(r_{i,j}^{(m-1)}{w}_l({\tau }_j)+\frac{1}{2}{\sigma }_{i,j}^{(m-1)}{w}_l^2({\tau }_j)\right)+\frac{\lambda }{2}{w}_l^2({\tau }_j)\right\}.$$

$$w_l^{(m)}({\tau }_j)=-\frac{{\sum }_{i\in N_j\cap I_l}{r}_{i,j}^{(m-1)}}{{\sum }_{i\in N_j\cap I_l}{\sigma }_{i,j}^{(m-1)}+\lambda },$$

Full text

1 Introduction

Chinese consumer finance market, with the rapid development of internet finance and e-commerce, has entered a period of market eruption in recent years. The consumer credit hit 8.45 trillion RMB, the historical high in October 2018, according to a research report released by Tsinghua University111Chinese consumer credit market research 2018 (in Chinese), Institute for Chinese Economic Practice and Thinking, Tsinghua University, https://att.dahecube.com/4a5ab7de-1aa3-4754-a4d7-6e707d943d69.. However, only approximately one third of Chinese consumers have credit ratings, in spite of the rising needs for financing, and therefore, the lack of credit infrastructure has made a critical challenge to Chinese credit providers and it calls for more advanced credit scoring techniques.

Credit scoring has been a commonly used risk measurement tool for decades (see Thomas et al., (2002) for a comprehensive introduction) and it has already been widely adopted by credit agencies all over the world. Credit scoring is traditionally solved by classification methods, e.g., logistic regression, decision tree, and more recently, gradient boosting decision tree (GBDT, Friedman,, 2002) and other boosting models, e.g., XGBoost (XGB, Chen and Guestrin,, 2016).

Survival analysis is a statistical method to deal with the analysis of time-to-event data, where the time to the occurrence of an event is of particular interest. In the context of credit scoring, the event is usually defined as “default”. Therefore, the primary advantage of applying survival analysis is that it can not only model whether to default, but also predict the probability of time when default occurs. The idea to use survival analysis in the field of credit scoring was first introduced by Narain, (1992). Since 2000, several studies (Cao et al.,, 2009; Dirick et al.,, 2017 and references therein) have applied survival analysis to quantify the credit risk and the results show a better performance on real credit data, which are collected by banks in developed countries like the UK and Belgium (Dirick et al.,, 2017), than the traditional credit scoring methods. In contrast to the developed countries where credit data are mostly provided by credit bureaus, the data collected for credit assessment in Chinese market are usually obtained from the Internet, which makes them high-dimensional, noisy, sparse and extremely heterogeneous.

The most widely applied model in survival analysis is the Cox proportional hazards model (Blakely and Cox,, 1972) with (generalized) linear functions (Cao et al.,, 2009) or with splines (Dirick et al.,, 2017). To relax the linear assumption imposed by Cox model, several extensions were developed. Among them, a boosting approach (CoxBoost) was introduced by Binder and Schumacher, (2008), while a gradient boosting algorithm called GBMCI was developed by Chen et al., (2013) to optimize the concordance index (C-index, Harrell et al.,, 1982) directly. Both algorithms are using Cox as base models.

An alternative category of survival analysis methods is based on trees, e.g., survival trees (Gordon and Olshen,, 1985) and random survival forests (RSF, Ishwaran et al.,, 2008) were proposed to quantify survival probabilities in medical studies. Wright et al., (2017) proposed the conditional inference survival forest model (CIF) to correct the bias in the RSF model, and empirical studies (Nasejje et al.,, 2017) showed that CIF is superior to RSF model on some medical datasets.

Recently several models originated from the machine learning community are adapted to survival analysis. For instance, Fernández et al., (2016) introduced a semi-parametric Bayesian survival model based on Gaussian processes, while an approach DeepHit based on deep neural networks was introduced by Lee et al., (2018) to learn the distribution of survival times. The former model relies on the assumption that the latent stochastic process follows a Gaussian process which is usually not true for real credit data. The problem of applying the latter approach to credit scoring is that deep neural networks perform in general worse on tabular data (like the credit data) than tree-based ensemble methods, unless the network architecture is specially designed (Pfister,, 2021; Popov et al.,, 2020). DeepHit introduced by Lee et al., (2018) (see also Blumenstock et al.,, 2020) is based on the fully-connected type of network, which may not be the best architecture for heterogeneous credit data.

On the one hand, to our best knowledge, there exists no gradient boosting algorithm that is applied to survival trees. On the other hand, studies on the Chinese consumer finance market (Li et al.,, 2019) indicate that ensemble methods outperform the traditional linear methods and the deep neural network as well, due to the characteristics of the credit data collected in Chinese market. This inspires us to develop a gradient boosting algorithm for survival tree models that optimizes the negative log-likelihood function. Hence, we propose in this paper a nonparametric ensemble tree model called gradient boosting survival tree (GBST) that extends the survival tree models with a gradient boosting algorithm. The proposed model optimizes the survival probability of each time period simultaneously, which can reduce the overall error of the model significantly.

The novelty of this paper is twofold. First, in order to deal with high-dimensional and heterogeneous credit data, we propose a gradient boosting algorithm to survival trees with likelihood as the optimization objective. Sepcifically, most of the boosting algorithms (e.g., Binder and Schumacher,, 2008; Chen et al.,, 2013) applied to survival analysis are based on linear Cox model rather than trees, and meanwhile the tree-based survival models (Ishwaran et al.,, 2008; Wright et al.,, 2017) employ the empirical log-rank split for generating trees rather than the split rule of optimizing likelihood objective. Second, we conduct experiments on two real large-scale credit datasets, where a comprehensive comparison is made between GBST and other popular time-to-event models, which fills the gap that most of literature in survival analysis methods are tested on small-scale clinical data.

The remainder of this paper is organized as follows. Section 2 introduces the methodology of gradient boosting survival tree (GBST). We start with a brief introduction of survival analysis, followed by a calculation of the negative log-likelihood loss function based on the hazard function represented by a survival tree ensemble. The loss function is optimized by a gradient boosting approach and the corresponding algorithms are derived. In Section 3, empirical experiments are conducted on two datasets: one is the lending club loan dataset that can be publicly retrieved from Kaggle and the other is a private dataset collected in Chinese consumer finance market. The results show that GBST outperforms the other candidate models (Cox, RSF, CoxBoost, XGB, CIF, GBMCI and DeepHit) on both datasets measured by the concordance index (C-index), Kolmogorov-Smirnov (KS) index, as well as by the area under the receiver operating characteristic curve (AUC) of each time period. Finally, we summarize this paper in Section 4.

2 Gradient boosting survival tree

In this section, we introduce the method of gradient boosting survival tree (GBST) that extends the traditional survival analysis models with a gradient boosting tree algorithm.

2.1 Survival analysis

The survival analysis aims, in general, for quantifying the distribution of time until some random event happens. In the field of consumer finance, the event is that a borrower defaults. Let $T$ denote the default time, i.e., the time period from the moment that an borrower obtains the money to the event that he defaults. Note that in general $T$ is a random variable and we assume it follows a probability distribution $\mathbb{P}$. Then for any time $t$, the survival function $S(t)$ is defined as follows,

[TABLE]

which quantifies the probability that the borrower “survives” longer than time $t$. Comparing with the traditional credit scoring methods, the survival analysis is able to predict not only the probability whether a borrower defaults, but also the probability of time when he defaults.

In the field of consumer finance, we need to consider the right censoring cases, i.e., the borrower does not default within the whole observation period, which is said to be “censored”. An indicator $\delta$ is used to denote whether the sample is censored ($\delta=0$) or not ($\delta=1$).

Throughout this paper, we consider a finite set of observation time, denoted by $\{\tau_{j},j=0,1,2,\ldots,J\}$ satisfying $0=\tau_{0}<\tau_{1}<\ldots<\tau_{J}<\infty$. In the context of consumer finance, the observation time could be, e.g., the monthly repayment due date. The discrete-time hazard function is defined as

[TABLE]

which quantifies the probability of the event that occurs within the time period $(\tau_{j-1},\tau_{j}]$ for the first time. By definition, the survival function $S$ can be represented by the hazard function $h$. Indeed, for each $j=1,2,\ldots$, we have

[TABLE]

Finally, the probability of the event that occurs within the time period $(\tau_{j-1},\tau_{j}]$ is given by

[TABLE]

Suppose that $h(t)\in(0,1)$ for each $t$ and we consider a logarithmic transformation of $h$ such as

[TABLE]

By defining the following two mappings according to the censoring indicator:

[TABLE]

[TABLE]

Eq. (2) implies

[TABLE]

On the other hand, if $t>\tau_{J}$, i.e., beyond the total observation period, its probability could be quantified as

[TABLE]

Combining the above two cases, we have that for any observed default time $t\in[0,\infty)$, its probability is given by

[TABLE]

where $J(t)\wedge J:=\min(J(t),J)$.

Remark. In many real applications, the exact “death” time might not be observable. Instead, with a discrete series of observation (sampling) time, one can usually determine in which period $(\tau_{j-1},\tau_{j}]$ one individual is “dead”. Then the output of $y_{j}(t)$ and $J(t)$ are observable without knowing the exact “death” time.

2.2 Learning objective

Let $\boldsymbol{x}\in\mathbb{R}^{n}$ denote the feature of each individual. In the field of consumer finance, it quantifies the characteristics of each borrower, such as age, sex, education level, etc. The fundamental task of survival analysis is then to investigate the optimal survival function for each individual $S(t;\boldsymbol{x})$. As shown in Eq. (1), it is sufficient to find out the best hazard function $h(t;\boldsymbol{x})$ and equivalently its logarithmic representation $f(t;\boldsymbol{x})$.

Now we state the major assumption of this paper: $f$ is assumed to be approximated by $\hat{f}:[0,\infty)\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ with a tree ensemble model such as

[TABLE]

where for each $k$, $f_{k}$ is the output of $k$th survival tree, and

[TABLE]

is the space of survival trees with leaf nodes as functionals of time $t$. Here $q$ represents the structure of each tree with $L$ leaf nodes that map a variable $\boldsymbol{x}$ to its corresponding leaf index. Each $f_{k}$ represents an independent tree structure $q_{k}$ and a leaf weight functional $w^{(k)}$. The difference between a survival tree and a conventional regression tree is that the leaf nodes of a survival tree are functionals of time $t$, whereas the leaf nodes of a regression tree are simply real values.

Suppose we observe samples $\{\boldsymbol{x}_{i},t_{i}\},i=1,2,\ldots,N$, where $t_{i}$ denotes the default time. Under the i.i.d. assumption, we can calculate the negative log-likelihood as in Eq. (8) as follows,

[TABLE]

For each $j=1,2,\ldots,J$, the set of samples that survive longer than $\tau_{j-1}$ (in other words, $J(t_{i})\geq j$), $N_{j}$, is defined as follows:

[TABLE]

Then it is easy to verify that Eq. (10) implies

[TABLE]

Finally, in order to avoid over-fitting, one $l^{2}$ regularization term is added to penalize the complexity of the model:

[TABLE]

where $\lambda>0$ is a hyperparameter controls how large the penalization is and $w^{(k)}$ is the leaf weight functionals for $k$th survival tree. It should be noted that since we consider in this paper merely the discrete observation time $\{\tau_{j}\}$, the $l^{2}$-norm of a leaf weight functional $w$ is calculated as $\lVert w\rVert^{2}:=\sum_{j=1}^{J}\sum_{l=1}^{L}w_{l}^{2}(\tau_{j}).$

2.3 Gradient boosting

Inspired by the gradient boosting method (Friedman,, 2001), the optimization problem stated in Eq. (12) is solved in an additive manner. More specifically, one starts with an initial survival tree $f_{0}$ and for $m=1,2,\ldots,$

[TABLE]

and $\hat{f}$ is updated by

[TABLE]

Define the logarithmic loss function $\varrho:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$ as

[TABLE]

Then $\mathcal{L}^{(m)}$ can be approximated by a Taylor expansion up to the second order:

[TABLE]

where the first and second order derivative $r^{(m-1)}_{i,j}$ and $\sigma^{(m-1)}_{i,j}$ are given by

[TABLE]

Removing the constant independent of $f$ in Eq. (14), the objective becomes

[TABLE]

Note that since $y_{j}(t_{i})$ takes value either 1 or -1 (see Eq. (7)), for the logarithmic loss function $\varrho$, it is easy to verify that

[TABLE]

where $h:=\frac{1}{1+e^{-f}}$ can be interpreted as the hazard probability (cf. Eq. (4)).

For a survival tree with $L$ leaf nodes, $f$ can be represented as

[TABLE]

where we define $I_{l}:=\{i\in\{1,2,\ldots,N\}|q(\boldsymbol{x}_{i})=l\}$ and $q$ represents the structure of the tree. Eq. (15) becomes

[TABLE]

As shown in Eq. (19), $\sigma^{(m-1)}_{i,j}>0,\forall i,j,$ and $\tilde{\mathcal{L}}^{(m)}$ is therefore a strictly convex function with respect to $w$. Hence, given a fixed structure $q$, for each $j$ and $l$, the optimal $w_{l}^{(m)}(\tau_{j})$ is

[TABLE]

and the corresponding $\tilde{\mathcal{L}}^{(m)}$ is

[TABLE]

where the explicit form of $r^{(m-1)}_{i,j}$ and $\sigma^{(m-1)}_{i,j}$ is

[TABLE]

with

[TABLE]

being the hazard probability estimated for $\boldsymbol{x}_{i}$ at time $\tau_{j}$ up to $(m-1)$th iteration.

Eq. (21) measures how well a survival tree $q$ fits the observed data. Based on it, we develop a greedy algorithm for constructing the optimal survival tree, since usually it is impossible to explore all possible tree structures. We start with a single leaf node and iteratively split it into two sets. Suppose that one set $I$ is split into two separate sets $I_{L}$ and $I_{R}$. Then the corresponding loss reduction is given by

[TABLE]

The pseudo code of the proposed greedy split algorithm is shown in Algorithm 1. Some techniques to speed up the split algorithm is demonstrated in Appendix.

To initialize the tree, one has to give an initial estimation of the hazard probability $h_{i,j}^{(0)}$. Without any prior knowledge, we assume each individual follows the same hazard distribution over time with the Kaplan-Meier estimator (Kaplan and Meier,, 1958). More specifically, let $d_{j}$ be the number of default individuals at $(\tau_{j-1},\tau_{j}]$ and $n_{j}$ be the number of individuals whose survival time is larger than $\tau_{j-1}$, i.e., formally,

[TABLE]

where $t_{i}$ is the default time of $i$th individual. Then, the hazard probability by the Kaplan-Meier estimator is calculated as

[TABLE]

which is used as our initial estimation $h_{i,j}^{(0)}=\hat{h}_{j},\forall i,j.$

Finally, we summarize the algorithm for GBST in Algorithm 2. All the codes of GBST are publicly available in Github222https://github.com/360jinrong/GBST.

3 Experiments

In order to show the applicability of the proposed gradient boosting survival tree (GBST) algorithm, we apply it to analyze two datasets. One is the lending club loan data333https://www.kaggle.com/sonujha090/xyzcorp-lendingdata?select=XYZCorp_LendingData.txt, which is publicly available from Kaggle, and the other is a private loan dataset provided by 360 Finance Inc. It is noteworthy that the former public dataset is collected from the US market, since to our best knowledge, there exists no public dataset available from Chinese consumer finance market.

We compare the performance of GBST with the performance of other existing models: Cox proportional hazards model (COX), random survival forest (RSF, Ishwaran et al.,, 2008), XGBoost (XGB, Chen and Guestrin,, 2016), CoxBoost (Binder and Schumacher,, 2008), gradient boosting machine for concordance index (GBMCI, Chen et al.,, 2013), conditional inference forest model (CIF, Wright et al.,, 2017) and DeepHit (Lee et al.,, 2018).

Credit scoring is usually solved by classification methods. It is, therefore, of great interest to see comparison of the proposed GBST model with classification methods. Among them, XGB dominates the Kaggle competitions on tabular data (Kaggle,, 2019). Therefore, we choose XGB as the best candidate among all classification methods.

There are several open-source packages implemented for the models mentioned above. Among them, there are implementations in R for CoxBoost, GBMCI and ICF, while Python packages exist for the other models. For readers’ convenience, the link addresses of the implemented packages of the models are listed in Appendix C. We also notice that lately there exists an in-progress Python package (Pölsterl,, 2020) called “gradient boosting survival analysis”, which provides several implementations of popular models, including Cox, RSF and CoxBoost, as well as a Python package “PyCox” by Geck, (2012), which provides in addition an implementation of DeepHit.

3.1 Lending club loan data

3.1.1 Dataset

The lending club loan dataset retrieved from Kaggle contains data of all loans issued from 2007 to 2015, including the loan status (normal, late, fully paid) and payment information. It totally contains 855,527 individual loans. The observation period we selected is 24 months. 124,219 loans issued before May 2013 are selected as the training set and 100,188 loans issued between May 2013 and December 2013 as the test set (see Table 1). Since the data ends at end of 2015, The loans issued after 2013 are not used due to the incomplete observation period which is less than 24 months.

Based on the last month payment (labeled by “last_pymnt_d” in the dataset), issued month (“issue_d”) and the default indicator444The default indicator is labeled by “default_ind” and is taken value 0 or 1, whereas the definition of default is not specified in the description of the dataset., we can calculate for each loan the survival time (or censoring time) and the censoring indicator. In this paper, we assume that if a loan defaults in some repayment month, it remains in the default state in the rest months, regardless of later repayments. The default rate in certain repayment month $t$ is defined to be the number of loans that default in this month or were so in previous months over the total number of accounts, i.e.,

[TABLE]

Note that the default rate defined here is accumulative over time, in accordance with default probability used in survival analysis. Hence, the curve of default rate grows with time, as depicted in Figure 1. We can also see in Figure 1 that the default rate curve of the training data is almost identical with that of the test data.

3.1.2 Pre-processing of data

In the lending club loan dataset, there are 55 variables for each loan, including its ID and default label. Among these variables, the ID-like variables and the variables with a missing rate higher than 80% are first excluded. Categorical variables, e.g., the loan purpose and employment length, are represented by numerical values with one-hot encoding. For each variable, the missing value is filled up with a default value and an additional binary variable is generated to indicate whether the value is missing. Eventually 42 variables among 55 are chosen as features for model inputs. The selected features mainly include:

• •

loan information, e.g., the loan amount, interest rate, loan grade;

• •

personal information, e.g., home ownership, the number of finance trades;

• •

historical credit information, e.g., the total number of credit lines in the borrower’s credit file, etc.

3.1.3 Convergence of GBST

In this subsection, we demonstrate that GBST algorithm converges after dozens of iterations when we apply the algorithm to the lending club loan data. Note that at each iteration of the GBST algorithm (see Algorithm 2), a survival tree is constructed by minimizing the loss function stated in Eq. (12) on the training dataset. This turns out to be very time-costly when the dataset is large. Hence, we apply a sampling approach as proposed in stochastic gradient boosting (Friedman,, 2002) to reduce the computational cost. Instead of using the entire training dataset, we randomly sample a subset of the training dataset and apply them to train the model, with certain sampling rate $\alpha$. Note that the loss defined in Eq. (12) measures how well the proposed GBST model fits the observed samples. To test the convergence behavior of the GBST algorithm with random sampling, we run the algorithm 1000 times independently. In this experiment, we use the sampling rate $\alpha=20\%$ and the regularization parameter $\lambda=0.001$, and the max tree depth is set to $6$.

Figure 2 plots the distribution of loss values over iterations of these 1000 independent runs. For each iteration, we calculate the minimal, maximal, 25% quantile, median and 75% quantile of the 1000 independent loss values and plot in Figure 2 with a box plot. Figure 2 shows that the loss values decrease very quickly over iterations and converges uniformly to a certain level near 0, as the iteration grows. Hence, the GBST algorithm with random sampling shows a promising property of convergence. It shows also that 30 iterations are sufficient to obtain the optimal solution.

3.1.4 Performance of GBST

We apply the GBST model to calculate the predicted survival probability of each loan in the test dataset for every repayment month. The loans are sorted in ascending order according to the predicted survival probabilities and are divided into 10 survival groups with equal sample size. Hence, the larger the group index is, the higher the predicted survival probabilities within the group are. For each survival group, we calculate the true default rate defined in Eq. (24) and the results are plotted in Figure 3. It shows that for each repayment month (labeled by different shapes and colors in Figure 3) the true default rate decreases monotonically as the predicted survival probability increases, which indicates that the GBST model has a good discriminating power in the test dataset over all months.

To examine the model performance more rigorously, we adopt the concordance index (C-index) proposed by Harrell et al., (1982) to measure the goodness of fit for survival models. C-index provides a global assessment of the model for continuous events. It takes values ranging from 0.5 (no discrimination) to 1.0 (perfect discrimination). The value of C-index is 0.6740 for GBST, showing that the GBST model has a certain ability to distinguish risky loans in the lending club loan dataset.

3.1.5 Comparison

Other existing models (COX, XGB, RSF, CoxBoost, GBMCI, CIF and DeepHit) are also employed to analyze the lending club loan dataset, and we compare the results of GBST with the results of the other models under several performance metrics. For all models, we apply the same procedure of data pre-processing as described in Section 3.1.2. It should be noted that XGB is a classification model and it cannot predict survival probability curves. Nevertheless, we use the following label as the learning target of XGB model: a loan is labeled with “default” if it defaults in any repayment month within the observation period of 24 months, and with “non-default” otherwise. After training, XGB model is able to predict non-default probability for each loan, which can then be used to calculate the performance metrics to be introduced below.

In order to make the comparison statistically solid, we conduct 10 leave-one-out cross-validation (LOOCV) experiments: all data is divided into 10 almost equal-size datasets according to the issued months (see Table 2), and for each of the 10 experiments, 9 of them are combined as the training set while the remaining one as the test set. In each experiment, the hyperparameters of all candidate models are optimized in the training dataset by a random grid search with 5-fold cross validation, which is used to avoid over-fitting. For instance, the hyperparameters of the GBST model to be optimized are the maximum tree depth, learning rate, number of trees and sub-sample ratio. The optimization objective for XGB is AUC (the area under the receiver operating characteristics curve), while for the other survival models including GBST the objective is C-index. In other words, we ensure that each candidate model is optimized separately and a fair comparison is guaranteed.

First, we calculate the C-indices for all models and Table 3 summarizes the C-indices in the 10 LOOCV experiments (in terms of mean values and 95% confidence intervals), which shows that all candidate models show some extent of discriminating power with C-index ranging from 0.6666 to 0.6867, while GBST achieves the highest C-index among all models. In order to test the significance of the differences, we conduct a T-test with the 10 C-indices on GBST and other methods. Table 3 shows that GBST outperforms other models significantly with all p-values $<0.05$.

Second, the AUC and Kolmogorov-Smirnov (KS) index are two widely used performance metrics for binary classification methods in credit scoring (Thomas et al.,, 2002). Note that most of the survival models are able to predict the survival probability of each loan for each repayment month, and therefore, we can calculate AUC and KS indices for each repayment month in the 10 LOOCV experiments (in terms of mean values and 95% confidence intervals). For each repayment month, we only take those “survived” loans into account in the calculation of AUC and KS index, and hence, the samples size decreases as time increases. It is also noteworthy that since GBMCI can only predict estimated survival time rather than survival probability, KS index and AUC are not computable by GBMCI.

Table 4 presents the results of AUC and KS index for every 2 repayment months by all candidate models except GBMCI. For each model, the first observation is that except for a few cases, the AUC (respectively, KS index) decreases, and hence, the discriminating power deceases, as time (month) increases. It can be explained by the fact that the prediction of future events is more difficult than that of recent events. The second observation from Table 4 is that except for the 3rd month, GBST achieves the highest score (in terms of mean value) among all models for all months in terms of both performance metrics, and the difference between GBST and the other models are statistically significant in terms of p-value ($<0.05$) with a few exceptions. Therefore, we conclude that GBST performs significantly better than all other models over all months (with one exception) in the lending club loan dataset.

Finally, in order to understand the performance variation in the models of interest, we conduct an analysis similar to the one presented in Section 3.1.4 by comparing the predicted survival probabilities with the true default rate. For each model, the loans are sorted in ascending order by the predicted survival probability of the 24th repayment month and are accordingly divided into 10 survival groups with equal sample size. It is noteworthy that since for each loan the survival probabilities predicted by different models may differ from each other, a loan can be categorized into different survival groups under different models. For each survival group predicted by each model, we calculate the default rate of the 24th month within the group. The results are plotted in Figure 4. We find that first, all models shows to some extent a discriminating power, since the lower survival group index is (i.e., the lower the average predicted probability within the group is), the higher the default rate is; and second, GBST shows a better discriminating power than the other models, as for the bottom 4 survival groups (1 – 4) predicted by GBST have higher default rates than those groups by other models, while the top 4 survival groups (7 – 10) predicted by GBST have lower default rates than those groups by other models.

To conclude this subsection, the proposed GBST model outperforms the other candidate models (COX, RSF, XGB, CoxBoost, GBMCI, CIF and DeepHit) on the lending club loan dataset, measured by three typical performance metrics (C-index, AUC, KS index), as well as by the group analysis conducted above.

3.2 360 Finance Dataset

3.2.1 Dataset

We use the data provided by 360 Finance Inc., who provides credit products mainly in form of installment loans. The loans are expected to be repaid monthly over a period of 3, 6 or 12 months. In order to make a full observation of repayment behavior, we consider only the accounts who borrowed money in January 2018 and March 2018. The loans issued in February 2018 are excluded to avoid the festival effect for the Chinese New Year Festival was in that month. All the loans were scheduled to be repaid in 12-month installments. Among them, 200,000 loans borrowed in January are randomly selected as the training set, and 120,000 in March are sampled as the test set (see Table 5). In total, 320,000 samples are drawn.

A loan is defined as a default loan in this dataset if on any scheduled repayment due date the borrower is once overdue for at least 10 days, no matter whether he pays off the installment later or not. Hence, the discretized observation time (see Section 2.1) applied in survival analysis is the monthly repayment due date with extra 10 days. Therefore, the default rate in certain repayment month is the number of accounts who is overdue for at least 10 days in this month, as depicted in Figure 5. In compliance with the data protection policy of 360 Finance Inc., the exact values of default rates are not plotted. Instead, we plot the values of default rates compared to a baseline, which is the default rate of the sample accounts from the training dataset cumulated to 12th month. It should also be noted that our definition of default is stricter than the conventional definition and therefore, the default rate in our definition is higher as well. We also see in Figure 5 that the default rate of the test data in March 2018 grows faster than that of the training data in January 2018, though two curves do not diverge significantly.

3.2.2 Pre-processing of data and feature selection

For each loan, over 400 raw variables are collected at application, including

• •

personal information provided by the borrower, e.g., sex, age, education level;

• •

credit report retrieved from People’s Bank of China (PBC), e.g., income score, credit score;

• •

device information collected by the 360 Finance App which is authorized by the borrower, e.g., device brand, memory size of the mobile phone;

• •

credit report from third-party credit rate agencies, e.g., number of loans applied in other platforms, travel intensity, etc.

We conduct a pre-processing procedure similar to the one described in Section 3.1.2. The variables with a missing rate higher than 80% are excluded and the categorical variables are transformed into numerical values with one-hot encoding. We fill up the missing values in each attribute with a default value and add a missing-value indicator. After pre-processing, ca. 1000 features are generated.

Although GBST and XGB can deal with high dimensional inputs as in this case more than 1000, all of the other models to be compared with can only work with inputs of a rather smaller dimensionality, which is usually less than 100. Hence, for the sake of fair comparison, we apply XGB to do a feature selection before training by survival models. More specifically, we train an XGB model with the training dataset to solve a classification problem, where the samples are labeled as “bad” if the account defaults within the observation period of 12 months, and are labeled as “good” otherwise. The hyperparameters of XGB model, such as max tree depth, are chosen based on a 5-fold cross validation. We finally select the 50 features with the highest importance (for details see Section 10.13, Friedman et al.,, 2001). Table 6 shows some of the most important features and their data sources.

3.2.3 Performance and comparison

In this subsection we follow the same steps of experiments done for the lending club loan dataset described in Subsection 3.1.

With 50 features selected XGB as inputs, we train GBST model on the training set of Jan. 2018 and apply the obtained model to the test set of Mar. 2018. For each repayment month, the loans are sorted in ascending order according to the probabilities predicted by GBST and are then divided into 10 groups with equal sample size. Therefore, the 1st group contains the loans with the lowest predicted survival probability, i.e., the highest predicted default rate, and vice versa. Figure 6 exhibits the plots of the true default rate of each group across the whole period of loan. The primary findings are as follows:

• •

The predicted survival probability is negatively related to the true default rate in all the repayment months, i.e., groups with higher (lower) predicted survival probability generally have lower (higher) real default rates.

• •

The slope of the real default rate line becomes much steeper in the interval between the 4th group and the 5th group, which implies that the survival probability in this interval has stronger discriminating power of default.

We then compare the performance with other models. We conduct a 10 leave-one-out cross-validation (LOOCV) experiments with 10 almost equal-size datasets (see Table 7). Similar to the training procedure described in Subsection 3.1, in each experiment, the hyperparameters of each model are optimized by a grid search with 5-fold cross validation on the training set.

We calculate the C-index to measure the performance of all candidate models in the 360 Finance dataset and the results are presented in Table 8. It shows that GBST achieves the highest score (0.7997) among all models and the difference between its score and other scores is significantly large in terms of p-value ($<0.05$). Hence, we can conclude that GBST outperforms all the other models significantly and it has a strong discriminating power for risk assessment.

Furthermore, comparing the results shown in Table 8 with those in Table 3, we find that the scores of all models on this dataset are much higher than the scores on the lending club loan dataset (cf. Table 3). It suggests that the learning task on the 360 Finance data is easier than the task on the lending club loan data, probably because the 360 Finance dataset contains not only more variables than the lending club loan dataset (400 vs. 50), but the variables in the 360 Finance dataset are also more relevant to the risk assessment than the variables in the latter dataset.

We also calculate AUC and KS indices for each candidate model and each repayment month. The results are summarized in Table 9. It shows that both AUC and KS indices of GBST are higher than the corresponding scores of the other models and the differences are statistically significant (p-value $<0.05$) with very few exceptions. Hence, GBST outperforms the other models significantly. Furthermore, both scores of GBST decays much more slowly with time than the scores of the other models, indicating that the GBST model tends to be more time-consistent on this dataset.

Finally, we compare the predicted survival probabilities with the true default rate. For each model, the loans are sorted by the predicted survival probability of the 12th repayment month and are in ascending order divided into 10 survival groups with equal sample size. For each survival group predicted by each model, we calculate the true default rate of the 12th month within the group compared to the baseline $b$ that is the default rate of the loans from the training dataset cumulated to 12th month. The results are depicted in Figure 4. The baseline $b$ can also be approximately viewed as the average default rate of all 10 groups. It is interesting to find that when the survival groups are controlled, the GBST model has higher real default rates in the lower-ranked groups (group 1 – 4, which are higher than the baseline) compared to the other models, while it has relatively lower real default rates in the higher-ranked groups, despite several exceptions from the RSF model. This finding proves that the GBST model is more capable of distinguishing the default loans from the others, which is in accordance with the previous results observed with the goodness-of-fit metrics.

4 Conclusion and future work

This paper presents a novel nonparametric ensemble tree model called gradient boosting survival tree (GBST) that extends the survival tree models with a gradient boosting algorithm. The survival tree ensemble is learned by optimizing the likelihood in an additive manner. As a test, the GBST model is applied to quantify credit risk on two large-scale real market datasets. The results on the both datasets show that the performance of GBST is better than that of previous models, measured by C-index, AUC and KS index of each time period.

Like traditional scoring methods, the proposed gradient boosting survival tree (GBST) model is based on the attributes of the accounts (called features in this paper) measured at the time of application. Yet, many of them change over time, especially when the modeling period is not short. To address the problem of changing attributes, Djeundje and Crook, (2019) propose a dynamic parametric survival model with B-splines. In the future, we plan to incorporate time-varying attributes into the GBST model.

In the usual setting of survival analysis, as well as in the setting of GBST, one considers a binary state space, i.e., either the event of interest happens (state = 1) or not (state = -1). However, in many applications (see e.g., Lee et al.,, 2018), as in the problem of credit scoring, additional events may be of importance as well. For instance, an account with early repayment is viewed as a non-default account. However, with respect to profits, an early-repayment account tends to differ from an account with on-time repayments. This problem can be dealt with by extending the binary state space to the multi-state space, which is left in our future work.

Appendix A Maximum weighted quantile spilt algorithm

The split algorithm for GBST stated in Algorithm 1 is a greedy algorithm exploring all possible split points, which can be extremely time-consuming when the value set of features is very large. In order to speed up, we propose here a maximum weighted quantile method by extending the method applied by Chen and Guestrin, (2016). Note that the objective in Eq. (13) can be rewrite as

[TABLE]

Hence, the objective can be viewed as a sum of the weighted squared loss with labels $-\frac{r_{i,j}}{\sigma_{i,j}}$ and weights $\sigma_{i,j}$.

At time $\tau_{j}$, we consider a multi-set $N^{k}_{j}:=\{(x_{1,k},\sigma_{1,j}),(x_{2,k},\sigma_{2,j}),\ldots,(x_{n_{j},k},\sigma_{n_{j},j})\}$ satisfying that $x_{1,k}\leq x_{2,k}\leq\ldots\leq x_{n_{j},k}$, where $n_{j}$ denotes the total size of samples that survive longer than $\tau_{j-1}$. In other words, the survived samples are sorted by the value of its $k$th feature. Then we define a rank function $g^{j}_{k}:\mathbb{R}\rightarrow[0,\infty)$ at time $\tau_{j}$ such that

[TABLE]

With the rank function, the candidate split points $\{s_{k,1},s_{k,2},...,s_{k,q}\}$ are given by

[TABLE]

where $\varepsilon$ is the approximation factor and one can prove that the maximum size of split points is $J\cdot\lceil\frac{1}{\varepsilon}\rceil$ (Chen and Guestrin,, 2016). The algorithm for generating split points is summarized in Algorithm 3. Accordingly, the revised split algorithm with maximum weighted quantile is demonstrated in Algorithm 4.

Appendix B Parallel execution

Another trick to speed up the GBST algorithm is to apply parallel computing. For boosting methods, trees must be generated sequentially since the successive tree depends on the training and prediction of the previous tree. Nevertheless, in the process of training a single tree, the problem of finding the best split points (see Algorithm 3) can be solved in parallel. Before training each tree, we can pre-sort the features and reuse the sorting result in subsequent iterations, which can significantly reduce the amount of calculation and improve algorithm performance.

Appendix C Links of open-source packages