CLoTH: a Simulator for HTLC Payment Networks

TL;DR

CLoTH is a simulator designed for HTLC payment networks like the Lightning Network, enabling performance prediction and optimization assessment before deployment.

Contribution

Introduces CLoTH, a novel simulator for HTLC payment networks, facilitating performance analysis and optimization testing.

Findings

Predicts payment completion times and failure probabilities.

Helps identify potential issues in HTLC network development.

Estimates effects of network optimizations.

Abstract

The Lightning Network (LN) is one of the most promising off-chain scaling solutions for Bitcoin, as it enables off-chain payments which are not subject to the well-known blockchain scalability limit. In this work, we introduce CLoTH, a simulator for HTLC payment networks, of which LN is the best working example. It simulates input-defined payments on an input-defined HTLC network and produces performance measures in terms of payment-related statistics, such as time to complete payments and probability of payment failure. CLoTH helps to predict issues that might arise in the development of an HTLC payment network, and to estimate the effects of an optimisation before deploying it. In upcoming works we'll publish the results of CLoTH simulations.

Figures

Tables

Table 1. Table 1 : Mapping between CL o TH simulator functions and lnd functions.

Simulator Function	Description	Simulated Functions	Peer	Message
find_route	Search for a payment route	SendPayment, RequestRoute, findPath, newRoute	Sender	-
send_payment	Send a payment	handleLocalDispatch, handleDownStreamPkt, SendMessage	Sender	HTLCAdd
forward_payment	Forward a payment	handleUpstreamMsg, processRemoteAdds, handlePacketForward, handleDownStreamPkt, SendMessage	Hop	HTLCAdd
receive_payment	Receive a payment	handleUpstreamMsg, processRemoteAdds, SendMessage	Receiver	HTLCAdd
forward_success	Forward the successful result of a payment	handleUpstreamMsg, processRemoteSettleFails, handlePacketForward, handleDownStreamPkt, SendMessage	Hop	HTLCFulfill
forward_fail	Forward the fail result of a payment	handleUpstreamMsg, processRemoteSettleFails, handlePacketForward, handleDownStreamPkt, SendMessage	Hop	HTLCFail
receive_success	Receive the successful result of a payment	handleUpstreamMsg, processRemoteSettleFails, handleLocalResponse	Sender	HTLCFulfill
receive_fail	Receive the fail result of a payment	handleUpstreamMsg, processRemoteSettleFails, handleLocalResponse	Sender	HTLCFail

Table 2. Table 2 : CL o TH simulator pre-processing phase input parameters.

Type	Symbol	Definition
Network	$N_p$	Number of peers
	$N_{ch}$	Average number of channels per peer
	${\sigma }_t$	Tuner of network topology
	$P_{{\overline{c}}_b}$	Uncooperative peers probability before HTLC establishment
	$P_{{\overline{c}}_a}$	Uncooperative peers probability after HTLC establishment
	$C_{ch}$	Average channel capacity
	$G$	Gini index of channel capacity
Payments	$r_{\pi }$	Transactions per second (off-chain)
	$N_{\pi }$	Number of payments
	${\sigma }_a$	Tuner of payment amounts
	$F_{sr}$	Fraction of same-recipient payments

Table 3. Table 3 : CL o TH simulator performance measures.

Symbol	Definition
$P_s$	Probability of payment success
$P_{f_r}$	Probability of payment failure for no route
$P_{f_b}$	Probability of payment failure for unbalancing
$P_{f_{\overline{c}}}$	Probability of payment failure for uncooperative peers
$P_{\overline{t}}$	Probability of payment failure for timeout expiration
$P_{\overline{k}}$	Probability of unknown payments
$T$	Payment complete time
$N_a$	Number of payment attempts
$L_r$	Payment route length

Table 4. Table 4 : CL o TH simulator execution time.

(a)
Peers	Execution Time (h)
100,000	6.64
200,000	15.92
500,000	49.25
700,000	64.68
1,000,000	104.15
(b)
Dijkstra calls	Execution Time (h)
58927	4.98
303374	28.3

Equations

$$r_k=\varphi (p_k,{\mathcal{G}}_k,b_k)$$

$${\mathcal{G}}_{k+1}=\mu (p_k,r_k)$$

Full text

††titlenote: A previous version Conoscenti et al. (2018) of this work has been published on September 3 2018.

\Title

CLoTH: a Simulator for HTLC Payment Networks

\AuthorMarco Conoscenti 1, Antonio Vetrò 1, Juan Carlos De Martin 1,

Federico Spini2, Fabio Castaldo2 , Sebastiano Scròfina2 \AuthorNamesMarco Conoscenti, Antonio Vetrò, Juan Carlos De Martin, Federico Spini, Fabio Castaldo, Sebastiano Scròfina

\lastrevision

1 Introduction

Bitcoin is a decentralised cryptocurrency that allows mistrusting peers to send/receive monetary value without the need for intermediaries Nakamoto (2008). Bitcoin relies on the blockchain, a distributed peer-to-peer public ledger where each peer stores all the history of Bitcoin economic transactions. Its distributed and replicated nature along with a limited block dimension entails capped transaction throughput Sompolinsky and Zohar (2013). As proven in the comprehensive study by Gervais et al. Gervais et al. (2016), a commonly accepted blockchain design that resolves the scalability issue is not yet known.

A way to address blockchain scalability is off-chain scaling Poon and Dryja (2016); Decker and Wattenhofer (2015); Miller et al. (2017); rai ; Burchert et al. (2017); Khalil and Gervais (2017); Prihodko et al. (2016), which enables off-chain payments, i.e., payments that do not need to be registered on the blockchain. Payment channels are key components for off-chain scaling approaches. A payment channel is a two-party ledger which is updated off-chain and uses the underlying blockchain only as a settlement or dispute resolution layer. It allows an unbounded number of off-chain payments to be sent/received between two involved parties (the channel endpoints), as long as they can jointly reach consensus.

Two-party payment channels can be linked together to build a payment network for off-chain payments. This allows parties not directly connected by a payment channel to send/receive off-chain payments which are routed across a network of linked payment channels.

The Lightning Network (LN) Poon and Dryja (2016) is the most developed off-chain solution for Bitcoin. The LN protocol leverages a contract called Hashed Timelock Contract (HTLC) to build a payment network. The HTLC scheme allows transferring off-chain payments through multiple payment channels linked together. It guarantees transfer atomicity: even if a payment is carried across multiple intermediate channels to reach the recipient, the payment either succeeds or fails in all the involved channels. Henceforth, we define an HTLC payment network as a network of linked payment channels where payments are transferred using the HTLC scheme.

At its current state of development, the LN protocol is characterised by features that, if not properly understood, implemented and controlled, might undermine the development of a healthy HTLC payment network, i.e., a network that supports fast and successful payments. The features that need to be studied are the following: (i) routing, i.e., a good routing path that does not cause a payment failure depends on the knowledge of an up-to-date network topology; (ii) channel capacity, which constrains the payment amount; (iii) channel unbalancing, namely, the condition of a skewed channel that has one endpoint depleted due to a number of unidirectional payments (an issue not internally addressed by the LN protocol); and (iv) uncooperative behaviour of peers involved in a payment route, which increases payment time.

LN developers stress the importance of protocol improvements and optimisation actions to guarantee correct functioning of the payment network: in particular, they outlined the need for special routing nodes, which are always online and contribute enough capital to route payments Vu . However, LN is an emergent network and the resulting lack of central coordination does not allow to easily steer network growth.

In this work we present CLoTH, an HTLC payment network simulator. The CLoTH simulator could be used to: predict issues and obstacles that might emerge in development stages of an HTLC payment network; assist in planning uncoordinated development of the network; estimate the effects of an optimisation before deploying it; predict the return on investment of adding a hub to the network.

Examples of questions that can be answered by CLoTH are: (i) “How many channels per peer are required to have a well connected network?”; (ii) “How do peers going offline affect performance?”; (iii) “How does payment amount influence the chance of payment failure?”; (iv) “How does mean payment time decrease by adding a peer with a specific set of payment channels in a specific section of the network?”.

The CLoTH simulator is a discrete-event simulator that simulates payments on HTLC payment networks. It takes as input parameters defining an HTLC network (e.g., number of peers and number of channels per peer) and parameters of payments to be simulated on the defined HTLC network (e.g., payment rate and payment amounts). It generates performance measures in the form of payment statistics, e.g., the probability of payment failure and the mean payment complete time. To the best of our knowledge, CLoTH is the first available simulator for HTLC payment networks.

The remainder of the paper is organised as follows. Section 2 is a background on Bitcoin, off-chain scaling solutions and the Lightning Network. Section 3 describes the LN code used as reference to develop the simulator. Section 4 presents the CLoTH simulator design. Section 5 reports about the usability of the simulator. Finally, in Section 6 we draw the conclusions of this work and outline possible future work.

2 Background

In this Section, we briefly outline the background of our research: Bitcoin and the blockchain; off-chain scaling solutions; and the Lightning Network.

2.1 Bitcoin

Bitcoin is a decentralised payment system running on a peer-to-peer network. Transactions are used to transfer bitcoin cryptocurrency and they are defined by an input state and an output state. The output is represented by the amount of currency to be transferred and by a Bitcoin script defining the spending conditions to claim that amount. The input is represented by the reference to an output of a previous transaction and the proof fulfilling the spending conditions of the referenced output. The blockchain is the distributed ledger technology storing all Bitcoin transactions. Each peer (that runs a full node) stores a whole copy of the ledger. Miners are special peers that can write on the ledger: (i) they validate transactions broadcast on the peer-to-peer network against the ledger in order to prevent double spending of coins; and (ii) they gather transactions and write on the ledger, by broadcasting a block (i.e., a collection of transactions), statistically once every 10 min. For more details on Bitcoin and the blockchain, we refer the reader to the detailed background study in Bonneau et al. (2015).

Bitcoin’s last year average throughput is slightly below 2.7 confirmed transactions per second111Data fetched from https://www.blockchain.com/en/charts/n-transactions on November 24, 2018.. In its current form, Bitcoin will hardly scale beyond 100 transactions per second because of storage, processing, latency, and bandwidth constraints Decker and Wattenhofer (2015). For Bitcoin to become a mainstream medium of exchange, transaction throughput should grow significantly.

2.2 Off-Chain Scaling Approaches

Off-chain scaling addresses scalability by moving as many transactions as possible off the blockchain.

2.2.1 Payment Channels

Payment channels pay are the building block of an off-chain payment network. Such channels are implemented as two-party ledgers that allow two parties (the channel endpoints) to update their balances off-chain, without the limit imposed to blockchain transactions. The blockchain is only used as a dispute resolution layer if and when consensus on the off-chain state cannot be reached between channel endpoints.

In the rest of this section we’ll use the following example of a payment channel.

Payment channel example. Alice and Berto open a payment channel. Alice funds the channel with 0.5 BTC, Berto funds the channel with 0.5 BTC, so Alice’s initial balance in the channel is 0.5 BTC and Berto’s is 0.5 BTC. The capacity of the channel is the total amount of bitcoins in the channels: 1 BTC. After opening the channel, Alice can transfer 0.1 BTC off-chain to Berto: to do so, they update their balances in the channel accordingly, so Alice’s balance is now 0.4 BTC and Berto’s balance is 0.6 BTC. Channels are bidirectional, so Berto can pay Alice as well. No transactions on the blockchain are required to accomplish these payments between Alice and Berto: they just update their bidirectional off-chain balances.

2.2.2 Payment Network

A payment network is a connected network of payment channels linked together. Compared to blockchain transactions, off-chain payments performed in payment networks are: cheaper, as lower fees are required to route payments in such networks; faster, as they do not need to be registered on the blockchain; and more privacy preserving, as they are not visible in the public blockchain.

In the rest of the section we’ll use the following example of a payment network.

Payment network example. Alice sends 0.1 BTC off-chain to Davide, without having a direct payment channel with him. She uses a route of already open channels linking her to Davide: the channel she has with Berto; the channel between Berto and Carola; and the channel between Carola and Davide.

2.3 The Lightning Network

The Lightning Network (LN) is the implementation of a network of payment channels for Bitcoin. It was introduced in 2015 by Joseph Poon and Thaddeus Dryja Poon and Dryja (2016).

2.3.1 LN Payment Channel Lifecycle

A payment channel lifecycle, as defined by the LN protocol, is presented in the following.

Channel funding. Alice and Berto create a funding transaction, the transaction required to fund the channel. The funding transaction is broadcast to the blockchain and, when confirmed, the channel is opened. Alice and Berto also create a commitment transaction representing their current balances in the channel: such transaction spends the bitcoins in the funding transaction and transfers 0.5 BTC to Alice and 0.5 BTC to Berto. The commitment transaction is not broadcast to the blockchain.

Payment execution. When Alice wants to pay 0.1 BTC to Berto, the two parties create a new commitment transaction which reflects the new balances: 0.4 BTC of Alice and 0.6 BTC of Berto. No transaction is sent to the blockchain, i.e., the operation is executed off-chain.

Channel closing. When the two parties want to close the channel, they send the latest commitment transaction to the blockchain.

Punishments to disincentivise misbehavior. Berto can punish Alice if Alice misbehaves (or vice versa), thanks to the Revocable Sequence Maturity Contract (RSMC), a contract implemented by a script in the spending condition of the commitment transaction. Alice could be tempted to broadcast an older commitment to the blockchain, for instance the commitment preceding her payment to Berto, when she owned more funds (0.5 instead of 0.4 BTC). If Alice tries to do so and Berto finds the older commitment transaction on the blockchain, Berto can withdraw all funds from the channel (including the 0.4 BTC of Alice’s balance), by using the RSMC output contract of the old commitment transaction broadcast by Alice. In practice, when an older commitment transaction is replaced by a newer one, Berto is provided with a special transaction, pre-signed by Alice, that he can unilaterally broadcast. This special transaction “invalidates” the older commitment transaction as it consumes all the older-commitment transaction outputs and assigns all channel funds to Berto. Thanks to this construction, Alice may lose all her funds if caught cheating, and she is therefore not incentivized to misbehave.

Channel unbalancing. A channel is said unbalanced or skewed when one balance is much higher than the other. If payments always flow from Alice to Berto and no payments flow from Berto to Alice, Alice’s balance may go to zero. Channel unbalancing is an issue, as a payment cannot traverse a channel in a given direction if the balance in that direction is lower than the payment amount.

2.3.2 Hashed Timelock Contract (HTLC)

HTLC is a contract for off-chain multi-hop payment through channels that connect the payment sender and the payment recipient. An off-chain multi-hop payment is the atomic composition of several off-chain payments, each one performed on an intermediate payment channel. An intermediate peer receives an inbound payment on one of its channels and forwards the same amount to the next payment hop (actually a little less since it retains an intermediation fee), until the payment reaches the recipient. The HTLC scheme creates a network of payment channels where off-chain payments can be routed.

In practice, the HTLC contract is implemented by a script in the spending condition of the commitment transaction of a payment channel and it executes conditional payments. Alice pays 0.1 BTC to Berto if Berto can demonstrate to know a value $R$ within a certain timeout (timelock, in the Bitcoin jargon): in this case, the HTLC is fulfilled; otherwise, 0.1 BTC returns back to Alice and the HTLC fails. Funds in the HTLC payment stay locked up to the disclosure of $R$ or, if $R$ is never disclosed, up to the timelock’s expiration. In the following, we describe the phases of a payment from Alice to Davide using HTLCs.

HTLC establishment. For the multi-hop payment originated by Alice and trying to reach Davide, the following HTLCs are established: (i) the HTLC between Alice and Berto, where Alice pays 0.1 BTC to Berto if Berto demonstrates to know $R$ within a timelock, e.g., three days; (ii) the HTLC between Berto and Carola, where Berto pays 0.1 BTC to Carola if Carola demonstrates to know $R$ within a timelock, e.g., two days; and (3) the HTLC between Carola and Davide, where Carola pays 0.1 BTC to Davide if Davide demonstrates to know $R$ within a timelock, e.g., one day.

HTLC fulfillment. Alice sends $R$ to Davide, and HTLCs from Davide to Alice are fulfilled: Davide shows $R$ to Carola within one day, and Carola pays him 0.1 BTC; Carola shows $R$ to Berto within two days, and Berto pays her 0.1 BTC; Berto shows $R$ to Alice within three days, and Alice pays him 0.1 BTC. At the end, 0.1 BTC has been transferred from Alice to Davide. For their work of forwarding the payment, Berto and Carola can withhold a small fee when transferring the payment (which we omitted in this example for the sake of simplicity). It is important to highlight that all the described operations are performed off-chain, without the need to interact with the blockchain. Finally, it is worth noticing that, from Davide to Alice, HTLCs have increasing timelocks. This ensures that each party has enough time to know $R$ and claim her funds: Davide shows $R$ to Carola in one day, Carola pays 0.1 BTC to Davide; after that, Carola still has one day to claim 0.1 BTC from Berto, as the timelock with Berto is set to two days.

HTLC failure. If $R$ is not disclosed by an uncooperative peer, the payment fails in all channels. A failure message is sent back to Alice and no payment occurs between the channels, as HTLC conditions have not been fulfilled. In such case funds in the HTLC are locked up to the timelock expiration. If Davide does not disclose $R$, Carola has to wait one day to get back 0.1 BTC according to the HTLC conditions. After that, Carola closes the channel with Davide by broadcasting the last commitment transaction to the blockchain and propagates back the failure to Alice.

2.4 Brief Literature

Although the most developed, the Lightning Network is not the only off-chain payment network in literature. In 2015 Christian Decker proposed Duplex Micropayment Channels for building an off-chain payment network based on HTLCs Decker and Wattenhofer (2015). Compared to this approach, the key innovation in LN is in punishing a misbehaving party. Raiden rai uses smart contracts to implement the same fundamental concepts of LN on the Ethereum blockchain. Sprites Miller et al. (2017) is an attempt to improve both LN and Raiden, as it aims at minimising the time during which funds are locked while being transferred via HTLCs.

2.5 Related Work

There are currently two publicly available projects Reynolds (2017); Di Stasi et al. labelled as Lightning Network simulators. However, they do not simulate the HTLC protocol: unlike CLoTH, they do not present a complete mapping of Lightning Network code (cfr. Section 4.3.1). They are simulators of generic networks where payments are routed: their purpose is to study specific aspects of the protocol, and they make no claim of generality nor completeness. To the best of our knowledge, CLoTH is the first publicly available HTLC payment network simulator.

3 Analysis

For implementation level details of HTLC mechanisms, we took as reference lnd, the Golang implementation of the Lightning Network. lnd, as the other LN implementations (e.g., c-lightning c-l , and eclair ecl ), fully conforms to the so-called Basis of Lightning Technology (BOLT) bol , the Lightning Network specifications.

In Figure 1, we show the multi-hop payment call graph resulting from our analysis of the release lnd-v0.5-beta. The call graph represents the main functions called when a payment flows from a sender to a receiver through an intermediate hop. These functions implement the HTLC mechanism that ensures the atomicity of the payment (cfr. Section 2.3.2).

We describe three peers, as they call different functions with different behaviors: the payment sender $\mathcal{A}$, the payment receiver $\mathcal{D}$ and a generic intermediate hop $\mathcal{H}$. This case can be easily generalised to n-intermediaries since the behaviour of each intermediate hop is always the same.

HTLC messages (i.e., HTLCAdd, HTLCFulfill and HTLCFail) that flow through peers are represented by arrows:

• •

HTLCAdd: sent from $\mathcal{A}$ to $\mathcal{D}$ through $\mathcal{H}$, for the establishment of HTLCs among the peers;

• •

HTLCFulfill: sent from $\mathcal{D}$ to $\mathcal{A}$ through $\mathcal{H}$, for the fulfillment of the HTLCs; and

• •

HTLCFail: sent in case of failures, for failing the HTLCs.

The function handleUpStreamMsg in the gray box represents the first function called by a peer when an HTLC message is received. The gray line represents the commitSig procedure that establishes an HTLC: the creation of a new commitment transaction containing the HTLC between the two involved parties.

In Appendix A, we describe details of the lnd code and of the functions of the call graph, while in the following we provide an high-level description of the flow of a payment.

Payment initialisation and sending. Source routing is run by $\mathcal{A}$ to search for an available route to $\mathcal{D}$. If no route is found, the payment is considered failed; otherwise, $\mathcal{A}$ sends an HTLCAdd message to $\mathcal{H}$ to establish the HTLC.

Payment relay (intermediate hop). $\mathcal{H}$ checks for possible errors (e.g., it checks whether it has enough balance to forward the payment). If there are errors, it sends an HTLCFail message back to $\mathcal{A}$, failing the HTLC; otherwise, it sends an HTLCAdd message to $\mathcal{D}$. (This step occurs multiple times in the general case of a payment going through multiple intermediate hops.)

Payment reception. $\mathcal{D}$ checks for possible errors. If there are errors, it sends an HTLCFail message to $\mathcal{H}$ (which in turn propagates it to $\mathcal{A}$), failing the HTLCs; otherwise, it sends an HTLCFulfill message to $\mathcal{H}$ (which in turn propagates it to $\mathcal{A}$), for fulfilling the HTLCs.

Payment re-attempt. If an HTLCFail is received by $\mathcal{A}$, $\mathcal{A}$ tries to re-attempt the payment by searching for a new route. If the failure was due to a channel which did not have enough balance to forward the payment, the channel is blacklisted. If the failure, instead, was due to an uncooperative peer, the peer is blacklisted. A new route is searched, excluding blacklisted peers and channels. If no new route is found, the payment is considered definitively failed.

4 Design

In this Section, we provide a detailed description of the CLoTH simulator design, which is, to the best of our knowledge, the first simulator for HTLC payment networks. By HTLC payment network, we mean a network where peers are connected by payment channels and off-chain payments are routed using HTLC contracts.

The simulator takes as input a definition of HTLC network and payment script to be played during simulation. It simulates payments in the HTLC network by locally running a discrete-event mapping of the lnd code. It produces performance measures in the form of payment-related statistics (e.g., the probability of payment failures and the mean payment complete time).

4.1 Assumptions

An HTLC payment network relies on a blockchain as a securing mechanism for all its payment channels. The underlying blockchain is therefore a fundamental prerequisite of each HTLC payment network. We assume channels are loaded with an amount of native tokens from the underlying blockchain and fees for payment forwarding are due to the intermediary hops in the same denomination.

CLoTH does not consider blockchain interactions during a simulation execution, since the performance measures produced by the simulator are related to payments, which are completely performed off-chain. Simulations run on a network in which no new channels are opened. This condition is implicitly guaranteed if the whole simulation time is shorter than the time required to fund a new channel with an on-chain Bitcoin transaction. An on-chain Bitcoin transaction is usually considered final after 6 confirmations, statistically 50 minutes after the first confirmation. This implies a relationship between simulation duration and likelihood of simulation results: the shorter the simulation, the more plausible the results. Without loss of generality, we do not take into account those channels that might have been established early enough before the simulation starting time to become operational during the simulation time window. With simulations of around 15 minutes we obtain a good compromise between meaningfulness of the experiment and accuracy of results.

Another assumption is that each peer has a complete and precise knowledge of all other peers and channels in the network. In reality, as specified in BOLT, new channels and peers are announced through a gossip protocol. Therefore, real performance may be slightly worse than the one measured by CLoTH, as peers may have a slightly imprecise knowledge of the network due the nature of the gossip protocol.

4.2 Formal Model

Having a network with $N$ nodes connected by payment channels and having $M$ payments to be executed on this network, the simulator formal model can be expressed using the following formulas:

[TABLE]

[TABLE]

The output $r_{k}$ is the route found for the payment $p_{k}$. Function $\phi$ is the function which finds a route for a payment. It takes as input the following parameters:

• •

$p_{k}=(n_{i_{k}},n_{j_{k}},a_{k})$ for $k=1,2,...,M$ is the payment for which a route has to be found: $n_{i_{k}}$ and $n_{j_{k}}$ are the sender node and the receiver node of the payment, for $i,j=1,2,...,N$ and $i\neq j$; and $a_{k}$ is the payment amount.

• •

$\mathcal{G}_{k}$ is the graph of nodes $n_{i}$ for $i=1,2,...,N$ connected by payment channels. The subscript $k$ indicates the state of the graph at the moment in which $p_{k}$ is processed, with the available nodes and channels and their attributes at that moment.

• •

$b_{k}$ is the blacklist of possible nodes and channels excluded when searching for a route for $p_{k}$ (cf. Section 3).

Given payment $p_{k}$ and the route $r_{k}$, function $\mu$ executes the payment $p_{k}$ along the route $r_{k}$. $\mu$ function produces a new state of the graph, namely $\mathcal{G}_{k+1}$, because the payment execution causes some changes in the channels involved in the payment, such as the update of channel balances according to payment amount $a_{k}$. A whole simulation is therefore the transition from $\mathcal{G}_{0}$ to $\mathcal{G}_{|M|+1}$, the initial state of the network and the final one (i.e., the state reached after the execution of the last payment), respectively.

4.3 Software Architecture

CLoTH is a discrete-event simulator Jain (1990). Events represent state changes of the simulated system. The event loop is the core of the simulation engine: it extracts the next event from a queue where events are sorted according to their occurring time; it advances the simulation time to the instant of occurrence of the event being currently processed; and it calls the function that processes the event.

The simulator generates an event each time a payment changes its state according to the state diagram in Figure 2. Each type of event in the diagram is processed by a function of the same name of the simulator, as explained in the following Section.

4.3.1 Computation Flow

Table 1 shows the mapping between each function of the lnd computation flow and functions of the CLoTH simulator codebase. The completeness of the mapping ensures the validity of the simulated results.

Table 1 makes a clear distinction between specific behaviors of each lnd function which depends on two parameters: (i) the type of peer that invokes it (i.e., sender, receiver or intermediate hop); and (ii) the type of the triggering message (HTLCAdd, HTLCFail, HTLCFulfill). For example, the function send_payment of the simulator simulates the functions handleLocalDispatch, handleDownStreamPkt and SendMessage of the lnd code, when they are called by the payment sender in the case of HTLCAdd message.

Uncooperative behaviour. Functions forward_payment, receive_payment and forward_success simulate the uncooperative behaviour of a peer. Using Bernoulli probability distributions, we determine whether a peer is uncooperative and, if so, whether it is uncooperative after or before establishing the HTLC. If the peer is cooperative, the function is normally executed. If the peer is uncooperative before establishing the HTLC, as specified by the LN protocol, an HTLCFail is propagated back to the payment sender and the payment is re-attempted with a new route that does not involve the uncooperative peer. Finally, if the peer is uncooperative after establishing the HTLC, as specified by the LN protocol, an HTLCFail is propagated back to the payment sender after the timelock expiration and the channel connecting to the uncooperative peer is closed. Then, the sender can re-attempt the payment, with a new route excluding the closed channel.

4.3.2 Data Structures

Figure 3 shows main attributes of the simulator data structures. A channel connects two peer (each one represented by an ID) and has a certain capacity. The endpoint of a channel describes how a peer behaves in a specific channel and contains: the current balance of the endpoint in that channel; base and proportional fee, which constitute the fee withheld by the endpoint for forwarding a payment (proportional fee is the part of the fee which depends on the payment amount, while base fee is the constant fee applied regardless of the payment amount); the timelock set in the HTLCs established by the channel endpoint; and the minimum value of the HTLC the endpoint accepts to forward. A payment is described by a sender, a receiver and the payment amount.

5 Usability

This Section describes the usability of CLoTH; a detailed description of input/output of a simulation run is provided along with documentation of the three simulation phases; simulator performances are also discussed.

5.1 Workflow

The workflow to interact with the CLoTH simulator is shown in Figure 4. It is made of three phases: (i) pre-processing phase; (ii) simulation phase; and (iii) post-processing phase. For the sake of clarity, we first introduce the simulation phase. An explanation of the pre-processing and post-processing phases will follow.

5.1.1 Simulation Phase

Input attributes. The simulation engine requires two inputs:

A complete specification of the HTLC network (Files 3, 4 and 5 in Figure 4). These files allow for a fine grained specification of the attributes of Figure 3 of peers (peers.csv), channels (channels.csv) and endpoints (endpoints.csv). 2. 2.

The detailed list of payments to be executed with their attributes (File 6 in Figure 4, payments.csv), specifying sender, receiver, payment amount and triggering time.

Although verbose, this input specification allows for the most detailed definition of the simulation scenario. For a succinct and statistically meaningful input configuration mode, the reader can resort to the pre-processing phase of the workflow described below.

Simulation engine execution. Given the HTLC network and the payments specified as input, the simulation engine executes the payments on the HTLC network, running the actual discrete-event simulation.

Output attributes. The simulation phase outputs detailed information about each simulated payment (File 7 in Figure 4, raw-per-payment-data.csv):

• •

payment end time;

• •

payment result: success or failure;

• •

number of times payment was attempted;

• •

whether payment encountered an uncooperative peer; and

• •

route traversed by the payment, if present.

5.1.2 Pre-Processing Phase

We wanted the CLoTH simulator to allow fine-grained simulations as well as the option to concisely describe a simulation from a statistical point of view. While the input described in Section 5.1.1 directly supports the first case, the second requirement is fulfilled by the pre-processing phase. This phase takes as input a few parameters that statistically define both the network and the payments to be simulated. Then, it randomly generates an instance of HTLC network and an instance of payment script that match the input description.

Input parameters. Input parameters of the pre-processing phase with their symbols are shown in Table 2. The following parameters deserve an explanation:

• •

$\sigma_{t}$ tunes the presence of hubs in the network topology. It is the width of the Gaussian distribution defining the probability of connection among peers. For each peer, this Gaussian probability distribution is used for choosing the other endpoint of one of the peer channels. Therefore, if the width of this Gaussian is zero, all peers have one channel open with the same peer, which, consequently, will be a hub. On the other hand, if the Gaussian width is infinite any peer has the same probability to be connected to any other, thus producing a totally decentralised network.

• •

$\sigma_{a}$ tunes payment amounts. It is the width of the Gaussian distribution whose tail is used to choose the orders of magnitude of payment amounts. The greater this width is, the higher the payment amounts.

• •

$F_{sr}$ is the fraction of the total payments directed toward the same recipient. This parameter allows to model the use case of many small payments sent to the same destination peer, e.g., to a provider of video-streaming services which is paid for each short segment of video streaming.

5.1.3 Post-Processing Phase

The post-processing phase transforms raw per-payment simulation output attributes into statistically meaningful performance measures (File 8 in Figure 4, payments-statistics.json). To do so, it applies the batch means method Jain (1990), which allows: (i) producing performance results that are not influenced by the initial transient state, where the system is not stable; and (ii) computing statistical mean, variance and 95% confidence interval for each measure. The batch means method consists in dividing a simulation run into multiple batches, which are statistically independent among each other. Output measures are zeroed and re-computed at each batch. Each final output measure is the statistical mean of that measure over the batches and comes also with variance and 95% confidence interval.

Simulation results format. The performance measures produced by the simulator are shown in Table 3 with their symbols. Some clarifications are presented in the following:

• •

$P_{f_{r}}$ is the probability that a payment fails due to the absence of a route connecting sender and receiver.

• •

$P_{f_{b}}$ is the probability that a payment fails because a channel in the route was unbalanced and an alternative route is not found.

• •

$P_{f_{\bar{c}}}$ is the probability that a payment fails because a peer in the route was uncooperative and an alternative route is not found.

• •

$P_{\bar{t}}$ is the probability that a payment fails because it took more than a timeout of 60 seconds to be completed.

• •

$P_{\bar{k}}$ represents the remaining fraction of payments for which we do not know whether they failed or succeeded, as they ended after the time validity window of our simulation. For example, this category can encompass payments delayed after a long timelock, as a peer was uncooperative after establishing the HTLC for the payment.

• •

$T$ is the mean time for a successful payment to complete.

• •

$N_{a}$ is the mean number of times a payment is re-attempted.

• •

$L_{r}$ is the mean route length traversed by a successful payment.

5.2 Performance Analisys

Table 4 shows the performance of the CLoTH simulator in terms of execution time. Simulation times refer to experiments run on a machine of model DGX-1, manufactured by Nvidia, located in Rome, Italy. The machine is equipped with 80 CPUs of model Intel® Xeon® CPU E5-2698 v4 @ 2.20 GHz and 512 GB of RAM. Table 4a shows that execution time increases with the increase of the number of peers of the simulated network. This is due to the fact that time spent by the Dijkstra’s algorithm, used to find payment routes, grows with the number of peers. With 1 million peers, a simulation run requires more than four days.

Table 4b shows that the execution time also increases with the increase of the number of calls to the Dijkstra’s algorithm. An execution of the Dijkstra’s algorithm is required each time a payment is attempted. As a payment is re-attempted, calls to the Dijkstra’s algorithm grow as well al simulation execution time. Execution of the Dijkstra’s algorithm is a simulator performance bottleneck. However, there are no limits to the values of input parameters. The only effect of setting high values (e.g., an high number of peers or payments) is a correspondingly long execution time.

6 Conclusions

The present research work focused on the Lightning Network, the mainstream proposal that aims to address the well-known scalability problem of the Bitcoin blockchain. This solution implements an HTLC payment network, i.e., a network to securely route off-chain payments through HTLC contracts.

We developed CLoTH, a simulator of HTLC payment networks. The simulator takes as input: (i) parameters representing the HTLC network to be studied (e.g., peers, channels, and channel capacities); and (ii) parameters defining characteristics of the payments to be simulated on the HTLC network (e.g., payment rate and payment amounts). CLoTH simulates the defined payments script on the input network and produces performance measures in the form of payment-related statistics (e.g., the probability of payment failures and the mean time to complete a payment).

Since LN is an emergent network with no central coordination, it’s important to understand in advance what’s needed to guarantee healthy network growth, that is quick and successful payments. We consider CLoTH a useful predicting tool to simulate possible troublesome configurations and to check in advance the effects of a routing optimisation action (e.g., a node that joins the network, and then establishes and funds several payment channels).

6.1 Future work

We plan to follow lnd code and LN mainnet changes with regular CLoTH upgrades, and LN mainnet snapshots. We’ll then simulate various payment scripts on such snapshots, in order to understand how LN mainnet might respond to different payment dynamics. We’ll also simulate different networks, to explore how the network could and should evolve.

To better understand the impact of new LN features, in future works we plan to simulate features that have been proposed but are not yet part of LN specifications (e.g., Atomic Multi-Path Payments Osuntokun , Split Payments Piatkivskyi and Nowostawski (2018)). Conversely, a bottom-up, data-driven approach could lead to empirically find desirable LN features which have not been proposed yet.

Finally, we plan a complete exploration of the whole space of meaningful LN configurations. To do so we’ll need to improve CLoTH performance and run simulations with millions of nodes in reasonable times. A performance bottleneck we will address is the inherently sequential application of the Dijkstra’s algorithm.

Acknowledgements.

We thank Prof. Michele Garetto, who gave us a fundamental help in understanding the theoretical concepts of simulations. \abbreviationsThe following abbreviations are used in this manuscript:

[TABLE]

\appendixtitlesyes \appendixsectionsmultiple

Appendix A Reference Code Explanation

In this section, we describe the details of the lnd code we took as reference to develop CLoTH. In lnd, there are two main code structures which manage payments: Switch and Link. Switch is the messaging bus of HTLC messages: it is in charge of forwarding HTLCs or redirecting HTLCs initiated by the local peer to the proper functions. Link is the service which drives a channel commitment update procedure according to the HTLCs that concern the channel. In the following, we describe the functions included in the call graph in Figure 1.

• •

SendPayment. As shown by the numbers in the arrows, which represent the order of function calls, this function first tries to find possible routes to transfer the payment to the receiver and then tries to send the payment through one of the routes found. If the payment fails, it re-attempts the payment through another viable route.

• •

RequestRoute. It attempts to find candidate routes which can route the payment to the receiver.

• •

findPath. It runs the Dijkstra’s algorithm, using timelock and fee as distance metric. In fact, each channel endpoint has a policy which defines the timelock and the fee that will be applied to any HTLC forwarded by that endpoint. The higher are the timelock and the fees in the endpoint policy, the higher the distance.

• •

newRoute. It attempts to transform a path into a route. A route is a path which connects sender and receiver and which can also transfer the payment. A path is considered capable of transferring a payment if all channels in the path have a capacity greater than or equal to the payment amount, taking fees into account. A fee is the amount of funds a channel endpoint withholds for forwarding a payment through that channel.

• •

handleLocalDispatch. Function of the Switch that processes HTLCs relative to payments initiated by the local peer. This function returns an error if there are no channels to the next hop with enough balance to forward the payment.

• •

handleLocalResponse. Function of the Switch that processes the result of a payment initiated by the local peer. It receives an HTLCFail or HTLCFulfill message and it propagates them back to SendPayment.

• •

handlePacketForward. Function of the Switch that processes HTLCs of payments initiated by other peers and to be forwarded by the local peer. It produces an HTLCFail if no channel to the next route hop has enough balance to forward the payment, or if the local peer policy is not respected. The HTLCFail message is then sent back to the payment sender.

• •

handleDownStreamPkt. Function of the Link which processes HTLCs coming from the Switch. It produces an HTLCFail if the channel does not have enough balance to forward the payment.

• •

handleUpStremMsg. Function of the Link, the first called by a peer upon the reception of an HTLC message by another peer.

• •

SendMessage. Function to send over the network an HTLC message from a peer to another.

• •

processRemoteAdds. It processes an HTLCAdd and checks for possible errors. When called by the payment receiver, the function decides whether to accept or not the payment.

• •

processRemoteSettleFails. It processes an HTLCFail or HTLCFulfill and checks for possible errors.

\reftitle

References \externalbibliographyyes