Automatically Generating Documentation for Lambda Expressions in Java

TL;DR

This paper studies the usage of lambda expressions in Java, finds they are under-documented, and introduces LambdaDoc, a tool that automatically generates natural language documentation for lambda expressions, improving developer understanding.

Contribution

It provides the first empirical analysis of lambda usage in Java repositories and presents LambdaDoc, a novel tool for automatic documentation generation for lambda expressions.

Findings

11% of Java GitHub repositories use lambda expressions

Only 6% of lambda expressions have source code comments

Developers find LambdaDoc's documentation to be complete and expressive

Abstract

When lambda expressions were introduced to the Java programming language as part of the release of Java 8 in 2014, they were the language's first step into functional programming. Since lambda expressions are still relatively new, not all developers use or understand them. In this paper, we first present the results of an empirical study to determine how frequently developers of GitHub repositories make use of lambda expressions and how they are documented. We find that 11% of Java GitHub repositories use lambda expressions, and that only 6% of the lambda expressions are accompanied by source code comments. We then present a tool called LambdaDoc which can automatically detect lambda expressions in a Java repository and generate natural language documentation for them. Our evaluation of LambdaDoc with 23 professional developers shows that they perceive the generated documentation to be complete, concise, and expressive, while the majority of the documentation produced by our participants without tool support was inadequate. Our contribution builds an important step towards automatically generating documentation for functional programming constructs in an object-oriented language.

Tables

Table 1. TABLE I: Survey questions. Each horizontal like indicates a page break of the survey. Note that Questions 9–12 were repeated five times for each of the lambda expressions and corresponding documentation shown in Table  II-B 3 .

	Question	Answer options
1	Is developing software part of your job?	Yes / No
2	What is your job title?	Open-ended
3	For how many years have you been developing software?	Less than one year / 1–2 years / 2–4 years / 4–6 years / More than 6 years
4	How many years of experience do you have in Java development?	Less than one year / 1–2 years / 2–4 years / 4–6 years / More than 6 years
5	How would you rate your expertise in Java?	Beginner / Intermediate / Expert
6	How confident are you in your ability to WRITE lambda expressions in Java?	Not confident at all / Slightly confident / Somewhat confident / Fairly confident / Completely confident
7	How confident are you in your ability to READ lambda expressions in Java?	Not confident at all / Slightly confident / Somewhat confident / Fairly confident / Completely confident
8	What is your area of software development? (e.g., web, systems, embedded)	Open-ended
9	Consider the following lambda expression in Java. Could you please write a one-sentence summary of what this lambda expression does?	Open-ended
10	Consider now the following sentence which aims to describe the lambda expression. Only focusing on the content of the sentence without considering the way it has been presented, do you think the description is COMPLETE?	The sentence is considered to be complete / The sentence misses some important information to understand the lambda expression / The sentence misses the majority of the important information to understand the lambda expression
11	Consider the same lambda expression and sentence. Only focusing on the content of the sentence without considering the way it has been presented, do you think the sentence is CONCISE?	The sentence is considered to be concise / The sentence contains some redundant/useless information / The sentence contains a lot of redundant/useless information
12	Consider the same lambda expression and sentence. Only focusing on the content of the sentence without considering the completeness and conciseness, do you think the sentence is EXPRESSIVE?	The sentence is easy to read and understand / The sentence is somewhat readable and understandable / The sentence is hard to read and understand
13	Please try LambdaDoc for any lambda expressions of your choice at link to LambdaDoc web application$>$. Please provide feedback on LambdaDoc here.	Open-ended
14	How likely is it that you would recommend this tool to a friend or colleague?	5-point Likert scale from “not likely at all” to “extremely likely”
15	How would you prefer this tool to be implemented?	As an Eclipse plugin / As a GUI Application / As a website/web service / Other (multiple-choice)
16	Which of the following software engineering tasks would you use this type of documentation of lambda expressions in Java for?	Implementation / Testing / Documentation / Maintenance / Other (multiple-choice)
17	Do you have any further comments about lambda expressions or this survey?	Open-ended

Table 2. TABLE II: Lambda expressions and corresponding documentation generated by LambdaDoc

Lambda expression	Generated documentation
⬇ callInContext( REPO_USER_2, repo2.getId(), MASTER_BRANCH, () -> createNode ( NodePath.ROOT, ”repo2Node” ) );	This lambda expression does not take any parameter and returns the result of the execution of the “create Node” method with two parameters “NodePath ROOT and “repo2Node” ”.

Full text

I Introduction and Motivation

Modern programming languages enhance existing features and add new ones on a regular basis. For example, as part of the release of Java 8 in 2014, several new features and enhancements were introduced, ranging from improved type inference and method parameter reflection to the introduction of lambda expressions. According to Oracle,111https://www.oracle.com/technetwork/java/javase/8-whats-new-2157071.html, last accessed 5 Jan 2019. lambda expressions “enable you to treat functionality as a method argument, or code as data. Lambda expressions let you express instances of single-method interfaces (referred to as functional interfaces) more compactly.”

The introduction of lambda expressions to Java was motivated by the suitability of functional programming for synchronised, parallel, and event-driven programming. Other programming languages, including Groovy, Scala, and Python, already supported functional programming. Lambda expressions are useful since they are shorter than anonymous classes and can result in a similar outcome.

The word lambda is derived from the Greek letter lambda ($\lambda$) to represent a function of the abstract [Sharan2014]. Lambda expressions in Java can be defined as nameless code blocks of functions that contain a collection of formal parameters and a body that is joined by an arrow (->). Listing 1 shows examples of lambda expressions in Java [Sharan2014]. A lambda expression that declares its parameters types is called an explicit lambda expression and a lambda expression that does not declare its parameters types is called an implicit lambda expression [Sharan2014].

One of the primary objectives of using lambda expressions is to maintain a short syntax and to allow the compiler to deduce the details. However, along with the compiler, developers are also often left to deduce the details on their own. Understanding lambda expressions is not trivial, and good software documentation has long been known to be rare [Lethbridge2003]. To make lambda expressions more accessible to developers, we follow in a long line of work on automatically documenting particular parts of source code (e.g., test cases [Li2016] and database code [Linares-Vasquez2016]) to develop a novel approach called LambdaDoc to automatically document lambda expressions in Java. To the best of our knowledge, LambdaDoc is the first tool of its nature.

In this paper, we contribute:

• •

An empirical study of how lambda expressions are used in 435 engineered software repositories hosted on GitHub.

• •

An empirical study of the documentation of lambda expressions in GitHub repositories via source code comments.

• •

LambdaDoc, a novel approach to automatically document lambda expressions in Java, along with an evaluation with 23 professional developers.

The remainder of this paper is structured as follows. We introduce our research questions as well as our methods for data collection and analysis in Section II. LambdaDoc is described in detail in Section LABEL:sec:lambdadoc and our findings are reported in Section LABEL:sec:findings, separately for each of our research questions. We discuss threats to validity in Section LABEL:sec:threats and related work in Section LABEL:sec:related, before we conclude the paper and outline future work in Section LABEL:sec:conclusions.

II Research Method

In this section, we outline our research questions as well as the data collection and analysis methods used to answer them.

II-A Research Questions

To study how lambda expressions are used by Java developers, we first needed to devise an approach for detecting lambda expressions in non-compilable code and code snippets in commits. We chose not to rely on existing static analysis approaches (e.g., PMD [Louridas2006]) since these approaches often require compilable code to identify lambda expressions. To evaluate our custom-built lambda expression detector, we first addressed the following research question:

RQ1 How accurate is our approach for detecting lambda expressions?

Since lambda expressions have been recently introduced in Java, little is known about the prevalence of lambda expression usage in Java projects. Hence, we aimed to establish how frequently lambda expressions are used by Java developers. In addition, we investigated the kind of lambda expressions that are commonly used:

RQ2 How are lambda expressions used?

RQ2.1 How many repositories use lambda expressions, and what is the amount of lambda expressions per repository?

RQ2.2 Are the same lambda expressions used in multiple repositories?

In addition to establishing how Java developers are making use of lambda expressions, a goal of our work was the automated generation of documentation for these expressions. To inspire our work on documentation generation, we next asked how lambda expressions are currently documented in the form of source code comments:

RQ3 How are lambda expressions documented?

RQ3.1 How many lambda expressions are accompanied by comments?

RQ3.2 What kind of comments accompany lambda expressions?

Finally, to evaluate our documentation generation approach for lambda expressions LambdaDoc, we asked professional developers to evaluate the automatically generated documentation:

RQ4 How well can LambdaDoc document lambda expressions?

RQ4.1 How do developers document lambda expressions when asked to provide comments?

RQ4.2 How is the documentation generated using LambdaDoc perceived by developers?

II-B Data Collection

II-B1 Repositories

Motivated by previous work on the perils of mining GitHub [Kalliamvakou2014], e.g., a large portion of repositories on GitHub are not for software development, we used the RepoReaper framework developed by Munaiah et al. [Munaiah2017] to select repositories for our work. RepoReaper was developed to address the difficulty to differentiate between repositories with engineered software projects and those with assignments and noise. The ratio of unwanted repositories in a stochastic sample could distort research and cause illogical and possibly incorrect conclusions. RepoReaper contains repositories classified as organisation and utility.

To select repositories for our study, we first obtained all 51,392 Java repositories which had been classified as containing engineered software projects by the Random Forest classification of RepoReaper. We then randomly sampled from these 51,392 repositories in batches of 1,000 until we had obtained at least 400 repositories which contained at least one lambda expression detected by our lambda expression detector (see Section II-B2). This way, we were able to ensure that our conclusions concerning the ratio of repositories with a specific characteristic would generalise to the entire population of engineered Java repositories on GitHub containing lambda expressions with a confidence level of 95% and a confidence interval of 5.222https://www.surveysystem.com/sscalc.htm, last accessed 5 Jan 2019. After cloning and analysing 4,000 repositories (i.e., four batches of 1,000, the number it took to find at least 400 repositories containing at least one lambda expression), we had retrieved a total of 435 repositories containing lambda expressions, i.e., 11%. These 435 repositories are a statistically representative sample of all engineered Java repositories containing lambda expressions. They contained a total of 497,108 Java files, out of which 9,933 contained at least one lambda expression. In total, we collected 54,071 lambda expressions across the 435 Java repositories.

II-B2 Detection of Lambda Expressions

We developed an approach to detect lambda expressions and collect their metadata, i.e., start line number, start character position, end line number, end character position, number of lines, number of parameters, and type (explicit vs. implicit).

To identify lambda expressions in the source code of each Java file, our lambda expression detector first reads source code line-by-line until encountering a lambda arrow (->) that is not part of a comment or a string. The detector then checks whether the lambda expression covers multiple lines—a multi-line lambda expression starts with an open-parenthesis or an open-bracket and ends with the corresponding closing symbol while a single-line lambda expression ends with a semicolon or a parenthesis. The detector then determines if the lambda expression is explicit or implicit by checking whether the parameter list contains parameter types. For example, in Listing 1, the lambda expression Ex1 is identified as a single-line lambda expression that has an implicit type, while the lambda expression Ex2 is identified as a multi-line lambda expression that has an explicit type. Finally, our lambda expression detector extracts source code comments written directly above the lambda expression. For multi-line lambda expressions, our detector also extracts source code comments written within the lambda expression.

II-B3 Practitioner Survey

To evaluate the documentation generated by our automated approach, we employed a practitioner survey which followed a similar structure used by Linares-Vásquez et al. [Linares-Vasquez2016]. We recruited participants through Amazon Mechanical Turk333https://www.mturk.com/, last accessed 22 Jan 2019. and the required qualification was “Employment Industry – Software & IT Services”. This methodology has been successfully used by previous work [Prana2019].