Conversational DevBots for Secure Programming: An Empirical Study on SKF Chatbot

TL;DR

This paper presents an empirical study of SKF chatbot, a conversational agent designed to assist developers with secure coding, analyzing its effectiveness, user needs, and factors influencing its development.

Contribution

It introduces SKF chatbot as a novel conversational DevBot for security, providing empirical evidence on its effectiveness and insights for future improvements.

Findings

SKF chatbot effectively answers security queries

Users expect personalized and context-aware security assistance

Identifies challenges in developing sophisticated conversational security bots

Abstract

Conversational agents or chatbots are widely investigated and used across different fields including healthcare, education, and marketing. Still, the development of chatbots for assisting secure coding practices is in its infancy. In this paper, we present the results of an empirical study on SKF chatbot, a software-development bot (DevBot) designed to answer queries about software security. To the best of our knowledge, SKF chatbot is one of the very few of its kind, thus a representative instance of conversational DevBots aiding secure software development. In this study, we collect and analyse empirical evidence on the effectiveness of SKF chatbot, while assessing the needs and expectations of its users (i.e., software developers). Furthermore, we explore the factors that may hinder the elaboration of more sophisticated conversational security DevBots and identify features for improving the efficiency of state-of-the-art solutions. All in all, our findings provide valuable insights pointing towards the design of more context-aware and personalized conversational DevBots for security engineering.