IReEn: Reverse-Engineering of Black-Box Functions via Iterative Neural Program Synthesis

TL;DR

This paper presents IReEn, a neural program synthesis method that reverse-engineers black-box functions by iteratively refining candidate programs, achieving high accuracy without privileged information.

Contribution

Introduces a novel neural program synthesis approach for black-box reverse engineering that outperforms existing methods without requiring privileged data.

Findings

Achieves 78% success in generating functionally equivalent programs.

Outperforms state-of-the-art methods on the Karel dataset.

Operates effectively with only input-output access to black-boxes.

Abstract

In this work, we investigate the problem of revealing the functionality of a black-box agent. Notably, we are interested in the interpretable and formal description of the behavior of such an agent. Ideally, this description would take the form of a program written in a high-level language. This task is also known as reverse engineering and plays a pivotal role in software engineering, computer security, but also most recently in interpretability. In contrast to prior work, we do not rely on privileged information on the black box, but rather investigate the problem under a weaker assumption of having only access to inputs and outputs of the program. We approach this problem by iteratively refining a candidate set using a generative neural program synthesis approach until we arrive at a functionally equivalent program. We assess the performance of our approach on the Karel dataset. Our results show that the proposed approach outperforms the state-of-the-art on this challenge by finding an approximately functional equivalent program in 78% of cases -- even exceeding prior work that had privileged information on the black-box.