Reinforcement Logic Rule Learning for Temporal Point Processes

TL;DR

This paper introduces a reinforcement learning-based framework for incrementally learning temporal logic rules to explain event sequences, optimizing rule sets for better likelihood fitting in temporal point processes.

Contribution

It presents a novel combination of reinforcement learning and temporal logic rule learning for point processes, with an efficient neural search policy for rule generation.

Findings

Effective rule learning on synthetic data

Promising results on healthcare datasets

End-to-end training of rule generation policy

Abstract

We propose a framework that can incrementally expand the explanatory temporal logic rule set to explain the occurrence of temporal events. Leveraging the temporal point process modeling and learning framework, the rule content and weights will be gradually optimized until the likelihood of the observational event sequences is optimal. The proposed algorithm alternates between a master problem, where the current rule set weights are updated, and a subproblem, where a new rule is searched and included to best increase the likelihood. The formulated master problem is convex and relatively easy to solve using continuous optimization, whereas the subproblem requires searching the huge combinatorial rule predicate and relationship space. To tackle this challenge, we propose a neural search policy to learn to generate the new rule content as a sequence of actions. The policy parameters will be trained end-to-end using the reinforcement learning framework, where the reward signals can be efficiently queried by evaluating the subproblem objective. The trained policy can be used to generate new rules in a controllable way. We evaluate our methods on both synthetic and real healthcare datasets, obtaining promising results.