Randomly Weighted, Untrained Neural Tensor Networks Achieve Greater Relational Expressiveness

TL;DR

This paper introduces Randomly Weighted Tensor Networks (RWTNs), which use untrained random tensors in neural tensor networks to improve relational reasoning and efficiency in semantic image interpretation tasks.

Contribution

The paper proposes RWTNs that replace trained tensors with random ones, achieving comparable or better performance than traditional LTNs with fewer parameters and enhanced scalability.

Findings

RWTNs outperform LTNs in part-of relation detection.

RWTNs match LTN performance in object classification with fewer parameters.

Shared random tensors enable scalable multi-task learning.

Abstract

Neural Tensor Networks (NTNs), which are structured to encode the degree of relationship among pairs of entities, are used in Logic Tensor Networks (LTNs) to facilitate Statistical Relational Learning (SRL) in first-order logic. In this paper, we propose Randomly Weighted Tensor Networks (RWTNs), which incorporate randomly drawn, untrained tensors into an NTN encoder network with a trained decoder network. We show that RWTNs meet or surpass the performance of traditionally trained LTNs for Semantic Image Interpretation (SII) tasks that have been used as a representative example of how LTNs utilize reasoning over first-order logic to exceed the performance of solely data-driven methods. We demonstrate that RWTNs outperform LTNs for the detection of the relevant part-of relations between objects, and we show that RWTNs can achieve similar performance as LTNs for object classification while using fewer parameters for learning. Furthermore, we demonstrate that because the randomized weights do not depend on the data, several decoder networks can share a single NTN, giving RWTNs a unique economy of spatial scale for simultaneous classification tasks.