Image Inspired Poetry Generation in XiaoIce

TL;DR

This paper presents a system that generates poetry from images by extracting keywords, expanding them through associations, and using neural networks, resulting in more artistic poems and over 12 million generated poems since 2017.

Contribution

The paper introduces a novel image-to-poetry generation system that mimics human poetic inspiration and demonstrates its effectiveness through human evaluation and large-scale deployment.

Findings

Generated poems are more artistic than baselines

Over 12 million poems generated since 2017

Published poetry collection claimed as first AI-written poetry book

Abstract

Vision is a common source of inspiration for poetry. The objects and the sentimental imprints that one perceives from an image may lead to various feelings depending on the reader. In this paper, we present a system of poetry generation from images to mimic the process. Given an image, we first extract a few keywords representing objects and sentiments perceived from the image. These keywords are then expanded to related ones based on their associations in human written poems. Finally, verses are generated gradually from the keywords using recurrent neural networks trained on existing poems. Our approach is evaluated by human assessors and compared to other generation baselines. The results show that our method can generate poems that are more artistic than the baseline methods. This is one of the few attempts to generate poetry from images. By deploying our proposed approach, XiaoIce has already generated more than 12 million poems for users since its release in July 2017. A book of its poems has been published by Cheers Publishing, which claimed that the book is the first-ever poetry collection written by an AI in human history.