# End-to-End Learning Using Cycle Consistency for Image-to-Caption Transformations

## TL;DR

This paper introduces an end-to-end learning approach for image-to-caption transformations using cycle consistency, ensuring generated captions contain enough information to reconstruct the original image, validated through experiments.

## Contribution

The study proposes a novel cycle-consistency based method for mutual image-text transformations, enhancing caption faithfulness and image reconstruction capabilities.

## Key findings

- Cycle consistency improves caption quality.
- Automatic and crowdsourced evaluations confirm effectiveness.
- Method outperforms non-cycle-consistent baselines.

## Abstract

So far, research to generate captions from images has been carried out from the viewpoint that a caption holds sufficient information for an image. If it is possible to generate an image that is close to the input image from a generated caption, i.e., if it is possible to generate a natural language caption containing sufficient information to reproduce the image, then the caption is considered to be faithful to the image. To make such regeneration possible, learning using the cycle-consistency loss is effective. In this study, we propose a method of generating captions by learning end-to-end mutual transformations between images and texts. To evaluate our method, we perform comparative experiments with and without the cycle consistency. The results are evaluated by an automatic evaluation and crowdsourcing, demonstrating that our proposed method is effective.