Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship
Joseph P. Robinson, Zaid Khan, Yu Yin, Ming Shao, Yun Fu

TL;DR
This paper introduces FIW MM, a multimodal kinship dataset with videos, audio, and captions, demonstrating significant performance improvements in kinship recognition and enabling more realistic, multi-faceted research in the field.
Contribution
It extends the existing FIW dataset with multimedia data and provides a new multi-task benchmark for kinship recognition, developed with minimal human effort.
Findings
Significant performance improvements with added modalities
First publicly available multimodal kinship dataset
Enhanced potential for real-world kinship recognition systems
Abstract
Kinship, a soft biometric detectable in media, is fundamental for a myriad of use-cases. Despite the difficulty of detecting kinship, annual data challenges using still-images have consistently improved performances and attracted new researchers. Now, systems reach performance levels unforeseeable a decade ago, closing in on performances acceptable to deploy in practice. Like other biometric tasks, we expect systems can receive help from other modalities. We hypothesize that adding modalities to FIW, which has only still-images, will improve performance. Thus, to narrow the gap between research and reality and enhance the power of kinship recognition systems, we extend FIW with multimedia (MM) data (i.e., video, audio, and text captions). Specifically, we introduce the first publicly available multi-task MM kinship dataset. To build FIW MM, we developed machinery to automatically…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Generative Adversarial Networks and Image Synthesis · Advanced Image and Video Retrieval Techniques
