Hi Sheldon! Creating Deep Personalized Characters from TV Shows
Meidai Xuanyuan, Yuwang Wang, Honglei Guo, Xiao Ma, Yuchen Guo, Tao, Yu, Qionghai Dai

TL;DR
This paper introduces a new task called Deep Personalized Character Creation (DPCC) that generates multimodal responses matching a character's personality from TV shows, supported by a large new dataset and baseline methods.
Contribution
The paper proposes the DPCC task, introduces the DPCD dataset with extensive multimodal character data, and develops baseline models for personalized multimodal character generation.
Findings
Baseline models can generate character-matched multimodal responses.
The DPCD dataset is approximately ten times larger than existing datasets.
Subjective and objective evaluations show promising results for personalized character responses.
Abstract
Imagine an interesting multimodal interactive scenario that you can see, hear, and chat with an AI-generated digital character, who is capable of behaving like Sheldon from The Big Bang Theory, as a DEEP copy from appearance to personality. Towards this fantastic multimodal chatting scenario, we propose a novel task, named Deep Personalized Character Creation (DPCC): creating multimodal chat personalized characters from multimodal data such as TV shows. Specifically, given a single- or multi-modality input (text, audio, video), the goal of DPCC is to generate a multi-modality (text, audio, video) response, which should be well-matched the personality of a specific character such as Sheldon, and of high quality as well. To support this novel task, we further collect a character centric multimodal dialogue dataset, named Deep Personalized Character Dataset (DPCD), from TV shows. DPCD…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Human Motion and Animation
