SoundScape: A Human-AI Co-Creation System Making Your Memories Heard

TL;DR

SoundScape is a user-friendly mobile system that enables people to easily create immersive sound memories by integrating AI-generated music and sound effects, enhancing the emotional and atmospheric quality of their multimedia content.

Contribution

It introduces a novel human-AI co-creation platform that simplifies sound design for memory recording, combining conversational AI and AI music generation on mobile devices.

Findings

Users preferred SoundScape over existing tools for sound memory creation.

SoundScape improved user experience and perceived quality of sound memories.

User studies confirmed the effectiveness of the system in real-world scenarios.

Abstract

Sound plays a significant role in human memory, yet it is often overlooked by mainstream life-recording methods. Most current UGC (User-Generated Content) creation tools emphasize visual content while lacking user-friendly sound design features. This paper introduces SoundScape, a human-AI co-creation system that allows users to easily create sound memories on mobile devices through innovative interaction. By integrating sound effects and music with visual scenes, SoundScape encourages users to enrich their creations with immersive sound elements, enhancing the atmosphere of their works. To support public creation, SoundScape incorporates a conversational agent and AI music generation technology. User studies indicate that our approach is effective for sound memory creation, with SoundScape outperforming existing tools in terms of user experience and the perceived quality of produced works.