Cooperative Task-Oriented Communication for Multi-Modal Data with Transmission Control

TL;DR

This paper introduces a cooperative, task-oriented communication approach for multi-modal data in IoT, reducing data transmission by leveraging lower-rate modality results to control higher-rate data transfer, exemplified in human activity recognition.

Contribution

It proposes a novel transmission control scheme for multi-modal data in IoT, improving efficiency by using received data to regulate subsequent transmissions.

Findings

Transmission reduced to 2% with control scheme

Maintains high accuracy in human activity recognition

Demonstrates effectiveness in smart home environment

Abstract

Real-time intelligence applications in Internet of Things (IoT) environment depend on timely data communication. However, it is challenging to transmit and analyse massive data of various modalities. Recently proposed task-oriented communication methods based on deep learning have showed its superiority in communication efficiency. In this paper, we propose a cooperative task-oriented communication method for the transmission of multi-modal data from multiple end devices to a central server. In particular, we use the transmission result of data of one modality, which is with lower rate, to control the transmission of other modalities with higher rate in order to reduce the amount of transmitted date. We take the human activity recognition (HAR) task in a smart home environment and design the semantic-oriented transceivers for the transmission of monitoring videos of different rooms and acceleration data of the monitored human. The numerical results demonstrate that by using the transmission control based on the obtained results of the received acceleration data, the transmission is reduced to 2% of that without transmission control while preserving the performance on the HAR task.