DevPiolt: Operation Recommendation for IoT Devices at Xiaomi Home

TL;DR

DevPiolt is a novel LLM-based system designed to generate personalized IoT device operation recommendations, improving user experience and engagement in Xiaomi Home app through domain knowledge integration and preference alignment.

Contribution

The paper introduces DevPiolt, a new LLM-based recommendation model with domain knowledge pre-training, preference optimization, and confidence control, tailored for IoT device operation suggestions.

Findings

Outperforms baselines with 69.5% average improvement across metrics.

Deployed in Xiaomi Home app, serving 255,000 users.

Online tests show 21.6% increase in device coverage and 29.1% in acceptance rates.

Abstract

Operation recommendation for IoT devices refers to generating personalized device operations for users based on their context, such as historical operations, environment information, and device status. This task is crucial for enhancing user satisfaction and corporate profits. Existing recommendation models struggle with complex operation logic, diverse user preferences, and sensitive to suboptimal suggestions, limiting their applicability to IoT device operations. To address these issues, we propose DevPiolt, a LLM-based recommendation model for IoT device operations. Specifically, we first equip the LLM with fundamental domain knowledge of IoT operations via continual pre-training and multi-task fine-tuning. Then, we employ direct preference optimization to align the fine-tuned LLM with specific user preferences. Finally, we design a confidence-based exposure control mechanism to avoid negative user experiences from low-quality recommendations. Extensive experiments show that DevPiolt significantly outperforms baselines on all datasets, with an average improvement of 69.5% across all metrics. DevPiolt has been practically deployed in Xiaomi Home app for one quarter, providing daily operation recommendations to 255,000 users. Online experiment results indicate a 21.6% increase in unique visitor device coverage and a 29.1% increase in page view acceptance rates.