FryNet: Dual-Stream Adversarial Fusion for Non-Destructive Frying Oil Oxidation Assessment

TL;DR

FryNet is a novel dual-stream neural framework that combines RGB and thermal imaging to assess frying oil oxidation in real-time, overcoming sensor noise issues and providing accurate chemical indices.

Contribution

It introduces a dual-encoder adversarial fusion approach with chemical grounding and spatial attention, enabling robust, non-destructive frying oil oxidation assessment.

Findings

Achieved 98.97% mIoU in oil-region segmentation

Attained 100% accuracy in serviceability classification

Reduced regression MAE to 2.32 across 7,226 frames

Abstract

Monitoring frying oil degradation is critical for food safety, yet current practice relies on destructive wet-chemistry assays that provide no spatial information and are unsuitable for real-time use. We identify a fundamental obstacle in thermal-image-based inspection, the camera-fingerprint shortcut, whereby models memorize sensor-specific noise and thermal bias instead of learning oxidation chemistry, collapsing under video-disjoint evaluation. We propose FryNet, a dual-stream RGB-thermal framework that jointly performs oil-region segmentation, serviceability classification, and regression of four chemical oxidation indices (PV, p-AV, Totox, temperature) in a single forward pass. A ThermalMiT-B2 backbone with channel and spatial attention extracts thermal features, while an RGB-MAE Encoder learns chemically grounded representations via masked autoencoding and chemical alignment. Dual-Encoder DANN adversarially regularizes both streams against video identity via Gradient Reversal Layers, and FiLM fusion bridges thermal structure with RGB chemical context. On 7,226 paired frames across 28 frying videos, FryNet achieves 98.97% mIoU, 100% classification accuracy, and 2.32 mean regression MAE, outperforming all seven baselines.