Lightweight Apex-based Enhanced Network (LAENet)神經網路 進行微表情辨識

Abstract

Abstract Micro-expression is an expression that reveals one's true feelings and can be potentially applied in various domains such as healthcare, safety interrogation, and business negotiation. The micro-expression recognition is thus far being judged manually by psychologists and trained experts, which consumes a lot of human effort and time. Recently, the development of the deep learning network has proven promising performance in many computer vision related tasks. Amongst, micro-expression recognition adopts the deep learning methodology to improve the feature learning capability and model generalization. This paper introduces a Lightweight Apex-based Enhanced Network (LAENet) that improves by extending one of the state-of-the-art, Shallow Triple Stream Three-dimensional CNN (STSTNet). Concretely, the network is first pre-trained with a macro-expression dataset to encounter the small data problem. The features extracted from the datasets are the optical flow guided features. Besides, an eye masking technique is introduced to reduce noise interference such as eye blinking and glasses reflection issues. The results obtained are accuracy of 79.19% and F1-score of 75.9%. Comprehensive experimentation had been conducted on the composite dataset that consists of CASME II, SMIC, and SAMM datasets. Moreover, a thorough recognition results comparison is provided by comparing it with recent methods. Detail qualitative and quantitative results are reported and discussed.

中文摘要 微表情可表露人們隱藏的真實情緒，可運用在醫療照顧、安全審訊、商業協商上。由心理學家等相關專家辨識微表情，將耗費相當多的時間和精力，近期透過神經網路進行深度學習已證實其可靠性。這份研究報告將介紹神經網路Lightweight Apex-based Enhanced Network (LAENet)，此為其前身Shallow Triple Stream Three-dimensional CNN (STSTNet)的延伸研究。具體來說，神經網路會先經過較為明顯的表情資料庫訓練，這解決了微表情資料庫過小的問題。表情特徵是由optical flow取得。另外，運用eye masking來減少眨眼與眼鏡反光所造成的雜訊。最終的精確度為79.19% F1-score為75.9%。全面性的實驗是運用在由CASME II、SMIC和SAMM所組成的複合資料庫。此外與近期其它研究結果進行比較，並報告與討論詳細結果。