Emotion, Age and Gender Prediction Through Masked Face Inpainting

Islam M. B., HOSEN M. I.

26th International Conference on Pattern Recognition, ICPR 2022, Montreal, Kanada, 21 - 25 Ağustos 2022, cilt.13643 LNCS, ss.37-48, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 13643 LNCS
  • Doi Numarası: 10.1007/978-3-031-37660-3_3
  • Basıldığı Şehir: Montreal
  • Basıldığı Ülke: Kanada
  • Sayfa Sayıları: ss.37-48
  • Anahtar Kelimeler: Emotion Prediction, Face detection, Face Inpainting
  • İstanbul Ticaret Üniversitesi Adresli: Evet

Özet

Prediction of gesture and demographic information from the face is complex and challenging, particularly for the masked face. This paper proposes a deep learning-based integrated approach to predict emotion and demographic information for unmasked and masked faces, consisting of four sub-tasks: masked face detection, masked face inpainting, emotion, age, and gender prediction. The masked face detector module provides a binary decision on whether the face mask is available or not by applying pre-trained MobileNetV3. We use the inpainting module based on U-Net embedding with ImageNet weights to remove the face mask and restore the face. We use the convolutional neural networks to predict emotion (e.g., happy, angry). Besides, VGGFace-based transfer learning has been used to predict demographic information (e.g., age, gender). Extensive experiments on five publicly available datasets: AffectNet, UTKFace, FER-2013, CelebA, and MAFA, show the effectiveness of our proposed method to predict emotion and demographic identification through masked face reconstruction.