HiMFR: A Hybrid Masked Face Recognition Through Face Inpainting

Hosen M. I., Islam B.

arXiv, cilt.2209.08930, ss.1-7, 2022 (Hakemsiz Dergi)

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 2209.08930
  • Basım Tarihi: 2022
  • Dergi Adı: arXiv
  • Sayfa Sayıları: ss.1-7
  • İstanbul Ticaret Üniversitesi Adresli: Hayır

Özet

To recognize the masked face, one of the possible solutions could be to restore the occluded part of the face first and then apply the face recognition method. Inspired by the recent image inpainting methods, we propose an end-toend hybrid masked face recognition system, namely HiMFR, consisting of three significant parts: masked face detector, face inpainting, and face recognition. The masked face detector module applies a pretrained Vision Transformer (ViT b32) to detect whether faces are covered with masked or not. The inpainting module uses a fine-tune image inpainting model based on a Generative Adversarial Network (GAN) to restore faces. Finally, the hybrid face recognition module based on ViT with an EfficientNetB3 backbone recognizes the faces. We have implemented and evaluated our proposed method on four different publicly available datasets: CelebA, SSDMNV2, MAFA, Pubfig83 with our locally collected small dataset, namely Face5. Comprehensive experimental results show the efficacy of the proposed HiMFR method with competitive performance.