Journal of Engineering Applications and Intelligent Systems
e-ISSN Not Available  ·  p-ISSN Not Available  ·  HafizLab
Sign in Register

Robustness and Generalization of Lightweight Face-Verification Models: A Reproducible Cross-Dataset Evaluation Framework for Resource-Constrained Deployment

System Admin 0000-0002-1825-0097
HafizLab
Md Hafizur Corresponding
hafizLab
8 Nov 2026
pp. 1-19
https://doi.org/10.64886/jeais.0001

Abstract

Lightweight face-verification models suit deployments with limited compute, including CPU-only servers, but performance on one benchmark may not transfer to other poses, ages, resolutions, compression levels or capture conditions. This paper specifies a reproducible framework for measuring robustness and generalization of lightweight face-verification models across datasets and capture conditions, with within-dataset, fixed-threshold cross-dataset, target-recalibrated, controlled-degradation and CPU-efficiency tracks, identity-aware bootstrap intervals, and an open toolkit implementing every metric and degradation. In an executed study, three lightweight models (MobileFaceNet-w600k, SFace and a Qualcomm MobileFaceNet) and an ArcFace-R50 reference were evaluated on LFW, CFP-FP and AgeDB-30 and on a 22-condition degradation grid on LFW. The lightweight models had overlapping clean LFW EER (0.43%, 0.57% and 0.67%; reference 0.30%) but differed from 5.51% to 14.60% on CFP-FP and from 3.70% to 5.70% on AgeDB-30; the paired difference between the Qualcomm model and MobileFaceNet-w600k on CFP-FP was -9.1 points (A − C). A threshold calibrated on LFW at FMR = 1% gave FMR above 1% on AgeDB-30 for all four models (1.83%–3.23%); recalibration restored FMR at the cost of higher FNMR and needed about 1,000 target pairs. Under strong blur, 20-pixel resolution and a lower-face band the models diverged by several-fold, and under the largest band a frozen clean threshold gave FMR of 32.53%, 4.20%, 93.13% and 18.73% (A, B, C, reference). Single-thread CPU latency was 13.7, 24.9, 11.3 and 287.6 ms per image; speed did not predict robustness. CFP-FP and AgeDB-30 came from an unofficial redistribution with approximated identities, so those results are provisional pending replication on the official releases. The framework helps practitioners identify where a lightweight model remains reliable, where thresholds become unstable, and when recapture or human review is preferable to automatic decisions.
Keywords: face verification; lightweight deep learning; robustness; generalization; cross-dataset evaluation; domain shift; threshold calibration; CPU inference

Full Text

Introduction

Face verification compares two facial samples and estimates whether they belong to the same person. Deep embedding models make this practical at scale, and lightweight architectures reduce latency and resource consumption. Yet computational efficiency does not guarantee robustness: a model can perform well on a familiar benchmark and degrade when the capture device, image-processing pipeline, pose, illumination, resolution, or compression changes.
A deployment threshold also affects generalization. A threshold calibrated on one dataset may not achieve the same FMR or FNMR on another. When a system supports admission, account access, or another consequential workflow, a shift in error rates creates both security risk (false accepts) and usability cost (false rejects). Robustness evaluation must therefore examine not only average verification accuracy but also operating-point stability, condition-specific errors, failure-to-process rates, and computational cost.
This paper proposes a protocol for assessing lightweight face-verification models across datasets and capture conditions. It emphasizes controlled comparisons, strict separation of calibration and test data, transparent reporting, and avoidance of unsupported claims. The protocol is released together with executable evaluation code so that any group can populate the result tables with identical procedures.

Prefer the typeset version? Switch to the PDF or download it.

References

  1. 1. Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 815–823.
  2. 2. Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). ArcFace: Additive angular margin loss for deep face recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4690–4699.
  3. 3. Chen, S., Liu, Y., Gao, X., & Han, Z. (2018). MobileFaceNets: Efficient CNNs for accurate real-time face verification on mobile devices. Chinese Conference on Biometric Recognition (CCBR), 428–438.
  4. 4. Huang, G. B., Ramesh, M., Berg, T., & Learned-Miller, E. (2007). Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. University of Massachusetts Amherst, Technical Report 07-49.
  5. 5. Sengupta, S., Chen, J.-C., Castillo, C., Patel, V. M., Chellappa, R., & Jacobs, D. W. (2016). Frontal to profile face verification in the wild. IEEE Winter Conference on Applications of Computer Vision (WACV).
  6. 6. Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., & Zafeiriou, S. (2017). AgeDB: The first manually collected, in-the-wild age database. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
  7. 7. Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (2000). The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1090–1104.
  8. 8. International Organization for Standardization / International Electrotechnical Commission. ISO/IEC 19795 series, Information technology — Biometric performance testing and reporting. Part 1: Principles and framework (ISO/IEC 19795-1:2021).
  9. 9. National Institute of Standards and Technology. Face Recognition Technology Evaluation (FRTE, formerly FRVT) reports. FRTE 1:1 Verification, https://pages.nist.gov/frvt/ (accessed 27 October 2026).
  10. 10. Efron, B., & Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Chapman & Hall.
  11. 11. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.
  12. 12. Zhong, Y., Deng, W., Hu, J., Zhao, D., Li, X., & Wen, D. SFace: Sigmoid-constrained hypersphere loss for robust face recognition. arXiv:2205.12010.
  13. 13. OpenCV Zoo. face_recognition_sface model page, https://github.com/opencv/opencv_zoo (accessed 27 October 2026). InsightFace model zoo README, https://github.com/deepinsight/insightface (accessed 27 October 2026). AgeDB resource page, https://ibug.doc.ic.ac.uk/resources/agedb/ (accessed 27 October 2026).