Facial recognition technology is applied in various fields nowadays, from IT security, to social media and personalized systems. To explore its functionality and possibilities, it was decided to combine it with information retrieval of details to identified individuals.
For the facial recognition/the facial embedding generation, FaceNet [1], a deep neural network transforming facial images into 512-dimensional embeddings in Euclidean space, was used. In combination with the Labeled Faces in the Wild (LFW) dataset [2] that contains labelled images of approx. 6000 people, the system is able to construct a database containing all the embeddings in order to identify individuals as accurately as possible.
To extend the functionality, after identification of the individual the label is used to retrieve the first section of the individual's Wikipedia article over the Wikipedia API [3]. The retrieved data is then processed by the Mistral 7B [4] large language model to generate context-aware responses. This combination of technologies enables seamless interaction from recognition to detailed knowledge generation.
The application is evaluated for its accuracy in facial identification and the precision of its responses. By integrating advanced techniques in computer vision and natural language processing, this project demonstrates the potential for systems that offer both recognition and enriched interaction.
- Dominik Hammler
- Fabian Frühwirth
- Klemens Armstorfer
- Sebastian Pack
[1] Florian Schroff, Dmitry Kalenichenko, and James Philbin. “FaceNet: A unified embedding for face recognition and clustering”. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2015, pp. 815–823. DOI: 10 . 1109 / cvpr . 2015.7298682. URL: http://dx.doi.org/10.1109/CVPR.2015.7298682.
[2] Gary B. Huang et al. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Tech. rep. 07-49. University of Massachusetts, Amherst, Oct. 2007.
[3] Martin Majlis. Wikipedia API. 2025. URL: https://github.com/martin-majlis/Wikipedia-API?tab=readme-ov-file.
[4] Albert Q. Jiang et al. Mistral 7B. 2023. arXiv: 2310.06825 [cs.CL]. URL: https://arxiv.org/abs/2310.06825.