How it works

The app tries to imitate how humans would rate attractiveness. It uses machine learning and computer vision algorithms to calculates attractiveness scores from images.

Architecture

The app consists of three components: a face extractor component and 2 Neural Networks of which one predicts age and gender and the other one estimates attractiveness.

Extract Face

The open-source tool OpenCV is used to locate and extract the face. If multiple faces are detected, one is randomly picked (We might improve this in the future). Black bars are added to the sides to make the result a squared image of the size 224 x 224 pixels.

Predict Age and Gender

For this component, we created a customized Deep Neural Network, inspired by this open-source project. Output is the estimated age and a label that indicates whether the person is male or female. Our experiments showed that first predicting age and gender improves the accuracy of the attractiveness score.

Predict Attractiveness

This Component is a Deep Neural Network (based on the ResNet architecture). It takes as input the extracted face and previously predicted age and gender to perform the attractiveness prediction.

The model is trained with a dataset consisting of images with faces that are annotated with labels for age, gender and attractiveness score. An optimization algorithm ensures that the model learns to generalize, meaning that it also learns to predict the attractiveness of images that are not part of the dataset (supervised learning).

Statistics

This chart shows the results of 100,000 predictions from real people. The average score is 6.0 (std=1.25). See more statistics here.

Technologies we use

All tools we use to operate the prediction model and the website to interact with it are part of the Google Cloud Platform. We created a containerized API, that holds the previously mentioned face extractor, attractiveness detection models and a simple web application that provides the API endpoints. Based on this Docker container, we create a VM within Google Compute Engine. The same API is used to operate the website and android app. For the website, we use App Engine, which is a managed environment that can be used to easily host simple web applications built with Flask. The Flask app connects to the API described above to predict the attractiveness score. Python libraries used are OpenCV, TensorFlow, Keras, Flask, Django, Django-rest, google-cloud-logging and google-cloud-storage.

Check out our attractiveness test