The problem Facepixer solves

The Police often releases sketches of suspects so that they can be released to the public for identification and so that people can report any sightings of such individuals. Such sketches are known as 'Facial Composites'. While certainly being better than nothing, humans are especially susceptible to picking up even small difference in facial structures and hence may find it difficult to correlate the sketch to an individual.

Facepixer uses GANs to generate photorealistic images from sketches of peoples faces. This would help people identify suspects and missing people much more easier than ever before.

Facepixer can also be used to quick-sketch photo-realistic images from plain, simple pencil sketches. This can be useful for manga and comic book writers as they can see how their characters might turn out in a movie/coloured book/animation.

It also makes an amazing personalized gift and just takes a simple sketch to be made.

Challenges we ran into

Finding the dataset of images and sketches corresponding to them proved to be a herculean task, as most of them were hidden behind paywalls. Finally we found a dataset from CUHK which was more than 20 years old and which had an altogether different purpose entirely.

Training Neural Networks, particularly Generative Adverserial networks (GANs) is a very time consuming process that can take days and even weeks. Having only 24 hours, and being further limited by the need to also build a webapp to make Facepixer a usable product meant we had to optimise our training strategy.

Integrating the models with the webserver threw up some errors, which we fixed by finding workarounds (writing scripts!).

The dataset was from china, and as such had majority of chinese faces. This got us in fear of introducing a bias into the dataset (such as skin tone). We took our own pictures and converted them to sketches with an app and ran it through the model and got relatively non-biased results, albeit the skin tone was similar to the training dataset.

Facepixer strives to be a full blown application which can convert a sketch into a whole photo-realistic picture. This, though, requires training of a bunch of classifiers and several GAN networks, which could not be done in the span of 24 hours. We strive to continue this project and complete it so one day we can see sketches being converted to photo-realistic pictures.