Abstract
Explosive performance and memory space growth in computing machines, along with resent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way a textual post on social media reveals individuals characteristics of its author, facial images may manifest same personality traits. This work is milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial image via deep learning. In particular, two deep learning models, included a standard feedforward neural network (SNN) and a convolution neural network (CNN) are applied to discriminant criminal and non-criminal facial images.Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent then the SNN in learning to reach best test accuracy, which 8% higher then the SNN's test accuracy. Next, to explore the classifier's hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualization conventional layer in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantages of by CNN in order to classify the two sets of images.
Introduction
Face is the primary mean of recognizing a person, transmitting information, communicating with other, and inferring with people's feelings, among others. Our faces might disclose more then what we except. A facial image can be informative of personal traits [1], such as race, gender, age, health, emotion, psychology, and profession.
This study is triggering by Lombroso's research [2], which showed that criminal could be identified by their facial structure and emotions. While Lombroso's study looked at this issue from a physiology and psychiatry perspective, our study investigated weather are not machine learning algorithm would be would be able to learn and distinguish between criminal and non-criminal facial image. More specifically, we will look for gender biases machine predictions. This is important for because criminal facial image is used to train the machines are mostly male. This is result of the large gap between the number of mugshots for arrested males and females, available to the public and used to train the machine.
It is noteworthy that this study's scope is limited to the technical and analytical aspects of this topic, while its stoical implications require more scrutiny and its practical applications demand even higher level of cautions suspicion. With that in mind, this paper explores the deep learning's capability distinguishing between criminal and non-criminal facial images. To this effect, two deep learning models, a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are trained with 10,000 neural-emotion mixed-gender, mixed-face, facial image. A neural or bank face expression is characterized by neural positioning of the facial feature. A neural face expression could be caused by a lack of emotions, boredom, depression, or slight confusion. A natural face expression is also referred to as a poker face. It is meant to conceal one's emotions when playing the card game poker [3]. While both neural network models are controlled for facial emotions by applying only neural emotion images, no control has been imposed on race, due to our small dataset and the difficulty and occasionally subjectivity of identifying the race from low-quality facial images. Both models are trained with and without controlling for gender. The results indicated that controlling for gender does not have much effect on accuracy or learning and both models reach high classification accuracies regardless. CNN achieves a tenfold cross-validation accuracy of 97%.
The strength of this study lies in its applications of neural network to investigate if a stuck of non-linear functions with thousands of parameter can find useful facial features to distinguish between criminal and non-criminal face shots. Its weakness however lies in its reliance on machine to learn these features and o limited number of images.
"Related works" section provides a review of related work. "Methodology" section elaborates on this study's methodology. " Data preparation " section describes the image dataset source and approach taken to prepare the dataset. "Neural network architecture" section describe the SNN's and CNN's architecture, proposed in this study, for criminal tendency recognition through facial images. "Visual criminal tendency detection results an discussion" section presents the results for both mixed gender and male only classification scenarios. "Contusion and feather directions" section concludes the paper by discussing the results and feature directions.
Related work
Machine learning has shown to be more effective then humans is discovering personality traits through facial images [4]. Geng et al. [5] trained a machine to estimate the age through facial image. Reese and Danforth [6] applied an ensemble for machine learning models and image processing to detect depression and psychiatric disorder in Instagram facial images.
The goal of facial emotion detection is to train a machine to distinguish among six emotional facial expression: happiness, surprise, sadness, disgust, anger, and fear [7]. Fuzzy inference system [8], hidden Markov model based on real-time tracking of the mouth shape [9], and Bayesian network [10] are among the approaches use for classifying facial emotions.
Criminal tendency another personality trait. Lombroso [2] was the first in 1871 to point out that criminal could be identified by their facial structure and emotions. Recently , Wu and Zhang [11] revisited this theory and quantitatively demonstrated the correlation between criminality and facial features. They trained four classifiers: logistic regressing, k nearest neighbor (KNN), support vector machines (SVM) convolutional neural network (CNN) and claimed that their machine identify a criminal face with a 90% accuracy. Their models was controlled for gender, race, and facial expression of emotion.
Neural networks have resurged and drown much attention in the last decade [12] with the new brand of deep learning, mainly due to the significant performance gain n visual recognition [13]. Deep learning has been applied to a wide range of applications, such as three disease recognition [14]. Among the most relevant applications of deep learning to our work, we can point to the application of CNN for force recognition [15, 16]. Cristani [17] and segalin et al. [18, 19] applied machines learning to predict the self-assessed personalty traits (openness to experience, conscientiousness, extroversion, agreeableness, and neuroticism) of a person from the image he/she uploads or likes on social media, such as Flicker, and what impressions in terms of personality traits those images trigger in unacquainted people. They performed their experiments with 60,000 images from 300 Flickers users. Cristani et al. [17] and segalin et al. [18] used a hybrid approach where generative models, used as latent representations of features (color, composition, textural, properties, etc.) extracted from images, are built and then passed to a discriminative classifier to predict each user's personality traits. Simplifying the problem into five distinct binary classification problems, one of each trait, segalin et al. [19] applied AlexNet [13], which is an eight-layer versions of CNN, pre-trained on ImageNet 2012 competition dataset. The problem they pose is to detect the personalty traits based on the images that one uploads or likes on social media, such as Flicker. Their results showed that the personality traits that others attribute to a person (based on the image that the individual uploads or likes on social media) can be predicated 10% more accurately then the personality traits that the individual attributes to him/her-self Wang and Kosinski [4] trained a deep neural network to classify facial images based on sexual orientation.
Explosive performance and memory space growth in computing machines, along with resent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way a textual post on social media reveals individuals characteristics of its author, facial images may manifest same personality traits. This work is milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial image via deep learning. In particular, two deep learning models, included a standard feedforward neural network (SNN) and a convolution neural network (CNN) are applied to discriminant criminal and non-criminal facial images.Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent then the SNN in learning to reach best test accuracy, which 8% higher then the SNN's test accuracy. Next, to explore the classifier's hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualization conventional layer in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantages of by CNN in order to classify the two sets of images.
Introduction
Face is the primary mean of recognizing a person, transmitting information, communicating with other, and inferring with people's feelings, among others. Our faces might disclose more then what we except. A facial image can be informative of personal traits [1], such as race, gender, age, health, emotion, psychology, and profession.
This study is triggering by Lombroso's research [2], which showed that criminal could be identified by their facial structure and emotions. While Lombroso's study looked at this issue from a physiology and psychiatry perspective, our study investigated weather are not machine learning algorithm would be would be able to learn and distinguish between criminal and non-criminal facial image. More specifically, we will look for gender biases machine predictions. This is important for because criminal facial image is used to train the machines are mostly male. This is result of the large gap between the number of mugshots for arrested males and females, available to the public and used to train the machine.
It is noteworthy that this study's scope is limited to the technical and analytical aspects of this topic, while its stoical implications require more scrutiny and its practical applications demand even higher level of cautions suspicion. With that in mind, this paper explores the deep learning's capability distinguishing between criminal and non-criminal facial images. To this effect, two deep learning models, a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are trained with 10,000 neural-emotion mixed-gender, mixed-face, facial image. A neural or bank face expression is characterized by neural positioning of the facial feature. A neural face expression could be caused by a lack of emotions, boredom, depression, or slight confusion. A natural face expression is also referred to as a poker face. It is meant to conceal one's emotions when playing the card game poker [3]. While both neural network models are controlled for facial emotions by applying only neural emotion images, no control has been imposed on race, due to our small dataset and the difficulty and occasionally subjectivity of identifying the race from low-quality facial images. Both models are trained with and without controlling for gender. The results indicated that controlling for gender does not have much effect on accuracy or learning and both models reach high classification accuracies regardless. CNN achieves a tenfold cross-validation accuracy of 97%.
The strength of this study lies in its applications of neural network to investigate if a stuck of non-linear functions with thousands of parameter can find useful facial features to distinguish between criminal and non-criminal face shots. Its weakness however lies in its reliance on machine to learn these features and o limited number of images.
"Related works" section provides a review of related work. "Methodology" section elaborates on this study's methodology. " Data preparation " section describes the image dataset source and approach taken to prepare the dataset. "Neural network architecture" section describe the SNN's and CNN's architecture, proposed in this study, for criminal tendency recognition through facial images. "Visual criminal tendency detection results an discussion" section presents the results for both mixed gender and male only classification scenarios. "Contusion and feather directions" section concludes the paper by discussing the results and feature directions.
Related work
Machine learning has shown to be more effective then humans is discovering personality traits through facial images [4]. Geng et al. [5] trained a machine to estimate the age through facial image. Reese and Danforth [6] applied an ensemble for machine learning models and image processing to detect depression and psychiatric disorder in Instagram facial images.
The goal of facial emotion detection is to train a machine to distinguish among six emotional facial expression: happiness, surprise, sadness, disgust, anger, and fear [7]. Fuzzy inference system [8], hidden Markov model based on real-time tracking of the mouth shape [9], and Bayesian network [10] are among the approaches use for classifying facial emotions.
Criminal tendency another personality trait. Lombroso [2] was the first in 1871 to point out that criminal could be identified by their facial structure and emotions. Recently , Wu and Zhang [11] revisited this theory and quantitatively demonstrated the correlation between criminality and facial features. They trained four classifiers: logistic regressing, k nearest neighbor (KNN), support vector machines (SVM) convolutional neural network (CNN) and claimed that their machine identify a criminal face with a 90% accuracy. Their models was controlled for gender, race, and facial expression of emotion.
Neural networks have resurged and drown much attention in the last decade [12] with the new brand of deep learning, mainly due to the significant performance gain n visual recognition [13]. Deep learning has been applied to a wide range of applications, such as three disease recognition [14]. Among the most relevant applications of deep learning to our work, we can point to the application of CNN for force recognition [15, 16]. Cristani [17] and segalin et al. [18, 19] applied machines learning to predict the self-assessed personalty traits (openness to experience, conscientiousness, extroversion, agreeableness, and neuroticism) of a person from the image he/she uploads or likes on social media, such as Flicker, and what impressions in terms of personality traits those images trigger in unacquainted people. They performed their experiments with 60,000 images from 300 Flickers users. Cristani et al. [17] and segalin et al. [18] used a hybrid approach where generative models, used as latent representations of features (color, composition, textural, properties, etc.) extracted from images, are built and then passed to a discriminative classifier to predict each user's personality traits. Simplifying the problem into five distinct binary classification problems, one of each trait, segalin et al. [19] applied AlexNet [13], which is an eight-layer versions of CNN, pre-trained on ImageNet 2012 competition dataset. The problem they pose is to detect the personalty traits based on the images that one uploads or likes on social media, such as Flicker. Their results showed that the personality traits that others attribute to a person (based on the image that the individual uploads or likes on social media) can be predicated 10% more accurately then the personality traits that the individual attributes to him/her-self Wang and Kosinski [4] trained a deep neural network to classify facial images based on sexual orientation.
Comments
Post a Comment