Share / Export Citation / Email / Print / Text size:

International Journal on Smart Sensing and Intelligent Systems

Professor Subhas Chandra Mukhopadhyay

Exeley Inc. (New York)

Subject: Computational Science & Engineering, Engineering, Electrical & Electronic


eISSN: 1178-5608



VOLUME 8 , ISSUE 3 (September 2015) > List of articles


Shaoping Zhu *

Keywords : Facial expression recognition, Active Appearance Model (AAM), Bag of Words model, LDA model, computer vision.

Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 8, Issue 3, Pages 1,464-1,483, DOI:

License : (CC BY-NC-ND 4.0)

Received Date : 15-April-2015 / Accepted: 01-August-2015 / Published Online: 01-September-2015



Automatic facial expression recognition from video sequence is an essential research area in the field of computer vision. In this paper, a novel method for recognition facial expressions is proposed, which includes two stages of facial expression feature extraction and facial expression recognition. Firstly, in order to exact robust facial expression features, we use Active Appearance Model (AAM) to extract the global texture feature and optical flow technique to characterize facial expression which is determined facial velocity information. Then, these two features are integrated and converted to visual words using “bag-of-words” models, and facial expression is represented by a number of visual words. Secondly, the Latent Dirichlet Allocation (LDA) model are utilized to classify different facial expressions such as “anger”, “disgust”, “fear”, “happiness”, “neutral”, “sadness”, and “surprise”. The experimental results show that our proposed method not only performs stably and robustly and improves the recognition rate efficiently, but also needs the least dimension when achieves the highest recognition rate , which demonstrates that our proposed method is superior to others.

Content not available PDF Share



[1] A. S. Ghotkar and G. K. Kharat, “Study of vision based hand gesture recognition using Indian sign language,” International Journal on Smart Sensing and Intelligent Systems, vol.7, no.1, 2014, pp. 96-114.
[2] Y. Wang and Y. Zhang, “Object tracking based on machine vision and Improved svdd algorithm,” International Journal on Smart Sensing and Intelligent Systems, vol.8, no.1, 2015, pp.677-696.
[3] F. Samaria and S. Young, “HMM-based architecture for face identification,” Image and Vision Computing, vol.12, No.8, 1994, pp. 537-543.
[4] C. Shan, S. Gong and P. W. McOwan, “Facial expression recognition based on local binary patterns: A comprehensive study,” Image and Vision Computing, vol.27, No.6, 2009, pp. 803-816.
[5] X. Feng, M. Pietikäinen and A. Hadid, “Facial expression recognition based on local binary patterns,” Pattern Recognition and Image Analysis, vol.17, No.4, 2007, pp.592-598.
[6] T. Xiang, M. K. H. Leung and S. Y. Cho, “Expression recognition using fuzzy spatio-temporal modeling,” Pattern Recognition, vol.41, No.1, 2008, pp.204-216.
[7] N. Neggaz, M. Besnassi and A. Benyettou, “Facial expression recognition,” Journal of Applied Sciences, vol.10, No.15, 2010, pp. 1572-79.
[8] G. Zhao and M. Pietikäinen, “Boosted multi-resolution spatiotemporal descriptors for facial expression recognition,” Pattern recognition letters, vol.30, No.12, 2009, pp. 1117-27.
[9] K. Yu, Z. Wang, L. Zhuo, et al, “Learning realistic facial expressions from web images,” Pattern Recognition, vol.46, No.8, 2013, pp.2144-2155.
[10] S. Y. Fu, G. S. Yang and X. K. Kuai, “A spiking neural network based cortex-like mechanism and application to facial expression recognition,” Computational Intelligence and Neuroscience, pp.1-13. Online publication date: 1-Jan-2012.
[11] M. N. Dailey, G. W. Cottrell, C. Padgett, et al, “EMPATH: A neural network that categorizes facial expressions,” Journal of Cognitive Neuroscience, vol.14, No.8, 2002, 1158-1173.
[12] D. C. Turk, C. Dennis and R. Melzack, “The measurement of pain and the assessment of people experiencing pain,” Handbook of Pain Assessment, ed D. C. Turk and R. Melzack, New York: Guilford, 2nd edition: 2001, pp. 1-11.
[13] L. Wang, R. F. Li, and K. Wang, “A novel automatic facial expression recognition method based on AAM,” Journal of Computers, vol.9, No.3, 2014, pp.608-617.
[14] K. M. Prkachin, “The consistency of facial expressions of pain: a comparison across modalities,” Pain, vol.3, No.5, 1992, pp. 297-306.
[15] K. M. Prkachin and P. E. Solomon, “The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain,” Pain, vol.2, No.139, 2008, pp.267-274.
[16] S. Zhang, B. Jiang and T. Wang, “Facial expression recognition algorithm based on active shape model and gabor wavelet,” Journal of Henan University (Natural Science), vol.40, No.5, 2010, pp. 521-524.
[17] W. Zhang and L. M. Xia, “Pain expression recognition based on SLPP and MKSVM,” Int. J. Engineering and Manufacturing, No. 3, 2011, pp.69-74.
[18] K. W. Wan, K. M. Lam, and K. C. Ng, “An accurate active shape model for facial feature extraction,” Pattern Recognition Letters, vol.26, No. 15, 2005, pp.2409-23.
[19] T. F. Cootes, G. J. Edwards, C. J. Taylor, “Active appearance models,” Computer Vision—ECCV’98, Springer Berlin Heidelberg, 1998, pp. 484-498.
[20] B. Fasel and J. Luettin, “Automatic facial expression analysis: a survey,” Pattern recognition, vol.36, No.1, 2003, pp. 259-275.
[21] B. Lucas and T. Kanade, “An iterative image restoration technique with an application to sereo vision,” Proceedings of the DARPA IU Workshop, 1981, pp.121-130.
[22] G. J. Burghouts and K. Schutte, “Spatio-temporal layout of human actions for improved bag-of-words action detection,” Pattern Recognition Letters, vol.34, No. 15, 2013, pp.1861-1869.
[23] D. M. Blei, A. Y. Ng and M. I. Jordan, “Latent dirichlet allocation,” The Journal of Machine Learning Research, No. 3, 2003, pp.993-1022.
[24] L. Shang and K. P Chan, “A temporal latent topic model for facial expression recognition,” Computer Vision–ACCV 2010, Springer Berlin Heidelberg, 2011, pp. 51-63.
[25] A. Bansal, S. Chaudhary and S. D. Roy, “A novel LDA and HMM-Based technique for emotion recognition from facial expressions,” Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction, Springer Berlin Heidelberg, 2013, pp. 19-26.
[26] F. Monay and D. Gatica-Perez, “Modeling semantic aspects for cross-media image indexing,” 2007 IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.29, No. 10, 2007, pp.1802-1817.
[27] G. Sen Gupta, S.C. Mukhopadhyay and M Finnie, Wi-Fi Based Control of a Robotic Arm with Remote Vision, Proceedings of 2009 IEEE I2MTC Conference, Singapore, May 5-7, 2009, pp. 557-562.
[28] H. Zhang, Z. Liu and H. Zhao, “Human activities for classification via feature points,” Information Technology Journal, vol.10, No. 5, 2011, pp.974-982.
[29] J. C. Niebles, H. Wang and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” International Journal of Computer Vision, vol.79, No3, 2008, pp.299-318.
[30] F. Cheng, J. Yu and H. Xiong, “Facial expression recognition in JAFFE dataset based on Gaussian process classification,” IEEE Transactions on Neural Networks, vol.21, No.10, 2010, pp.1685-1690.