Human-Computer Interaction: Overview on State of the Art


Share / Export Citation / Email / Print / Text size:

International Journal on Smart Sensing and Intelligent Systems

Professor Subhas Chandra Mukhopadhyay

Exeley Inc. (New York)

Subject: Computational Science & Engineering, Engineering, Electrical & Electronic


eISSN: 1178-5608



VOLUME 1 , ISSUE 1 (March 2008) > List of articles

Human-Computer Interaction: Overview on State of the Art

Fakhreddine Karray * / Milad Alemzadeh * / Jamil Abou Saleh * / Mo Nours Arab *

Keywords : Human-Computer Interaction, Multimodal HCI, Ubiquitous Computing

Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 1, Issue 1, Pages 137-159, DOI:

License : (CC BY-NC-ND 4.0)

Published Online: 13-December-2017



The intention of this paper is to provide an overview on the subject of Human-Computer Interaction. The overview includes the basic definitions and terminology, a survey of existing technologies and recent advances in the field, common architectures used in the design of HCI systems which includes unimodal and multimodal configurations, and finally the applications of HCI. This paper also offers a comprehensive number of references for each concept, method, and application in the HCI.

Content not available PDF Share



[1] D. Te’eni, J. Carey and P. Zhang, Human Computer Interaction: Developing Effective Organizational Information Systems, John Wiley & Sons, Hoboken (2007).
[2] B. Shneiderman and C. Plaisant, Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th edition), Pearson/Addison-Wesley, Boston (2004).
[3] J. Nielsen, Usability Engineering, Morgan Kaufman, San Francisco (1994).
[4] D. Te’eni, “Designs that fit: an overview of fit conceptualization in HCI”, in P. Zhang and D. Galletta (eds), Human-Computer Interaction and Management Information Systems: Foundations, M.E. Sharpe, Armonk (2006).
[5] A. Chapanis, Man Machine Engineering, Wadsworth, Belmont (1965).
[6] D. Norman, “Cognitive Engineering”, in D. Norman and S. Draper (eds), User Centered Design: New Perspective on Human-Computer Interaction, Lawrence Erlbaum, Hillsdale (1986).
[7] R.W. Picard, Affective Computing, MIT Press, Cambridge (1997).
[8] J.S. Greenstein, “Pointing devices”, in M.G. Helander, T.K. Landauer and P. Prabhu (eds), Handbook of Human-Computer Interaction, Elsevier Science, Amsterdam (1997).
[9] B.A. Myers, “A brief history of human-computer interaction technology”, ACM interactions, 5(2), pp 44-54 (1998).
[10] B. Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction (3rd edition), Addison Wesley Longman, Reading (1998).
[11] A. Murata, “An experimental evaluation of mouse, joystick, joycard, lightpen, trackball and touchscreen for Pointing - Basic Study on Human Interface Design”, Proceedings of the Fourth International Conference on Human-Computer Interaction 1991, pp 123-127 (1991).
[12] L.R. Rabiner, Fundamentals of Speech Recognition, Prentice Hall, Englewood Cliffs (1993).
[13] C.M. Karat, J. Vergo and D. Nahamoo, “Conversational interface technologies”, in J.A. Jacko and A. Sears (eds), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Application, Lawrence Erlbaum Associates, Mahwah (2003).
[14] S. Brewster, “Non speech auditory output”, in J.A. Jacko and A. Sears (eds), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Application, Lawrence Erlbaum Associates, Mahwah (2003).
[15] G. Robles-De-La-Torre, “The Importance of the sense of touch in virtual and real environments”, IEEE Multimedia 13(3), Special issue on Haptic User Interfaces for Multimedia Systems, pp 24-30 (2006).
[16] V. Hayward, O.R. Astley, M. Cruz-Hernandez, D. Grant and G. Robles-De-La-Torre, “Haptic interfaces and devices”, Sensor Review 24(1), pp 16-29 (2004).
[17] J. Vince, Introduction to Virtual Reality, Springer, London (2004).
[18] H. Iwata, “Haptic interfaces”, in J.A. Jacko and A. Sears (eds), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Application, Lawrence Erlbaum Associates, Mahwah (2003).
[19] W. Barfield and T. Caudell, Fundamentals of Wearable Computers and Augmented Reality, Lawrence Erlbaum Associates, Mahwah (2001).
[20] M.D. Yacoub, Wireless Technology: Protocols, Standards, and Techniques, CRC Press, London (2002).
[21] K. McMenemy and S. Ferguson, A Hitchhiker’s Guide to Virtual Reality, A K Peters, Wellesley (2007).
[22] Global Positioning System, “Home page”,, visited on 10/10/2007.
[23] S.G. Burnay, T.L. Williams and C.H. Jones, Applications of Thermal Imaging, A. Hilger, Bristol (1988).
[24] J. Y. Chai, P. Hong and M. X. Zhou, “A probabilistic approach to reference resolution in multimodal user interfaces”, Proceedings of the 9th International Conference on Intelligent User Interfaces, Funchal, Madeira, Portugal, pp 70-77 (2004).
[25] E.A. Bretz, “When work is fun and games”, IEEE Spectrum, 39(12), pp 50-50 (2002).
[26] ExtremeTech, “Canesta says “Virtual Keyboard” is reality”,,1558,539778,00.asp, visited on 15/10/2007.
[27] G. Riva, F. Vatalaro, F. Davide and M. Alaniz, Ambient Intelligence: The Evolution of Technology, Communication and Cognition towards the Future of HCI, IOS Press, Fairfax (2005).
[28] M.T. Maybury and W. Wahlster, Readings in Intelligent User Interfaces, Morgan Kaufmann Press, San Francisco (1998).
[29] A. Kirlik, Adaptive Perspectives on Human-Technology Interaction, Oxford University Press, Oxford (2006).
[30] S.L. Oviatt, P. Cohen, L. Wu, J. Vergo, L. Duncan, B. Suhm, J. Bers, T. Holzman, T. Winograd, J. Landay, J. Larson and D. Ferro, “Designing the user interface for multimodal speech and pen-based gesture applications: state-of-the-art systems and future research directions”, Human-Computer Interaction, 15, pp 263-322 (2000).
[31] D.M. Gavrila, “The visual analysis of human movement: a survey”, Computer Vision and Image Understanding, 73(1), pp 82-98 (1999).
[32] L.E. Sibert and R.J.K. Jacob, “Evaluation of eye gaze interaction”, Conference of Human-Factors in Computing Systems, pp 281-288 (2000).
[33] Various Authors, “Adaptive, intelligent and emotional user interfaces”, Part II of HCI Intelligent Multimodal Interaction Environments, 12th International Conference, HCI International 2007 (Proceedings Part III), Springer Berlin, Heidelberg (2007).
[34] M.N. Huhns and M.P. Singh (eds), Readings in Agents, Morgan Kaufmann, San Francisco (1998).
[35] C.S. Wasson, System Analysis, Design, and Development: Concepts, Principles, and Practices, John Wiley & Sons, Hoboken (2006).
[36] A. Jaimes and N. Sebe, “Multimodal human computer interaction: a survey”, Computer Vision and Image Understanding, 108(1-2), pp 116-134 (2007).
[37] I. Cohen, N. Sebe, A. Garg, L. Chen and T.S. Huang, “Facial expression recognition from video sequences: temporal and static modeling”, Computer Vision and Image Understanding, 91(1-2), pp 160-187 (2003).
[38] B. Fasel and J. Luettin, “Automatic facial expression analysis: a survey”, Pattern Recognition, 36, pp 259-275 (2003).
[39] M. Pantic and L.J.M. Rothkrantz, “Automatic analysis of facial expressions: the state of the art”, IEEE Transactions on PAMI, 22(12), pp 1424-1445 (2000).
[40] J.K. Aggarwal and Q. Cai, “Human motion analysis: a review”, Computer Vision and Image Understanding, 73(3), pp 428-440 (1999).
[41] S. Kettebekov and R. Sharma, “Understanding gestures in multimodal human computer interaction”, International Journal on Artificial Intelligence Tools, 9(2), pp 205-223 (2000).
[42] Y. Wu and T. Huang., “Vision-based gesture recognition: a review”, in A. Braffort, R. Gherbi, S. Gibet, J. Richardson and D. Teil (eds), Gesture-Based Communication in Human-Computer Interaction, volume 1739 of Lecture Notes in Artificial Intelligence, Springer-Verlag, Berlin/Heidelberg (1999).
[43] T. Kirishima, K. Sato and K. Chihara, “Real-time gesture recognition by learning and selective control of visual interest points”, IEEE Transactions on PAMI, 27(3), pp 351-364 (2005).
[44] R. Ruddaraju, A. Haro, K. Nagel, Q. Tran, I. Essa, G. Abowd and E. Mynatt, “Perceptual user interfaces using vision-based eye tracking”, Proceedings of the 5th International Conference on Multimodal Interfaces, Vancouver, pp 227-233 (2003).
[45] A.T. Duchowski, “A breadth-first survey of eye tracking applications”, Behavior Research Methods, Instruments, and Computers, 34(4), pp 455-470 (2002).
[46] P. Rubin, E. Vatikiotis-Bateson and C. Benoit (eds.), “Special issue on audio-visual speech processing”, Speech Communication, 26, pp 1-2 (1998).
[47] J.P. Campbell Jr., “Speaker recognition: a tutorial”, Proceedings of IEEE, 85(9), pp 1437-1462 (1997).
[48] P.Y. Oudeyer, “The production and recognition of emotions in speech: features and algorithms”, International Journal of Human-Computer Studies, 59(1-2), pp 157-183 (2003).
[49] L.S. Chen, Joint Processing of Audio-Visual Information for the Recognition of Emotional Expressions in Human-Computer Interaction, PhD thesis, UIUC, (2000).
[50] M. Schröder, D. Heylen and I. Poggi, “Perception of non-verbal emotional listener feedback”, Proceedings of Speech Prosody 2006, Dresden, Germany, pp 43-46 (2006).
[51] M.J. Lyons, M. Haehnel and N. Tetsutani, “Designing, playing, and performing, with a vision-based mouth interface”, Proceedings of the 2003 Conference on New Interfaces for Nusical Expression, Montreal, pp 116-121 (2003).
[52] D. Göger, K. Weiss, C. Burghart and H. Wörn, “Sensitive skin for a humanoid robot”, Human-Centered Robotic Systems (HCRS’06), Munich, (2006).
[53] O. Khatib, O. Brock, K.S. Chang, D. Ruspini, L. Sentis and S. Viji, “Human-centered robotics and interactive haptic simulation”, International Journal of Robotics Research, 23(2), pp 167-178 (2004).
[54] C. Burghart, O. Schorr, S. Yigit, N. Hata, K. Chinzei, A. Timofeev, R. Kikinis, H. Wörn and U. Rembold, “A multi-agent system architecture for man-machine interaction in computer aided surgery”, Proceedings of the 16th IAR Annual Meeting, Strasburg, pp 117-123 (2001).
[55] A. Legin, A. Rudnitskaya, B. Seleznev and Yu. Vlasov, “Electronic tongue for quality assessment of ethanol, vodka and eau-de-vie”, Analytica Chimica Acta, 534, pp 129-135 (2005).
[56] S. Oviatt, “Multimodal interfaces”, in J.A. Jacko and A. Sears (eds), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Application, Lawrence Erlbaum Associates, Mahwah (2003).
[57] R.A. Bolt, “Put-that-there: voice and gesture at the graphics interface”, Proceedings of the 7th Annual Conference on Computer Graphics and Interactive Techniques, Seattle, Washington, United States, pp 262-270 (1980).
[58] M. Johnston and S. Bangalore, “MATCHKiosk: a multimodal interactive city guide”, Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions, Barcelona, Spain, Article No. 33, (2004).
[59] I. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard and D. Zhang, “Automatic analysis of multimodal group actions in meetings”, IEEE Transactions on PAMI, 27(3), pp 305-317 (2005).
[60] S. Meyer and A. Rakotonirainy, “A Survey of research on context-aware homes”, Australasian Information Security Workshop Conference on ACSW Frontiers, pp 159-168 (2003).
[61] P. Smith, M. Shah and N.D.V. Lobo, “Determining driver visual attention with one camera”, IEEE Transactions on Intelligent Transportation Systems, 4(4), pp 205-218 (2003).
[62] K. Salen and E. Zimmerman, Rules of Play: Game Design Fundamentals, MIT Press, Cambridge (2003).