FPGA-Based Implementation of Real Time Optical Flow Algorithm and Its Applications for Digital Image Stabilization


Share / Export Citation / Email / Print / Text size:

International Journal on Smart Sensing and Intelligent Systems

Professor Subhas Chandra Mukhopadhyay

Exeley Inc. (New York)

Subject: Computational Science & Engineering , Engineering, Electrical & Electronic


eISSN: 1178-5608



VOLUME 3 , ISSUE 2 (June 2010) > List of articles

FPGA-Based Implementation of Real Time Optical Flow Algorithm and Its Applications for Digital Image Stabilization

Robert Piotrowski * / Stanislaw Szczepanski * / Slawomir Koziel * / Robert Piotrowski * / Stanislaw Szczepanski * / Slawomir Koziel *

Keywords : Optical flow, correlation algorithm, digital image stabilization, FPGA

Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 3, Issue 2, Pages 253-272, DOI: https://doi.org/10.21307/ijssis-2017-391

License : (CC BY-NC-ND 4.0)

Published Online: 12-December-2017



An efficient simplification procedure of the optical flow (OF) algorithm as well as its hardware implementation using the field programmable gate array (FPGA) technology is presented. The modified algorithm is based on block matching of subsets of successive frames, and exploits one-dimensional representation of subsets as well as the adaptive adjustments of their sizes. Also, an l1-norm-based correlation function requiring no multiplication/division operations is used. As a result, it was possible to reduce the computational complexity of the algorithm without compromising the processing accuracy. Both the accuracy and the limitations resulting from the introduced simplifications have been verified based on several examples of both synthetic and real movie samples. The presented algorithm has been implemented using VirtexII-1000 FPGA to realize a digital stabilization system for the CMOS camera images. Experimental results fully confirm the efficiency of the presented algorithm when working with limited computational resources. This demonstrates the possibility of using our algorithm in the autonomous navigation and other battery-powered systems.

Content not available PDF Share



[1]S. Chhaniyara, P. Bunnun, L. Seneviratne and M. Ichikawa, ”Optical Flow Algorith for Velocity Estimation of Ground Vehicles: A Feasibility Study”, International Journal on Smart Sensing and Intelligent Systems, Vol. 1, No. 1, March 2008.
[2]G. Barrows, C. Neely: ”Mixed-mode VLSI optic flow sensors for inflight control of a micro air vehicle”, Proc. SPIE Vol. 4109, Critical Technologies for the Future of Computing; pp. 52-63, 2000.
[3]Diaz, J., Ros, E., Pelayo, F., Ortigosa, E. M., Mota, S.: ”FPGA based realtime optical-flow system”; IEEE Transactions on Circuits and Systems for Video Technology, 16, 2 (2006), 274-279.
[4]B. McCane, K. Novins, D. Crannitch, and B. Galvin, ”On benchmarking optical flow”, Comput. Vis. Image Understanding, vol. 84, pp. 126-143, 2001.
[5]Barron J. L, Fleet D. J, and Beauchemin S. S. , ”Performance of optical flow techniques.” International Journal of Computer Vision, Vol. 12, pp. 43-77, 1994.
[6]B.D. Lucas and T. Kanade, ”An Iterative Image Registration Technique with an Application to Stereo Vision (IJCAI),” Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI ’81), April, 1981, pp. 674-679.
[7]Jangheon Kim, Thomas Sikora: ”Hybrid recursive energy-based method for robust optical flow on large motion fields.”, ICIP (1) 2005: 129-132
[8]Bruhn, J. Weickert, C. Feddern, T. Kohlberger, C. Schnrr, “Real- Time Optic Flow Computation with Variational Methods”, CAIP 2003, LNCS, vol. 2756, pp. 222-229, 2003.
[9]P. Cobos and F. Monasterio, ”FPGA implementation of the Horn & Schunck optical flow algorithm for motion detection in real time images”, Proc. DCIS’98 XIII, pp.616-621, 1998.
[10]S.J. Ko, S.H. Lee, and K.H. Lee, ”Digital image stabilizing algorithms based on bit-plane matching”, IEEE Trans. Consumer Electron., vol. 44, no. 3, pp.796-800, Aug., 1998.
[11]L. Xu and X. Lin, ”Digital image stabilization based on circular block matching”, IEEE Transactions on Consumer Electronics, vol. 52, no. 2, pp. 566-574, 2006.
[12]N. Ancona and T. Poggio, ”Optical Flow from 1D Correlation: Application to a simple Time-To-Crash Detector”, Fourth International Conference on Computer Vision, IEEE Computer Society Press, May 11-14, 1993, Berlin, Germany, pp. 209-214.
[13]K. Janschek, V. Tchernykh, M. Beck: ”Optical Flow based Navigation for Mobile Robots using an Embedded Optical Correlator”, Preprints of the 3rd IFAC Conference on Mechatronic Systems - Mechatronics 2004, 6-8 September 2004, Sydney, Australia, pp.793-798.
[14]P. C. Arribas and F. M. H. Maciá. FPGA implementation of the Horn & Shunk Optical Flow Algorithm for Motion Detection in real time Images. Proceedings of the XIII Design of Circuits and Integrated Systems Confer- ence, pages 616_621, 1998.
[15]S. Erturk, ”Digital image stabilization with sub-image phase correlation based global motion estimation”, IEEE Trans. Consumer Electron., vol. 49, no. 4, pp.1320-1325, Nov., 2003.
[16]Intel Corporation, ”Open Source Computer Vision Library”, Intel Corporation, http://developer.intel.com, 2000.
[17]Xilinx Company, ”Virtex-II Pro and Virtex-II Pro X FPGA User Guide”, 2004.
[18]Z. Wei, D.-J. Lee, B. Nelson, James K. Archibald, and Barrett B. Edwards, ”FPGA-Based Embedded Motion Estimation Sensor”, International Journal of Reconfigurable Computing Volume 2008 (2008).
[19]Z. Wei, D.-J. Lee, B. Nelson, and M. Martineau, “A fast and accurate tensor-based optical flow algorithm implemented in FPGA,” in Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV ’07), p. 18, Austin, Tex, USA, February 2007.
[20]S. Baker, D. Scharstein, J.P. Lewis, S. Roth, M. J. Black, and R. Szeliski, “A Database and Evaluation Methodology for Optical Flow”, IEEE International Conference on Computer Vision, Oct. 2007.