International Journal of Image, Graphics and Signal Processing (IJIGSP)
IJIGSP Vol. 9, No. 5, 8 May 2017
Cover page and Table of Contents: PDF (size: 612KB)
Author(s)
Index Terms
Face super-resolution, Face Hallucination, Reconstruction, Learning
Abstract
Accurate recognition and tracking of human faces are indispensable in applications like Face Recognition, Forensics, etc. The need for enhancing the low resolution faces for such applications has gathered more attention in the past few years. To recognize the faces from the surveillance video footage, the images need to be in a significantly recognizable size. Image Super-Resolution (SR) algorithms aid in enlarging or super-resolving the captured low-resolution image into a high-resolution frame. It thereby improves the visual quality of the image for recognition. This paper discusses some of the recent methodologies in face super-resolution (FSR) along with an analysis of its performance on some benchmark databases. Learning based methods are by far the immensely used technique. Sparse representation techniques, Neighborhood-Embedding techniques, and Bayesian learning techniques are all different approaches to learning based methods. The review here demonstrates that, in general, learning based techniques provides better accuracy/ performance even though the computational requirements are high. It is observed that Neighbor Embedding provides better performances among the learning based techniques. The focus of future research on learning based techniques, such as Neighbor Embedding with Sparse representation techniques, may lead to approaches with reduced complexity and better performance.
Cite This Paper
Sithara Kanakaraj, V.K. Govindan, Saidalavi Kalady,"Face Super Resolution: A Survey", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.9, No.5, pp.54-67, 2017. DOI: 10.5815/ijigsp.2017.05.06
Reference
[1]K. Nasrollahi and T. B. Moeslund, “Super-resolution: a comprehensive survey,” Machine vision and applications, vol. 25, no. 6, pp. 1423–1468, 2014.
[2]S. Baker and T. Kanade, “Hallucinating faces,” in Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000. Proceedings, pp. 83–88, IEEE, 2000.
[3]C. Liu, H.-Y. Shum, and C.-S. Zhang, “A two-step approach to hallucinating faces: global parametric model and local nonparametric model,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. vol. 1, pp. I–192.
[4]W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” Computer Graphics and Applications, IEEE, vol. 22, no. 2, pp. 56–65, 2002.
[5]F. W. Wheeler, X. Liu, and P. H. Tu, “Multi-frame super-resolution for face recognition,” BTAS 2007. First IEEE International Conference on in Biometrics: Theory, Applications, and Systems, 2007, pp. 1–6, IEEE, 2007.
[6]P. H. Hennings, Yeomans, S. Baker, and B. Kumar, “Recognition of low-resolution faces using multiple still images and multiple cameras,” in. BTAS 2008. 2nd IEEE International Conference on Biometrics: Theory, Applications and Systems, 2008, pp. 1–6, IEEE, 2008.
[7]Z. Jia, J. Zhao, H. Wang, Z. Xiong, and A. Finn, “A two-step face hallucination approach for video surveillance applications,” Multimedia Tools and Applications, vol. 74, no. 6, pp. 1845–1862, 2015.
[8]S. Shijo and V. Govindan, “An improved approach to super-resolution image reconstruction,” in International Journal of Engineering Research and Technology, vol. 3, ESRSA Publications, December 2014.
[9]J. Satiro, K. Nasrollahi, P. L. Correia, and T. B. Moeslund, “Super-resolution of facial images in forensics scenarios,” in 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 55–60, IEEE, 2015.
[10]P. H. Hennings, Yeomans, S. Baker, and B. Kumar, “Simultaneous super-resolution and feature extraction for recognition of low-resolution faces,”. CVPR 2008. IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
[11]C. Wang, W. Li, L. Wang, and Q. Liao, “An effective framework of ibp for single facial image super resolution,” 2013.
[12]S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290, no. 5500, pp. 2323–2326, 2000.
[13]S. Tang, L. Xiao, P. Liu, and H. Wu, “Coupled learning based on singular-values-unique and hog for face hallucination,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1315–1319.
[14]X. Ma, J. Zhang, and C. Qi, “Hallucinating face by position-patch,” Pattern Recognition, vol. 43, no. 6, pp. 2224–2236, 2010.
[15]J. Yang, H. Tang, Y. Ma, and T. Huang, “Face hallucination via sparse coding,” 15th IEEE International Conference on Image Processing, ICIP 2008. pp. 1264–1267, IEEE, 2008.
[16]D. D. Lee and H. S. Seung, “Learning the parts of objects by non-negative matrix factorization,” Nature, vol. 401, no. 6755, pp. 788–791, 1999.
[17]J. Jiang, R. Hu, Z. Wang, and Z. Han, “Noise robust face hallucination via locality-constrained representation,” IEEE Transactions on Multimedia, vol. 16, no. 5, pp. 1268–1281, 2014.
[18]L. Zhang, M. Yang, and X. Feng, “Sparse representation or collaborative representation: Which helps face recognition?,”, 2011 IEEE International Conference on Computer Vision (ICCV), pp. 471–478, IEEE,2011.
[19]Q. Zhang, F. Zhou, F. Yang, and Q. Liao, “Face super-resolution via semi-kernel partial least squares and dictionaries coding,”, 2015 IEEE International Conference on Digital Signal Processing (DSP), pp. 590–594, IEEE, 2015.
[20]F. Juefei, Xu and M. Savvides, “Single face image super-resolution via solo dictionary learning,” in (ICIP), 2015 IEEE International Conference on Image Processing, pp. 2239–2243, IEEE, 2015.
[21]S. Qu, R. Hu, S. Chen, Z. Wang, J. Jiang, and C. Yang, “Face hallucination via cauchy regularized sparse representation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 1216–1220, IEEE, 2015.
[22]Z. Wang, R. Hu, S. Wang, and J. Jiang, “Face hallucination via weighted adaptive sparse regularization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 5, pp. 802–813, 2014.
[23]H. Chang, D.-Y. Yeung, and Y. Xiong, “Super-resolution through neighbor embedding,” in CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004., vol. 1, pp. I–I, IEEE, 2004.
[24]L. Fang and D. Yu, “Hallucinating facial image based on adaptive neighbourhood selection,” in AASRI International Conference on Industrial Electronics and Applications (IEA 2015), Atlantis Press, 2015.
[25]Y. Deng and F. Liu, “Hallucinating faces based on adaptive neighborhood selection and dual tree complex wavelet transform,” Optik-International Journal for Light and Electron Optics, vol. 127, no. 2, pp. 525–534, 2016.
[26]V. Nguyen, C.-C. Hung, and X. Ma, “Super resolution face image based on locally linear embedding and local correlation,” ACM SIGAPP Applied Computing Review, vol. 15, no. 1, pp. 17–25, 2015.
[27]Y. Li, J. Liu, W. Yang, and Z. Guo, “Multi-pose face hallucination via neighbor embedding for facial components,” in (ICIP), 2015 IEEE International Conference on Image Processing, pp. 4037–4041, IEEE,2015.
[28]S. Song, Y. Li, Z. Gao, and J. Liu, “Face hallucination based on neighbor embedding via illumination adaptation,” in Proceedings of APSIPA Annual Summit and Conference, vol. 16, 2015.
[29]M. Tanveer and N. Iqbal, “A bayesian approach to face hallucination using dlpp and krr,” in (ICPR), 2010 20th International Conference on Pattern Recognition, pp. 2154–2157, IEEE, 2010.
[30]C.-C. Hsu, C.-W. Lin, C.-T. Hsu, H.-Y. M. Liao, and J.-Y. Yu, “Face hallucination using bayesian global estimation and local basis selection,” in (MMSP), 2010 IEEE International Workshop on Multimedia Signal Processing, pp. 449–453, IEEE, 2010.
[31]M. F. Tappen and C. Liu, “A bayesian approach to alignment-based image hallucination,” in Computer Vision–ECCV 2012, pp. 236–249, Springer, 2012.
[32]P. Innerhofer and T. Pock, “A convex approach for image hallucination,” arXiv preprint arXiv:1304.7153, 2013.
[33]A. Chambolle and T. Pock, “A first-order primal-dual algorithm for convex problems with applications to imaging,” Journal of Mathematical Imaging and Vision, vol. 40, no. 1, pp. 120–145, 2011.
[34]M. Li, D. Xu, R. Yi, and X. He, “Face hallucination based on nonparametric bayesian learning,” in (ICIP), 2015 IEEE International Conference on Image Processing, pp. 986–990, IEEE, 2015.
[35]J. Tang, C. Huang, Y. Liu, H. Fan, and J. Zuo, “Face super-resolution algorithm based on svm-improved learning,” Computer Modelling and New Technologies, 2014.
[36]C.-T. Tu, M.-C. Ho, and J.-R. Luo, “Face hallucination through ensemble learning,” in (DSP), 2015 IEEE International Conference on Digital Signal Processing, pp. 1242–1245, IEEE, 2015.
[37]F. Liu and Y. Deng, “Face hallucination based on two-dimensional joint learning,” Digital Signal Processing, vol. 48, pp. 310–321, 2016.
[38]E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin, “Learning face hallucination in the wild,” in Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
[39]C. H. J. Tang, “A human face super-resolution reconstruction algorithm based on niche genetic algorithm,” Advanced Science and Technology Letters, vol. 121, pp. 1–5, 2016.
[40]S. Shijo and V. K. Govindan, “Optimization of mathematical morphology based super-resolution image reconstruction using genetic algorithm,” International Journal of Advanced Information Science and Technology, vol. 41, no. 41, 2015.
[41]W. Zhang and W.K. Cham, “Hallucinating face in the dct domain,”, IEEE Transactions on Image Processing, vol. 20, no. 10, pp. 2769–2779, 2011.
[42]Z. Hui and K.-M. Lam, “Eigentransformation-based face super-resolution in the wavelet domain,” Pattern Recognition Letters, vol. 33, no. 6, pp. 718–727, 2012.
[43]X. Wang, Y. Zhang, and X. Guo, “Very low resolution face image super-resolution based on dct,” Journal of Information and Computational Science, vol. 11, no. 11, pp. 3807–3813, 2014.