Citations


Contact Us

For Marketing, Sales and Subscriptions Inquiries
2637 E Atlantic Blvd #43110
Pompano Beach, FL 33062
USA

Conference List

Review of Computer Engineering Research

December 2020, Volume 7, 2, pp 86-95

An Overview of Advances in Image Colorization Using Computer Vision and Deep Learning Techniques

Rashi Dhir

,

Meghna Ashok

,

Shilpa Gite

Rashi Dhir 1 Meghna Ashok 1 Shilpa Gite 1 ,
;

  1. CS Department, Symbiosis Institute of Technology, Symbiosis International (Deemed University), Pune, India. 1

Pages: 86-95

DOI: 10.18488/journal.76.2020.72.86.95

Share :

Article History:

Received: 18 September, 2020
Revised: 08 October, 2020
Accepted: 27 October, 2020
Published: 16 November, 2020


Abstract:

Automatic image colorization as a process has been studied extensively over the past 10 years with importance given to its many applications in grayscale image colorization, aged/degraded image restoration etc. In this study, we attempt to trace and consolidate developments made in Image colorization using various computer vision techniques and methodologies, focusing on the emergence and performance of Generative Adversarial Networks (GANs). We talk in depth about GANs and CNNs, namely their structure, functionality and extent of research. Additionally, we explore the advances made in image colorization using other Deep Learning frameworks ranging from LeNets to MobileNets in order of their evolution in detail. We also compare existing published works showcasing new advancements and possibilities, and predominantly emphasize the importance of continuing research in image colorization. We further analyze and discuss potential applications and challenges of GANs to tackle in the future.
Contribution/ Originality
This study attempts to trace and consolidate developments made in Image colorization using various computer vision techniques and methodologies, focusing on the emergence and performance of Generative Adversarial Networks (GANs).

Keywords:

Deep learning, Computer vision architectures, Image colorization, Generative adversarial networks (GANs), Neural networks, Machine Learning.

Reference:

[1]           J. G. Ian, P.-A. Jean, M. Mehdi, X. Bing, W.-F. David, O. Sherjil, C. Aaron, and B. Yoshua, "Generative adversarial networks," arXiv: 1406.2661, 2014.

[2]           M. Y. Liu and O. Tuzel, "Coupled generative adversarial networks," In Advances in Neural Information Processing Systems, pp. 469-477, 2016.

[3]           T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation," arXiv preprint arXiv:1710.10196, 2017.

[4]           A. Brock, J. Donahue, and K. Simonyan, "Large scale gan training for high fidelity natural image synthesis," CoRR abs/1809.11096, 2018.

[5]           Y. Jin, J. Zhang, M. Li, Y. Tian, H. Zhu, and Z. Fang, "Towards the automatic anime characters creation with generative adversarial networks," arXiv preprint arXiv:1708.05509, 2017.

[6]           P. Isola, J. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," in In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1125-1134.

[7]           L. Ma, X. Jia, Q. Sun, B. Schiele, T. Tuytelaars, and L. Van Gool, "Pose guided person image generation," In Advances in Neural Information Processing Systems, pp. 406-416, 2017.

[8]           G. Antipov, M. Baccouche, and J. Dugelay, "Face aging with conditional generative adversarial networks," in 2017 IEEE International Conference on Image Processing (ICIP), Beijing, 2017, pp. 2089-2093.

[9]           C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, and Z. Wang, "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4681-4690.

[10]         C. Vondrick, H. Pirsiavash, and A. Torralba, "Generating videos with scene dynamics," In Advances in Neural Information Processing Systems, pp. 613-621, 2016.

[11]         R. Child, S. Gray, A. Radford, and I. Sutskever, "Generating long sequences with sparse transformers," arXiv preprint arXiv:1904.10509, 2019.

[12]         T. Park, M.-Y. Liu, T.-C. Wang, and J.-Y. Zhu, "GauGAN: semantic image synthesis with spatially adaptive normalization," In ACM SIGGRAPH 2019 Real-Time Live!, pp. 1-1, 2019.

[13]         B. Jan, H. Farman, M. Khan, M. Imran, I. U. Islam, A. Ahmad, S. Ali, and G. Jeon, "Deep learning in big data Analytics: A comparative study," Computers & Electrical Engineering, vol. 75, pp. 275-287, 2019.

[14]         M. Mirza and S. Osindero, "Conditional generative adversarial nets," arXiv preprint arXiv:1411.1784, 2014

[15]         A. Radford, L. Metz, and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv:1511.06434, 2015.

[16]         K. Nazeri, E. Ng, and M. Ebrahimi, "Image colorization with generative adversarial networks," presented at the In International Conference on Articulated Motion and Deformable Objects. Springer, Cham, 2018.

[17]         H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, and D. N. Metaxas, "StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks," in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 5907-5915.

[18]         B. Kratzwald, Z. Huang, D. P. Paudel, A. Dinesh, and L. Van Gool, "Improving video generation for multi-functional applications," arXiv preprint arXiv:1711.11453, 2017.

[19]         Y. Lecun, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, pp. 2278–2324, 1998.

[20]         A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," In Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.

[21]         D. C. Cireşan, U. Meier, L. M. Gambardella, and J. Schmidhuber, "Deep, big, simple neural nets for handwritten digit recognition," Neural Computation, vol. 22, pp. 3207-3220, 2010.

[22]         P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," arXiv preprint arXiv:1312.6229, 2013.

[23]         K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.

[24]         M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:1312.4400, 2013.

[25]         K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for imagerecognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.

[26]         C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," arXiv preprint arXiv:1602.07261, 2016.

[27]         B. Wu, F. Iandola, P. H. Jin, and K. Keutzer, "SqueezeDet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 129-137.

[28]         F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1251-1258.

[29]         A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.

[30]         M. Arjovsky, S. Chintala, and L. Bottou, "Wasserstein gan," arXiv preprint arXiv:1701.07875, 2017.

[31]         H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, "Self-attention generative adversarial networks," presented at the International Conference on Machine Learning. PMLR, 2019.

[32]         Z. Jin, L. Min, M. K. Ng, and M. Zheng, "Image colorization by fusion of color transfers based on DFT and variance features," Computers & Mathematics with Applications, vol. 77, pp. 2553-2567, 2019.

[33]         X. Kuang, J. Zhu, X. Sui, Y. Liu, C. Liu, Q. Chen, and G. Gu, "Thermal infrared colorization via conditional generative adversarial network," Infrared Physics & Technology, p. 103338, 2020.

[34]         D. Nie, Q. Ma, L. Ma, and S. Xiao, "Optimization based grayscale image colorization," Pattern Recognition Letters, vol. 28, pp. 1445-1451, 2007.

[35]         D. Liu, Y. Jiang, M. Pei, and S. Liu, "Emotional image color transfer via deep learning," Pattern Recognition Letters, vol. 110, pp. 16-22, 2018.

[36]         M. M. Johari and H. Behroozi, "Context-aware colorization of gray-scale images utilizing a cycle-consistent generative adversarial network architecture," Neurocomputing, 2020.

[37]         S. Liu and X. Zhang, "Automatic grayscale image colorization using histogram regression," Pattern Recognition Letters, vol. 33, pp. 1673-1681, 2012.

[38]         R. Zhang, P. Isola, and A. A. Efros, "Colorful image colorization," presented at the European Conference on Computer Vision. Springer, Cham, 2016.

Statistics:

Google Scholor ideas Microsoft Academic Search bing Google Scholor

Funding:

This study received no specific financial support.

Competing Interests:

The authors declare that they have no competing interests.

Acknowledgement:

All authors contributed equally to the conception and design of the study.

Related Article

( 1 ) An Overview of Advances in Image Colorization Using Computer Vision and Deep Learning Techniques
( 3 ) Lossless Image Compression and Decompression to Improve the PSNR and MSE Values Using Architecture
( 4 ) Survey of Image Processing Based Applications in AMR
( 7 ) Investigate of Mechanical Fuse in Cardan Shaft Using FEM
( 8 ) Vlsi Architecture of Mimo Detector Using Fixed Complexity Sphere Decoding
( 9 ) Predict Survival of Patients with Lung Cancer Using an Ensemble Feature Selection Algorithm and Classification Methods in Data Mining
( 10 ) Postbuckling Analysis of Functionally Graded Beams Using Hyperbolic Shear Deformation Theory
( 11 ) A Detailed Analysis of Software Cost Estimation Using Cosmic-FFP
( 12 ) Received Signal Strength in a Macrocell in Lagos Environs Using Finite Element Method
( 13 ) RLS Fixed-Lag Smoother Using Covariance Information Based on Innovation Approach in Linear Continuous Stochastic Systems
( 15 ) PAPR Reduction Using Eight Factors Rotating Phase Shift Technique Based on Local Search Algorithm in OFDM
( 16 ) Using ICT Policy Framework as a Panacea for Economic Recession and Instability in Nigeria
( 17 ) Detection and Prevention of Phishing Attack Using Linkguard Algorithm
( 18 ) A Survey on Efficient Power Management Using Smart Socket and IoT
( 19 ) Characterisation of Propagation Loss for a 3G Cellular Network in a Crowded Market Area Using CCIR Model
( 20 ) Network Traffic Analysis Using Queuing Model and Regression Technique
( 21 ) Stability Analysis of Type-2 Fuzzy Process Control Using LMI
( 22 ) Intensive Patient Monitoring Using LabVIEW
( 23 ) Quality Assessment and Monitoring of Networks Using Passive Technique
( 25 ) Activity Recognition and Creation of Web Service for Activity Recognition using Mobile Sensor Data using Azure Machine Learning Studio