Introduction

Artificial Intelligence, also widely known as ‘AI’, is intelligence executed by machines which take actions to achieve the prescribed goals to the maximum extent based on the perceived environment [1, 2]. The latest applications and products in many fields are increasingly practicing Artificial Intelligence (hereinafter referred to as AI). Deep learning, regarded as the latest AI technology, is also becoming widespread. This essay will illustrate the development of AI from the classical rule-based systems through to deep learning, the applications of deep learning used in AI, the language translation in deep learning and potential challenges of deep learning.

Development of AI

From the classical rule-based systems to deep learning, the core of AI, has been increasingly improving. From the early stage to the idea of ‘Artificial Intelligence’ established at a workshop at Dartmouth College in 1956, the scholars considered that the computers could be programmed to execute intelligent tasks, including to learn and to demonstrate any feature of intelligence precisely [3]. In the early 1970’s, in rule-based systems represented by the expert system, the probabilistic methods were early applied, followed by the more popular heuristic approaches developed later on [4]. The latter methods aforementioned, heuristic approaches in rule-based expert systems, drove the scholars to focus on the optimality of the system performance and the methods which are under uncertain situation increasingly [4, 5]. A significant feature of the expert systems at that time was the application of the production-rule architecture to actual diagnosis, which provided the flexible scheme as an ‘expert’ representative [4].

However, with the improvement of AI discipline, various scholars hold the view that the rule-based expert systems were insufficient for the development of AI, thus the concept of ‘deep learning’ came out – which allows the computational models that contain numerous processing layers to learn the data representation that is composed of multiple levels of abstraction [6]. Several cases reported that deep learning excels on discovering the complex structures in the data of high-dimensionality, and at the same time was making significant progress in overcoming the challenges caused by rule-based systems in AI [6]. Currently, deep learning is improving the fields dramatically including speech recognition, facial recognition, object detection, visual object recognition and many other domains. In addition to this, especially, deep learning contributes to natural language understanding, which includes but is not limited to topic classification and language translation [6]. Language translation in deep learning will be illustrated and evaluated later on in detail.

Applications of Deep Learning Used in AI

In the past several years, from computer visualization to natural language processing, deep learning has been widely used in AI discipline. One significant achievement of deep learning in computer visualization is real-time face capture and reenactment, which was demonstrated by a pioneer group at Stanford University in 2016 [7]. The system named Face2Face is able to capture the faces in original digital materials and take the new faces, which can be the face of user or prescribed stickers, as alternatives [7]. Currently, the technique is applied not only in some main streaming mobile applications, such as Instagram and Snapchat, but also used in film effects and 3-dimensional scene reconstruction [8]. Another important application in deep learning is the generation of natural language descriptions for a specific image and its regions [9]. The model has the ability to give some basic descriptions of the objects appearing in the image, such as a cup of coffee, a bottle of water, a tablet and even a person with his or her real name [9]. In addition to this, taking ‘a glass of water with ice and lemon’ as an example, it can process the complicated objects and give the corresponding and suitable descriptions [9]. Other than computer visualization and natural language processing, deep learning can predict the possibilities to happen of an event. Through the training of a deep neural network, the model can predict a potential upcoming earthquake [10]. Through a similar approach to this, it also can predict the future population of a region or a city, and even the election results of a country [11].

Multiple giant enterprises have been investing in deep learning for years, meanwhile more companies start to implement deep learning technology in their products. As a social media empire, Facebook, firstly practiced ‘Custom Audiences’ on their advertising fields in 2012 [12]. Straightforwardly saying, the algorithm implemented for ‘Facebook Custom Audiences’ named Lookalike enables to find presumed intriguing advertisements for target users [12]. The group of users aforementioned, then, is willing to explore the recommended products in the advertisement because it fits their interests to the maximum extent [12]. Additionally, the users of YouTube may be curious why the videos YouTube recommended to them were mostly intriguing for them. It is also because of the dramatic performance improvements brought by deep learning, which was applied on the recommendation algorithm of YouTube [13]. In summary, deep learning improves the user experience toward to the product, and the users will be attracted by more interesting contents so they will spend more time on their devices, and consequently, the product will find itself competitively stronger in the market, having the chance to be seen and used by a wider audiences; more importantly, macroscopically, deep learning will also increase the income of the company, as the example ‘Facebook Custom Audiences’ mentioned above – the major advantages that deep learning brings with it are leading more companies to invest into its research.

Language Translation in Deep Learning

In the language translation field, the involvement of deep learning improved the performance of machine translation. As pioneers, the group of Kyung Hyun Cho at New York University, they found an approach to construct a black box system, which aims to learn how to translate based on the training data, by using deep learning technique [14, 15]. This deep learning model used a parallel corpus to learn how to translate between two languages without human intervention [14, 15]. The approach, mentioned before, contains two core concepts behind – Recurrent Neural Networks (RNN) and Encodings [15]. RNN is an improved neural network, meaning that a previous state will be one of the inputs in the next calculation [15]. Taking ‘Automatic Correction’ function on virtual keyboards as an example, through RNN, the previous words in the memory will influence the next prediction. Based on the concept of RNN, RNN Encoder-Decoder, which is composed of two RNNs, came out. The former RNN is responsible for encoding the prescribed text to a vector representation with fixed-length, and the latter RNN will decode from vector representation to target language text – while it can be trained by a parallel corpus [15]. Compared to traditional statistical machine translation, this model brings the possibility to have different translation results based on the same texts at different time – it is not immutable and better suits the development of reality. More significantly, it does not have complex procedure to develop and to maintain, resulting that the expenses on hiring the ‘expensive’ linguists and programmers can be decreased.

The limitation of RNN Encoder-Decoder approach usually is that the performance of the approach depends on the amount of training data and the computing power invested in it [15]. However, with several years of development, the performance of the approach currently put it in the same league as traditional statistical machine translation system which has been developed for several decades [16]. Additionally, the approach may translate the words based on trends potentially but unnecessary actually. Under these circumstances, the translation with the older expression seems better. For example, a famous saying by a celebrity. Therefore, the modern deep learning or other machine learning approaches are possible to work with classical rule-based systems jointly – for those words which are better to keep the older expression, add them all into the rule-based systems. Ideally, to use the best system for each situation.

Potential Challenges of Deep Learning

Despite the fruitful results brought by deep learning research, there are still several challenges for deep learning to overcome. The first challenge is the lack of data so far for deep learning to train [17, 18]. Different from human brains, deep learning does not have the ability to learn abstract concepts based on clean definition in language description. Consequently, the performance of deep learning is positively correlated with the amount of training data. With the increasing training data examples, from thousands, millions to even billions, the performance of deep learning will improve dramatically. Secondly, deep learning is currently fragile under the unknown world, or non-highly-stable environment [17]. However, the real world is full of uncertainties. In various fields, including politics and business, the only stable phenomenon is that there are always changes. If deep learning is applied to predict the prices of stock now, it may happen the same result as Google predicted flu by deep learning – obtained the wrong prediction and crushed [17]. Thirdly, deep learning is not fully transparent at this stage [17]. As mentioned before in language translation, the latest approach needs to construct a black box system in neural networks. Therefore, it also demonstrates in another way that deep learning cannot be practiced in financial transactions, medical diagnosis and other fields which need to provide specific and precise processing details, until an optimal solution is found, otherwise it will very likely result in a catastrophe.

Conclusion

In conclusion on the demonstration and evaluation above, from classical rule-based systems to deep learning, AI improved tremendously over the past several decades. Although AI is not fully mature at the moment, the excellent performance on various industrial applications and products proved that AI has the potentiality to perform better – bringing many enterprises to invest in deep learning. Furthermore, through further analysis on language translation of deep leaning example, the appeal and possible limits of deep learning has been revealed. It is also hold to be possible to apply classical rule-based system and modern machine learning methods jointly so that the performance of AI may eventually reach its peak. Lastly the potential challenges of deep learning are illustrated, including the lack of training data and the characteristics of instability.

References

[1] Poole, D. L., Mackworth, A. K., & Goebel, R. (1998). Computational intelligence: a logical approach (Vol. 1). New York: Oxford University Press.

[2] Persson, S. (1964). An Introduction to Artificial Intelligence. Ekonomisk Tidskrift, (r 2), 88-112.

[3] Moor, J. (2006). The Dartmouth College artificial intelligence conference: The next fifty years. Ai Magazine, 27(4), 87-87.

[4] Horvitz, E. J., Breese, J. S., & Henrion, M. (1988). Decision theory in expert systems and artificial intelligence. International journal of approximate reasoning, 2(3), 247-302.

[5] Müller, V. C., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. In Fundamental issues of artificial intelligence (pp. 555-572). Springer, Cham.

[6] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436.

[7] Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016). Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2387-2395).

[8] Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017). 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1802-1811).

[9] Karpathy, A., & Fei-Fei, L. (2015). Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3128-3137).

[10] DeVries, P. M., Thompson, T. B., & Meade, B. J. (2017). Enabling large‐scale viscoelastic calculations via neural network acceleration. Geophysical Research Letters, 44(6), 2662-2669.

[11] Gebru, T., Krause, J., Wang, Y., Chen, D., Deng, J., Aiden, E. L., & Fei-Fei, L. (2017). Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences, 114(50), 13108-13113.

[12] Liu, H., Pardoe, D., Liu, K., Thakur, M., Cao, F., & Li, C. (2016, August). Audience expansion for online social network advertising. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 165-174). ACM.

[13] Covington, P., Adams, J., & Sargin, E. (2016, September). Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 191-198). ACM.

[14] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

[15] Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.

[16] Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., … & Klingner, J. (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144.

[17] Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631.

[18] Hinton, G. (2018). Deep learning—a technology with the potential to transform health care. Jama, 320(11), 1101-1102.