深度学习: 智能时代的核心驱动力量

本书有两个相互交织的主题：人类智能是如何进化的，以及人工智能会如何演变。这两种智能之间的巨大差异在于，人类智能的进化经历了数百万年的时间，而人工智能在最近几十年才发展起来。尽管对于文化演变来说，这个速度仍然是快得出奇，但是过于谨小慎微可能并不是个正确的选择。

深度学习在近期取得的突破，并不是你从新闻报道中读到的那种一夜成功。从基于符号、逻辑和规则的人工智能向基于大数据和学习算法的深度学习网络的转变，其背后的故事通常并不为人所熟知。本书介绍了这个故事，并从我的角度探讨了深度学习的起源和成果。作为20世纪80年代开发神经网络学习算法的先行者和NIPS基金会的主席，我亲身经历了过去30年机器学习和深度学习的发展过程。我和同在神经网络领域的同事多年来都未能取得令人瞩目的成就，但坚持和耐心最终给我们带来了回报。

1. “啊，美丽的新世界，有这么美的人在里头！”（“O brave new world that has such people in’t!”）来自莎士比亚的剧作《暴风雨》中米兰达的台词（5.1.182–183 [Oxford Standard Authors Shakespeare]）。
2. Bill Vlasic, “G.M. Wants to Drive the Future of Cars That Drive Themselves,” New York Times, June 4, 2017, https://www.nytimes.com/2017/06/04/business/generalmotors-self-driving-cars-mary-barra.html.
3. “Full Tilt: When 100% of Cars Are Autonomous,” New York Times Magazine,November 8,2017, https://www.nytimes.com/interactive/2017/11/08/magazine/tech-design-autonomous-future-cars-100-percent-augmented-realitypolicing.html?hp&action=click&pgtype=Homepage&clickSource=storyheading&module=second-column-region®ion=top-news&WT.nav=top-news/.
4. Christopher Ingraham, “The Astonishing Human Potential Wasted on Commutes,”Washington Post,February 24, 2016, https://www.washingtonpost.com/news/wonk/wp/2016/02/25/how-much-of-your-life-youre-wasting-on-your-commute/?utm_term=497dfd1b5d9c..
5. “Full Tilt: When 100% of Cars Are Autonomous,” New York Times Magazine ,November 8,2017, https://www.nytimes.com/interactive/2017/11/08/magazine/tech-design-autonomous-future-cars-100-percent-augmented-realitypolicing.html?hp&action=click&pgtype=Homepage&clickSource=storyheading&module=second-column-region®ion=top-news&WT.nav=top-news/.
6. “Google’s Waymo Passes Milestone in Driverless Car Race,” Financial Times ,December 10,2017, https://www.ft.com/content/dc281ed2-c425-11e7-b2bb322b2cb39656/.
7. B. A. Golomb, “Will We Recognize It When It Happens?” in Brockman, J., (ed.),What to Think About Machines That Think (New York: Harper Perennial, 2015),533– 535.
8. Pierre Delforge, “America’s Data Centers Consuming and Wasting Growing Amounts of Energy,” Natural Resources Defense Council Issue Paper, February 6, 2015, https://www.nrdc.org/resources/americas-data-centers-consuming-andwasting-growing-amounts-energy/.
9. W. Brian Arthur, “Where Is Technology Taking the Economy?” McKinsey Quarterly, October,2017, https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/Where-is-technology-taking-the-economy/.
10. 乞力马扎罗山是一座雪山，高19 710英尺，据说是非洲最高的山峰。它的西峰被马赛人称作“Ngaje Ngai”，意为“上帝的家”。靠近西峰有一具干燥、冰冻的豹子尸体。没有人解释过这只豹子在那个高度是要寻找什么。
11. 乞力马扎罗山是一座高达19 710英尺的雪山，据说是非洲最高的山峰。它的西峰在马赛语里被称为“Ngaje Ngai”，上帝的家。西峰附近有一具干燥、冷冻的豹子尸体。从来没有人解释过豹子在那个高度想要找什么。
12. Gideon Lewis-Kraus, “The Great A. I. Awakening,” New York Times Magazine ,December 14, 2016, https://www.nytimes.com/2016/12/14/magazine/the-greatai-awakening.html.
13. Aleksandr Sergeevich Pushkin, Eugene Onegin:ANovel in Verse,2nd ed., trans.Vladimir Nabokov (Princeton: Princeton University Press, 1991).
14. 关于这一做法的早期尝试，请参阅Andrej Karpathy, “The Unreasonable Effecti-veness of Recurrent Neural Networks,” Andrej Karpathy Blog,posted May 21, 2015,http://karpathy.github.io/2015/05/21/rnn-effectiveness/.
15. G. Hinton, L. Deng, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, et al., “Deep Neural Networks for Acoustic Modeling in Speech Recognition,” IEEE Signal Processing Magazine 29, no. 6(2012): 82–97.
16. W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, et al., “Achieving Human Parity in Conversational Speech Recognition,” Microsoft Research Technical Report MSR-TR-2016-71, revised February 2017, https://arxiv.org/pdf/1610.05256.pdf.
17. A. Esteva, B. Kuprel, R. A. Novoa, J. Ko J, S. M. Swetter, H. M. Blau, and S.Thrun,“Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks,” Nature 542, no.7639 (2017): 115–118.
18. Siddhartha Mukherjee, “A.I. versus M.D.: What Happens When Diagnosis Is Automated?”New Yorker,April 3, 2017, http://www.newyorker.com/magazine/2017/04/03/ai-versus-md/.
19. Dayong Wang, Aditya Khosla, Rishab Gargeya, Humayun Irshad, Andrew H. Beck,Deep Learning for Identifying Metastatic Breast Cancer,arXiv: 1606.05718. 他们使用的测量方法在信号检测理论里被称为“曲线下面积”，它对假阴性和假阳性都很敏感。

    https://arxiv.org/abs/1606.05718/.

20. Anthony Rechtschaffen and Alan Kales, eds.,AManual of Standardized Terminology, Techniques and Scoring System for Sleep Stages of Human Subjects ,National Institutes of Health publication no. 204 (Bethesda, MD: U.S. National Institute of Neurological Diseases and Blindness, Neurological Information Network, 1968).
21. 参阅Ian Allison, “Former Nuclear Physicist Henri Waelbroeck Explains How Machine Learning Mitigates High Frequency Trading,” International Business Times,March 23, 2016,http://www.ibtimes.co.uk/former-nuclear-physicist-henriwaelbroeck-explains-how-machine-learning-mitigates-high-frequency-1551097.;Bailey McCann, “The Artificial-Intelligent Investor: AI Funds Beckon,” Wall Street Journal, November 5, 2017, https://www.wsj.com/articles/the-artificialintelligent-investor-ai-funds-beckon-1509937622/.
22. Sei Chong, “Morning Agenda: Big Pay for Hedge Fund Chiefs despiteaRough Year,” New York Times,May 16, 2017, https://www.nytimes.com/2017/05/16/business/dealbook/hedge-funds-amazon-bezos.html.
23. 除了雇用了数千名数学家的美国国家安全局。Alfred W. Hales, personal communication, May 4, 2016.
24. Sarfaz Manzoor, “Quants: The Maths Geniuses Running Wall Street,” Telegraph, July 23,2013, http://www.telegraph.co.uk/finance/10188335/Quants-the-mathsgeniuses-running-Wall-Street.html.
25. D. E. Shaw, J. C. Chao, M. P. Eastwood, J. Gagliardo, J. P. Grossman, C. Ho, et al., “Anton:A Special-Purpose Machine for Molecular Dynamics Simulation,”Communications of the ACM 51, no.7 (2008): 91–97.
26. D. T. Max, Jim Simons, “The Numbers King,” New Yorker,December 18 &25, 2017, https://www.newyorker.com/magazine/2017/12/18/jim-simons-thenumbers-king/.
27. 很快就会被拍成电影。
28. 约翰·冯·诺依曼，引自根据雅各布·布罗诺夫斯基（Jacob Bronowski）的著作改编的电视纪录片《人类的攀升》 （The Ascent of Man ），第13集（1973年）。
29. 参阅M. Moravík, M. Schmid, N. Burch, V. Lisý, D. Morrill, N. Bard, et al.,“Deep-Stack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker,”Science 356, no. 6337 (2017): 508–513。标准偏差是钟形曲线的半峰宽。只有16% 的样本落入平均值的一个标准偏差之外的区域。只有万分之三的样本在平均值的四个标准偏差以外。
30. 脑海中浮现出了1983年科幻电影《战争游戏》（War Game s）中的场景。参阅https://en.wikipedia.org/wiki/WarGame.。
31. 参阅D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. v. d. Driessche,et al.,“Mastering the Game of Go with Deep Neural Networks and Tree Search,”Nature 529, no. 7587(2016): 484–489.
32. “我不知道今天该说些什么，”李世石告诉媒体记者，“但我想我必须先表示歉意。我应该能表现得更好，在比赛中占据更多的主动。我也很抱歉，让很多人失望了。我觉得很无助。

    如果回顾三场比赛，即使第一场比赛能够重赛，我也不认为自己会取胜，因为那时我误判了AlphaGo的能力。” Jordan Novet, “Go Board Game Champion Lee Sedol Apologizes for Losing to Google’s AI,” Venture Beat,March, 12, 2016, https://venturebeat.com/2016/03/12/goboard-game-champion-lee-sedol-apologizes-for-losing-to-googles-ai/。

33. “勘测者1号”于1966年6月2日，国际标准时间6点17分36秒（美国东部时间凌晨1点17分36秒）登陆月球表面。着陆点位于弗拉姆斯蒂德火山口（Flamsteed Crater）以北一个直径为100公里的陨坑内。
34. Ke Jie, as quoted in Selina Cheng, “The Awful Frustration ofaTeenage Go Champion Playing Google’s AlphaGo,” Quartz,May 27, 2017, https://qz.com/993147/the-awful-frustration-of-a-teenage-go-champion-playing-googlesalphago/.
35. Ke Jie, as quoted in Paul Mozur, “Google’s A.I. Program Rattles Chinese Go Master As It Wins Match,” New York Times,May 25, 2017, https://www.nytimes.com/2017/05/25/business/google-alphago-defeats-go-ke-jie-again.html.
36. 老版的“乒乓外交”指1971年中国和美国乒乓球队开展互访的友好往事，不仅推动了中美两国关系正常化的进程，也加速了中国走向世界的步伐。——编者注
37. Paul Mozur, “Beijing Wants A.I. to Be Made in China by 2030,” New York Times ,July 20,2017, https://www.nytimes.com/2017/07/20/business/china-artificialintelligence.html.
38. Silver D., J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T.Hubert, L.Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, “Mastering the Game of Go Without Human Knowledge,” Nature 550 (2017): 354–359.
39. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis, Mastering Chess and Shogi by Self-Play with a general Reinforcement Learning Algorithm,arXiv: 1712.01815(2017).
40. Harold Gardner, Frames of Mind: The Theory of Multiple Intelligences,3rd ed. (New York: Basic Books, 2011).
41. J. R. Flynn, “Massive IQ Gains in 14 Nations: What IQ Tests Really Measure,”Psychological Bulletin 101, no. 2 (1987):171–191.
42. S. Quartz and T. J. Sejnowski, Liars, Lovers and Heroes: What the New Brain Science Has Revealed About How We Become Who We Are (New York: Harper Collins, 2002).
43. Douglas C. Engelbart, Augmented Intelligence: Smart Systems and the Future of Work and Learning,SRI Summary Report AFOSR-3223 (Washington, DC:Doug Engelbart Institute, October 1962), http://www.dougengelbart.org/pubs/augment-3906.html.
44. M. Young, “Machine Learning Astronomy,” Sky and Telescope,December (2017):20–27.
45. “Are ATMs Stealing Jobs?” The Economist,June 15, 2011,https://www.economist.com/blogs/democracyinamerica/2011/06/technology-and-unemployment/.
46. John Taggart and Kevin Granville, “From ‘Zombie Malls’ to Bonobos: What America’s Retail Transformation Looks Like,” New York Times,April 15, 2017.
47. E. Brynjolfsson and T. Mitchell, “What Can Machine Learning Do? Workforce Implications,” Science (2017): 358:1530–1534. doi: 10.1126/science.aap8062.
48. “Technology Is Transforming What Happens WhenaChild Goes to School:Reformers Are Using New Software to ‘Personalise’ Learning,” Economist,July 22, 2017,https://www.economist.com/news/briefing/21725285-reformers-areusing-new-software-personalise-learning-technology-transforming-what-happens/.
49. 教育市场的估值超过1.2万亿美元，主要包含三个领域：早期儿童教育（700亿美元），K–12（即从幼儿园到高中三年级的基础教育，6 700亿美元），以及高等教育（4 750亿美元）。参阅Arpin Gajjar, “How Big Is the Education Market in the US: Report from the White House,” Students for the Future, October 10, 2008, https://medium.com/students-for-the-future/how-big-is-the-educationmarket-in-the-us-report-from-white-house-91dc313257c.。
50. “Algorithmic Retailing: Automatic for the People,” Economist, April 15, 2017,
51. T. J. Sejnowski, “AI Will Make You Smarter,” in Brockman, J. (ed.), What to Think About Machines That Think (New York: Harper Perennial, 2015), 118–120.