Thursday, 19 February 2015, Cambridge, UK
Demis Hassabis – A well known scientist, innovator, and entrepreneur in the field of artificial intelligence and founder of Deepmind technologies (acquired by Google), delivered a talk at Cambridge University Technology & Enterprise Club. The talk was attended by more than 250 people from all over the UK (as far as from Edinburgh). Demis, who is also a University of Cambridge alumnus, spoke about his research on deep learning algorithms, future of artificial intelligence (AI), and entrepreneurship for blue sky initiatives.
In 1997, Demis finished his BA in Computer Science Tripos with a double degree from the Computer Lab (Cambridge University). He always had a strong passion for programming, certainly for computer games and AI. Even before he started studying (at the age of 16), he worked with Bullfrog Productions on the development of Syndicate (1993) and Theme Park (1994). After obtaining his degree, Demis joined Lionhead Studios as lead AI programmer for Black & White (2001). In 1998, he founded Elixir Studios and continued developing best-selling games such as Republic (2003) and Evil Genius (2004).
After Elixir Studios, Demis returned to academia. In 2009, he obtained his PhD in Cognitive Neuroscience from University College London (UCL). His thesis was on “The neural processes underlying episodic memory” with Prof. Eleanor Maguire. After he completed his degree, he continued his research in neuroscience and artificial intelligence as a Welcome Trust Research Fellow at the Gatsby Computational Neuroscience Unit (UCL) and as a visiting scientist jointly at MIT and Harvard. His work mainly focussed on autobiographical memory and amnesia. His most cited papers include:
- “Patients with hippocampal amnesia cannot imagine new experiences” (PNAS, 2007)
- “Using Imagination to Understand the Neural Basis of Episodic Memory” (Journal of Neuroscience, 2007)
- “Deconstructing episodic memory with construction” (Trends in Cognitive Sciences, 2007)
In 2011, Demis left academia to found DeepMind Technologies. His startup focussed on the development of machine-learning algorithms using deep learning. In January 2014, his company was acquired by Google for £400 million. Currently, Demis is continuing his research as Vice President of Engineering at Google, leading their general AI projects.
Demis is also an accomplished chess, shogi, and poker player. Before 2003, he won the World Games Championships at the Mind Sports Olympiad a record 5-times, and cashed 6 times at the World Series of Poker.
DeepMind, General-Purpose AI, and Deep Learning Algorithms
DeepMind Technologies was founded by Demis Hassabis together with Shane Legg and Mustafa Suleyman. Their goal was to create an environment that would facilitate a leap forward in AI research. In the words of Demis: “an Apollo Program effort for AI”. Their institute employed 150 top scientists and synthesized the best from academia with the best from industry. Their goals were big: creating a general-purpose AI.
What makes a general-purpose AI different from other AI’s is that it does not have to be trained, but trains itself after being given a certain task (also known as machine learning), and that it can be used for multiple purposes. Imagine for instance that you plug a general-purpose AI into a flight simulator and give it the task to fly a planes from A to B in the safest, quickest, and most energy-efficient manner. After many hours of repetition, the AI trains itself to do so in the best possible way. After being trained, you can use this trained “agent” to fly planes. The benefit is that you do not have to program everything. The machine learning algorithm programs itself. This is different from other AIs such as Siri or Deep Blue which are coded in a clever way to perform certain tasks very well by using strong computational power.
So, how did they approach this? The short answer is: deep learning or hierarchical learning. This process uses different layers of nonlinear processing units for feature extraction and transformation. It is hierarchical in the sense that the output from previous layer is used as input for the next. This processes strongly mimics the functioning of the brain (certainly with regards to visual input), and are therefore sometimes called neural networks.
Initially, they used Atari 8-bit computer games from the 80s to test there AI programs. The “agent” (AI program) got the raw pixels as input and could interact with the “environment” by controlling the game character with the input buttons. Interestingly, when plugged in, the agents does not know what these pixels mean. It does not know that certain pixels next to each other define an object and that the changing of pixels in time defines movement. It also has no idea what the different input buttons do. The only thing it “knows” is its goal: to maximize the score. In the first rounds, there is not much happening. However, after hours of training, the agents quickly learns how to beat the game – often much better than humans. By now, DeepMind’s agents are able drive cars in 3D racing simulations.
Demis’ dream is that in the future, these AI would not only play games. Apart from processing visual information, they should be particularly good in working with big amounts of data, and with complex systems. It could thus very well be that these systems would solve complex diseases such as cancer by mining and processing all the data that is currently available. In working with Google, Demis believes that these goals are closer then ever. He said that he would be incredibly proud if one of his agents would ever publish a paper in Nature magazine.
Entrepreneurship for Moonshots and Blue Sky Initiatives
What DeepMind accomplished is astonishing, not only from a research/technology point of view, but also from an entrepreneurship point of view. DeepMind was created as an independent research institute that would investigate general-purpose AI. This left many people with questions. How do you get investors excited about ventures that are so Blue Sky without an immediate view on the exit strategy? How do you accomplish selling the enterprise to Google for £400 million without having any product ready? Here are a couple of point of advise Demis had to share:
- Be about 5 years ahead. You need novelty and competitive/intellectual advantage. Demis achieved this by combining computer science and cognitive neuroscience. He advised everyone to keep on learning and improving, which would allow to identify a specific niche that you can overtake. Flirting with disciplines different than the main one you are focussing on would give you an edge, a fresh point of view, and a strong advantage compared to your peers. However, he suggested, you should not be too innovative (e.g. 50 years ahead), because people need to understand your technology and the implications of your Blue Sky initiative.
- Be passionate and obsessed with your goals. When starting a new venture, you need to be incredibly excited about what you are trying to achieve. This is even more true for Blue Sky initiatives. You need to stay ahead of the game, which you can only achieve by momentum and continuously pushing forward. If you are not passionate about what you are doing, your drive will be lower, you will not reach your milestones as quick, you will not be able to motivate your team as well, and you will have more difficulties attracting investors.
- Seek a lot of investment in a short time frame. For his previous startup, Demis went through the typical process of different rounds of funding. He received investment from one VC, reached the milestones, and then went for the next round. What he did for DeepMind was completely different. Once investment was secured from one source, he immediately went to seek more investment. In a quite short period of time, they were able to secure over £60 million in investment. This did not only allow them to pursuit Blue Sky goals in an accelerated manner (keeping ahead of the game), but also create momentum and awareness.
- Seek investment from the right sources. Most typical investors and VCs aim for a 10x return within a couple of years. This was obviously not something that DeepMind could deliver on. There was no clear exit strategy set in place, nor did they focus their efforts on creating a marketable product as soon as possible. For this reason, Demis approached investors such as Peter Thiel, Elon Musk, Li Ka-Shing, and Jaan Tallinn. These people understood the mission from DeepMind, including the impact it would have in terms of technological revolutions and financial return.
After about 2 years DeepMind delivered on the research and technology that they were prospecting. Key in achieving this was again the momentum they gained from their background, passion, and investment. Although different organization interested in acquiring DeepMind, they closed the deal with Google for an astonishing £400 million, Google’s biggest European acquisition. Demis explained that Google offered them the freedom to continue on their research. Furthermore, the resources they gained from Google would mean an additional leap forward in realizing their dream of general-purpose AIs that would revolutionize the world as we know it.
We thank Demis for such an insightful and inspiring talk.