2017 has been a great year for Deep Learning. What could possibly go wrong for Deep Learning that has spurred recent advances in a range of applications with every recent breakthrough being attributed to DL– autonomous cars, speech recognition, image recognition, neural machine translation. Much of the hype in Artificial Intelligence should be credited to DL, seen as the silver bullet in the current technological landscape. So much so that despite the general limitations (data-intensive & lack of generalization) AI today is synonymous with Deep Learning — the one-trick-pony solutions in the business world.

Researchers feel businesses are overselling the abilities of Deep Learning

Gary Marcus, professor of psychology at NYU and ex-director of Uber’s AI lab noted in his paper Deep Learning: A Critical Appraisal that even though “deep learning has historical roots going back decades, neither the term Deep Learning nor the approach gained popularity just over five years ago. He writes that the field was rekindled by Krizhevsky, Sutskever and Hinton’s famed paper on deep network model of Imagenet published in 2012. Marcus believes that against the backdrop of so much success, Deep Learning runs the risk of hype and could potentially hinder researchers from experimenting with new ideas. Rather than viewing it as a silver bullet, businesses should view it as “one tool among many”.

Well, Marcus is not the only detractor. Carlos E Perez, author of Artificial Intuition and the Deep Learning Playbook and founder of Intuition Machine Inc believes 2018 could be the year when Deep Learning’s hype could go kaput and everything could change dramatically. Even though Deep Learning has advanced way beyond theory, the methods still have a long way to go before they become industrialized.

For some researchers, end-to-end Deep Learning is a fascinating area of research, but for now hybrid systems are going to be more effective in application domains. New approaches such as meta-learning and transfer learning are gaining more credence and a cure to DL’s data problem and the year 2017 saw a renewed interested in Convolution Networks that are nudging out recurrent neural networks in some areas. In a way, Deep Learning became the answer to AI’s limitations and acted as a bridge between human brain and computer science, noted MIT scientist Josh Tenenbaum notes.  

But Perez is casting a speculative look at what the future may hold for Deep Learning. Let’s have a look at some his alarming predictions vis a vis India that’s on a cusp of DL awakening with a veritable ecosystem of GPU-based startups:

1)Hardware is Hard, majority of DL hardware startups will shut: This Silicon Valley cliché came to pass with one of the most promising hardware startups shutting down in 2017. Noah Kraft-founded, San Francisco-based Doppler Labs, well-known for its tagline of putting a “mic, speaker and computer in everyone’s ear” flatlined last year in November. The core product Here One earbuds, wireless headphones that gave users a way to change the volume of the real world wowed many a investor, but not enough to raise another series of funds. Known for its advanced engineering hardware and AI techniques, the startup wanted to capitalize on the hearing health market, that already has competition from Google’s Pixel Buds and Apple’s AirPods.  

AIM View: Now, the rapid success of Deep Learning in Computer Vision spurred a lot of startup activity in India with companies like Mad Street Den, Arya.ai, Artifacia, ParallelDots, Netradyne, Uncanny Vision, Qure.ai and Kochi-based Agrima Infotech among others driving innovation with their cutting-edge solutions. Known for leveraging deep learning-based algorithms, these startups made promising advances in healthcare, medical imaging and chatbots. Last July, Mountain View tech titan Google snapped up Bengaluru-based AI startup Halli Labs. Unlike North America that has seen a lot of activity in hardware, India seems more focused on applying DL at scale to current business problems. In the future, one can expect to see more startups talking about their internal infrastructure and how they are leveraging DL at scale.  

2) DL Research Deluge: According to Perez, deep learning research is facing an information deluge with submissions for ICLR 2018 conference crossing around 4,000 papers. Keeping up with the information overload wouldn’t be easy for researchers, who would have to cover 10 papers a day to catch up with just this conference. Another problem is that the theoretical frameworks are a work in progress, Perez emphasizes adding that most DL researchers lack the right mathematical background to correctly understand the complexity of these systems. Perez believes Deep Learning research papers will either increase 3x or 4x in 2018.

AIM View: Academically, there is a surge in interest around Deep Learning and leading institutes such as IISc, IITs, IIIT-Bangalore & IIIT-Hyderabad are leading in research. Alongside, companies like Microsoft India, Flipkart, Amazon have data labs that are at the forefront of cutting-edge research. According to Ramesh Loganathan, Professor Co-Innovation at IIIT Hyderabad Chief Innovation Officer for the Telangana State, IIT-H also boasts of the largest AI research lab cluster in India. The institute also has strong (ranking) research labs and faculty in NLP/info extraction/translations, video and image analytics/computer vision, Machine Learning, Robotics and Cognitive sciences. While Canada, USA and China are probably toe-to-toe in publishing DL papers, India is at the cusp of a DL revolution.

3) Faking explainability: Perez strongly believes this is an area of deep concern and is an unsolvable problem. He explains how there are concepts that a machine generates that will be perhaps defy explanation, citing the example of AlphaGo Zero where it is difficult to explain the concept behind the logic. All in all, Perez summarizes that eventually machines will become very good at ‘faking explanations’, which means in some cases there will be no explanations for certain problems.

In the case of Alpha Go, it was trained on thousands of human amateur and professional games. Now AlphaGo Zero skipped this step and learnt to play by playing games against itself, starting from completely random play. This way, the system surpassed human level of play and defeated the previously published champion-defeating version of AlphaGo by 100 games to 0, notes the DeepMind blog.

AIM View: Deep Learning’s un–explainability problem has been well-chronicled and companies are grappling with biased results in some cases. However, there are few use cases emerging where companies talk about getting unbiased results. There are cases of setbacks in DL techniques being used in business settings but there is very little information on Indian companies grappling with explainability issues.