Deep learning education and tools are becoming more and more democratic each day. There are only a few major deep learning frameworks; and among them, PyTorch is emerging as a winner. PyTorch is an open-source machine learning library inspired by Torch. It has primarily been developed by Facebook‘s artificial intelligence research group, and Uber‘s Pyro software for probabilistic programming is built on it.
1. PyTorch Is Based On Python
Python is the most popular coding language used by data scientists and deep learning engineers. Python continues to be a very popular language even among academics. PyTorch creators wanted to create a great deep learning experience for Python which gave birth to a cousin Lua-based library known as Torch. Hence PyTorch wants to become a Python-based deep learning and machine learning library which is open source. At the same time, it can give every Python user to build great machine learning applications for research prototyping and also production deployment. The creators of the library say, “Deep integration into Python allows popular libraries and packages to be used for easily writing neural network layers in Python.”
I've been using PyTorch a few months now and I've never felt better. I have more energy. My skin is clearer. My eye sight has improved.
— Andrej Karpathy (@karpathy) May 26, 2017
2. Dynamic Approach To Graph Computation
PyTorch builds deep learning applications on top of dynamic graphs which can be played with on runtime. Other popular deep learning frameworks work on static graphs where computational graphs have to be built beforehand. The user does not have the ability to see what the GPU or CPU processing the graph is doing. Whereas in PyTorch, each and every level of computation can be accessed and peaked at. Jeremy Howard from Fast.ai says, “An additional benefit of Pytorch is that it allowed us to give our students a much more in-depth understanding of what was going on in each algorithm that we covered. With a static computation graph library like TensorFlow, once you have declaratively expressed your computation, you send it off to the GPU where it gets handled like a black box. But with a dynamic approach, you can fully dive into every level of the computation, and see exactly what is going on.”
3. (Almost) Faster Deep Learning Training Than TensorFlow
TensorFlow and PyTorch are very close when it comes to speed of deep learning training. Models with many parameters require more operations. Much computation is needed to perform each gradient update, hence with growing number of parameters, training time will grow very fast. One study that compared deep learning frameworks said, “When you need high-performance models, which can probably be further optimized and speed is of the utmost importance, consider spending some time on developing your TensorFlow or Pytorch pipeline.”
4. Increased Developer Productivity
PyTorch is very simple to use and gives us a chance to manipulate computational graphs on the go. Jeremy Howard from Fast.ai who teaches deep learning using PyTorch said, “The key was to create an OO class which encapsulated all of the important data choices along with the choice of model architecture. Suddenly, we were dramatically more productive, and made far fewer errors, because everything that could be automated, was automated. With the increased productivity this enabled, we were able to try far more techniques, and in the process, we discovered a number of current standard practices that are actually extremely poor approaches.”
5. Easier To Learn And Simpler To Code
PyTorch is considerably easier to learn than any other deep learning library out there because it doesn’t travel far off from many conventional program practices. The documentation of PyTorch is also very brilliant and helpful for beginners.
6. Small Community Of Focussed Developers
Although the community of developers working on PyTorch is smaller than other frameworks it is safely in the house at Facebook. The organisation gives the creators much-needed freedom and flexibility to work on bigger issues of the tool rather than optimise smaller parts. Because of this reason, PyTorch has been able to successfully achieve bigger wins in the field with a focussed group of developers.
7. Simplicity and transparency
Because of the dynamic graph comes transparency for developers and data scientists. Programming deep neural networks are much easier in PyTorch than in TensorFlow because of the steep learning curve the latter requires.
8. Easy To Debug
The computational graph in PyTorch is defined at runtime and hence many popular regular Python tools are easier to use in PyTorch. This is a huge advantage because now our favourite Python debugging tools such as pdb, ipdb and PyCharm debugger can be used with the freedom to debug PyTorch code.
9. Data Parallelism
PyTorch has one of the most important features known as declarative data parallelism. This feature allows you to use torch.nn.DataParallel to wrap any module. This will be parallelised over batch dimension and the feature will help you to leverage multiple GPUs easily.
Outlook
PyTorch 1.0 is set to release very soon. It introduces lots of amazing features, including native C++ API, JIT compilation and ONNX integration. This means that you will be able to write production-ready services and do what TensorFlow Serving does. This is a big step to PyTorch and surely will empower its position as a fully featured framework for both research and production purposes.
