Abstraction is a common trait amongst the now widely used machine learning libraries or frameworks. Dusting off the nitty-gritty details under the rug and concentrating on implementing algorithms with more ease is what any data scientist would like to get their hands on.
TensorFlow rose into prominence for the very same reason — abstraction. Now with its latest library TensorFlow Graphics, it aims to address key computer vision challenges by incorporating the knowledge from graphics in the images, which in turn result in robust neural network architectures.
TensorFlow’s machine learning platform has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
During the last couple of years, neural network architectures have taken giant strides into a handful of domains by both contributing to and imbibing from the field it enters. In case of graphics, there has been a rise in novel differentiable graphics layers which can be inserted in neural network architectures.
Neural network architectures can be made more efficient by leveraging the knowledge acquired from computer vision and graphics say the researchers at Google.
“Training machine learning systems capable of solving these complex 3D vision tasks most often require large quantities of data. As labelling data is a costly and complex process, it is important to have mechanisms to design machine learning models that can comprehend the three-dimensional world while being trained without much supervision. Combining computer vision and computer graphics techniques provides a unique opportunity to leverage the vast amounts of readily available unlabelled data,” wrote the team behind TensorFlow graphics.
What Does TensorFlow Graphics Have To Offer
The following are one of the few functionalities of the new library that TensorFlow boasts of:
Transformations
Object transformations control the position of objects in space. In the illustration below, the axis-angle formalism is used to rotate a cube.
Modelling Cameras
Camera models greatly influence the appearance of three-dimensional objects projected onto the image plane. As can be observed below, the cube appears to be scaling up and down, while in reality, the changes are only due to changes in focal length.
Materials
Now, TensorFlow Graphics allows users to drop virtual furniture in their environment and have the pieces photo-realistically blend with their interior with this new feature.
TensorBoard 3D
Visual debugging is a great way to assess whether an experiment is going in the right direction. To this end, TensorFlow Graphics comes with a TensorBoard plugin to interactively visualize 3D meshes and point clouds.
Geometry — 3D convolutions and pooling
TensorFlow Graphics comes with two 3D convolution layers, and one 3D pooling layer, allowing, for instance, the training of networks to perform semantic part classification on meshes.
Installing TensorFlow graphics:
pip install tensorflow_graphics
With the experiments such as Tossing Bot and depth maps showing promising results, the release of TensorFlow graphics couldn’t have come at a better time.
From 3D reconstruction to adding video effects like synthetic defocus,as the domain of computer vision extends it reach, collecting every innovation into its arsenal, the future looks bright for AGI.
You can experiment with the library here.
