Join gaming leaders, alongside GamesBeat and Facebook Gaming, for their 2nd Annual GamesBeat & Facebook Gaming Summit | GamesBeat: Into the Metaverse 2 this upcoming January 25-27, 2022. Learn more about the event. 

Facebook AI Research (FAIR) today unveiled PyTorch3D, a library that enables researchers and developers to combine deep learning and 3D objects.

As part of the release, Facebook is also open-sourcing Mesh R-CNN, a model introduced last year capable of rendering 3D objects from 2D shapes in images of interior spaces. PyTorch3D was inspired by Mesh R-CNN and recent 3D work by Facebook AI Research, FAIR engineer Nikhila Ravi said.

Working in 3D is important for rendering 3D objects or scenes that appear in mixed reality or virtual reality. It can also be used to tackle AI challenges like robotic grasping or helping autonomous vehicles understand the position of nearby objects.

PyTorch3D comes with frequently used 3D operators and loss functions for 3D data and a differentiable mesh renderer for creating 3D objects. PyTorch3D also has a differentiable rendering API, some CUDA support, and heterogeneous batching capabilities unavailable in any existing 3D library, Ravi told VentureBeat in a phone interview.


The 2nd Annual GamesBeat and Facebook Gaming Summit and GamesBeat: Into the Metaverse 2

January 25 – 27, 2022

Learn More

“With PyTorch3D, researchers can input all these functions and use them with the existing deep learning system in PyTorch and it greatly reduces on the time to work on 3D planning research, which requires a lot of expertise in order to get started, and we want to try and reduce that ramp-up time,” she said.

PyTorch3D uses meshes, a data format for interoperability of vertices and faces that make up 3D objects, and can use a patch tensor to collapse all vertices for meshes in a batch into a single tensor involved with batching, a common process for deep learning research.

The premiere of PyTorch3D follows the launch of the PyRobot robotics framework last year, and FAIR 3D research that extracts characters from real-world videos.


VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more
Become a member