AI research with PyTorch

OpenAI sets PyTorch as its new standard deep learning framework

Maika Möbus
© Shutterstock / italianestro

The AI research organization OpenAI has declared to standardize which deep learning framework to use in its projects, and the winner is PyTorch. Let’s take a closer look and see not only why OpenAI selected PyTorch, but also what benefits the standardization itself should offer.

PyTorch is now set to be OpenAI’s standard deep learning framework, as the capped-profit research organization for artificial intelligence announced in a blog post.

SEE ALSO: OpenAI finally releases “dangerous” large-scale unsupervised language model GPT-2

In general, OpenAI’s framework standardization is meant to benefit the creation and sharing of models, but why was PyTorch chosen for this task and what does this imply for its future development?

Performance benefits and developer community

The open source framework PyTorch, developed by Facebook and first released in 2016, comes with a Python and a C++ interface. The latest version, PyTorch 1.4, landed earlier this month and introduced some new features including experimental Java bindings.

OpenAI sees PyTorch’s main advantage in that it can increase research productivity at scale on GPUs. This is illustrated by the organization’s example of cutting down iteration time in generative modeling from weeks to days.

And also, OpenAI is “excited to be joining a rapidly-growing developer community, including organizations like Facebook and Microsoft, in pushing scale and performance on GPUs.”

In addition to using PyTorch in its research projects, OpenAI is also planning to contribute to the open source framework in the future. It remains to be seen how the developer community will react to this, as OpenAI has received some backlash in the past after initially keeping its NLP model GPT-2 under lock and key, until it was released nine months after its first public announcement.

SEE ALSO: “BERT is a system than can be tuned to do practically all tasks in NLP”

The transition from a non-profit to a “capped-profit” organization in March 2019 also led to some controversy on Twitter:

Although PyTorch is now OpenAI’s standard framework, the organization clarified that exceptions are still possible when other frameworks are better suited for a certain project.

The OpenAI blog provides further details on the switch to PyTorch.

Maika Möbus
Maika Möbus has been an editor for Software & Support Media since January 2019. She studied Sociology at Goethe University Frankfurt and Johannes Gutenberg University Mainz.

Inline Feedbacks
View all comments