OpenAI sets PyTorch as its new standard deep learning framework
The AI research organization OpenAI has declared to standardize which deep learning framework to use in its projects, and the winner is PyTorch. Let’s take a closer look and see not only why OpenAI selected PyTorch, but also what benefits the standardization itself should offer.
PyTorch is now set to be OpenAI’s standard deep learning framework, as the capped-profit research organization for artificial intelligence announced in a blog post.
In general, OpenAI’s framework standardization is meant to benefit the creation and sharing of models, but why was PyTorch chosen for this task and what does this imply for its future development?
Performance benefits and developer community
The open source framework PyTorch, developed by Facebook and first released in 2016, comes with a Python and a C++ interface. The latest version, PyTorch 1.4, landed earlier this month and introduced some new features including experimental Java bindings.
OpenAI sees PyTorch’s main advantage in that it can increase research productivity at scale on GPUs. This is illustrated by the organization’s example of cutting down iteration time in generative modeling from weeks to days.
And also, OpenAI is “excited to be joining a rapidly-growing developer community, including organizations like Facebook and Microsoft, in pushing scale and performance on GPUs.”
In addition to using PyTorch in its research projects, OpenAI is also planning to contribute to the open source framework in the future. It remains to be seen how the developer community will react to this, as OpenAI has received some backlash in the past after initially keeping its NLP model GPT-2 under lock and key, until it was released nine months after its first public announcement.
The transition from a non-profit to a “capped-profit” organization in March 2019 also led to some controversy on Twitter:
We’ve created OpenAI LP, a new “capped-profit” company that allows us to rapidly increase our investments in compute and talent while including checks and balances to actualize our mission. https://t.co/OAkatL2vyy pic.twitter.com/loqckUZPzB
— OpenAI (@OpenAI) March 11, 2019
Although PyTorch is now OpenAI’s standard framework, the organization clarified that exceptions are still possible when other frameworks are better suited for a certain project.
The OpenAI blog provides further details on the switch to PyTorch.