How Facebook's funding for PyPI is being used

Python Package Index now has automatic malware checks on board

Maika Möbus
© Shutterstock / MicroOne

The Python Software Foundation members have been working on improving the Python Package Index (PyPI). In 2018, they announced that Facebook Research was funding security improvements, so let’s see how far they have come and what future plans they have.

The Python Package Index (PyPI) is home to over 200,000 Python projects and has over 400,000 users. Developed and maintained by the Python community, it is under copyright of the Python Software Foundation (PSF).

In 2018, the PSF announced that they had received a monetary gift from Facebook that was to go towards implementing security features in PyPI.

SEE ALSO: Python data visualization with Bokeh

Here is what has happened so far.

Automatic malware checks

Last month, Milestone 2 of the updates was completed. This means that PyPI now has a system for automatic malware checks on board.

This high-level diagram shows how it works:


Automatic malware checks. Source.

As seen in the diagram, there are three different ways the malware checks can be triggered: Either when a PyPI user uploads a new file, release or project, when a PyPI admin initiates an evaluation run, or on a schedule. Ultimately, the check should then lead to the removal of malicious packages, releases and files.

Future plans

PEP 458 was accepted in February 2020. The Python Enhancement Proposal calls for secure PyPI downloads with signed repository metadata and proposes how to integrate The Update Framework (TUF), a CNCF graduated project, with PyPI.

The work has therefore begun and should be completed within the following months. This will, according to the PSF, “enable clients like pip to ensure that they have downloaded valid files from PyPI and equip the PyPI administrators to better respond in event of a compromise.”

SEE ALSO: Python Software Foundation: Mozilla and Chan Zuckerberg Initiative are funding pip with $407,000

See more about the PyPI updates in the PSF blog post.

Maika Möbus
Maika Möbus has been an editor for Software & Support Media since January 2019. She studied Sociology at Goethe University Frankfurt and Johannes Gutenberg University Mainz.

Inline Feedbacks
View all comments