What's up Python: uv light speed, stand alone pythons and a new lock file format

February, 2024

Summary

• Astral releases uv, a new Python package manager

• Stand alone Pythons on the move

• New lock file proposal

Astral releases uv, a new Python package manager

Given the gigantic success of ruff and the loud noise uv release made, it's safe to assume most of you already know, but that wouldn't be a great monthly review without the most important news of the last 30 days, wouldn't it?

So, context first, last year Astral released ruff, a rust powered ultra fast python linter and formatter, and saw almost immediate mass adoption. The tool has sane defaults, provides the value of most of pylint/flake8/isort/black, and is super easy to install.

This February, they did it again, publishing uv, a software meant to replace package and virtual env managers.

After my disappointment with pipenv and poetry and urging people to going back to simpler ways, you would think I would curb my enthusiasm on this one.

XKCD 927, right?

But while I'm not going to proclaim the messiah just yet, I'm carefully optimistic, and here's why:

• They have a great track record. Ruff is an absolute success, not just because it's used everywhere, but also because I found it to be a productive tool with a good design. The API is well made, and it's not by chance, they designed it carefully. Andrew Gallant is likely a big part of those wins, being the guy behind the excellent ripgrep as well.

• Like with Ruff, they smartly embraced the existing ecosystem, and the main sub commands are essentially copying pip and venv. You do uv pip install stuff, so we all know how it works. They have been criticized for that, but I think it's smart and show they care about the community: it's going to be a lot of work to keep supporting that on the long run, but they agreed to pay this price for an easier adoption. And it frees the namespace for more opinionated additions.

• uv shows the same great qualities as ruff. Not only it does the job correctly, but on top of that it's independent from your Python setup, so fast you doubt it actually executed, has sane defaults and good error messages.

• Charlie Marsh has shown more than dev qualities, he has proven to be a superb community liaison: joyful, humble, open to discussion. It's a tough job since the FOSS crown is entitled, ungrateful and prone to critics.

No matter how it turns out, I have a lot of respect for this team.

Needlessness to say, I'm currently coding a new project with it to get a sense of whether or not to recommend it on the long run. I'll drop a definitive feedback after a year of concrete practical use, trials with beginners in training sessions, and the biggest test of all: a crappy corporate setup.

Stand alone Pythons on the move

It happens regularly and should not be worth a news, but I think it's a good opportunity to talk more about indygreg's Stand Alone Python, as it made a new release.

In short, this project compiles Python (currently from 3.9 to 3.12) for various OS+platform combinations, including Mac, Windows and Linux, with pip, openssl, setuptools, sqlite, etc. The results are archives that you can unpack on the machine, and run as is, no installation necessary, providing an homongeous experience every time, everywhere.

It's an important work because it brings opportunities for alternative ways to provide Python distributions that are not linked to the OS (like with apt/yum), are normalized (unlike with windows app store or homebrew), require no compilation (unlike pyenv) and expose the same features everywhere (pip is there, no matter the platform).

This is what Armin's rye, a project we talked about previously, uses to install Python.

And it turns out that rye uses uv to install packages now, while in turn, astral took stewardship of the rye project.

They didn't make any announcement yet, but I fully expect Astral to come up with a Python installer themselves. I just hope they won't use shims as rye did, which causes the usual problems.

Because bootstrapping Python is the root cause of so much suffering, having one clean and simple way to install Python, packages, and deal with the venv has been the holy grail. pyflow and rye were a PoC for that, but Astral has a real chance to pull it off.

I still hope I can one day retire “Relieving your packaging pain”.

New lock file proposal

Talking about package management, another good news that could lead to solving some hard problems is Brett Canon's new lock file proposal.

Lock files are, well, files, that contain pinned dependencies for your project. You usually want top dependencies, with minimal constraints, in your pyproject.toml, so that you can easily change and update your deps. But for deployment, it’s important to mirror your dev setup on the server, so precise constraints for the whole package tree, including the sub-sub-sub-dependencies, are useful. 3rd party tools like poetry and pip-tools provide this, but no official standard exist.

He already tried to suggest one in 2021 with PEP 665 which was rejected for lack of source distribution support. You would think with wheels being now the de facto standard it's not a big deal, but we are talking about a community that took 15 years for the 2=>3 transition: pythonistas move slowly, and they made clear they were not ready to drop the tar.gz option yet.

Without any surprise, the current proposal include sdists, but also, a different file format.

The author was adament it's his last attempt.

I told you the FOSS crowd was ruff. Err, rough.