Recently I read a few articles and have a few questions

Or how to write an article when you have Covid and want to stay in bed

I received an interesting email from a reader who asked pragmatic questions about packaging in Python. It’s very fair, I have a lot of strong opinions on the matter, but how are they applied in real life?

I took a long time to answer and realized it could be an article. So here is how it went (adjusted for obvious typos on my side so I can pretend I’m a big boy):

Hello,

Thanks for your awesome blog.
Recently I read a few articles and have a few questions.
In the https://www.bitecode.dev/p/whats-the-deal-with-setuptools-setuppy
you reviewed the history of packaging and various tools and standarts.
Also I provided an example of the pyproject.toml. How do you create
it? 

It depends on your stack. If you use just pip and venv, you create it manually. If you use things like poetry or uv, it creates one for you.

For years I had a pyproject.toml template I copied/pasted around. But nowadays you can just prompt chatgpt and it gives one for you.

I just typed 'Create a pyproject.toml file for project "blabla" with dependencies on "requests" from bash', and here is the output. It works fine:

$ echo '[project]

name = "blabla"
version = "0.1.0"
description = ""
authors = []
dependencies = ["requests"]

[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
'

I may write an article on how pyproject.toml is structured one day.

The point is: we introduce whole tools just to avoid typing a few keys for an operation we don't even do that often. And indirections are never free.

And how do you install dependencies using this file?

You activate your venv, and you pip install it from the root directory:

$ pip install .

How do you manage deps without poetry?

Depends on the project. For most projects, I used to manually craft the pyproject.toml and pip freeze for deployment. It's enough for the vast majority of code bases.

For more fancy projects, I used pip-tools.

Of course, my clients may mandate the stack, I don't always have a choice in the matter. Anaconda is unfortunately everywhere.

All this will likely change now that uv is getting more and more popular.

And in the second article
https://www.bitecode.dev/p/why-not-tell-people-to-simply-use you
shared thoughts on how the python should be properly installed. I used
pyenv for that purpose.

But decided to take a look and install python with version 3.9.19 and
there is no prepared macOS installer for version, only for the 3.9.13
version.
How do you propose to install it in a proper way, just install 3.9.13
and somehow update or just build it for macOS using tar or gzip
package?  It's a really interesting question.

(Some context that wasn’t in the email: Python 3.9 is in security mode, meaning it will only receive security patches. When this happens, python.org doesn’t provide installers for minor releases anymore, so this is a really good question. Because there are no bug fixes, installing an old version is fine on your dev machine as it is highly unlikely that it will cause compat problems. On the server, it’s another matter, since you may want those security patches, and this is why I advice to use deadsnake on Ubuntu, which makes the lastest versions available).

It all depends on your requirements. For the general population, installing 3.9.13 and not upgrading is fine. In fact, it's likely fine for 99% of the developers. We tend to focus on details like this to perfectionism, but the chances that it actually cause problems are very low.

Now, there are some cases where this is not acceptable. E.G:

- Your project has a deployment size so big the probability of you hitting a problem is high.

- You already know there is a bug below 3.9.19 that is not acceptable to you, and you need this version.

- You have legal requirements to do this.

- Etc.

In those cases, you are expected to have the experience and expertise to use advanced tools for what they can provide. Pyenv is an excellent tool for those situations. But this means you also must have the skill to deal with its modes of failure.

All the articles have been written with in mind the fact that people who need it and can ignore their advice will self-exclude from them.

It's all a matter of really needing it and your entire team, present and future, being capable of handling it.

We are quite bad at evaluating this in the community. People will say they need something when they really just want it. Or they will say their team can handle it, ignoring the juniors, interns or offshore colleagues.

Also, frustration plays a role in bad decisions.

If you decide to use 3.9.13 and hit a bug that makes you waste hours, you have to resist the temptation to make your whole stack more complicated just because you've been inconvenienced by something once.

Finally, devs are terrible at evaluating how much complexity they introduce by those choices. I still hear regularly that pyenv is easy to use, despite the fact it falls back on compilation for half the configurations in the world. It's too easy to live in a bubble when you know what you are doing.

Thank you in advance!

Best regards,
Alex

Cheers,

PS: that was longer than I expected. Would you be ok if I publish this conversation in the newsletter?

(Obviously, he said yes).