What's up Python? New packaging proposal, Python in Excel, a piano in the terminal...

August 2023

Summary

• Microsoft announces Python support in Excel

• Single file script deps declaration: a new pyproject.toml compatible proposal

• Bumping security for pypi: 2FA now activated and one more hire

• Milestones: wheels are used a lot, vscode gets better at mypy and 3.12 RC1 is out

• New textual-based project: a piano in the terminal

Python support in Excel

The big new of this August was, of course, Microsoft announcing official Python support in Excel.

Python being super popular for data manipulation is hardly a secret, but when it comes to cols and rows, the story was always a conundrum:

• Manipulate everything in Python/DB, and dump the result in a CSV or a very basic spreadsheet.

• Manipulate things with a spread sheet, and then automate Excel itself using xlwings or the com interface.

• Read the Excel raw result and use Python as a dumb pipe.

• Throw LibreOffice at the task, at least it's natively programmable in Python.

Now we can add the ability to drop Python code directly into an *.xls cell. Whether it will be a good programming experience or not had yet to be seen.

Since Excel must run in a web browser nowadays, and given most people don't follow "Relieving packaging pain", MS went for the nuclear option: running the Python snippet actually executes it in the cloud, in Anaconda's, no less.

I wonder how they will tackle the indentation problem, the fact Python has as many statements as expressions, and that cells are quite small. But we will have to join the insider program to check it ourselves.

This also moves the firm's strategy one step further. They hired Guido van Rossum, added the default python command in cmd.exe if no VM is installed, acquired github (and so copilot), then major stakes in OpenAI, and now this.

What's next, buying Continuum?

PEP 723, follows up on PEP 722

Last month, we talked about PEP 722, a proposal to allow a special syntax to define dependencies requirements for single file scripts in comments. There was a lot of back and forth on it, and finally the idea got incremented upon with PEP 723.

In this new version, the syntax has been chosen to reuse the semantics of pyproject.toml files, so that we keep some standardization:

# /// pyproject
# [run]
# requires-python = ">=3.10"
# dependencies = [
#   "requests<3",
#   "pandas"
# ]

import requests
import pandas
...

I like this a lot. It will make using single file scripts much easier will allow existing tools to implement it quickly; plus it's backward compatible with pretty much the whole modern ecosystem.

I also have hopes this will lead to more tools actually using this kind of data to provide zipapps or stand alone executables.

Bumping security for pypi

Pypi's focus on security has been steadily increasing for a while, and we get a few changes. There was the announcement to ramp up 2FA, and starting from today, newly registered users must enable 2FA before they can perform any management actions on PyPI.

On top of that, Mike Fiedler is joining the team as Safety & Security Engineer. He was already a contributor and maintainer of Pypi for a couple of years, and can go full time for the PSF thanks to a donation from Amazon.

Cheesy milestones

On bitecode, there are quite a few articles about packaging, but we never really made a retrospective of everything that has changed. Despite the fact people believe it's always been broken, I think most devs don't realize it used to be worse.

Way worse.

For a long, long time, the installing things relied on zipped sources, or, God had mercy on your soul, .eggs files. If you don't know what this means, it's an old parseltongue word for "toss a coin and pray".

Nowadays, we have the much better .whl format, or "wheels", named like this because Pypi used to be called "the cheese factory". Monty Python jokes never end. Wheels are good stuff: they make it easier for packagers to ship binaries to you, so you can pip install numpy without much trouble.

Well, this month 357 of the most popular 360 packages are now wheels. This has been tracked on the python wheel leaderboard, and is a testament of the inertia of such big communities.

The last 3 remaining ones are:

• future: a portability layer to make code that works python 2 and 3. I doubt people will ever be motivated to port that to a wheel.

• pyspark: a lib to interact with Apache Spark and make big calculations on arrays. Lot's of exotic compiled things, so hard to port, but also highly valuable to have as a wheel. Installing pyspark is hit or miss.

• sagemaker: the SDK to deal with the ML AWS service. Never used it, so can't really tell.

Other than that, the usual bumps. Mypy 15 is out with improved dataclasse and dict support. VSCode gets better at dealing with pytest test discovery quirks.

And Python 3.12 is in RC1, RC2 is this monday, as we are closing on November’s release. Python 3.11.5, 3.10.13, 3.9.18, and 3.8.18 are also now available

Brace for a flow of articles rewording the release notes without even testing the new version and posting it on reddit and HN :)

Textual is upping the coolness

Yes, I talk a lot about textual. No, Will McGugan is not my secret sponsor. Rumor has it he pays in spicy food, and there are only so many tears in my body.

It's just that it's a good project that keeps inspiring more and more fun. Last time, it was paint, and today, I'd like to share upiano with you a terminal based piano app:

Come on, tell me it's not awesome.