What's up Python? Hatch inline deps, an UI to write pandas, stand alone builds...

May 2024

Summary

• Hatch v1.10.0 supports inline dependencies

• Generate Pandas scripts from a VSCode UI

• A new tool in the Python distribution arena

• Quality of life improvement from mypy 1.10

Hatch v1.10.0 supports inline dependencies

Hatch is a Python project management tool, with goals encompassing those of poetry and pipenv, but also spreading to formatting, task running, and testing.

It didn't start that way and was, initially, mostly a build tool. So it fell off my radar for quite some time.

Recently though, they released version 1.10 with inline deps support, following one of my favorite proposals of the last few years: PEP 723.

We talked about this before, but the idea is to be able to put part of a pyproject.toml in scripts comments so that you can embed the dependencies requirements. E.G.:

# /// script
# requires-python = "3.6"
# dependencies = [
#   "requests",
#   "rich",
# ]
# ///

import requests
from rich.pretty import pprint

resp = requests.get("https://peps.python.org/api/peps.json")
data = resp.json()
pprint([(k, v["title"]) for k, v in data.items()][:10])

Then if you run:

hatch run script.py

This will install the deps in a temporary venv, then print:

[
│   ('1', 'PEP Purpose and Guidelines'),
│   ('2', 'Procedure for Adding New Modules'),
│   ('3', 'Guidelines for Handling Bug Reports'),
│   ('4', 'Deprecation of Standard Modules'),
│   ('5', 'Guidelines for Language Evolution'),
│   ('6', 'Bug Fix Releases'),
│   ('7', 'Style Guide for C Code'),
│   ('8', 'Style Guide for Python Code'),
│   ('9', 'Sample Plaintext PEP Template'),
│   ('10', 'Voting Guidelines')
]

I like that the community is starting to adopt the concept. The more we have it in the wild, the more we can play with it and polish it, and maybe eventually integrate that with the standard pip/venv stack.

Right now, as you may know, I recommend having a big venv dedicated to all your little scripts.

The new version also puts on the table a test command (that delegates to pytest under the hood), and uv support which makes the inline deps feature super fast.

This tells me I should take another look at Hatch, because it does many interesting things:

• Complex behaviors are provided by proven tooling. E.G.: hack fmt will call ruff by default. But this is pluggable.

• It normalizes up very common operations: env creating, version bumping, running tox/nox... Something I typically use doit for, in each project.

• It supports custom scripts, so it's a small task runner. True, it's much more basic than doit, but npm proved you can already do a lot with this approach.

Between this and Astral work, I'm starting to get good vibes about the Python project management story. I'm still hopping to declare "Relieve your Python packaging pain" obsolete one day.

Generate Pandas scripts from a VSCode UI

Microsoft released an official VSCode extension, Data Wrangler, that lets you manipulate tabular data (E.G.: from a CSV, Excel or Parquet) with standard operations such as ordering, filtering, aggregating, or cleaning the data. While you do so, it writes the equivalent pandas code in a split tab for you.

My favorite dataset when I give a Python for Data Scientists training: the stats on plane crashes due to collisions with birds! Lots of “correlation is not causation” traps, but easy numbers to manipulate.

While the ergonomics are not great, not terrible, for now, it's already an interesting tool for all those coders that are not professional devs and just want to automate number massaging. I can see it being also useful for people not familiar with pandas yet, as the lib’s API is quite obscure.

Given Python is in Excel now, I'm guessing this is a way for MS to preview the enthusiasm for a tool that may end up in Office eventually.

A new tool in the Python distribution arena

I stumbled upon "Replacing pyinstaller with 100 lines of code" during my weekly tech review, with very interesting insights on what can be done with python-build-standalone, something I've been talking about for some time. The kicker here, is that using those static binary Python distributions coupled with self-extracting archives, you can ship a single executable for your Python program.

And yes, Nuitka and PyOxidizer (mentioned in the article) do this quite well, but the approach, published as the packaged lib, has the benefit of being simple enough to understand and may inspire others.

Again, I don't think we have reached peak user-friendlyness, but I'm ecstatic that the state of the art is advancing at this pace recently.

Still waiting for Astral to jump the gun on this.

Exciting times.

Quality of life improvement from Mypy 1.10

V1.10 is out, and while Mypy releases don't always catch my attention, this one did because it brings...

Support for TypeIs (PEP 742)

Basically, a better version of TypeGuard that lets you say "if I'm in this branch, the type is this, if I'm in that branch, the type is that":

from typing_extensions import TypeIs # until python 3.13

def is_string(s: object) -> TypeIs[str]:
    return isinstance(s, str)

def a_super_function(param: str | int) -> None:
    if is_string(param):
        # Type of param is 'str'
    else:
        # Type of param is 'int'

Doesn't seem like much, but the fact you couldn't do this before was irritating in dynamic code bases.

Support for TypeVar Defaults (PEP 696)

If you make a generic collection, you can now say that by default it will contain a certain type of objects:

from typing import Generic
from typing_extensions import TypeVar # until 3.13

T = TypeVar("T", default=int)

class BunchOfStuff(Generic[T]):
    ...

x = BunchOfStuff()
y: BunchOfStuff[str] = BunchOfStuff()
reveal_type(x)  # BunchOfStuff[int], because of the default
reveal_type(y)  # BunchOfStuff[str]

I realize many readers may not know what the heck is a generic type, which is why I will likely do a series of articles on typing at some point. But the series on testing is far from done so it's in a distant future.