Environment variables for beginners

Yes, it's for Python too

Summary

Environment variables are variables that can be set from outside a program and read from inside a program. They are very useful, but more and more devs don't know how they work. Hence this article.

You can read them:

• On windows using the "Edit the system environment variables" entry from the start menu or %VAR_NAME% in cmd.exe.

• On Linux and Mac with $VAR_NAME in a shell.

• In Python using os.environ['VAR_NAME'].

You can set them:

• On windows using set VAR_NAME=value in cmd.exe.

• On Linux and Mac with export VAR_NAME=value in a shell.

• On Python, it is possible but rare to set them.

You can persist them:

• On windows using the "Edit the system environment variables" entry from the start menu or setx VAR_NAME= alue in cmd.exe.

• On Linux, there are many ways, but the most likely one is putting export VAR_NAME=value in $HOME/.bashrc.

• On Mac, there are many ways, but the most likely one is putting export VAR_NAME=value in $HOME/.zshrc.

• On Python, it is possible but rare to persist them.

Lots of programs use env vars to let technical users to configure behavior, and you can do so in your script too. You can control the behavior of Python itself through 11 env vars, such as PYTHONSTARTUP which will run a script every time a shell starts.

PATH is the most important environment variable, as it controls what program you can run from anywhere in the shell, on Windows, Mac and Linux.

The forgotten configuration knob

As a freelancer, moving from company to company regularly exposes me to some surprising truths that my colleagues often don't believe when I share them. Things like:

• Windows is still on the majority of dev machines. Not Mac. Certainly not Linux.

• A lot of devs are productive without ever touching the terminal.

• Choosing your stack and tooling is a luxury many don't have.

Today I write to support the remarkably high number of coders that don't know what an environment variable is, and how to use one. Windows users can read Ultra beginners first steps for the terminal on Windows to refresh their memory before starting.

Environment variables are variables that you can define in the programs that run other programs.

Think about it.

The programs you run don't start from nothing. The overwhelming majority of them, including all the scripts ever written, are executed from another one: a shell, an editor, a button/menu provided by your desktop manager, etc.

Environment variables are variables, but variables you can set at the level of your shell, your editor, or even your entire OS, so that any program you start from there can read them.

Initially it was a mean of communication between a parent and a child process, but it has become a super useful way of passing configuration values to a program, such as Python code, from outside of this program.

It's a useful thing to know how to manipulate in general, even if you don't program at all. But as we will see later on, it's very useful in programming, with or without Python.

Reading environment variables

Like any variable, environment variables can be read and written, but unlike them, you often write one in one program, and read it in a different program.

Your computer starts with a lot of environment variables already set, the most important one being PATH, a variable that contains a list of directories.

Let's try to read the content of PATH

On Windows

On Windows you can read any en variables in two ways, using a graphical interface, or the terminal.

Let's start with the GUI:

• Open the start menu and search for "Environment Variables"

• Click "Edit the system environment variables"

You will see a windows looking like this (on Windows 10 or 11):

There are 2 sections. The top one is for variables for you only, while the bottom one is for the entire system. Focus on the first one, which is specifically your user.

You can select "PATH" and click the "EDIT" button to read the content of the PATH variable.

Don't modify it though, it has consequences on your system. Close all windows.

It can be done from the terminal as well. On Windows, environment variables can be used using the %ENV_VAR_NAME% syntax in cmd.exe. So open cmd.exe (again, check there if you forgot how), and type:

echo %PATH%

Any machine can be configured differently, but the result will be a bunch of directory paths (sometimes with variables themselves), separated by ";". E.G.:

%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\

On mac and Linux

Mac and Linux don't typically invite you to manipulate environment variables from a GUI, it's all from a terminal.

The good news is that most MacOS are configured to use ZSH, and most Linux boxes to use Bash, so they have the same syntax by default to use an environment variables: $ENV_VAR_NAME.

So open a terminal and type:

echo $PATH

On my machine, it looks like:

/home/user/.bun/bin:/home/user/.local/bin:/home/user/.nvm/versions/node/v16.14.0/bin:/home/user/.local/bin/:/home/user/.cargo/bin:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Again a list of path directories, but this time separated by ":".

In Python

Python can read environment variables from the os.environ dictionary. In a Python shell:

>>> import os
>>> os.environ['PATH']
'/home/user/.bun/bin:/home/user/.local/bin:/home/user/.nvm/versions/node/v16.14.0/bin:/home/user/.local/bin/:/home/user/.cargo/bin:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin'

Because Python is portable, the way you read the variable is the same on Windows, Linux or Mac, but the value will be specific to the OS you are on.

Setting environment variables

On windows

We will see the graphical method in the "Persisting an environment variable".

For now it will be about the terminal.

You can use set VARNAME=value to set an environment variable for the current cmd.exe session:

set QUEST=grail

It can be read in the same terminal:

echo %QUEST%
grail

Any program that is started from this terminal can read the variable:

>>> import os
>>> print(os.environ['QUEST'])
grail

But it can't be read outside of this terminal, and will disappear once you close it.

On mac and linux

You can use export VARNAME=value to set an environment variable for the current shell session:

export QUEST=grail

It can be read in the same shell:

echo %QUEST%
grail

Any program that is started from this shell can read the variable:

>>> import os
>>> print(os.environ['QUEST'])
grail

But it can't be read outside of this shell, and will disappear once you close it.

In Python

While one may set environment variables for modules like "subprocess", this is an advanced use case. I will simplify the situation by considering Python usually reads environment variables, but seldom need to write them, so you can ignore this part.

Persisting an environment variable

All those variables you have created so far will disappear next time your machine reboot. What if you want to keep them around?

On windows

Using the graphical interface, adding or setting a environment variable is not much harder than reading it. You go to the exact same place, remember this?

But now you click "New" and choose a name and a value:

This quest probably uses graalvm. Definitly not a typo.

After you restart your computer, all programs you start will have access to the QUEST variable.

You can do the same in cmd.exe with setx VARNAME value. Note the absence of the = sign. E.G.:

setx QUEST grail
SUCCESS: Specified value was saved.

On mac

On mac, there is no central mechanism to persist environment variables, but there are several files that are loaded at different times when your machine boots. It's actually quite a complicated topic.

For this tutorial though, you can focus on editing (or creating) a file named ".zshrc" in your user directory. Open it with any text editor, and put this at the very bottom of the tile:

export QUEST=grail

Now, every time you start a new shell, this variable will be available.

On Linux

Like mac, there is no central mechanism to persist environment variables, but there are several files that are loaded at different times when your machine boots. It's even a more complex matter because you have many different linux distributions with all kinds of setup.

A huge majority of linux distros uses bash though, so let's focus on editing (or creating) a file named ".bashrc" in your user directory. Open it with any text editor, and put this at the very bottom of the tile:

export QUEST=grail

Now, every time you start a new shell, this variable will be available.

In Python

Technically, you can do that with things like .env files, but again, let's simplify and say we don't need it in Python. Keep Python for reading environment variables, not writing them.

So what's the big deal?

Ok, so you can set variables from outside a program and they can be read inside the program, why is that nice?

Well, it's a way to configure programs that is between a configuration file (persistent) and a command line option (easy to change).

You can use it in your own scripts to allow a user to change some option from outside, and they can decide if they want to keep it forever that way, just for this session, just for this run, etc. It's very flexible.

So flexible in fact, it's used everywhere.

Python alone has 13 environment variables that will change the behavior of the interpreter itself!

E.G:

• PYTHONPATH lets you set a directory (or a list of directories). Anything inside those directories are now importable, now matter where you are, or they are, on the system. I often set that to ".".

• PYTHONSTARTUP takes a path to a python script. When you start a shell, this script runs. Tired of always importing the same thing in the shell? You can put that in your startup script, and it's automatically imported.

• PYTHONBREAKPOINT controls how the python debugger works. You can set PYTHONBREAKPOINT to ipdb.set_trace to use ipdb instead of pdb.

Once you know how environment variables work, there is a world of possibilities that opens to you.

PATH, the most important environment variable

PATH is special. Firstly, it's an environment variable that does the same thing on Windows, Linux and Mac, which is rare.

Secondly, PATH contains a list of directories, and any program in those directories can be started as a command in a shell. If you type the name vlc, your system will look for a program with that name in each of those directories, and run it when it finds it.

When you try to run a command and it says "not found", it's because the command directory is not in PATH.

It controls what you can and can't do as main actions in your terminal, and what program have priority on what.

Indeed, the order in PATH is important. If you call the program "vlc", and it exists in two directories in PATH, only the first one is called.