Reading the stack trace to understand python errors

And how to warm up your loved ones spam folder

Reading the stack trace to understand python errors

Summary

Stack traces are your friends. Just remember to:

• read them from bottom to top;

• identify the exception raised;

• then follow up the chain of events that lead to the error;

• and maybe put a little break point in there.

Good news everyone!

You were running a simple script but it explodes, and you get something like this:

Traceback (most recent call last):
  File "example.py", line 9, in perform_calculation
    result = divide_numbers(numerator, denominator)
  File "example.py", line 4, in divide_numbers
    return a / b
ZeroDivisionError: division by zero

You see this horrible flow of error lines, and you think of a wave of YouTube demonetization slurs.

Well, believe it or not, this is a good thing.

Not that your program crashed, no.

But that you have those lines.

Those lines, called the stack trace, are not the enemy. There are not even your friend, they are your brothers in arms and they got your back.

Reading the stack trace

There is no such thing as an obvious concept, and stack traces are no exceptions. Regularly, I encounter Python coders that fear them, because they think it's bad.

Errors are bound to happen, and you will have to solve them. What would be bad would be to do it blind.

So if you were apprehensive about the stack trace so far, let's learn how to read it together.

First, the most important tip:

Stack traces are read in reverse, from bottom to top:

Then you have to understand there are really two parts in it:

• The first line (the one at the very bottom) is the error that happened.

• All the other lines above are the list of events that lead to this error.

What to do with this

What happened?

The first line contains information about the nature of the error. First the name of the exception, second, a short explanation.

In our example, the name of the exception is ZeroDivisionError, while the explanation is division by zero.

So what are you supposed to do with this?

First, check if the exception is documented. Google the name (or use ChatGPT), to figure out what is this type of error.

In this case, ZeroDivisionError is documented here, and we can learn in which context this exception is raised.

(Side note, the stdlib exception page is a very good read, as you will encounter those exceptions very often, so being familiar with them will make your life a lot easier)

Then, reading the explanation "division by zero", you get that somewhere the code ends up trying to divide by something by the number 0, which is mathematically impossible.

In this basic example, it's simple, because the exception name is very clear, it's in the standard library, and the explanation is crystal clear.

Sometimes, it's not that clean, or even cryptic.

You can increase your chances of understanding what it means by installing a recent Python, as they have way better messages than older ones (and the trend will continue with the next releases).

If the error is simple like here, it can be sufficient to understand the problem. In real life code, it's not necessarily the case, and we need to move to the next section.

How did it happen?

The part:

Traceback (most recent call last):
  File "example.py", line 9, in perform_calculation
    result = divide_numbers(numerator, denominator)
  File "example.py", line 4, in divide_numbers
    return a / b

Is an history of all the steps that lead to this error.

First, the error manifested here:

  File "example.py", line 4, in divide_numbers
    return a / b

Which means, in the file "example.py", on line 4, in the function "decide_numbers", there is the code return a / b, which broke.

This may be enough for you to figure it out. If you know this file well, and you know where b comes from, then you know why b is equal to 0, and therefore you have found the bug.

If not, you need to know where does this b come from, and move to the line above:

  File "example.py", line 9, in perform_calculation
    result = divide_numbers(numerator, denominator)

So this return a / b was part of a divide_numbers function, and now we can see that this function was called on line 9 in "example.py" in the function "perform_calculation", passing denominator which is likely where the 0 was passed.

And that way, you go up and up, from file to file, from line to line, until you figure out where this damn problem comes from.

If you can't just by reading the code, you may even use the stack trace to choose a good spot to put a breakpoint and debug from there. If you don't know how to do it, we got a good article on that.

A concrete example

Let's imagine a script to send a few emails:

import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from pathlib import Path


def send_newsletter(
    subscriber, article, smtp_server, smtp_port, smtp_username, smtp_password
):
    msg = MIMEMultipart()
    msg["From"] = "contact@bitecode.dev"
    msg["T0"] = subscriber
    msg["Subject"] = "[Bite Code!] New article"

    msg.attach(MIMEText(article, "plain"))

    with smtplib.SMTP(smtp_server, smtp_port) as server:
        server.starttls()
        server.login(smtp_username, smtp_password)
        server.send_message(msg)


subscribers = (
    "mum@bitecode.dev",
    "test@bitecode.dev",
    # "guido@python.org"  # he said no, but maybe in comic sans, later?
)

article = Path("article.md").read_text()
smtp_server = "mail.gandi.net"
smtp_port = 587
smtp_username = "contact@bitecode.dev"
smtp_password = "admin123"

for subscriber in subscribers:
    send_newsletter(
        subscriber, article, smtp_server, smtp_port, smtp_username, smtp_password
    )

I run it, and boom, I get:

Traceback (most recent call last):
  File "/path/to/my/script.py", line 36, in <module>
    send_newsletter(
  File "/path/to/my/script.py", line 20, in send_newsletter
    server.send_message(msg)
  File "/usr/lib/python3.10/smtplib.py", line 986, in send_message
    return self.sendmail(from_addr, to_addrs, flatmsg, mail_options,
  File "/usr/lib/python3.10/smtplib.py", line 901, in sendmail
    raise SMTPRecipientsRefused(senderrs)
smtplib.SMTPRecipientsRefused: {}

I have zero idea of what went wrong.

First I read:

smtplib.SMTPRecipientsRefused: {}

And I find that SMTPRecipientsRefused is documented here, stating it means "All recipient addresses refused".

WTF?

I have a recipient address, and I know it's good because I write to my mum every day! Ok, I don't, but it worked when she sent me conspiracy links last month.

Let's see the explanation:

{}

Err... This is... Not terribly helpful.

Ok, let's follow the events that lead to this problem.

First:

  File "/usr/lib/python3.10/smtplib.py", line 901, in sendmail
    raise SMTPRecipientsRefused(senderrs)

Well, that's not my code, that the stdlib. Let's open that file. Line 901, I got:

if len(senderrs) == len(to_addrs):
    # the server refused all our recipients
    self._rset()
    raise SMTPRecipientsRefused(senderrs)

Oh, so somehow the servers rejected all my recipients. But look, senderrs is what's passed to the exception, which explains our {} as the message explanation. It's supposed to list all the senders that didn't work, but it's an empty set!

But why is it empty?

Let's put a breakpoint here and check where senderrs is comming from (I'll need admin rights, and take a mental note to put everything back cleanly as those are my system python files!):

> /usr/lib/python3.10/smtplib.py(939)sendmail()
-> self._rset()
(Pdb) l .
934                     self.close()
935                     raise SMTPRecipientsRefused(senderrs)
936             if len(senderrs) == len(to_addrs):
937                 breakpoint()
938                 # the server refused all our recipients
939  ->             self._rset()
940                 raise SMTPRecipientsRefused(senderrs)
941             (code, resp) = self.data(msg)
942             if code != 250:
943                 if code == 421:
944                     self.close()

(Pdb) up
> /usr/lib/python3.10/smtplib.py(1035)send_message()
-> return self.sendmail(from_addr, to_addrs, flatmsg, mail_options, rcpt_options)
(Pdb) l .
1030                    mail_options = (*mail_options, "SMTPUTF8", "BODY=8BITMIME")
1031                else:
1032                    g = email.generator.BytesGenerator(bytesmsg)
1033                g.flatten(msg_copy, linesep="\r\n")
1034                flatmsg = bytesmsg.getvalue()
1035 ->         return self.sendmail(from_addr, to_addrs, flatmsg, mail_options, rcpt_options)
1036
1037        def close(self):
1038            """Close the connection to the SMTP server."""
1039            try:
1040                file = self.file
(Pdb) to_addrs
[]

So our empty variables from from this self.sendmail() call line 1035, but to_addrs is empty. Where is this to_addrs created?

(Pdb) l 980,1030
980             # option allowing the user to enable the heuristics.  (It should be
981             # possible to guess correctly almost all of the time.)
982
983             self.ehlo_or_helo_if_needed()
984             resent = msg.get_all("Resent-Date")
985             if resent is None:
986                 header_prefix = ""
987             elif len(resent) == 1:
988                 header_prefix = "Resent-"
989             else:
990                 raise ValueError("message has more than one 'Resent-' header block")
991             if from_addr is None:
992                 # Prefer the sender field per RFC 2822:3.6.2.
993                 from_addr = (
994                     msg[header_prefix + "Sender"]
995                     if (header_prefix + "Sender") in msg
996                     else msg[header_prefix + "From"]
997                 )
998                 from_addr = email.utils.getaddresses([from_addr])[0][1]
999             if to_addrs is None:
1000                addr_fields = [
1001                    f
1002                    for f in (
1003                        msg[header_prefix + "To"],
1004                        msg[header_prefix + "Bcc"],
1005                        msg[header_prefix + "Cc"],
1006                    )
1007                    if f is not None
1008                ]
1009                to_addrs = [a[1] for a in email.utils.getaddresses(addr_fields)]
1010            # Make a local copy so we can delete the bcc headers.
1011            msg_copy = copy.copy(msg)
1012            del msg_copy["Bcc"]
1013            del msg_copy["Resent-Bcc"]
1014            international = False
1015            try:
1016                "".join([from_addr, *to_addrs]).encode("ascii")
1017            except UnicodeEncodeError:
1018                if not self.has_extn("smtputf8"):
1019                    raise SMTPNotSupportedError(
1020                        "One or more source or delivery addresses require"
1021                        " internationalized email support, but the server"
1022                        " does not advertise the required SMTPUTF8 capability"
1023                    )
1024                international = True
1025            with io.BytesIO() as bytesmsg:
1026                if international:
1027                    g = email.generator.BytesGenerator(
1028                        bytesmsg, policy=msg.policy.clone(utf8=True)
1029                    )
1030                    mail_options = (*mail_options, "SMTPUTF8", "BODY=8BITMIME")

Apparently from this snippet:

1000                addr_fields = [
1001                    f
1002                    for f in (
1003                        msg[header_prefix + "To"],
1004                        msg[header_prefix + "Bcc"],
1005                        msg[header_prefix + "Cc"],
1006                    )
1007                    if f is not None
1008                ]
1009                to_addrs = [a[1] for a in email.utils.getaddresses(addr_fields)]

Let's see what gives:

(Pdb) header_prefix
''
(Pdb) msg[header_prefix + "To"]
(Pdb) msg["To"]

Wait, why is this giving me nothing?

I'm certain I set it in the script.

(Pdb) msg.keys()
['Content-Type', 'MIME-Version', 'From', 'T0', 'Subject']

So I do set it.

Is it case sensitive? I'm pretty sure SMTP doesn't care.

(Pdb) msg['TO']
(Pdb) msg['T0']
'mum@bitecode.dev'
(Pdb) 'TO' == 'T0'
False

Ah, damn it, I used a zero instead of an "O". Stupid typo.

Now my spammer legitimate newsletter script is working fine. I received an email stating my own mother ghosted me:

This is the mail system at host relay8-d.mail.gandi.net.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<mum@bitecode.dev>: host spool.mail.gandi.net[217.70.178.1] said: 550 5.1.1
    <mum@bitecode.dev>: Recipient address rejected: User unknown in virtual
    mailbox table (in reply to RCPT TO command)

So you get the gist:

• read the stack trace from bottom to top;

• try to figure out what type of error you have;

• move along the chain of events to figure out where everything goes wrong;

• if needed, put a break point to investigate.

That reminds me I have to remove that breakpoint(). I'll be right back.

By the way

Don't use that script IRL, sending a lot of emails like this is going to be not only very slow, blocking your program, but will also likely get you marked as spam. This is just an artificial example for this article.

Also you don’t really have to edit files with admin rights, you can use the breakpoint command in PDB or a debugger with a GUI to avoid this, and it’s less risky. But harder to describe in a blog post.