A little taste of HTMX (part 1/5)

It's a long tutorial and the result is ugly. Wanna read it?

Summary

Since the entire Web has voted for to-do lists to be the canonical frontend toy example, this is what we are going to do.

To get into the state of mind required to program in HTMX, we must pretend Harry Potter has not been written yet, and make a site that operates with raw HTTP. Not even HTMX, just pure GET and POST back and forth. It will work, mind you. It will suck, but it will work.

For this we will need a few functions to add/delete/update tasks in a database, and a few URLS calling those functions. The backend will be in Python, but fear not, you don't need to know Python to follow. I worked hard to strip down the thing to the most minimal and clear logic I could. Just read the names, you'll be fine.

The source code is tagged and ready for download if you want to jump on it, but I recommend rather to read this article step by step if you started coding with post-2015 frontend tech stacks. I'll ease you into it.

Once this monster from the past works and you understand I actually want good things for you despite my wooden club and wolf-fur swag, we'll do a first timid HTMX attempt as a teaser.

Prerequisites

To follow this tutorial, you only need essential web dev skills. I'll drop the context of what you need on the way.

But you do need those basics. I can't explain what the DOM and Ajax are, that would take too much time as this tutorial will already likely take 4 or 5 articles, which means between 20 and 30 hours to write.

However, you DON'T need to know backend programming.

That's because HTMX is completely neutral to the backend technology you use, it sees only HTTP, and you will get better at using it if you think that way. Having Node, PHP or C# on the other side of the fence is an implementation detail.

Understand that our goal is only to give you a taste on you how to use HTMX. We don't aim for perfs, correctness, great error handling and accessibility concerns, as I would need 10 more articles. Although they are surprisingly decent out of the box.

The stack

There is a long tradition of using the venerable To Do List example to demonstrate how front end tech works. You can easily find examples of the same app for all JS frameworks, so I'm going to use exactly that for familiarity, simplicity, and giving you a point of comparison. Take it with the usual suspicion about toy examples, but it's a useful one.

Yes, I did say the server will be written in Python, but don't be afraid if it's not your fav language. I carefully picked the requirements to be minimal and wrote code in a barebone way to prevent noise from burying what's important to understand. I will give plenty of context on the way, and again remember that for HTMX, what's important is how the Web works, not the tech you use to power the machinery.

You can decide to skip installing Python at all. The final backend code is less than a hundred lines with explicit naming. It's OK if you just get the general idea.

However, if you do have Python installed, you can run the project as is. There is no need to select a particular version of Python (yep, it works also with Python 2), you don't need to create a virtualenv or install anything. Download the zip, go in one folder, then execute server.py.

Similarly there is no need to install anything for the JS or CSS part. No npm, no compilation. It's all in there already. That's what made the browser such a popular and powerful platform in the first place: a standardized programmable VM installed on virtually every machine in the world with self-contained scripting and update abilities.

Here are the specific libs we will use:

• bottle for the server. It's a Python lib to make web dev that fits in one file, no deps, no install procedure. I drop that in a folder and import it.

• picocss. It's a CSS lib that gives a decent style to the page with minimal markup. I stick a script tag to a CDN and to can have a PoC while writing almost raw HTML.

• HTMX. Of course.

If you started Web dev after 2015

Feel free to skip this wall of text if you are a veteran, it's just a bit of context for our younger readers.

Today it may feel like there is no other way than popping a JS framework to write some frontend code and interact with the backend through Ajax calls or SSR, but it hasn't always been that way.

In fact, it's a very recent trend.

The Web started to really heat up when Microsoft published Internet Explorer, eventually killing the popular Netscape with love (IE 5.5 was indeed a much better product) and monopoly abuses.

At the end of the nineties, The Matrix and Amazon already existed, but JS was slow as hell, there was no debugger or console in the browser and CSS support was terrible. So the web was mosty... HTML.

The layout was done with a ton of <table>, inline styling (that puts a twist on tailwind, doesn't it?) and few human sacrifices.

Here is an extract of code from the famous 1996 space jam website:

<center><nobr>
<table border=0 cellpadding=5 cellspacing=0><tr>
<td align="center" colspan=3><!--#include virtual="/html.ng/site=spacejam&spacedesc=button.top" --></td>
</tr><tr>
<td align="center"><!--#include virtual="/html.ng/site=spacejam&spacedesc=button.left" --></td>
<td align="center" width="10"></td>
<td align="center"><!--#include virtual="/html.ng/site=spacejam&spacedesc=button.right" --></td>
</tr></table>

And here is what Apple, master of design, could do the same year:

Skeuomorphism was not an iPhone thing. Visit webdesignmuseum for more of this. 

Why do I say all that? First, because it already allowed a lot of the web as you know it today to work You could login, post comments, buy stuff and listen to sounds (cheating a little bit with Flash). You could also search for women between 18 and 45 on Youtube. Yeah, everybody forgot that one.

But more than that, because it still works. The principles that made all that stuff operate are the foundation of today's web dev, we just added layers on top.

Don't worry, I'm not advocating returning back to living like digital cavemen, I'm just saying the primitives are still available, efficient, elegant and robust.

HTMX realization is that there is nothing wrong with those primitives. We could embrace them more!

But without JS, only <a> and <form> can send requests. And just GET and POST, no DELETE, no PATCH. Plus, only when the user asks for it. When the request succeeds, you lose the entire page, and the new page replaces it completely.

So when Microsoft came up with XMLHttpRequest (because of Outlook for God’s sake!), manual requests won.

However you can do a lot with just the DOM, events and HTTP requests.

Remember it's not HTMX or SPA, though. It's HTMX and SPA. And static websites. And stuff that works with lynx, Figma style web assembly monsters or whatever Unity produces.

HTMX also doesn't mean you stop using JS: all my HTMX powered apps, which are only a fraction of what I dev, involve me writing some JS. You can even mix HTMX and react, like a mad man.

HTMX does make it easy to not use JS explicitly or even create dynamic sites that degrade gracefully when JS is disabled, but the ship of progressive enhancement has sailed last decade. I'm not here for that. I just like to get stuff done.

So here is the deal: you know how you send Ajax requests with fetch(), axios.get and $.ajax? We are basically going to do this with HTMX, but instead of exchanging JSON, passing it manually to some JS logic that will insert that in the page, we are going to ask the server to give us the final result directly, and insert that. HTMX does the JS logic, though, it's a trick. But it's close enough to the native experience.

Because the web started that way, browsers are super good at rendering it, and with the progress of internet speed and compression, a well-made HTMX app can actually feel local. All that by stacking primitives together.

The initial template

The template will change a lot during the series of tutorials, but the general look stays the same:

The blue things are buttons. Don’t laugh.

I hard-coded an HTML template so you can get a sense of what it looks like with zero logic in there (colored version here):

<!DOCTYPE html>
<html>
<head>
    <title>Todo List</title>

    <!-- Minimal clean style from pico css -->
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@1/css/pico.min.css">

    <style>
      /* Pico css keeps bullets in lists, we hide them */
      li {
        list-style-type: none !important;
      }

      /* If a task is done, we display strike it */
      input:checked + label {
        text-decoration: line-through;
      }
    </style>
</head>
<body>
    <!-- pico css needs this class for centering  -->
    <div class="container">

      <br> <!-- Lazy spacing from top of the screen  -->

        <!-- The title. We hardcode the number of tasks and the tasks themselves here,
        but we will change that will templating later -->

        <h1>Todo <span>(3/4)</span></h1>


        <!-- The form to add a new task. button-primary is pico css styling -->
        <form method="post"  >
          <input type="text" name="title" placeholder="New task" value="" autocomplete="off">
          <button type="submit" value="Add" class="button-primary" role="button">Add</button>
        </form>


        <!-- The list of tasks, again hardcoded for the moment. It's in a form
        with a button to update the status (done/not done) manually, like grandma
        used to do. -->
        <form method="post">
            <ul>

                <li>
                    <input type="checkbox" id="task432432"
                           name="task"
                           value="432432"
                    >
                    <label for="task432432">Call the secret services</label>
                    <a>X</a>
                </li>

                <li>
                    <input type="checkbox" id="task1"
                          name="task"
                          value="1"
                          checked
                    >
                    <label for="task1">Save orphans</label>
                    <a>X</a>
                </li>

              <li>
                  <input type="checkbox" id="task43"
                        name="task"
                        value="43"
                  >
                  <label for="task43">Leg day</label>
                  <a>X</a>
              </li>

            </ul>

            <button type="submit" class="button-primary">Update</button>

        </form>
    </div>
</body>
</html>

The weird part is the last button. This form can (and will!) work without any JavaScript, but remember that without JS ,browsers can only send data when the user requests it. So for now, we need this button.

The model

At this point you understand that I need to warm you up to the whole HTMX thinking. In this first article, we are not going to do awesome stuff. We are going to do boring stuff. Still, useful stuff.

Useful to understand from where we start, to where we are going, and why we do things that way.

I suppose you can picture the typical to-do list example requirements:

• a way to list tasks, each with a title and a boolean stating if it's done or not

• a way to add the task to the list

• a way to delete a task from the list

We can represent the tasks with a mapping that looks like this:

{
    id: 76807890,
    title: "Buy a raspberry pi",
    done: False
}

So, our first job is to have functions to manage all that, without a UI, without even a server. Something completely neutral. Our raw business logic.

Python comes with everything that is needed for that, therefore I'm going to create a module named todo.py that will contain one function to add a task, another to delete one, one more to list them, etc.

You don't need to understand how it works in detail, but here are the functions for each operation (colored version here):

from __future__ import unicode_literals

import sqlite3
import tempfile
from pathlib import Path

# This creates the database in a temporary directory
db_path = Path(tempfile.gettempdir()) / "bitecode_htmx_todo.db"
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
c = conn.cursor()
c.execute(
    "CREATE TABLE IF NOT EXISTS todos (id INTEGER PRIMARY KEY AUTOINCREMENT, title TEXT NOT NULL, done BOOL NOT NULL)"
)
conn.commit()


# From there, each function is an operation on the list of things to do
# The texts between """ tell you what each function does.
def add_todo(title):
    """Add a task to the things to do in the database and return it"""

    c.execute("INSERT INTO todos (title, done) VALUES (?, 0)", (title,))
    conn.commit()
    return {"id": c.lastrowid, "title": title, "done": False}


def set_task_status(task_id, done):
    """Set a task as done, or to be done, and return it partially"""

    c.execute("UPDATE todos SET done=? WHERE id=?", (done, task_id))
    conn.commit()
    return {"id": task_id, "done": done}


def delete_todo(task_id):
    """Delete a task completely"""

    c.execute("DELETE FROM todos WHERE id=?", (task_id,))
    conn.commit()


def list_todos():
    """Return all tasks"""

    return c.execute("SELECT * FROM todos ORDER BY id").fetchall()


def get_task(task_id):
    """Return one single task"""

    return c.execute("SELECT * FROM todos WHERE id=?", (task_id,)).fetchone()


def count_todos():
    """Return a count of all tasks and a count of the ones remaining to be done"""

    c.execute("SELECT COUNT(*) FROM todos")
    total = c.fetchone()[0]
    c.execute("SELECT COUNT(*) FROM todos WHERE done")
    done = c.fetchone()[0]
    return {"done": done, "total": total}

This module will never change for the whole project. It's not something I would use in production (correct DB handling in a Web app is more complicated than that), but to explain HTMX, we don't need more.

Those damn millennials, ruining the servers

The server code will change from one article to another to adapt to the progression of our architecture, but we are going to start with an old-style backend with big single page rendering to make a point.

We need the server to:

• have one index page on "/" that displays… everything

• have one "/add" URL to add a task

• have one "/set_task_status" URL where we can sell all (yes all) tasks to a combination of done/not done

• have one "/delete/7809790" URL that will delete task 7809790

We really only have one page, mind you, the index. All the other URLs perform their duty when the user navigates by clicking a link or a form, then once they succeed, they redirect to the index. The index always regenerates an entire new page, which by definition, is fully up to date.

We also have verbs in our URL, a bad practice in HTTP, but we can't fix that now: our browser doesn't have any JS yet, and without it, it can only do POST and GET. In fact, we will have to misuse POST and GET as well.

Again, you don't need to understand all the code for this part, but here is the gist of it (colored version here):

import sys
from pathlib import Path

### BOILERPLATE TO MINIMIZE DEMO SETUP FOR YOU, IGNORE ###
CUR_DIR = Path(__file__).resolve().parent
sys.path.append(str(CUR_DIR.parent))

import todo
from bottle import TEMPLATE_PATH, debug, redirect, request, route, run, template

TEMPLATE_PATH.append(str(CUR_DIR))
### END BOILER PLATE ###


@route("/")
def index():
    """List all the tasks, count them, then render the index page template"""

    tasks = todo.list_todos()
    counts = todo.count_todos()
    return template("index", tasks=tasks, counts=counts)


@route("/add", method="POST")
def add():
    """Add one task, and redirect to the index"""

    title = request.forms.get("title")
    if title:
        todo.add_todo(title)
    redirect("/")


@route("/set_task_status", method="POST")
def set_task_status():
    """Set the status of all the task, then redirect to the index"""

    all_tasks = todo.list_todos()
    done_task_ids = set(int(task_id) for task_id in request.forms.getall("task"))
    for task in all_tasks:
        new_status = task["id"] in done_task_ids
        todo.set_task_status(task["id"], new_status)
    redirect("/")


@route("/delete/<task_id:int>", method="GET")
def delete(task_id):
    """Delete one task and redirect to the template"""

    todo.delete_todo(task_id)
    redirect("/")

# Start the server
if __name__ == "__main__":
    run(host="localhost", port=8080)

A functional UI, without HTMX

A hard-coded template will only take us so far, so let's insert the values from our server in there, as well as the URLs.

There are only two variables in the templates:

• tasks, a list of tasks mappings;

• counts, that contains our total of tasks and tasks to be done.

There are also 3 URLs:

• "/add": we need to put that on the form to add a task.

• "/set_task_status": we need to put that on the form listing tasks to update them.

• "/delete/ID_OF_A_TASK": we need to put that on the little "x" next to each task.

With our template system, we can use {{}} to insert variables and % to do if/else or loops, which gives us this (colored version here):

<!DOCTYPE html>
<html>
<head>
    <title>Todo List</title>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@1/css/pico.min.css">

    <style>
      li {
        list-style-type: none !important;
      }
      input:checked + label {
        text-decoration: line-through;
      }
    </style>
</head>
<body>
    <div class="container">
      <br>

        <!-- Insert the counts from the server -->
        <h1>Todo <span>({{counts['done']}}/{{counts['total']}})</span></h1>

        <!-- Triggering the form will send the data to action URL, here /add -->
        <form action="/add" method="post"  >
          <input type="text" name="title" placeholder="New task" value=""  autocomplete="off">
          <button type="submit" value="Add" class="button-primary" role="button"  >Add</button>
        </form>

        <!-- Here the action set the tasks status. You know see why we set
        the status on ALL the tasks. To do them individually, we would need a form
        and a button for each task. -->
        <form action="/set_task_status" method="post">
            <ul>
                % for task in tasks:
                <li >
                    <input type="checkbox" id="task{{task['id']}}"
                           name="task"
                           value="{{task['id']}}"
                           % if task['done']:
                           checked
                           % end
                    >
                    <label for="task{{task['id']}}">{{task['title']}}</label>
                    <a href="/delete/{{task['id']}}"  >X</a>
                </li>
                % end
            </ul>
            % if tasks:
            <button type="submit" class="button-primary">Update</button>
            % end
        </form>
    </div>
</body>
</html>

At this point, we have zero JS, and not even HTMX, but we do have a working product!

That’s because if the user clicks a X, it’s a link, so it will make a GET request to /delete, our server will react, delete the task, and send us back to the index. Same with the first form, if the user click the button, it sends the content of the input to /add, the server gets that, processes it, then redirects us to the index.

The index always regenerate the whole HTML, so the users see the up to date page every time.

Want to try it out?

You can clone the "no-htmx" branch or just download and extract the zip.

Then open the terminal, cd into the 1_no_htmx directory, and execute the server.py. If you don't know how to run install Python and run a file, we have a tutorial for that.

And if you don't want to try it out, you'll have to take my word for it.

We can add a task, update the status, delete a task, see the task count... Sure, we reload the entire page every time, and updating the tasks needs a big ugly button at the bottom, but it works.

Having action, href and name attributes properly filled is all what it takes for HTTP to do the job.

HTMX into the MIX

Kudos if you read this entire thing, cause it was long, and we still have zero HTMX in sight.

We had to pretend 5G never existed because HTMX thrives on those kinds of designs, but also so you can see the raw Web in action, since that's how you need to think with HTMX. Not in terms of updating the client, but in terms getting hypermedia out of the server.

Ok, I don’t want to lose you forever, so let’s use at least a tiny bit of HTMX. The most basic thing we can do: boosting.

Boosting adds quick and dirty dynamism to an old school site. It's not amazing, but, it's a first step.

For this we have to add the HTMX lib to our <head> and mark the links and forms we want HTMX to boots (colored version here):

<!DOCTYPE html>
<html>
<head>
    <title>Todo List</title>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@1/css/pico.min.css">

    <!-- Drop in the HTMX lib-->
    <script src="https://unpkg.com/htmx.org@1.9.6"></script>

    <style>
      li {
        list-style-type: none !important;
      }
      input:checked + label {
        text-decoration: line-through;
      }
    </style>
</head>
<body>
    <div class="container" >
      <br>
      <!-- Sprinkle hx-boost="true" on all the things we want to be dynamic -->
        <h1>Todo <span>({{counts['done']}}/{{counts['total']}})</span></h1>
        <form action="/add" method="post" hx-boost="true" >
          <input type="text" name="title" placeholder="New task"  value="" autocomplete="off" >
          <button type="submit" value="Add" class="button-primary" role="button"  >Add</button>
        </form>
        <form action="/set_task_status" method="post" hx-boost="true" >
            <ul>
                % for task in tasks:
                <li >
                    <input type="checkbox" id="task{{task['id']}}"
                           name="task"
                           value="{{task['id']}}"
                           % if task['done']:
                           checked
                           % end
                    >
                    <label for="task{{task['id']}}">{{task['title']}}</label>
                    <a href="/delete/{{task['id']}}" hx-boost="true"  >X</a>
                </li>
                % end
            </ul>
            % if tasks:
            <button type="submit" class="button-primary">Update</button>
            % end
        </form>
    </div>
</body>
</html>

To try it, clone the minimal-htmx tag, or download the zip and run server.py from inside the directory "2_minimal_htmx".

You'll notice everything works the same, but now the page don't reload every time. HTMX detect the boosted forms and links interactions and intercept them. It then creates an Ajax request, get the response, extract the <body>, and swap it with the current page.

We don't need to change anything from the server, it works out of the box.

The benefits are small for now, but they exist. The changes load faster, there is no re-parsing of the CSS file, some of the state of the page stays, and there is no flickering for the user.

Of course, we will do much better, yet you can already see the whole concept: we set attributes in the HTML to tell HTMX what to manage, and it performs partial updates for us, delegating the whole state to the server.

That long introduction out of the way, we will see something a bit more exciting in the next article.