Monday, May 2, 2022

Simulating a differential drivetrain, part 1 - If it's stupid but it works...

There’s a lot going on in a drivetrain.

On the surface, they seem pretty simple: “Rotary power comes from somewhere. It’s moved through mechanical links. The links connect to wheels. Wheels make thing go.” But there’s a lot of subtle complexity that underpins all of that. Drivetrains can actually be one of the more complicated things to simulated on a robot.

In a FIRST competition, every robot has some method of getting around, so it’s worth the time to invest in learning how to simulat them and what is involved in simulating them. Over the next few weeks of posts, we’re going to dig deep on this topic. I’m going to start from a simple simulation, then move onto a more complicated one, and then finally onto one using built-in WPILib classes that tries to capture many, many details of the drivetrain. Along the way, we’ll compare the approaches and the results we get and see what the tradeoffs are.

Full disclosure: I’m as exploring as I am explaining here, dear reader. At the moment, I have only implemented one of these approaches before, so I don’t really know what we’ll discover. It’ll be exciting to see where we end up.

Before we start in with our first model, let’s talk a bit about what a differential drive is and what we’ll be comparing.

What’s a differential drive?

[A picture of a robot with a differential drive train, courtesy WPILib docs

A differential drive (“diff drive”) is a drivetrain where the motors and wheels are divided into two independently controlled sets: left and right. By varying the power to each half, you can get the robot to move forward and backward or rotate (generally, for convenience, we want to build the robot so that point is in the middle of the chassis when the left and right trains are driven with equal but opposite inputs).

Diagram of the motions a diff drive can make: forward, backward, turn left, turn right

Since there’s no way for a diff drive to “sideslip”, we say diff drive is a non-holonomic system (which is a fancy way of saying there are paths through space an air-hockey puck could take but the robot can’t; it can’t move in an arbitrary direction while continuously spinning). But since the robot can turn on a dime, it should still be able to get everywhere (as opposed to a robot using car-style steering, which has some parking spots it can’t reach).

The simplest model of a differential drive is just two big wheels controlled by two motors, so that’s what we’re going to use. We’ll assume our wheels have a diameter of 4" (so about 0.1016 meters) so the robot goes about π * 0.1016 ≈ 0.3192 meters per revolution if both wheels are driven at the same speed.

The comparison

For each simulation, we’ll compare the following:

  • Driving straight for three rotations (about one meter total, more precisely 95.76cm)
  • Driving in a circle for three rotations (one motor full forward, one full back). Assuming a track width (distance between wheels) of 50 centimeters (about 19.6 inches), three wheel rotations are a distance along an arc of 31.92 centimeters in a circle with diameter 50 centimeters. Since the circumference of a circle is π * diameter, we can find the angle by
      length of arc / total circumference
    = 31.92 / π * 50
    ≈ 31.92 / 157.07
    ≈ 20.32% of the circle
    
    … which is about (2π * 20%) radians = 2/5π radians = 72 degrees.
  • driving in a lazy arc (one motor full power, one motor half power) for three rotations. This one, I’m not going to lay out the math to predict what will happen yet (future post); we’ll just see what it does.

We’ll consider for each kind of simulator

  • What results it gives us for our inputs
  • How we set it up
  • Velocity curves, acceleration curves, and final positions

And we’ll discuss some pros and cons.

Having established what we’re doing here, let’s start off with a very simple model.

The simple interpolator

So at first glance, the question of simulating a diff drive doesn’t feel too complicated. We know that

  • If both motors go full-forward, the robot goes max-speed forward
  • If both motors go full-backward, the robot goes max-speed backward
  • If one motor is full forward and one full backward, the robot turns on a dime at max speed

The interpolation simulator just assumes that any motion the robot could be doing between these extremes is a simple linear interpolation of these motions. We establish a top speed for the wheel based on motor input (something simple, like “The robot goes at most 1 meter / second forward and rotates twice a second at max speed”). Then we split the motion in question into linear motion and turning motion.

  • To figure out linear, we take the average of the sum of the left and right motor power and multiply that by top linear speed
  • To figure out rotation, we take the average of the difference of the left and right motor (i.e. (left - right) / 2)) and multiply that by the top rotational speed

Then we update the robot’s speed and then update the robot’s position and heading based on that speed. Updating the position involves changing simulated encoder values, and updating the rotation involves changing a simulated gyro value.

There’s an implementation of this approach split between PoseEstimator.java and SimpleSimulatedChassis.java in this simulator demo.

Comparison and results

The end result feels okay; when hand-controlled; it’s not obviously broken. Here’s what it looks like on the tests of motion:

Note: one spot of trouble I got into testing this: you can change the refresh rate for graphs in Shuffleboard. If you set it too high, Shuffleboard becomes unresponseive and cannot edit the preference again. The fix is to delete the prefs from Java’s user prefs store (in my case, it’s stored in the system prefs in ~/.java/userPrefs/edu/api/first/shuffleboard/plugin/base/prefs.xml.

Linear motion

Graph of the drive operation


The “Drive” command (which causes the robot to drive forward until its encoder passes three rotations) causes the robot to drive forward and then stop; no problem. It gets pretty close to spot-on for the final value; the overage is likely due to the fact that the simulation happens in steps of 0.02 seconds; if the robot passes the target encoder rotation in one of those time steps, it can’t stop on a dime with simple on-off control and so stops a bit after the target.

Graph of the turn operation

Final number of turn operation: 85.314


The “Turn” command (which attempts to cause the robot to turn precisely 72 degrees) is similarly accurate. It also overshoots because turning can’t stop in the middle of a time step.

Angular velocity and acceleration

Worth noting is the acceleration and velocity result. Because the robot accelerates instantaneously, they are very spiky. This is unrealistic motion; real robots don’t immediately stair-step from zero velocity to some high velocity; they have to overcome inertia and static friction (and similarly, once moving, inertia will keep them moving and motors have to fight that to stop the machine). So our simulation isn’t particularly realistic.

Graph of lazy turn position indciators

Final lazy turn numbers

Finally, we take a look at our lazy arc, which drives with the right motor full power and the left half power until we pass 72 degrees. The robot does trace a little arc, and we find our final position to be about a quarter-meter out in the x and y position with a heading of 76 degrees, closer to the 72-degree target than in raw turn (this makes sense; the rate of turn is lower on the lazy-turn mode).

Analysis

While this is good enough for practicing, it does not reflect how an actual robot behaves. For one thing, the motors cut on and off immediately, which doesn’t really reflect how motors work in the real world (see the discussion of the flywheel for details). For another, the robot’s path through space looks like a bunch of small line segments, since the facing is only allowed to update between steps and the robot always moves straight along its current facing. If the steps are small enough, this might be fine, but it doesn’t reflect well the curves that the robot actually travels in space.

Next week’s simulation approach will try to follow those curves.

Monday, April 25, 2022

Simulating a flywheel part 2: friction and time-step updates

On a previous post, I discussed creating a simulator for a simple flywheel in the FIRST Robotics WPILib framework in Java. We last discussed the high-level framework and simulating a motor; here, we’ll go into simulating friction and how we put it all together.

Friction is deceptively complicated

On the surface, friction is extremely simple. As is taught in high school physics, it’s the force that resists motion proportional to the normal force of the surface pushing against gravity. Which is to say, it’s just -K<sub>d</sub> * m * g, for g the gravitational constant (acceleration) and m the mass of the object, and K<sub>d</sub> the dynamic friction coefficient. Since the flywheel’s axle is pushed down by gravity on its socket, the friction force becomes a counter-torque opposing spin. So, no problem. We calculate the counter-torque, subtract it from the motor’s forward torque, and update every step. Punch that in and… Oh dear.

Graph of the flywheel velocity when motor on. When motor is off, the flywheel oscillates back and forth rapidly


The problem is that friction is a continuous effect: the force opposing motion applies until the object gets very close to zero velocity (relative to the surface it’s rubbing against). At that point, the molecules of the two objects “gum up” with each other and the object stops moving. But the simulator is a discrete time-step simulation: every time step, the forces acting on the flywheel are summed up, velocity is changed, and then it’s assumed the flywheel velocity stays the same until the next time step. So the velocity “wiggles” back and forth around zero without ever reaching zero and the flywheel never actually stops.

There are several solutions of various levels of complexity for handling this aspect of friction, but for our purposes we can use a fairly simple one:

  • Calculate the new angular velocity from the torques and counter-torques.
  • Check to see if the sign of the velocity has flipped. If it has, we know the velocity crossed through the zero boundary.
  • If we did cross zero, check to see if the driven torque exceeds the counter torque.
  • If it doesn’t, the motor seizes; set velocity to zero.

Code as follows:

if (Math.signum(newAngularVelocityRps) != Math.signum(flywheelAngularVelocityRps)
    && flywheelAngularVelocityRps != 0
    && newAngularVelocityRps != 0) {
    // Crossed zero boundary; if torque is too low to overcome friction, motor seizes
    if (Math.abs(advantagedTorque) < Math.abs(counterTorque)) {
        newAngularVelocityRps = 0;
	}
}

One other aspect of friction to account for: Static friction generally exceeds dynamic friction, and applies if the flywheel is starting from zero velocity. We account for that by selecting the right friction coefficient based on whether the motor is moving and comparing the torque to the counter-torque from the friction; if driven torque does not exceed counter-torque, the motor doesn’t start.

var frictionCoefficient = m_motorSpeedRpm == 0 ?
          m_flywheelStaticFrictionConstant.getDouble(1.0) :
          m_flywheelDynamicFrictionConstant.getDouble(1.0);
// ...
var totalTorque = advantagedTorque - counterTorque;
if (m_motorSpeedRpm == 0 && Math.abs(advantagedTorque) < Math.abs(counterTorque)) {
    // stalled motor does not overcome static friction
    totalTorque = 0;
}

Putting it all together

Having accounted for friction, we now only need to take the total torques computed and apply them to how the velocity and encoder count changes over time.

To start, the total torque changes the angular velocity. Similar to the famous F = ma equation, torque = moment of inertia * angular acceleration. So we divide our torque by the moment of inertia to get an acceleration, and add it (times the time delta step) to the velocity:

var angularAccelerationRadsPerSec = totalTorque / m_flywheelMomentOfInertiaKgMSquared.getDouble(1.0);
var flywheelAngularVelocityRps = Units.rotationsPerMinuteToRadiansPerSecond(m_motorSpeedRpm) / gearRatio;

// Update angular velocity, parceling the change by the delta-T
var newAngularVelocityRps = flywheelAngularVelocityRps + angularAccelerationRadsPerSec * TIMESTEP_SECS;

Then, we just need to update the velocity and position for the encoder, and update the RPM graph:

m_motorSpeedRpm = Units.radiansPerSecondToRotationsPerMinute(newAngularVelocityRps * gearRatio);

// multiply by timestep and ratio of rotations to radians to get new flywheel position
var flywheelSpeedRotationsPerSec = newAngularVelocityRps / (2 * Math.PI);
var flywheelDelta = flywheelSpeedRotationsPerSec * TIMESTEP_SECS;

// encoder values are relative to flywheel, not motor
m_encoder.setDistance(m_encoder.getDistance() + flywheelDelta);
m_encoder.setRate(newAngularVelocityRps);

m_flywheelSpeedRpm.setDouble(flywheelSpeedRotationsPerSec * 60);

And we’re done! We have a self-updating flywheel simulation that we can experiment with.

Monday, April 18, 2022

Simulating a Flywheel part 1: overview and simulating a motor

Well, the off-season of FIRST Robotics is upon us. A time when young people’s minds turn away from robots and towards other things. Like simulating robots!

To give the team something to play with on the software side, I’m putting together a couple of simulators of robot subsystems to experiment with. The first one is a simple flywheel connected to one motor and encoder.

The interface for the simulator in Shuffleboard
Interface for the simulator, showing the graph of flywheel RPM over time

I’m going to break the discussion of how I did this into two parts. Here, I’ll go over the top-level architecture of the program and how it tries to simulate, and talk about the equations to simulate a DC electric motor. In a follow-up post, I’ll go over the physics of friction simulation.

Full disclosure: I’m a rank amateur at this, putting together half-remembered high school physics, a barely-used robotics minor, and some hazy memories of graphs from my first years playing FIRST Robotics. Definitely room for improvement, and I welcome feedback on this.

You can get the simulator from the GitHub repsitory.

Overview

Program structure

The program is a simple TimedRobot operating at the default frequency of 50 times a second. It includes one automatically-running Flywheel subsystem that lets a motor be turned on and off (at max power or 0 power), which is controlled by a single joystick button.

In addition to Flywheel, a SimulatedFlywheel class is constructed if the robot is operating in simulation mode. This class takes in the motor from the Flywheel and a simulation wrapper around the encoder, allowing us to explicitly set the encoder’s position and velocity values.

The simulation

At a high level of abstraction, the torque on a motor-driven flywheel at any moment is simple:

Total torque = motor torque - friction torque

Spinning flywheel, with forward force and opposed force
Opposed forces acting on a spinning flywheel

Once we have calculated this torque, we can update the flywheel’s angular velocity by adding total torque / moment of inertia to the flywheel’s previous velocity. Then we update the flywheel’s position as well (accounting, in both cases, for the step of time over which this change is occurring, in our case 20ms between updates).

That’s the high-level approach; let’s drill down on the details.

Simulating motors

A DC motor is a relatively simple system, but it has a bit of complexity making simulating one interesting.

The basic principle by which a DC motor operates is that half of the motor uses electromagnets, while the other half uses permanent magnets. Varying the current through the motor’s electromagnets creates moving magnetic fields that the permanent magnets then “chase,” turning electricity into motion. The highest torque such a motor can give is when it’s “stalled” (i.e. not moving) and the voltage across it is maximized. At this configuration, the motor is pushing with its “stall torque” (note: It’s generally a bad idea to leave a motor stalled like this; many DC motors aren’t designed to run current through with no motion, and doing so for long periods of time can heat up and damage components).

So the total torque a motor can give is equal to some constant times the current through the motor:

motor torque = I * k_M

Where I is current through the motor and kM is a torque constant. Note: equations for motor behavior are mostly sourced from here. We can quickly calculate kM as just the stall torque divided by the current, but for the purposes of the simulation i find it easier to just consider a relationship between stall torque and voltage by subsituting current through V = IR, then considering the motor with 12 volts across it and finding the value of a constant q which is resistance divided by KM:

q = 12 / motor stall torque

What’s interesting about electric motors is that when I say “some loops of wire acting as electromagnets near some permanent magnets,” I’m also describing a type of generator. And indeed, a spinning electric motor tries to induce a current in the direction opposite the spin! This opposing force, known as “back-EMF” (EMF: electromotive force), creates a counter-voltage to the voltage across the motor, resulting in a drop in current through the circuit. The physics of why this happens can be a little confusing, but there’s (I think) an elegant way to remember it: the fact that motors have back-EMF sort of falls naturally out of conservation of energy once we remember that motors with a voltage across them don’t spin up to infinite speed. Since a motor spinning at top speed only needs a little bit of energy to overcome resistance and maintain that speed, and since energy lost across a component of a circuit is the power (P=IV), we know that when the motor is spinning near top speed, P across the motor must be small to keep it spinning, so current and/or voltage diminished.

Diagram of an electric generator and an electric motor
An electric motor and electric generator differ mostly in where the energy comes from. (Images sourced from Wikimedia)

The back-EMF on the motor is proportional to the motor’s rotational speed and some back-EMF constant ke.

Diagram of an electric generator and an electric motor

A motor allowed to spin up to full speed under no load and with maximum voltage across it will spin at a “free speed” which varies from motor to motor. At free speed, the back-EMF is approximately equal to the forward voltage, so we can set the voltage and speed in the above equation and re-arrange to solve for the back-EMF constant:

back-EMF constant = voltage / free speed

We’re closing in on simulating the torque. The remaining ingredient is the full torque equation.

To approach the final torque equation, we start by considering that the total voltage across the motor is the V=IR relationship through the motor plus the back-EMF value.

voltage = (current * resistance) + (motor speed * back-EMF constant

Substituting in the relationship between torque and current, we find

voltage = (motor torque * resistance / motor torque constant) + (motor speed * back-EMF constant)

Now we have voltage, motor speed, and motor torque related. Doing the algebra to make torque the dependent variable and substituting our constant for resistance divided by kM, we get:

motor torque = motor voltage - motor speed * back-EMF constant / q

That gives us the motor torque in almost all circumstances, save one: what happens when the voltage is set to zero?

Brake or coast?

It’s a subtle bit of architecture, but an important one: as we’ve established, a motor acts like a generator. So what happens when the motor should be applying no power?

Motor controllers used in the FIRST Robotics competition are generally configurable to “brake” or “coast” mode. In “brake” mode, the controller short-circuits the motor to itself, allowing the back-EMF to try and drive the motor in reverse. This will resist motion in the motor and drag it to a halt much quicker. The mathematics of this is, honestly, something I don’t know.

“Coast” mode is much easier to model: in “coast” mode, the controller opens the circuit. With no ability for electricity to flow from one end of the motor to the other, the motor basically contributes no torque and the flywheel will spin down normally. So we special-case the zero-voltage condition to eliminate the motor torque, simulating “coast mode.”

Next steps

This gives us half the flywheel simulation. Next time, we’ll talk about the friction counter-torque (and how annoying friction subtleties are!) and tie it all together with the acceleration and velocity-update logic.


Monday, April 11, 2022

Piping Linux audio to a file

I recently had to pipe some audio from my browser to a file. This may not be the most elegant way, but I found it works.

Using SimpleScreenRecorder

For capturing audio and video, I use SimpleScreenRecoder. It’s a pretty no-frills recording program that is intended for (among other things) streaming content from a machine. While it supports both audio and video, I only care about the audio portion for this project.

To minimize bandwidth wasted recordin video, I set “Record a fixed rectangle” and set a 2x2 rectangle at 0,0. I then enabled “Record audio” with the PulseAudio backend and a Source of Monitor of sof-hda-dsp Speaker + Headphones. That output to my computer’s main speakers.

Image of the SimpleScreenRecorder UI, showing the configuration

On the next screen, I set up an Output profile declaring the destination, use of MP4 as the container, an H.264 video codec with constant rate factor of 23, superfast preset, and “Allow frame skipping”, and an Audio codec of MP3, Bit-rate 128 kbit/s). 

Once set up, I could just start recording.

Monday, April 4, 2022

Resetting my monitor in Linux

Recently, I had an Ubuntu laptop I use regularly develop a weird quirk: sometimes when I plugged it into USB-C, neither the monitor nor power would connect. I discovered I could fix this by simply running lspci on the laptop, which somehow forced both to come back.

As this made no sense, I dug down a bit more. I can’t say I found an answer, but I did find some more things worth questioning.

PCI devices

On my laptop, devices are exposed at /sys/bus/pci/devices. Searching around a bit, I found that device /sys/bus/cpi/devices/0000:3b:00.0 was reporting itself as USB controller: Intel Corporation JHL6540 Thunderbolt 3 USB Controller (C step) [Alpine Ridge 4C 2016] (rev 02). Navigating into that subdirectory, I found I could sudo -s, then echo 1 > reset, which fixed the system in apparently the same way as running lspci.

What the hell?

At this point, I don’t really know. Sometimes the Linux architecture just puzzles me. I don’t know that lspci is running a reset on devices, but perhaps when it tries to poll details on the device, the act of polling itself trips a fault in the driver (becuase the driver realizes it can’t communicate with the device) and forces a reset to heal itself? I haven’t found any code confirming or falsifying this hypothesis; it’s still an open question for me.

On not giving up

Owning a Linux machine is like this. When things go wrong, the only person who’s responsible for fixing it is the owner. This is, ultimately, true of all self-owned computers, of course.

When I was younger, the ecosystem of Linux users was too hard to find to make this task easy; certainly not as easy as it was for Windows ownership, when everyone on the block had one of those machines. But the nice thing about living in the Internet era is Linux use bloomed, and at the same time users can find each other much more easily. I learned about the existence of the PCI list poking around on various sites.

There’s much more to learn, but it’s nice that resources are far easier to find now.

Monday, March 28, 2022

Visualizing access logs from my blog server

One of the things I’ve missed, having moved my blog off of Blogger, is the metrics. I don’t use the metrics for much, but there’s a nonzero serotonin hit to knowing that my content is read by someone. It’d be nice to be able to restore at least that piece of the Blogger feature-set.

Fortunately, I have access logs and a log analyzer.

I’ve settled on goaccess for my log analysis; it’s pretty straightforward, takes HTTP access logs as input, and presents the data visually (including on the command line). It’s installable on my local machine via the package manager (sudo apt-get install goaccess), so no problems there.

The steps are pretty straightforward:

  • Get the logs
  • Dump them into goaccess

The script to do that is short and sweet:

#!/bin/bash

SERVER=fixermark.com
LOGPATH=logs/personal-blog.fixermark.com/http
DESTINATION=logfiles

mkdir -p "$DESTINATION"
scp $SERVER:$LOGPATH/access.log* $DESTINATION
pushd $DESTINATION
# this is redundant because it's a symlink on the server to the most recent logfile
rm access.log.0
# gunzip will confirm replacing files
yes | gunzip -f *.gz
popd
goaccess $DESTINATION/access.log*

I run that, and I’m presented with a nice terminal interface for viewing the logs.

Terminal interface, showing categories for Requested Files, static requests visitor hostnames and IPs

This is a good start!

Filtering

goaccess doesn’t support any filtering directly, but access logs are relatively simple to filter with command-line tools, and goaccess does support receiving its logs from the command line. Here’s a simple script to drop the logs related to various static content pieces:

cat $DESTINATION/access.log* | \
    grep -v /lib/ | \
    grep -v /css
	grep -v /images/ | \
    grep -v /js/ | \
    goaccess --log-format=COMBINED -

Adding this to the fetch script, the logs are now honed in on just posts.

To do next

Only a couple things I'd like to improve in this flow:

  • scrubbing logs—after 21 days, I’d like to substitute the IP addresses with 0.0.0.0 to increase user anonymity.
  • Run these server-side (or have goaccess pull them remotely, if possible) so logs aren’t living on more machines than strictly necessary.

Monday, March 21, 2022

Self Hosted Hugo Comments: data rendered with partials

In an earlier post, I added self-hosted static comments via shortcodes in Hugo. This approach had some benefits, but I didn’t like how it required modifying every blog page to support comments, even if no comments were present.

Hugo has a system of partials and templates to allow for similar pages to have the same layout. We can take advantage of these to handle comments on every blog page. This will pull the comments out of the main flow of the blog posts; we could move them into the front matter of the pages, but insted I’m going to knock out another con of the previous approach and consolidate all comments into one data file.

The method

We have a few steps to go through here:

  • Consoolidate comments into a data file
  • Build comments.html and comment.html as partials
  • Build a new blogpost template to use the comments partial
  • Use cascading front-matter to shift all the blog posts to the new template

Consolidate comments into a data file

To make it easy to work with comments as a separate construct from posts, we’ll shift all of them into a new file at data/comments.yaml. Hugo automatically parses files in the data directory and makes their content available for the site builder as site.data.<name of file>.

I’m using yaml because it splits the difference a bit: easy to use, but allows for multi-line strings without a lot of hassle (and it place nicely with my emacs config). here’s a snippet of the resulting yaml file.

"/posts/2021/this-is-year-of-linux-on-desktop/":
  - id: 1
    username: "Anon 1"
    date: 2021-10-21T16:54:40.122Z
    comment: You have working audio on your GNU/Linux laptop?  Must be nice.
    replies:
      - id: 2
        username: Mark T. Tomczak
        date: 2021-10-21T17:56:00.084Z
        comment: I used to, but I changed my window manager and now I'm not so sure. :-p
"/posts/2021/marks-gallery-of-facebook-infractions-3/":
  - id: 1
    username: Anon-2
    date: 2021-06-14T15:34:10.877Z
    comment: |
      My vote is for "kill the filibuster." This is a failure of the algorithm to differentiate actual calls for violence from figurative language.  I wonder if you could post a comment about "Killing the Lights" when discussing what you might do before a movie or bedtime.

      Reminds me of when I tried to sell a dart board on FB Marketplace, and I included a photo of the darts themselves. I had my post removed for trying to sell weapons.

      There is an ENTIRE CATEGORY devoted to "Darts Equipment." Oh, Zuck...      

Worth noting:

  • The top-level object is a dictionary mapping post paths to a list of the top-level comments in the posts
  • comment IDs are unique within the post (they’re used to build URLs to email replies in)
  • We preserved the tree structure from the previous short-code solution, but since the replies are now a separate field from the comment text body, we’ll be able ot use Markdown on the comment without mangling replies.

Build the comments.html partial

The partial at layouts/partials/comments.html finds the comments for the current page. If they exist, it stitches them in.


{{ with site.Data.comments }}
{{ $comments := index . $.Page.RelPermalink }}
<div class="comments">
  <h1 class="comments">Comments</h1>
  <div class="comments-menu">
    <ul>
      <li>
        <a href="mailto:blog+personal-comment@fixermark.com?body=Your Name:%0d%0aIcon:%0d%0aComment:&subject=Comment on {{ $.Page.Permalink }}">
          Add comment
        </a>
      </li>
      <li>
        <a href="/how-to-comment">How to comment</a>
      </li>
    </ul>
  </div>
  <div class="comments">
  {{ with $comments }}
    {{ $sorted := sort $comments "date" "desc"}}
    {{ range $sorted }}
      {{ partial "comment.html" (dict "comment" . "permalink" $.Page.Permalink) }}
    {{ end }}
  {{ else }}
      <div class="no-comments"><i>This article has no comments</i></div>
  {{ end }}
  </div>
</div>
{{ end }}

Once we fetch the list of comments, we check for any comment list with a key matching this page. If we find any, we sort them by date and render them ({{ range $sorted}}). This partial also renders a header for the comments section and a link to add a comment to the post.

Partials receive only the state given to them by their invoking template. When we render individual comments with the comment.html partial, we only give it the two pieces of information it needs (in the form of a new dictionary): the comment data as comment and the link to this page as permalink. The link is used to build replies to comments.

Build the comment.html partial

Rendering individual comments is delegated to a second partial.

<div class="comment">
  <div class="user-info">
    {{ if .comment.usericon }}
      <img src="{{.usericon}}">
    {{ end }}
    <span class="username">{{ .comment.username }}</span> <span class="post-time">{{ time.Format "2006-01-02 15:04 Z" .comment.date }}</span>
  </div>
  <div class="comments-menu"><a href="mailto:blog+personal-comment@fixermark.com?body=Your Name:%0d%0aIcon:%0d%0aComment:&subject=Comment on {{ .permalink }}?comment-id={{ .comment.id }}">Reply</a></div>
  <div class="comment-body">
    {{ .comment.comment | markdownify }}
  </div>
    {{ with .comment.replies }}
    <div class="replies">
      {{ $sorted := sort . "date" "desc" }}
      {{ range $sorted }}
        {{ partial "comment.html" (dict "comment" . "permalink" $.permalink) }}
      {{ end }}
    </div>
    {{ end }}
</div>

We pretty up the time representation of the comment using time.Format and run the body of the comment through markdownify to convert any special characters. We also add a reply link taking advantage of the comment ID.

Note that to render replies to this comment, this partial re-invokes itself passing the reply as the comment. This sort of recursion is fine in Hugo as long as it’s not infinite (the nature of the tree data structure this function is running on makes such infinite recursion impossible).

Build a new blogpost template to use the comments partial

Hugo allows pages to specify their type, which determines which of several templates Hugo will use to render the content of the page. Now that we have comments, I copied the single.html template from the theme I’m using into layouts/blogpost/single.md and replaced its invocation of a comment partial with my own:

{{ partial "comments.html" . }}

At this top level, I give the partial everything the template has for convenience.

Use cascading front-matter to shift all the blog posts to the new template

Hugo uses a speciall-named _index file to allow for application of front-matter to every page in a subdirectory of the site. Using that, it’s straightforward to shift all my blog posts to the new tepmlate. I add the file content/posts/index.md:

---
cascade:
  type: blogpost
---

Now, every page under posts/ has its type set by default.

Putting that all together (and adding a bit of CSS to clean the formatting), we now have comments on every page without changes to every page. Very happy with the result!

A couple of comments underneath an article


Pros and cons

Pros

Thread flow still clear
Replies are nested under their comments. I’m glad I didn’t have to lose this from the inline solution
Comment text is just markdown
I’m much happier with markdown as the comment body text than markup; easier to read, and modestly harder for end-users to find a way to accidentally break the whole sight flow

Cons

Comments no longer live on their articles
I’m considering this a pro in the overall assessment. It’d be nice if comments lived right next to their articles, but with comments consolidated in one file it’s much easier to manage them as an entity (including scrubbing one if a user asks to have it removed; I only have to purge one file through all archives).

Final thoughts

I’m really finding Hugo very straightforward to use. It’s nice to have my toolchain more tightly integrated and this level of control over both content and presentation.