The Importance of Being a Sysadmin

The Importance of Being a Sysadmin
Photo by Rubaitul Azad / Unsplash

When I rebuilt my main VPS, I wanted to improve my CI/CD Pipeline.

Before that point, any time my website changed, I'd SSH into the server, run git pull and then rebuild the container, bring it back up and disconnect.

This was taking a lot of time, especially during rapid iterations, so I vowed to change the architecture when I rebuilt.

But How Would I Do It?

There were many options available to me, but three stood out:

  1. Jenkins Pipelines
  2. GitHub action to automate what I was doing
  3. Watchtower

Jenkins

I love Jenkins, I use it at work, and wanted to further understand how one of the best pieces of Open Source software worked.

I am also a full-time student when I'm not working, and realised the initial configuration & hardening of Jenkins would take a long time and would require bespoke pipelines and multiple steps for each service that I developed and wanted to host.

GitHub Action to Automate What I was Doing

This was almost what I went with, in fact, I set that up for CS++'s Plume.

It's an action that runs whenever there's a commit to the main branch, it connects to the box by SSH, pulls down the changes, builds the container and uses compose to bring it up.

It's a really quick and dirty method, but it works really well! Code can go from committed to up within about 2 minutes.

But early versions of the workflow would fail silently (Docker build would fail but would still mark the step as a success), and it also requires the entire source code to be on the box as well as the built image, which wastes space.

Watchtower

Watchtower is a neat little container that monitors all running containers tagged with latest. It occasionally checks the upstream container registry for newer builds. If it finds one, it pulls down the image, and restarts the container with the new image.

This ended up being the solution I use on my production box, and it works STUNNINGLY well.

Once code is committed to the main branch of any of my projects, a GitHub action runs to build the container and store it on the GitHub Container Registry (GHCR).

It takes Watchtower up to a minute to detect this change, pull it down and recreate the container.

When it's not doing much, it sits out of the way, occasionally pinging upstream registries for newer containers. It's REALLY neat.

Enter Starboard 2.0

I've wanted to rebuild my FriendConstellation project ever since I finished V1, and with the winter break I had the perfect excuse!

Svelte 5 had come out, and I'd been doing a lot of Web Development on other projects, which completely changed my Visual Style as well as the tools I used, most notably I moved from IBM Carbon over to TailwindCSS and ShadCN.

There were some other enhancements I wanted to add, like a more secure login system and the ability to add Discord Webhooks by UI.

The old Starboard Homepage
The new Starboard Homepage

Starboard 2.0 was built mobile-first, as opposed to the desktop-first design mantra of Starboard 1.0 (I'll be calling them SB1 and SB2 from now on).

A few new features were added, such as a 'Logging' system for the Champions. Whenever the admin adds or removes stars, they add a reason, and that is shown when you click the Champion's Arrow.

An example of the log

In the Admin panel, they are now able to add and remove Discord Webhooks through the UI. This previously needed to be done by manually attaching to the Database container and writing a new row.

Webhooks are hit whenever a task is added, so if you had no webhooks and added a task, the UI would return a 500 error.

Webhooks were added late into the SB1 development cycle, well after initial deployment to add further interaction and enhancements.

Deployment, What We're All Here For

SB2 was a complete overhaul, and would test the capabilities of Watchtower.

When I finally merged the 2 development branches into main (and after I fixed a lockfile issue), the image built successfully on the GHCR and within a minute, version 2 was live on the production box! You can find Starboard 2025 here.

Watchtower made it so simple, I literally just have to build an image through a GitHub workflow and Watchtower takes care of the rest.

A Little Detour to Fix RJM.ie

Since there was new architecture underpinning Starboard, I had to update rjm.ie to reflect the tools I was using now.

While I was doing that, I found that there was a visual bug causing white bars to appear on the bottom and right-hand side of some pages.

All I had to do was fix the incorrect Tailwind styling, commit the change and the new version of the site was online in a minute. So handy!

Conclusion

I really enjoy learning more technologies outside of the Lecture Theatres, and one of the chronically under-taught areas in Computer Science is Systems Architecture and Administration.

I sought to resolve that by introducing the Sysadmin role in TU Dublin's Computer Science Society when I was Chair, and I frequently learn more through implementing CI/CD, building servers and more, both for myself and the society.

It's all well and good to write code, make websites and fill a portfolio, but it's another thing to be able to showcase it.

In industry, Systems Administration and Development have diverged, but it's still important to understand what happens when a developer commits their code.

It's one of my favourite things to teach and show anyone who wants to learn, as the CI/CD pipelines, tests, and hardware is the backbone to everything we do these days.