That curiosity ended up going further than expected. We eventually put FreeNAS into production alongside Synology at work. Synology was the licensed, official box — the one we could actually support and rely on. But early on we also ran some unlicensed Synology setups alongside it, which, looking back, wasn’t exactly the right way to do things. Not something I’d repeat now that I think about it more carefully, but it was part of how that environment came together at the time.
FreeNAS itself held up well next to Synology. Different philosophy — one’s an appliance with a polished UI and a support contract behind it, the other is more hands-on, more “you own every decision here.” Both had their place.
Now at home it’s TrueNAS running my homelab storage, the direct descendant of that same FreeNAS I first poked around back in 2019. Feels like a full circle moment — from being curious about ZFS on a spare box to it being the thing quietly holding my homelab’s data together years later.
Found myself thinking about all of this again after watching this video — good reminder of how far this journey with NAS has actually gone.
]]>OpenStack is not a gentle system. It’s layered, it’s distributed, and when something breaks, it rarely breaks in isolation. A single Mnesia state loss in RabbitMQ can cascade into Neutron agents going dark and Nova compute services dropping out simultaneously — and suddenly what looked like “one queue had an issue” is actually half your control plane looking unhealthy at once.
I’ve spent hours chasing MariaDB Galera split-brain scenarios, trying to figure out which node actually holds the safe state to bootstrap from — running wsrep_recover, checking safe_to_bootstrap flags, hoping I picked the right node before forcing a new cluster into existence. Get it wrong, and you’re rebuilding from a backup instead of a quick recovery.
Then there’s the networking layer. Neutron L3 agents and OVS agents behave differently when they’re running inside Kubernetes pods instead of on bare metal — the usual troubleshooting playbook doesn’t fully apply, so you end up building a new mental model for how the same components fail in a containerized context.
I’ve also dealt with a dual SSD failure — correlated RAID 1 wear, both drives dying close together — that took down VMs running local qcow2 ephemeral storage. No shared storage to fall back on for those workloads, so it meant forcing state resets on the stuck instances and evacuating them to another host, then writing up an incident summary for the hardware team afterward. And on the Kubernetes side underneath all of it, I’ve had to recover a control plane after losing a master node — watching etcd leadership flap between the two remaining members before quorum settled back down.
None of these problems live in a single layer. That’s the part that makes OpenStack uniquely exhausting to operate.
That’s exactly what pulls me in, though. OpenStack forces you to understand systems, not just tools. You can’t troubleshoot Neutron without understanding how it talks to Nova through the message bus. You can’t fix a compute service outage without understanding what RabbitMQ and Mnesia are doing underneath it. Every incident is a lesson in distributed systems design — quorum, state consistency, failure domains — playing out in real infrastructure instead of a textbook diagram.
There’s a specific kind of satisfaction in finally tracing a cascading failure back to its actual root cause. Not the symptom — Nova compute showing DOWN, Neutron agents unresponsive — but the actual originating event, like a single Mnesia table going corrupt. It’s the same feeling as solving a hard puzzle, except the puzzle is also serving real workloads, and getting it right means someone’s VM comes back online.
Migrations are another place where the complexity turns into something almost elegant when it works. Block migration for VMs on local ephemeral storage — moving a running instance’s disk state across hosts without shared storage — feels like it shouldn’t be possible until you watch it actually happen cleanly.
I think what keeps me coming back isn’t that OpenStack is easy — it’s clearly not. It’s that every time I fix something, I understand the whole stack a little better. Bare metal, host Kubernetes, the OpenStack control plane, tenant clusters running inside OpenStack VMs, Ceph underneath all of it as shared storage — it’s a lot of moving parts, but each incident makes the overall picture clearer than the last one did.
If you’re starting out with OpenStack and it feels overwhelming — it is. But that difficulty is also where the actual learning lives. Stick with it long enough, and the moments where everything clicks into place, where you finally see how the message bus, the database, the network agents, and the compute layer all fit together, make all the 2 AM debugging worth it.
]]>That project taught me more about how networking actually works than anything I’d read beforehand, and a lot of that credit goes to a colleague who was patient enough to walk through the concepts with me while we were both under deadline pressure. Being new during a probationary period made it feel higher stakes than it probably needed to be. But that pressure is also what pushed me to actually understand the pieces instead of just copying configs that “worked” — because it usually takes a good scare, or a messy upgrade, to close that gap between configuring something and understanding it.
MetalLB solves a problem bare-metal Kubernetes clusters have that cloud-managed clusters don’t: there’s no cloud load balancer to hand out external IPs. MetalLB fills that gap by advertising IP addresses from a pool you define, using either Layer 2 mode (ARP/NDP) or BGP. When you create a Service of type LoadBalancer, MetalLB is the thing that actually assigns it a real, routable IP and makes the network aware that IP lives on a specific node.
Emissary-Ingress (built on the Ambassador API Gateway/Envoy) sits above that, handling actual HTTP/HTTPS routing — host-based routing, TLS termination, path rewriting, rate limiting. It reads Kubernetes annotations on Mapping and Host custom resources to configure Envoy’s routing rules dynamically.
ExternalDNS closes the loop. It watches your Ingress/Service resources and automatically creates matching DNS records in an external provider — in our case, that meant connecting to AWS (Route53), depending on which zone a given domain lived in. So the full chain looks like: MetalLB gives a Service a real IP → Emissary-Ingress routes traffic to the right backend based on hostname → ExternalDNS notices the Ingress and creates a DNS record pointing at that IP, all without a human manually touching DNS.
I spent real time studying how these three pieces actually interconnect — not just what each one does on its own, but how a change in one ripples into the others. The way I eventually explained it to myself was like a mall: MetalLB is the mall assigning your store a physical address in the building. Emissary-Ingress is the directory board and the hallways — it decides which door a visitor gets routed through based on which store they’re actually looking for. ExternalDNS is the signage outside on the street, automatically updating so people searching for your store from outside the building actually get pointed to the right entrance. None of the three do much alone. The value is entirely in how they hand off to each other.
That mental model is what let me properly prepare the RCA for the incident below and deliver it in a way the team could actually follow, instead of just listing “I found a bug” without explaining the mechanism behind it.
Back in the second half of last year during my probationary period, I was pulled into investigating something stranger: an intermittent, fluctuating connectivity issue across our dev and staging clusters. Services would be reachable, then suddenly not, then reachable again, with no clear pattern. Nothing in the application logs pointed anywhere useful. It looked like a phantom problem.
I was still new to Kubernetes at the time, and figuring this out took real digging — with a lot of help from a colleague who was patient enough to walk through concepts with me while we were both under pressure.
I went deep into MetalLB’s Layer 2 mode internals to understand what was actually happening. MetalLB in L2 mode works by having one node ARP-announce ownership of a given service IP on the local network segment — it’s literally answering “who has this IP” with “I do,” the same way any host claims an IP on a LAN.
Digging further, our team found the actual root cause: our dev and staging clusters were configured with MetalLB IP address pools drawn from the same subnet, and both clusters were sitting on the same underlying network segment. Whoever had provisioned the second cluster had missed narrowing the IP range, so both clusters had overlapping address space to hand out. That meant two different nodes — one in dev, one in staging — could end up both believing they owned the same IP, and would both send ARP replies claiming it. Depending on which reply won the race at any given moment, traffic would go to one cluster or the other, which is exactly the “sometimes works, sometimes doesn’t” symptom we were chasing.
Once we understood that ARP was the mechanism actually being fought over, the fix was straightforward: correct the IP pool ranges so dev and staging had properly separated, non-overlapping subnets. I used the mall analogy above to explain the whole chain in the RCA — two stores had accidentally been assigned the same address, so visitors kept getting routed to whichever one answered first. It landed well with the team, and it was one of the first times I felt like I could take something genuinely confusing and make it click for other people too.
This quarter, I moved from investigating a networking bug to owning an actual upgrade of the platform foundation module — and the biggest lesson here was completely different: how much of a safe upgrade actually depends on documentation discipline, not just running a new version.
I learned to slow down and actually read release notes and upgrade guides properly — what changes between versions, what’s now required that used to be optional, what breaking changes exist, and what the migration path actually looks like before touching anything.
MetalLB turned out to be the clearest lesson in “read the docs first.” We were running the Bitnami-packaged MetalLB chart, and moving to the upstream metallb/metallb chart wasn’t a simple in-place upgrade — the two aren’t structured the same way, and trying to upgrade over the top of the old install risked leaving conflicting resources behind. The only safe path was a full uninstall of the Bitnami version and a clean install of the upstream chart. Knowing that ahead of time, instead of finding out mid-upgrade, saved us from a much messier cleanup.
Emissary-Ingress turned into the most complex piece. During a POC upgrade in the dev cluster, we found a lot more Services were affected than we expected — the newer version required annotations to move to the getambassador.io/v3alpha1 API, which meant every Mapping and Host resource using the old annotation format needed updating. For services we owned directly — logging, and a few other internal tools — we went ahead and updated the annotations ourselves. But several affected services belonged to other teams, so that part is still pending coordination with them before we can move forward cluster-wide. It was a good reminder that a platform upgrade isn’t just a technical task — it’s also a communication and sequencing problem across teams.
ExternalDNS, by contrast, was refreshingly smooth. We found it had no dependency conflicts blocking a direct jump — we could simply uninstall the older version and install the latest release cleanly, without needing to step through every intermediate version one at a time. Not every upgrade needs to be complicated, and it was a nice contrast to see that confirmed directly instead of just assuming the worst going in.
Two very different problems — one a quiet, slow-burn investigation into a networking collision, the other a structured, cross-team version upgrade — but both forced the same kind of learning. You can’t safely operate MetalLB, ExternalDNS, or Emissary-Ingress by treating them as black boxes. You have to understand what’s happening underneath: ARP announcements, chart architecture differences, annotation contracts between resources, and who else depends on the thing you’re about to change.
I went into last year only knowing Kubernetes basics — Services, Pods, Deployments — and came out understanding how load balancer IP advertisement, DNS automation, and ingress routing all interlock as one system, well enough to explain it with something as simple as a mall analogy. And I got there because I had colleagues willing to explain the pieces and sanity-check my theories along the way. Being new doesn’t mean you can’t dig into the hard stuff — it just means it’s more fun with someone next to you.
That’s still the thing I love about this work. Every “weird intermittent bug” or “this upgrade touches more than we thought” is really just a system telling you exactly what’s connected to what, in a language you haven’t fully learned yet. You just have to be willing to go down the rabbit hole — and sometimes pick up the phone to another team — until it all makes sense.
]]>Working on IaaS infrastructure with Ceph as the shared storage backend, one of the recurring questions I have to answer whenever something looks off is deceptively simple to ask and genuinely hard to answer: is this a disk problem, a host problem, or just noise?
Our cluster or environment isn’t single-vendor — I support HPE, Supermicro, and Dell hardware, though HPE makes up most of what I troubleshoot day to day. Each vendor has its own out-of-band management flavor and its own quirks in how it surfaces hardware health, which means the underlying OS-level tools end up being the one consistent language across all three.
Ceph will tell you an OSD is flapping, slow, or down. What it won’t tell you outright is why — that part means going a layer deeper into the actual hardware underneath the OSD.
Over time I built up a real workflow for this, mostly leaning on:
smartctl — pulls SMART data directly off the disk: reallocated sector counts, pending sectors, uncorrectable errors, drive temperature history. This is usually the first place a disk starts confessing it’s dying, often well before it fails outright.dmesg — kernel-level messages, great for catching I/O errors, SCSI/SATA reset events, or controller-level complaints happening in real time.journalctl — systemd’s journal, useful for correlating disk errors with service-level symptoms — did the OSD process crash right when the kernel logged a read error? That correlation is often the smoking gun.Here’s the case that really drove the lesson home. I was troubleshooting a host with a suspicious OSD, and iLO reported the disk status as fine — green light, no fault flagged, nothing alarming in the summary view.
But the RAID CLI physical disk view and the kernel logs told a different story. smartctl showed climbing reallocated sector counts. dmesg had scattered I/O errors tied to that specific disk. The RAID controller’s own physical disk status, read directly rather than through iLO’s summary layer, showed predictive failure indicators that iLO simply wasn’t surfacing.
That gap — hardware management interface says fine, actual logs and controller-level data say otherwise — is a real trap if you only ever check the dashboard. iLO aggregates and simplifies; it doesn’t always propagate every signal a drive is generating at the SMART or controller level.
Armed with the RAID CLI physical disk read and the SMART/log evidence, I made the case to DC support to get the disk replaced through the vendor — even though the “official” hardware management view said everything was okay. That’s not always an easy sell; vendors and DC support teams naturally lean on the tool that’s supposed to be authoritative for hardware health. But bringing actual SMART counters, kernel error logs, and RAID controller output made the case concrete instead of “I have a feeling this disk is bad.”
The replacement went through, and it reinforced something I now treat as a rule rather than a suggestion: the management UI is a summary, not the ground truth. If the story from smartctl, dmesg, journalctl, and the RAID CLI’s physical view all point the same direction, that’s the signal to trust — even if the fancy dashboard is still showing green.
There’s something genuinely satisfying about this kind of troubleshooting. It’s detective work with hard evidence instead of guesses — cross-referencing SMART counters against kernel timestamps against RAID controller state against Ceph’s own OSD health signals, until the pieces line up into one coherent story instead of four separate half-answers.
It’s also a good reminder that infrastructure health isn’t one number on one screen. It’s a stack of signals — application layer, storage layer, kernel layer, controller layer, hardware layer — and the real skill is knowing which layer to check when the one you’re looking at isn’t giving you the full picture.
That’s the beauty of RAID CLI and iLO, honestly — not that either one is perfect on its own, but that between the two of them, plus the OS-level tools sitting alongside them, you can actually reconstruct what’s really happening on a disk that’s trying very hard to look fine.
]]>I picked up 5 M710q units to build this out — one dedicated as a firewall box, and four to run the actual infra workloads. A mix of sources: some bought online, some picked up secondhand from a colleague. I also grabbed a Cisco switch to tie everything together properly instead of relying on a consumer-grade switch. Small footprint, low power draw, and enough of them to actually simulate a real multi-node environment instead of just one box doing everything.
One of the units had an issue with its onboard Intel Ethernet controller: a corrupted NVM (non-volatile memory) checksum. The network adapter would report checksum errors, and depending on the exact failure mode, that can mean anything from cosmetic warnings to the NIC refusing to negotiate properly.
Repairing an NVM checksum on Intel controllers isn’t something most people ever have to think about — until suddenly it’s the reason your homelab node won’t get a reliable link. It pushed me to actually learn how Intel’s NIC firmware and NVM structures work, instead of just treating the network port as a black box that either works or doesn’t.
At some point I swapped in a USB-based adapter using a Realtek chipset, mostly out of convenience — they’re cheap and common. And it mostly worked. Mostly.
The frustrating part is that Realtek reliability seems to depend entirely on what you’re trying to run on top of it. Some workloads and apps handle Realtek’s driver quirks fine. Others — particularly anything sensitive to specific offload features, driver stability, or consistent throughput — would flake out in ways that took real time to diagnose. It’s the kind of problem that doesn’t show up in a spec sheet: the chipset “works,” right up until the specific thing you’re deploying exposes exactly the edge case it doesn’t handle well.
This is where things got clearest. When I set up pfSense as the firewall on one of the M710q boxes, I learned quickly that it has a strong preference for Intel network adapters. Realtek chipsets are notorious in the pfSense community for inconsistent driver support — dropped packets, poor performance under load, or outright instability, especially compared to Intel’s NICs, which have mature, well-tested drivers across FreeBSD (which pfSense is built on).
So the same Realtek adapter that was “good enough” for one use case became a real liability the moment pfSense was the thing depending on it. That’s the pattern that stuck with me: reliability isn’t a fixed property of hardware, it’s a property of hardware paired with a specific workload.
None of this was fun in the moment. Chasing an NVM checksum issue, or debugging why a Realtek NIC works under one app but chokes under another, isn’t glamorous work. But it’s exactly the kind of hands-on hardware/network troubleshooting that builds real intuition — the kind you don’t get from reading a datasheet.
It’s also just a good reminder: in a homelab or in production, “network adapter” is never really one thing. It’s a chipset, a driver, a firmware revision, and the specific software stack riding on top of it — and any one of those layers can be the reason something that should just work, doesn’t.
That’s the whole appeal of the homelab, honestly. Five little boxes, a Cisco switch, and enough networking headaches to actually understand why things break — you get to hit these walls on your own hardware, on your own time, and come out the other side actually understanding it.
]]>Gogs was the first one I tried, mostly because everyone describes it as lightweight — single binary, barely any footprint, up in minutes. And it is exactly that. Setup was painless on both Windows Server and later on an Ubuntu VM I spun up. No complaints there.
But it started feeling stale pretty fast. Releases are slow, the community’s quiet, and I kept bumping into stuff that just… isn’t there. It works fine as a bare-minimum git host, but I wasn’t looking for bare minimum, I was looking for something I could actually build a workflow around.
If Gogs was minimal, GitLab was the opposite extreme — CI/CD pipelines, container registry, issue boards, merge request workflows, the whole DevOps platform in a box. But running it at home made the tradeoff obvious fast. GitLab wants resources — real memory, real CPU, a database and a handful of background services all working in concert — and on a homelab built around Lenovo ThinkCentre M710q Tiny units, that’s a lot to ask for a system whose only job, most days, was hosting a handful of personal repos. It felt like bringing a production platform to do a hobby’s job.
Gitea sat right in the middle, and that’s exactly where I needed something to be. It’s actually forked from Gogs, so the lightweight feel is still there, but it didn’t stagnate — regular releases, actions/CI support, package registry, active community, and a UI that feels finished instead of half-abandoned. I tried it on Windows Server first, then again on an Ubuntu VM, and both installs were quick enough that I basically forgot I was “testing” something and just started using it for real repos.
It was the balance. Light enough to run comfortably on modest hardware, capable enough that I’m not missing anything GitLab offered for my use case, and maintained actively enough that I trust it as a long-term home for my repos rather than a placeholder. It’s now my main self-hosted Git server, and every project I care about keeping outside of GitHub lives there.
Haven’t tried Forgejo yet, but it’s next on the list. It’s the community fork of Gitea that split off to run its own governance, and from what I’ve read the config and feel are close enough to Gitea that switching shouldn’t be painful. Low risk to test.
The bigger thing I actually want to do is get git hosting off a VM and into my bare-metal Kubernetes homelab cluster — same cluster I’ve been wrestling with on the Cilium/kube-vip/kubeadm side, fighting DaemonSet crashes and CIDR overlaps. Both Gitea and Forgejo have Helm charts with statefulset persistence and external Postgres support, so once the cluster itself stops being on fire, moving git hosting in should mostly be a matter of doing it properly — PVCs instead of a VM disk, actual resource limits, ingress instead of “just hit the IP.”
We’ll see how Forgejo goes. Might replace Gitea entirely, might not. Either way it’s one more thing running on infra I built myself instead of a VM I forgot to snapshot.
]]>I picked it up in college, back in the Dota 1 days — the era of downloading map files, hopping into Garena or LAN setups, and learning the game almost entirely through trial, error, and getting yelled at by teammates who’d clearly been playing longer than you had. There was no in-game tutorial worth mentioning. You learned by dying to the same hero combo three times and slowly figuring out why.
That era taught me something I didn’t appreciate until much later: the game rewards understanding systems, not just mechanical skill. Item timings, map control, hero synergies, the rhythm of when to fight and when to farm — it’s less “twitch reflexes” and more “read the whole board and make a decision,” which honestly isn’t that different from a lot of what I do now in infrastructure work.
What’s interesting is that I never really stopped. The game evolved — Dota 1 became Dota 2, the whole interface and engine changed, new heroes got added, old ones got reworked entirely — but the core of why I liked it stayed the same. It’s a game where every match is a little different puzzle: this draft, this matchup, this set of teammates, figure out the win condition and execute on it.
There’s also just something satisfying about a hobby with that much continuity. Very few things I did in college are still part of my life in the same form today. Most of my early teammates have moved on, disappeared, or only show up for the occasional nostalgia match. But the game itself is still there, still getting patches, still throwing new problems at me to solve.
I think what’s kept me around this long is that Dota rewards the same kind of thinking I enjoy in my actual work — pattern recognition, adapting a plan when the situation changes, understanding a complex system well enough to predict what happens next. A teamfight that goes sideways because of one missed timing isn’t that different from a cascading infra failure that started with one small missed configuration. Different stakes, obviously, but the same instinct: trace it back, understand what actually happened, do better next time.
It’s also just fun. Not everything needs a deeper meaning — sometimes it’s just good to load into a game with friends, make questionable decisions, and laugh about it after a bad game instead of a good one.
From LAN cafes running Warcraft III maps to whatever the current meta looks like today, it’s been a genuinely long ride. I don’t know how many more years I’ll keep playing, but if the last one-plus decade is any indication — probably a lot more.
]]>Naturally, I needed Git on it. And naturally, yum install git was never going to be the answer.
RHEL 5.1 predates any package repo that ships a usable Git version, so getting it running meant going back to basics — pulling source, compiling it by hand, and working around a system that was never designed with today’s tooling in mind. Here are two ways I got it done.
If you want full control over the installation process, this is the way to go.
Steps:
tar -xvzf git-2.9.5.tar.gz # Extract the tarball
cd git-2.9.5 # Navigate into the extracted folder
make prefix=/usr/local all # Compile Git
make prefix=/usr/local install # Install Git
git --version # Verify installation
Watching a decade-old OS successfully compile a modern-ish Git version from source was oddly satisfying — proof that even ancient systems can still be taught a few new tricks.
If you want a faster, no-nonsense approach — and the box actually has outbound internet access, which not all legacy machines do — run this as root:
cd /tmp && wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.9.5.tar.gz && tar -xvzf git-2.9.5.tar.gz && cd git-2.9.5 && make prefix=/usr/local all && sudo make prefix=/usr/local install && git --version
This single command downloads, extracts, compiles, and installs Git in one shot — a nice shortcut when you don’t want to babysit four separate steps.
Installing Git on RHEL 5.1 isn’t straightforward, but it’s absolutely doable — and honestly, there’s something fun about keeping a legacy system alive and useful instead of just writing it off as too old to bother with. Pick whichever method fits your situation:
Legacy systems have a way of showing up right when you least expect them. Knowing how to work with them, instead of around them, is half the job.
]]>Mine started because I was playing an NFT restaurant simulator game called Dine Together, built on the FactoryChain network, and got tired of doing math in my head every time I wanted to figure out tips, ingredient costs, or whether a piece of furniture was actually worth the price. So I built a calculator. Then another. Then a bunch more.
Looking at the repo now, it’s genuinely fun to see how far it grew from “let me just calculate my tips.” There’s a regular tip calculator, a bonus tip calculator, a combined tip-with-bonus calculator, a chef checker, a farming calculator (with a v2 after I clearly wasn’t happy with the first version), a furniture info page, a dish calculator, marketplace pages for chefs and land, an exchange rate calculator, an energy calculator, and a synergy calculator. What began as a single utility page turned into a small suite of tools covering nearly every mechanic in the game.
That’s one of the things I love about side projects like this — you rarely plan the scope up front. You just keep noticing “oh, I could calculate that too,” and before long you’ve built something a lot bigger than you set out to make.
Along the way, I spent a lot of time in the game’s marketplace — buying and selling items, checking prices, trying to spot good deals. The more time I spent there, the more curious I got about what was actually happening behind the scenes, not just what the marketplace showed me.
So I did what a lot of self-taught devs eventually do — I popped open the browser’s dev tools, went to the Network tab, and started watching what requests the site was actually making. That’s how I first ran into GraphQL. I hadn’t worked with it before, and seeing real query payloads go back and forth, instead of just reading about it in a tutorial, made it click in a way documentation alone never had.
That was the real turning point. Once I understood the marketplace was being driven by GraphQL queries pulling from a backend data source, I started thinking differently about my own project — not “what calculator should I build next,” but “what if I built something that pulled from the marketplace data itself, backed by a real database instead of hardcoded numbers.” That shift is what eventually pushed the project from a handful of static calculator pages toward something with an actual backend behind it — the reason a server.js and package.json quietly showed up in the repo.
Most of the calculators are plain HTML and JavaScript — no framework, no build step, just straightforward DOM manipulation and math. That constraint turned out to be a gift. It’s easy to reach for a framework by default, but building these reminded me how much you can accomplish with vanilla JS when the problem is well-scoped: read some inputs, run the formula, update the page. It also meant I got comfortable organizing a growing set of independent-but-related pages without everything turning into spaghetti — a skill that mattered a lot more later than I expected.
Looking back, a few things really stuck with me. Small annoyances make great starting points — you don’t need a grand idea to start building, just something mildly irritating enough to be worth solving once. Constraints teach fundamentals — working without a framework forced me to actually understand JavaScript instead of leaning on abstractions I didn’t fully grasp yet. And curiosity compounds — poking around someone else’s marketplace requests is what led me to poke around my own backend, which is really where everything I later learned about GraphQL, APIs, and databases got its start.
Dine Together, like a lot of NFT games from that era, has since faded. The hype cycle around blockchain gaming moved on, and the game’s economy isn’t what it once was. But none of the value I got out of building these tools disappeared with it. The JavaScript fundamentals, the instinct to automate an annoying manual task, the early curiosity about GraphQL, APIs, and databases — all of that stuck around and quietly shaped how I approach problems today, infrastructure work included.
That’s the real lesson. A “throwaway” side project for a game you’re playing casually isn’t throwaway at all. Even if the game itself doesn’t last, the muscle you build solving real — if small — problems sticks with you long after the thing you built it for is gone.
The calculator is available at dinetogether-calc.vercel.app and the Official Dine Substack Newsletter — a small time capsule of a game that’s mostly gone now, and proof that the best lessons sometimes come from the projects you never expected to matter.
]]>But “hand it off to a vendor” isn’t where the responsibility ends. Before anything left our hands, we needed to make sure our side of the job was done first — the data itself had to be securely wiped, so there was zero exposure even before the physical destruction step ever happened. You don’t get to assume a vendor’s shredder is your only line of defense; sanitization needs to happen on your end, first.
That’s where PartialVolume/ShredOS came in. It wasn’t something we built — it’s an existing, free, open-source tool, one that lines up with NIST’s guidance on secure media sanitization. The team looked at it as a solid option that fit what we actually needed: boot it from USB, point it at a drive, and it handles secure overwriting without requiring us to build anything custom.
Before rolling it out across the whole disposal backlog, we ran a proper POC first — no point trusting a new tool at scale without validating it actually does what it claims.
I set up four host machines for the proof of concept and configured them to run the wipe process, with reporting built in so every wipe had a record attached to it — which drive, which method, and confirmation that it completed. Once the setup was solid, I shared the results with the IT asset management team.
The output was successful across the board. Drives wiped cleanly, reports generated as expected, and the team had what it needed to sign off. That gave us the green light to proceed with rolling ShredOS out across the rest of the disposal backlog before those drives went to the vendor.
It’s easy to treat asset disposal as an afterthought — old hardware, low priority, just get rid of it. But this project was a good reminder that disposal is still a data-handling process, not just a logistics one. Every one of those drives had, at some point, held real data, and the only way to guarantee it stayed protected was to verify the wipe ourselves instead of assuming the destruction step downstream would cover it.
It was also a nice change of pace from day-to-day operations. Instead of keeping something running, the goal here was making sure something was properly, verifiably gone — different mindset, same level of care.
Sometimes the most important infrastructure work isn’t about building something new — it’s making sure something old is retired the right way, with the evidence to back it up.
]]>