From efcdf387d87234fb201ac971493d453c6ea66205 Mon Sep 17 00:00:00 2001 From: Danny O'Brien Date: Tue, 2 Jan 2024 00:17:05 -0800 Subject: [PATCH] Moved fact-checking.md to 0PASS.org --- .../fact-checking.md | 143 ------------------ 1 file changed, 143 deletions(-) delete mode 100644 assets/editorial/01edits/integrity-new-years-backups/fact-checking.md diff --git a/assets/editorial/01edits/integrity-new-years-backups/fact-checking.md b/assets/editorial/01edits/integrity-new-years-backups/fact-checking.md deleted file mode 100644 index 6524a9a..0000000 --- a/assets/editorial/01edits/integrity-new-years-backups/fact-checking.md +++ /dev/null @@ -1,143 +0,0 @@ - ---- -Author: Integrity Jones -Title: Looking Back Up -Published: 2024-01-01 ---- - -To Old Danny, greetings: - -The solar nights have barely stopped closing in, and we see even large language -models have been taking it easy for the winter. But for the faithful maintainer -of systems, there's still work to be done, here in the cooling embers of the year. - -> Solar nights -- does this mean anything -> Link to LLM stories - -Now is a fine time of year to dust off your backup scripts, and see if they're -working as they should. An untested backup is a no backup at all, said the wise -ghosts of the Scary Devil Monastery, and if we want to set the year off to its -best start, we should ensure we can pause, tear-down, and re-start it at its lowest -points. - -> Link to Scary Devil Monastery - -When examining my backup and restore process, took the opportunity this year to -quadruple the size of my root partition on my server. - -One last manual backup of the 256GiB SSD in my trusty server `boat` for the -sake of the old, and then an attempted restore for the sake of the new: to a -spare machine, `tub`, temporarily hosting a new, 1TiB SSD. If all goes well, -the restored backup in tub would have new room to grow, and I could swap -out that drive back into `boat` with minimal downtime. - -> Box on hard drives? Link to goods? - -That moment of hardware-swapping would mean that `boat` would have to be -shutdown and then restarted anew. Humans need their respite over the holiday break: -but should I have permitted my server the same indulgence? My plan accepted -that boat would be offline for, one hopes, a small slice of time. -There are more convoluted ways to ensure that none of my websites, file syncing, and -miscellaneous tools did not flicker, even for a moment. - -I could have switched my DNS settings to the fresh clean `tub`, for instance, -while overwriting `boat`. Or perhaps just repurposed boat for gentler, less -demanding tasks, giving it the end-of-year gift of a well-deserved retirement, -and switched to `tub` a new year's responsibility of my hosting my main home -processes. - -At the smallest scale, I do believe that uptime is overrated. We are surrounded -by tools bent into the shapes demanded by large tech companies, for whom site -reliability is their first commandment. - -But we are humans, not corporations. If even Homer nods, so can my home lab. -Christmas has its own folklore of visitors rebuffed, but hopefully my friends -have better things to do at this time than hit reload on my websites. Yes, a -personal webserver can go down for a few moments -- as long as it comes back up. - -> Link to Home nods line - -Which, with restored filing systems, is ever the question. Will the backup -truly come back up? A restored and rebooted machine may also be a time for -failure and death, just as, morbidly, we should note that human coronary deaths -seem positively correlated with the change of pace of the holidays. - - -> Link to coronary death paper - -I use the more positively-framed "Relax-And-Recover" (REAR), a recover system -from when sysadmins wrote shell code and /liked it/. REAR is a sprawling bash -scripts that runs your choice of backup code -- from rsync to borg-- over your -entire linux root, on a regular, cron-determined, basis. Having seen to that -prosaic task, it will also create minimal, bootable blob. The blob, stuffed -onto an USB drive, CD ROM, or networked drive, will boot into a minimal Linux, - and gently lead you through reconstructing a re-partitioning of a drive - that will. At its end, the partition map on the machine you have booted - will be exactly the same shape as your original machine. Then it will pull - down your precious backups, and restore this drive to the precise state - that your backups recalled it. A perfect, royal, restoration. - - > Link to Relax and Recover - -Younger readers born into our age of strong types and weak stomachs, may be -balking at the idea of entrusting restoration to a bunch of stringly-typed Bash -scripts. REAR's 20K(!) lines of shellcode intimate that it has reached the -edges of complexity beyond that you might think bashism can bear. - -> Link to definition of stringly -> Link to proof of REAR's 20K of KLOCs - -But this is /sysadmin/ shell code. That terrifying KLOC is -defensive, modular coding of the highest order. For the casual shell user, -REAR's operation is comprehensible, the failure modes anticipated, and -tweaks and errors are semi-obvious. REAR's authors emphasise "a relaxing -recovery", and ghosts of sysadminning past do not emphasise enhancing your -calm in those moments lightly. - - Link to relaxing recovery - -I was never /not/ relaxed during my holiday restoration, but I did -occasionally emit a high-pitched carol or two of concern. Firstly, I had REAR create ISO -files for burning onto a CD-ROM that I could restore from. CD stands for -"Cretaceous Disk". Even I have not used one of them in anger in over a decade. -Converting a bootable ISO into a bootable USB drive drive turned out to be -surprisingly tricky, and I can never remember how to do it. In the end I was -forced, humiliatingly, to read REAR's FAQ, where they recommended a heavy -utility, https://unetbootin.github.io/, for achieving this. In the future, I've -set REAR to output those bootable blobs as RAWDISK, which can be burned -(warmed?) onto a USB. - -> Link to REAR's FAQ -> Link to UnetBootlin - -REAR's ecumenical acceptance of multiple backup programs introduces additional -complexity into its scripts. My use of borg backup tripped up the restore. REAR -scans executables that it plans to include on its rescue bootable blob, so it -can detect what libraries they need and copy those over. Sadly, the borg -executable can be either a binary executable (for which this works), or just a -Python script, whose demands REAR cannot fathom. This is undoubtedly a bug a -future REAR will fix, but in the meantime I just copied over the binary borg -into /usr/local/bin and used that instead of the Pythonic borg of the Debian -repos. - -> Link to binary executable -> Link to executable/python script issue on github - -After those tweaks, tub was filled with the form of boat's Christmas past. I swapped -over the two drives, holding my breath, and losing the little SSD screw as -always (these plastic nubbin replacements tempted me in the fallout). Boat -wobbled and then, like a plucky North Sea veteran, bobbed back up. - -> Link to nubbin ali express sale -> Link to North Sea tiktok music - -Testing your recovery plans during your own end-of-year downtime and recovery -gives you a moment to peer at what has been, and prepare for the ups and downs -of the coming year. What will be the same? What will change? What parts of your -life can you simply hard link to the habits of the past? And what will you have -to incrementally add and integrate into your ever-evolving life? - -Until next time, I am, - -~Integrity Mather -