The Cathedral and the Bizarre,
or
Dual-Booting for Fun and Prophet

Using Linux to Enhance TempleOS

Toby Betts

LinuxFest Northwest, 2024-04-27

Disclaimer

This presentation is for educational purposes only and does not contain any professional advice or recommendations and is not endorsed by the event organizer
Use at your own risk, yes this means you
Your actual mileage may vary
Make verified backups before implementing any system or workflow changes

Difficulty Level

Technical:

1/5 multitools: Mastering boot records and boot loaders

Difficulty Level

Booting:

4/5 boots: This talk was made for bootin' and that's just what we'll do

Genesis - In the beginning...

In the beginning God created the heaven and the earth and the earth was without form and void and darkness was upon the face of the deep

Genesis - In the beginning...

An average disk layout

A basic DOS-era disk layout defines contiguous blocks of storage called partitions

Define the partition table with fdisk, gparted
Reserve space at beginning to store the Master Boot Record
Different partitions are treated as "volumes" in DOS/Windows
Different partitions can store different operating systems

https://pikchr.org/home/pikchrshow boxht = .2; boxwid = 1.3; down box "MBR"; box ht 0.7*boxht "safety gap"; box ht 2*boxht "partition 1"; box ht 3*boxht "." "." "." L: box ht 2*boxht "partition 4"; box ht 0.8*boxht "unallocated space"; box invis wid .1*boxwid "0" with .e at 1st box .nw box invis wid .2*boxwid "512" with .e at 2nd box .nw box invis wid .2*boxwid "1M" with .e at 3rd box .nw box invis wid .2*boxwid "1G" with .e at 3rd box .sw box invis wid .2*boxwid "end" with .e at 6th box .sw

An average dual-boot layout

The traditional way to dual-boot

Create two partitions
Install DOS/Windows to first partition
Install Linux to second partition
Install Linux-aware boot loader

https://pikchr.org/home/pikchrshow boxht = .2; boxwid = 1.3; down box "MBR"; box ht 0.7*boxht "safety gap"; box ht 3*boxht "DOS/Windows (FAT32)"; L: box ht 2*boxht "Linux (ext2)"; box ht 0.8*boxht "unallocated space"; box invis wid .1*boxwid "0" with .e at 1st box .nw box invis wid .2*boxwid "512" with .e at 2nd box .nw box invis wid .2*boxwid "1M" with .e at 3rd box .nw box invis wid .2*boxwid "1G" with .e at 3rd box .sw

Why Boot Loaders are important

The first 512 bytes of the hard disk are essential

Holds the partition table, defining contiguous storage blocks and what they contain
It's why you never run dd if=/dev/zero of=/dev/sda unless you really want to choose violence
MBR tells to computer where to look next instructions

Why Boot Loaders are important

MBR points to the boot loader

Not limited to 512 bytes
Allows for sophisticated configuration details
Can be OS-dependent or OS-agnostic
Examples: GRUB, LILO, NTLDR, EasyBCD
Responsible for finding and loading OS kernel

Exodus

Boot loaders let you run multiple operating systems. Maybe something:

Fast, free
64-bit architecture
Developer-friendly
Non-proprietary, license unencumbered
Customizable
Creator has reputation for yelling at people

Exodus

TempleOS is:

A free, public domain
64-bit native operating system with
a custom kernel
built with a custom compiler, both are
written in a custom C-like language (HolyC)

Exodus

TempleOS has:

transparent file compression
custom file system (RedSea)
custom datetime library
custom documentation format with
custom 2D and 3D graphics libraries
impervious network security

Exodus - Transparent File Compression

"Files with names ending in .Z are individually compressed"
To compress a file:

Move("MyFile.TXT","MyFile.TXT.Z");

Exodus - RedSea

"The RedSea filesystem allocates just contiguous files -- you load and save whole files at once."
"The RedSea file system is a simple, 64-bit, file system which is similar to FAT32, but with absolute block addresses instead of [clusters], fixed-sized 64-byte directory entries and no FAT table, just an allocation bitmap."
"Files are stored in contiguous blocks and cannot grow in size."
Incompatible with FAT32, but TempleOS supports both

Exodus - POSIX time

Linux/UNIX measure time by counting seconds since January 1, 1970
This is the root of the Y2038 problem. 2038 - 1970 = 68 years
68 * 365.25 * 86,400 is approx. (2^31)-1 seconds

Exodus - The TempleOS datetime object

public I64 class CDate { U32 time; I32 date; };

Elapsed time is measured as (2^31)-1 calendar days and (2^32)-1 moments per day
TempleOS epoch is January 2, 0 AD
There was no year 0

Numbers - Gregorian calendars

Western secular timekeeping is based on the Gregorian calendar.

Adopted October 1582 by Pope Gregory XIII (1502-1585)
Dates before this used the previous Julian calendar
Most sovereign states adopted this calendar... eventually
0 is less than 1582

Numbers - Why is TempleOS?

Began in 2003 as J Operating System
Then became LoseThOS, SparrowOS, finally TempleOS
"[Terry] Davis claims much of the operating system's development was dictated by God, even claiming that the OS was meant to be God's third temple."

Vice - God's Lonely Programmer

https://knowyourmeme.com/memes/people/terry-davis-templeos

Numbers - Be not afraid

"My vision is a souped-up C64, not a 1970's mainframe"
— Terry A. Davis

Lamentations - Switches, lights, and knobs

"We've all got our switches, lights, and knobs to deal with, Stryker. I mean down here there are literally hundreds and thousands of blinking, beeping, and flashing lights, blinking and beeping and flashing. They're flashing and they're beeping... I can't stand it anymore, the blinking and beeping and flashing!"

Acts - We can fix the blinking

Find and change $$BK,1$$ to $$BK,0$$ in:

/Kernel/KernelA.HH
/Adam/DolDoc/DocEd.HC
/Adam/DolDoc/DocTerm.HC

Rebuild the kernel and restart

Acts - TipOfDay();

/Doc/Tips.DD contains helpful hints, print one with TipOfDay;

I copy my files to a mirrored ident partition, periodically with CopyTree() commands in scripts. I do merge commands with a menu entry like this:
Merge("C:/*","D:/*","+r+d"); to check my changes.
Merge() can be used to see what's changed. The +d flag will show differences of files which have changed and allow you to merge code. (The +r flag will recurse.)

Lamentations - Problem Statement

TempleOS is a platform for coding, gaming, learning, and exploring, but:

Exceptionally small footprint (<20MB, ~82KLOC), limited toolkit
TempleOS tools are not cross-platform compatible
No network stack makes sharing code difficult
Official suggestion is to maintain a full copy of system on separate partition
Backups != source control

Solution? Add another operating system!

Disk partitioning options

Option #1	Option #2

1 Samuel - No more Davids, only Goliaths

Multiple alternative OSes considered:

Devuan daedalus (340MB)
Void Linux (280MB)
OpenBSD 7.4/amd64 (1.1GB)
Damn Small Linux: "The new goal of DSL is... a hard limit of 700MB"

Pre-reqs: must be (a) tiny, (b) FAT32-support in core

1 Samuel - Five smooth stones

"And David prevailed over the Philistine, with a sling and a stone"

Tiny Core Linux: http://tinycorelinux.net

64-bit
Very tiny (<20MB)
Minimum install is 2 files (corepure64.gz & vmlinuz64)
Extensible via packages (tce-load -wi openssh)
GUI optional

Ideal TempleOS dual-boot layout

Single FAT32 partition
Combines TempleOS files & full Linux install
In the same file system
OS chosen at boot time

Install TempleOS install (the right way)

Boot a Linux live session
fdisk /dev/sda
mkfs.vfat -F32 /dev/sda1
Boot TempleOS ISO
Install (#include /Misc/OSInstall.HC)
Choose NOT to format C: drive
Choose NOT to install TempleOS bootloader
Reboot

Install TempleOS install (the right way)

Boot a Linux live session
Mount hard drive under /mnt/sda1
mkdir -p /mnt/sda1/tce/boot
copy corepure64.gz & vmlinuz64 to /mnt/sda1/tce/boot
Install syslinux bootloader

syslinux - The key to all of this

syslinux: the bootloader you could always use but never have

Included by default in most distros
cd /usr/lib/syslinux/modules/bios
cp menu.c32 libutil.c32 /mnt/sda1
vi /mnt/sda1/syslinux.cfg
dd if=/usr/lib/syslinux/mbr/mbr.bin \ of=/dev/sda bs=440 count=1 conv=notrunc
syslinux -i /dev/sda1

syslinux.cfg

UI menu.c32
PROMPT 1

MENU TITLE Boot Menu
MENU ROWS 2

LABEL TempleOS
  MENU LABEL TempleOS
  BOOT /Kernel.BIN.C

LABEL tc
  MENU LABEL Tiny Core Linux
  KERNEL /tce/boot/vmlinuz64
  INITRD /tce/boot/corepure64.gz
  APPEND loglevel=3 vga=789 lang=en_US.UTF-8

syslinux - This also works on Windows

Install Windows to FAT32 partition
dd if=/dev/sda1 of=sda1.bss bs=512 count=1

LABEL Win98
          MENU LABEL Windows 98
          BOOT sda1.bss

Boot Linux from DOS with LOADLIN.EXE
Use Tiny Core v13

Why do this?

Advantages to merging TempleOS with Linux

Goldilocks partition table: not too small, not too large
No need for intermediary space to transfer files
Best of both worlds: use TempleOS & Linux tools at will

But what about data integrity?

Data integrity

All TempleOS data is stored in either RedSea or FAT32.

File systems have bugs
Bespoke file systems are hard to debug
It sure would be nice to know if a file is corrupt before you need it

TempleOS needs a file checksum tool. Linux has plenty!

What is a checksum?

Read every byte of a file & compute a mathematical equation with those bytes. If any of the bytes change, or their order changes, the result of the equation will be different. Example (don't do this):

Convert "DOG" to ASCII: 68, 79, 71
For each value a_i
Take the prime numbers P in order, compute P_i^a_i
Sum each P_i^a_i
2⁶⁸ + 3⁷⁹ + 5⁷¹
"DOG" =>
42,351,647,362,764,286,563,220,907,008,715,940,411,151,060,669,848

Different answers for different inputs

"DOG" => 2⁶⁸ + 3⁷⁹ + 5⁷¹
42,351,647,362,764,286,563,220,907,008,715,940,411,151,060,669,848

"CAT" => 2⁶⁷ + 3⁶⁵ + 5⁸⁴
51,698,788,284,564,229,679,463,043,264,673,729,808,888,368,337,868,155,321,396

"GOD" => 2⁷¹ + 3⁷⁹ + 5⁶⁸
338,813,178,950,989,745,432,110,977,071,734,429,657,507,013,340

Some algorithms make reversing output back into input difficult

Checksums in Linux

Multiple checksum utilities

MD5: md5sum
SHA-1: sha1sum
Many others, but these are still popular and prevalent

These are old & insecure algorithms, but still functional, still useful for file checking

But there are no TempleOS equivalents!

TempleOS: Now with md5sum

MD5 in HolyC

Tracking files in TempleOS

But what is there besides MD5?

MD5 is deprecated
Common protocols use SHA-based hashes (BitTorrent, git)
SHA-1 uses a expansion function to create longer digest
But there is no TempleOS equivalent!

TempleOS: Now with sha1sum, too!

Metadata matters

Checksums are only part of the equation:

git computes checksums, detects corruption...
...but git only worries about content
files aren't only content, have modification times, modes
these are things a file system tracks, if you trust it
git is not an ideal archiving tool

mtree

mtree is a BSD utility, ported to Linux

apt-get install mtree-netbsd
https://github.com/vbatts/go-mtree
given a path, creates "spec file" of names, modes, mtimes, and more; like "ls -lR"
a spec file can be used to check or repair a file tree
file tree metadata in a spec file is portable
But there is no TempleOS equivalent!

TempleOS: Now with mtree, too!

Revelation - New TempleOS apps

MD5
SHA1
Mtree

Available at https://su.bze.ro/software.html

Revelation - What's next?

"OSS maxim: Is there something you are looking for and it's not available? Then build it."
UNIX utilities are still lacking native TempleOS ports

head, tail, grep
rsync-like incremental file copy
git, zlib compression

Porting third-party programs to TempleOS - jwhitham.org
Tiny Core NTFS integration - yalis.fr via Wayback Machine

Revelation - What's next?

Slides available online

https://su.bze.ro/talks.html

The Cathedral and the Bizarre,orDual-Booting for Fun and Prophet

Using Linux to Enhance TempleOS

Toby Betts

LinuxFest Northwest, 2024-04-27

Disclaimer

Difficulty Level

Difficulty Level

Genesis - In the beginning...

Genesis - In the beginning...

Genesis

Genesis

Genesis

Genesis

Genesis

An average disk layout

An average dual-boot layout

Why Boot Loaders are important

Why Boot Loaders are important

Exodus

Exodus

Exodus

Exodus - Transparent File Compression

Exodus - RedSea

Exodus - POSIX time

Exodus - The TempleOS datetime object

Numbers - Gregorian calendars

Numbers - Why is TempleOS?

Numbers - Be not afraid

Lamentations - Switches, lights, and knobs

Acts - We can fix the blinking

Acts - TipOfDay();

Lamentations - Problem Statement

Disk partitioning options

1 Samuel - No more Davids, only Goliaths

1 Samuel - Five smooth stones

Ideal TempleOS dual-boot layout

Install TempleOS install (the right way)

Install TempleOS install (the right way)

syslinux - The key to all of this

syslinux.cfg

syslinux - This also works on Windows

Why do this?

Data integrity

What is a checksum?

Different answers for different inputs

Checksums in Linux

TempleOS: Now with md5sum

MD5 in HolyC

Tracking files in TempleOS

But what is there besides MD5?

TempleOS: Now with sha1sum, too!

Metadata matters

mtree

TempleOS: Now with mtree, too!

Revelation - New TempleOS apps

Revelation - What's next?

Revelation - What's next?

Revelation - What's next?

Revelation - What's next?

Slides available online

The Cathedral and the Bizarre,
or
Dual-Booting for Fun and Prophet