The Cathedral and the Bizarre, or
Dual-Booting for Fun and Prophet
Using Linux to Enhance TempleOS
Toby Betts
SeaGL, 2024-11-08
Disclaimer
- This presentation is for educational purposes only and does not contain any professional advice or recommendations and is not endorsed by the event organizer
- Use at your own risk, yes this means you
- Your actual mileage may vary
- Make verified backups before implementing any system or workflow changes
Difficulty Level
Technical:
1/5 multitools: Mastering boot records and boot loaders... mostly
Difficulty Level
Booting:
4/5 boots: This talk was made for bootin' and that's just what we'll do
Genesis - In the beginning...
In the beginning God created the heaven and the earth and the earth was without form and void and darkness was upon the face of the deep
Genesis - In the beginning...
Genesis - In the beginning...
An average disk layout
A basic DOS-era disk layout defines contiguous blocks of storage called partitions
- Define the partition table with fdisk, gparted
- Reserve space at beginning to store the Master Boot Record
- Different partitions are treated as "volumes" in DOS/Windows
- Different partitions can store different operating systems
|
|
https://pikchr.org/home/pikchrshow
boxht = .2; boxwid = 1.3;
down
box "MBR"; box ht 0.7*boxht "safety gap";
box ht 2*boxht "partition 1";
box ht 3*boxht "." "." "."
L: box ht 2*boxht "partition 4"; box ht 0.8*boxht "unallocated space";
box invis wid .1*boxwid "0" with .e at 1st box .nw
box invis wid .2*boxwid "512" with .e at 2nd box .nw
box invis wid .2*boxwid "1M" with .e at 3rd box .nw
box invis wid .2*boxwid "1G" with .e at 3rd box .sw
box invis wid .2*boxwid "end" with .e at 6th box .sw
An average dual-boot layout
The traditional way to dual-boot
- Create two partitions
- Install DOS/Windows to first partition
- Install Linux to second partition
- Install Linux-aware boot loader
|
 |
https://pikchr.org/home/pikchrshow
boxht = .2; boxwid = 1.3;
down
box "MBR"; box ht 0.7*boxht "safety gap";
box ht 3*boxht "DOS/Windows (FAT32)";
L: box ht 2*boxht "Linux (ext2)"; box ht 0.8*boxht "unallocated space";
box invis wid .1*boxwid "0" with .e at 1st box .nw
box invis wid .2*boxwid "512" with .e at 2nd box .nw
box invis wid .2*boxwid "1M" with .e at 3rd box .nw
box invis wid .2*boxwid "1G" with .e at 3rd box .sw
Why Boot Loaders are important
The first 512 bytes of the hard disk are essential
- Holds the partition table, defining contiguous storage blocks and what they contain
- It's why you never run dd if=/dev/zero of=/dev/sda unless you really want to choose violence
- MBR tells to computer where to look next instructions
Why Boot Loaders are important
MBR points to the boot loader
- Not limited to 512 bytes
- Allows for sophisticated configuration details
- Can be OS-dependent or OS-agnostic
- Examples: GRUB, LILO, NTLDR, EasyBCD
- Responsible for finding and loading OS kernel
Exodus
Boot loaders let you run multiple operating systems. Maybe something:
- Fast, free
- 64-bit architecture
- Developer-friendly
- Non-proprietary, license unencumbered
- Customizable
- Creator has reputation for yelling at people
Exodus
TempleOS is:
- A free, public domain
- 64-bit native operating system with
- a custom kernel
- built with a custom compiler, both are
- written in a custom C-like language (HolyC)
Exodus
TempleOS has:
- transparent file compression
- custom file system (RedSea)
- custom datetime library
- custom documentation format with
- custom 2D and 3D graphics libraries
- impervious network security
Exodus - Transparent File Compression
- "Files with names ending in .Z are individually compressed"
- To compress a file:
Move("MyFile.TXT","MyFile.TXT.Z");
Exodus - POSIX time
- Linux/UNIX measure time by counting seconds since January 1, 1970
- This is the root of the Y2038 problem. 2038 - 1970 = 68 years
- 68 * 365.25 * 86,400 is approx. (2^31)-1 seconds
Doc/Strategy.DD, Doc/RedSea.DD
Exodus - The TempleOS datetime object
public I64 class CDate
{
U32 time;
I32 date;
};
Elapsed time is measured as (2^31)-1 calendar days and (2^32)-1 moments per day
TempleOS epoch is January 2, 0 AD
There was no year 0
Doc/Strategy.DD, Doc/RedSea.DD
Numbers - Gregorian calendars
Western secular timekeeping is based on the Gregorian calendar.
- Adopted October 1582 by Pope Gregory XIII (1502-1585)
- Dates before this used the previous Julian calendar
- Most sovereign states adopted this calendar... eventually
- 0 is less than 1582
Numbers - Gregorian calendars
When will a CDate timestamp roll over?
CDate cdt=0;
CDateStruct ds;
while (cdt.date<I32_MAX) {
cdt.date<<=1;
cdt.date++;
"0x%08x ",cdt.date;
"%10d ",cdt.date;
Date2Struct(&ds,cdt);
"%3t2 %2d, %d\n"
,ds.mon-1,"ST_MONTHS",ds.day_of_mon,ds.year;
}
Numbers - Gregorian calendars
When will a CDate timestamp roll over?
cdt.date decimal ds
0x00000001 1 Jan 3, 0
0x00000003 3 Jan 5, 0
0x00000007 7 Jan 9, 0
0x0000000f 15 Jan 17, 0
0x0000001f 31 Feb 2, 0
0x0000003f 63 Mar 5, 0
0x0000007f 127 May 8, 0
0x000000ff 255 Sep 13, 0
Numbers - Gregorian calendars
When will a CDate timestamp roll over?
cdt.date decimal ds
0x000001ff 511 May 27, 1
0x000003ff 1023 Oct 21, 2
0x000007ff 2047 Aug 10, 5
0x00000fff 4095 Mar 20, 11
0x00001fff 8191 Jun 6, 22
0x00003fff 16383 Nov 9, 44
0x00007fff 32767 Sep 18, 89
0x0000ffff 65535 Jun 7, 179
Numbers - Gregorian calendars
When will a CDate timestamp roll over?
cdt.date decimal ds
0x0001ffff 131071 Nov 12, 358
0x0003ffff 262143 Sep 23, 717
0x0007ffff 524287 Jun 15, 1435
0x000fffff 1048575 Nov 26, 2870
0x001fffff 2097151 Oct 23, 5741
0x003fffff 4194303 Aug 15, 11483
0x007fffff 8388607 Mar 30, 22967
0x00ffffff 16777215 Jun 28, 45934
Numbers - Gregorian calendars
When will a CDate timestamp roll over?
cdt.date decimal ds
0x01ffffff 33554431 Dec 22, 91868
0x03ffffff 67108863 Dec 14, 183737
0x07ffffff 134217727 Nov 28, 367475
0x0fffffff 268435455 Oct 26, 734951
0x1fffffff 536870911 Aug 22, 1469903
0x3fffffff 1073741823 Apr 11, 2939807
0x7fffffff 2147483647 Jul 20, 5879614
Conclusion: Need to solve the Y-5,879,614 problem... eventually.
Exodus - RedSea
- "The RedSea filesystem allocates just contiguous files -- you load and save whole files at once."
- "The RedSea file system is a simple, 64-bit, file system which is similar to FAT32, but with absolute block addresses instead of [clusters], fixed-sized 64-byte directory entries and no FAT table, just an allocation bitmap."
- "Files are stored in contiguous blocks and cannot grow in size."
- Incompatible with FAT32, but TempleOS supports both
Doc/Strategy.DD, Doc/RedSea.DD
Exodus - RedSea
RedSea FS stores the core kernel dir entry data structure
#define CDIR_FILENAME_LEN 38
public class CDirEntry
{
CDirEntry *next,*parent,*sub;
U8 *full_name;
I64 user_data,user_data2;
U0 start;
U16 attr;
U8 name[CDIR_FILENAME_LEN];
I64 clus,size;
CDate datetime;
};
Kernel/KernelA.HH, Kernel/BlkDev/FileSysRedSea.HC
Numbers - Why is TempleOS?
- Began in 2003 as J Operating System
- Then became LoseThOS, SparrowOS, finally TempleOS
- "[Terry] Davis claims much of the operating system's development was dictated by God, even claiming that the OS was meant to be God's third temple."
Vice - God's Lonely Programmer
https://knowyourmeme.com/memes/people/terry-davis-templeos
Numbers - Be not afraid
Quick notes about TempleOS:
- Kernel & compiler are binaries (.BIN.C)
- Everything else is JIT-compiled
- Programs are kept as source (.HC) & headers (.HH)
- The shell is an interactive compiler
- Scripts are also HolyC (.IN)
- Text is either .TXT or .DD
Numbers - Be not afraid
Quick notes about TempleOS:
- Single-user, no user accounts
- No passwords, no encryption
- No sudo, no privilege escalation
- "Ring-0-only. Highest CPU privileged mode at all times."
- "640x480 16 color VGA graphics."
- "Keyboard & Mouse support."
See C:/Doc/Features.DD.Z
Numbers - Be not afraid
"Linux is a semi... Windows is a car.... TempleOS is a motorcycle -- if you lean-over too far, a motorcycle will crash. Don't do that!"
- C:/Doc/Welcome.DD.Z
- C:/Doc/FAQ.DD.Z
- C:/Doc/CutCorners.DD.Z
- C:/Doc/Strategy.DD.Z
- C:/Doc/Tips.DD.Z
Welcome.DD.Z
"TempleOS is a x86_64, multi-cored, non-preemptive multi-tasking, ring-0-only, single-address_mapped (identity-mapped), operating system for recreational programming. The people whom can most benefit are:
- Professionals doing hobby projects
- Teenagers doing projects
- Non-professional, older-persons projects"
Numbers - Welcome.DD.Z
"It's a kayak, not a Titanic -- it will crash if you do something wrong. You quickly reboot, however.""
Numbers - Be not afraid
"My vision is a souped-up C64, not a 1970's mainframe"
— Terry A. Davis
Lamentations - Switches, lights, and knobs
|
"We've all got our switches, lights, & knobs
to deal with, Stryker. I mean down here there
are literally hundreds and thousands of blinking,
beeping, & flashing lights, blinking & beeping
& flashing. They're flashing & they're beeping...
I can't stand it anymore, the blinking & beeping
& flashing!" |
Acts - We can fix the blinking
Find and change $$BK,1$$ to $$BK,0$$ in:
- /Kernel/KernelA.HH
- /Adam/DolDoc/DocEd.HC
- /Adam/DolDoc/DocTerm.HC
Rebuild the kernel and restart
Acts - TipOfDay();
/Doc/Tips.DD contains helpful hints, print one with TipOfDay;
- I copy my files to a mirrored ident partition, periodically with CopyTree() commands in scripts. I do merge commands with a menu entry like this:
Merge("C:/*","D:/*","+r+d"); to check my changes.
- Merge() can be used to see what's changed. The +d flag will show differences of files which have changed and allow you to merge code. (The +r flag will recurse.)
Lamentations - Problem Statement
TempleOS is for coding, gaming, learning, exploring, but:
- Exceptionally small footprint (<20MB, ~82KLOC)
- TempleOS tools not cross-platform compatible
- No network stack makes sharing code difficult
- Official suggestion: maintain a full copy on 2nd partition
- Backups != source control
Solution? Add another operating system!
Disk partitioning options
1 Samuel - No more Davids, only Goliaths
Multiple alternative OSes considered:
- Devuan daedalus (340MB 17x)
- Void Linux (280MB 14x)
- OpenBSD 7.4/amd64 (1.1GB 55x)
- Damn Small Linux: "The new goal of DSL is... a hard limit of 700MB"
Pre-reqs: must be (a) tiny, (b) FAT32-support in core
1 Samuel - Five smooth stones
"And David prevailed over Goliath with a sling and a stone"
Tiny Core Linux: http://tinycorelinux.net
- 64-bit
- Very tiny (<20MB)
- Minimum install is 2 files (corepure64.gz & vmlinuz64)
- Extensible via packages (tce-load -wi openssh)
- GUI optional
Ideal TempleOS dual-boot layout
- Single FAT32 partition
- Combines TempleOS files & full Linux install
- In the same file system
- OS chosen at boot time
|
|
Install TempleOS (the right way)
- Boot a Linux live session
- fdisk /dev/sda
- mkfs.vfat -F32 /dev/sda1
- Boot TempleOS ISO
- Install (#include /Misc/OSInstall.HC)
- Choose NOT to format C: drive
- Choose NOT to install TempleOS bootloader
- Reboot
Install TempleOS (the right way)
- Boot a Linux live session
- Mount partition under /mnt/sda1
- mkdir -p /mnt/sda1/tce/boot
- Copy corepure64.gz & vmlinuz64 to /mnt/sda1/tce/boot
- Install syslinux bootloader
syslinux - The key to all of this
syslinux: the bootloader you could always use but never have
- Included by default in most distros
- cd /usr/lib/syslinux/modules/bios
- cp menu.c32 libutil.c32 /mnt/sda1
- vi /mnt/sda1/syslinux.cfg
- dd if=/usr/lib/syslinux/mbr/mbr.bin \
of=/dev/sda bs=440 count=1 conv=notrunc
- syslinux -i /dev/sda1
syslinux.cfg
UI menu.c32
PROMPT 1
MENU TITLE Boot Menu
MENU ROWS 2
LABEL TempleOS
MENU LABEL TempleOS
BOOT /Kernel.BIN.C
LABEL tc
MENU LABEL Tiny Core Linux
KERNEL /tce/boot/vmlinuz64
INITRD /tce/boot/corepure64.gz
APPEND loglevel=3 vga=789 lang=en_US.UTF-8
syslinux - This also works on Windows
Why do this?
Advantages to merging TempleOS with Linux
- Goldilocks partition table: not too small, not too large
- No need for intermediary space to transfer files
- Best of both worlds: use TempleOS & Linux tools at will
But what about data integrity?
Data integrity
All TempleOS data is stored in either RedSea or FAT32.
- File systems have bugs
- Bespoke file systems are hard to debug
- It sure would be nice to know if a file is corrupt before you need it
TempleOS needs a file checksum tool. Linux has plenty!
What is a checksum?
Read every byte of a file & compute a mathematical equation with those bytes. If any bytes change or their order changes the result will be different. Example: Adler32 (See RFC 1950).
I64 i,s1=1,s2=0;
U8 *buf=StrNew("DOG")
for(i=0;i<StrLen(buf);i++) {
s1+=buf[i]%65521;
s2+=s1%65521;
}
"%04x %04x\n",s2,s1;
Free(buf);
Different answers for different inputs
Each byte and its order in the sequence affects the result.
"DOG" => 01b4 00db
"CAT" => 01a2 00d9
"GOD" => 01ba 00db
Some checksums "mix" bytes faster, some make reversing output back into input difficult
Checksums in Linux
Multiple checksum utilities
- MD5: md5sum
- SHA-1: sha1sum
- Many others, but these are still popular and prevalent
These are old & insecure algorithms but still useful for file checking
But there are no TempleOS equivalents!
TempleOS: Now with md5sum
Tracking files in TempleOS
But what is there besides MD5?
- MD5 is deprecated
- Common protocols use SHA-based hashes (BitTorrent, git)
- SHA-1 uses a expansion function to create longer digest
- But there is no TempleOS equivalent!
TempleOS: Now with sha1sum, too!
Metadata matters
Checksums are only part of the equation:
- git computes checksums, detects corruption...
- ...but git only worries about content
- files aren't only content, have modification times, modes
- these are things a file system tracks, if you trust it
- git is not an ideal archiving tool
mtree
mtree is a BSD utility, ported to Linux
- apt-get install mtree-netbsd
- https://github.com/vbatts/go-mtree
- given a path, creates "spec file" of names, modes, mtimes, and more; like "ls -lR"
- a spec file can be used to check or repair a file tree
- file tree metadata in a spec file is portable
- But there is no TempleOS equivalent!
TempleOS: Now with mtree, too!
Revelation - What's next?
"OSS maxim: Is there something you are looking for and it's not available? Then build it."
UNIX utilities are still lacking native TempleOS ports
- head, tail, grep
- rsync-like incremental file copy
- git, zlib compression
Porting third-party programs to TempleOS - jwhitham.org
Tiny Core NTFS integration - yalis.fr via Wayback Machine
Revelation - What's next?
Revelation - What's next?
Revelation - What's next?
