blog dot information dash superhighway dot net

I recently wrote a small 16-bit Forth for 8086 PCs running DOS. I built the basic one-liner loop words that can trivially be built with just “branch if zero” and “goto”: begin, while, repeat, until, again. But I held off on implementing do / loop at first.

It didn't seem like too much of a hardship. In a previous Forth I'd built, I'd implemented do / loop using the return stack, but it was... ugly. The code to implement it was ugly, the code it generated was ugly (and large!), and I didn't find a lot of places where it was actually much nicer to use than explicit begin-based loops. I was able to implement an 8086 assembler and a Minesweeper game without bothering to build do / loop. I didn't really miss it, but I had a design percolating in the back of my mind that I wanted to try.

At some point I came across some writing that suggested that Forth had a “loop control stack”. Wouldn't it be nice if I could implement some kind of loop control stack that worked for all kinds of iteration?

The thing I built has blown me away with how flexible, composable, and useful it's turned out to be. It's way more powerful than I was expecting. And the code that leverages it is inevitably much simpler and easier to read.

Read more...

I fairly frequently see people who are taking an interest in Forth struggle with the idea of programming without local variables. I struggled with it when I started writing Forth! I feel like there's an unspoken assumption for people coming to Forth from other languages, and if I were to speak it aloud, it would sound something like “temporary data should go on the stack”.

Because... functions should be re-entrant by default! They should clean up after themselves! Global variables are bad and must be avoided at all costs! Functions should be “pure” and take all of their inputs as parameters, avoiding hidden dependencies!

All of these ideas of what “good code” looks like are wrong in Forth.

It is actually extremely common for Forth words to rely on implicit context, which is globally accessible through other Forth words. This is often how you build DSLs!

Read more...

I wrote this analysis of Return to Monkey Island's ending on October 4th, a couple of weeks after finishing it, in a kind of manic fugue state, writing well past the time I should have been in bed. After I'd written it, I decided it probably needed an editing pass before I posted it, but life immediately got in the way and I never looked at it again. Today I picked it up and read through it and decided not a word needed to be changed. So, without further ado, here it is.

HERE BE SPOILERS. You've been warned.

Read more...

On Feb 9, I made the following comment on my Mastodon:

still can't believe the shithead behind Pupkin managed to get his webcomic turned into a movie with Jennifer Lopez

Nobody responded to this in any way, because, quite frankly, nobody who wasn't an absolute fanatical follower of webcomics discourse 20 years ago has any idea what the fuck this means. Hell, I barely understand my feelings about this. But to do them justice, we are going to have to unpack 25 years of webcomics history.

The process of researching and writing this has rearranged my view of the formative years of webcomics. I've done my best to cite my sources, but primary sources are not always readily available anymore. I ended up using the Wayback Machine so much that it locked me out for a few hours. Support the Internet Archive.

Read more...

or, discovering and recovering a lost programming language over the course of a weekend

I go down rabbit holes. One of the great pleasures of this dumb future we live in is that you can dig through the milk crates of our culture forever, following whatever interests you, and there is no bottom. My latest rabbit hole looked something like this:

Wait, hold on a minute. This kind of sounds exactly like the weird idealized programming environment I keep in my head. Obviously with serious restrictions and caveats, but, as a learning tool? They implemented this on an Apple II in 1984? How far exactly did they go?

Read more...

Honeylisp is a programming environment for the Apple II which attempts to leverage the absurd computational horsepower of a modern laptop to make programming a 1mhz 8-bit processor more magical.

Step 1: Write a 6502 assembler

The Honeylisp assembler is written in Fennel, which is a Lisp dialect that compiles to Lua. My goal in writing it was that the input to the assembler would be simple lists, making Fennel into a super-powerful macro assembler basically for free. 6502 assembly is treated as simple data, a kind of embedded DSL that can trivially be generated by simple Fennel functions.

Step 2: Integrate into a text editor

The Honeylisp environment runs on top of lite, a very small and extensible programmer's text editor written in Lua. It's trivial to add new commands that trigger whatever kind of interactive build process I want. It's also straightforward to extend to create custom UI; I put together the bones of a simple tile editor in an evening. I ended up porting lite to the love2d runtime to allow me more flexibility in how I could build these custom editors.

It is also a fairly straightforward matter to add hot code reloading, though it requires a bit of forethought to take full advantage of it. When I put together the tile editor, I could make changes to it, hit Alt-R, and those changes were immediately live. It's cool to be able to use your editor to live-edit your editor.

Step 3: Integrate into an emulator

Back in July, I came across this video of Dagen Brock talking about an Apple IIgs emulator project he was working on that had an interesting property. Instead of integrating a debugger into the emulator, he decided he would implement a socket-based debugging interface that anyone could write a front-end for. Any external program could use this to gain full control of the emulated computer – read and write memory, set registers, set breakpoints, anything.

It turns out this idea kind of languished, and he never quite got to polishing up and releasing his debugger. But this emulator, GSPlus, exists, and it gave me the core idea at the heart of Honeylisp – if I can arbitrarily read and write memory, and I have the ability to augment my assembler, then I can do anything. I can take snapshots and jump back and forth between them, recovering from hard crashes. I can push code updates while the game is running; I just need to be able to specify an appropriate “merge point”. I can map variables from one version of the program to another, so even if data moves around in memory, I can compensate. I can put breakpoints on memory I know should be read-only, to catch wild pointer accesses. I can implement a REPL entirely on the PC side; assemble tiny snippets of code and trigger them. I can integrate with my tools, so making changes to a graphical tile in my editor instantly updates the screen of my emulated Apple II. The possibilities are truly vast.

Step 4: Integrate with hardware

One of the pretty unique features of the Apple II is that it comes with a built-in machine language monitor, complete with a mini assembler and disassembler. It turns out that this can trivially be controlled over a serial port to do things like... write arbitrary memory. So I can plug my laptop directly into a stock Apple II, type IN #1, and then run a single command to bootstrap my environment. In fact writing arbitrary memory in response to serial port commands is not very complicated, and many of the magical abilities I imagine for my environment can be done directly on real hardware.

Step 5: Write a game

There is no point in investing all of this effort into software development tools if you don't plan to develop any software. I intend to port my MS-DOS game Neut Tower to the Apple II using this system. Neut Tower was written using a hand-rolled Forth system; I've built a simple Forth-like stack VM to allow for non-performance-critical code to be written compactly. Because my assembler is so easily extensible, I can also easily use simple Fennel code to generate VM bytecode, which means I can do without a huge amount of the overhead of a full Forth as all of the compiler-y bits live on my laptop. This VM can also be the basis of my interactive REPL, which will be much nicer to write than little assembly snippets.

Step 6: Tell people about it

It's fairly early stages for all of these steps – I have a good start in all of them, but there is still lots of work to be done. I'm really excited about this project, though, and I want to talk about it! Hopefully this blog will be a useful place to do that. I'm looking forward to continuing to share my progress.

#lisp #honeylisp #retrocomputing #apple2 #neuttower

So I should probably have a blog post that I can point to about this whole retrocomputing project that I've been up to the past year and a half.

I wrote a game on an MS-DOS 286 PC, using only tools I built myself or tools that were available during the era where they were still selling 286 PCs. It's called Neut Tower and you can play it on your MS-DOS PC, in DOSBox, or in your browser. As part of this project, I implemented a Forth system, and built most of my game and its tools using it.

My motivation at the start of the project was this: I was enjoying using my 286. I liked the single-tasking workflow; there were no distractions. I was downloading games and apps and it was fun! So I figured I'd take the next step and write a little game or something.

Read more...

I've been seriously writing Forth, with my homebrew Forth dialect, for about a year now, off and on, and I've noticed something interesting with how things end up structured.

Forth and Lisp are often spoken of as though they are similar in some deep way. In Lisp circles, you often hear “code is data.” This is generally held to mean “Lisp has macros”, more or less – a Lisp program's source code is a syntax tree made of Lisp lists, that your Lisp program can introspect into and transform into new syntax trees and execute. Your program is literally a data structure.

My Forth code has very few things I would refer to as “data structures”. There is no significant language for defining them – I write one-off words that do pointer arithmetic. I only have a handful, so I haven't felt the need to generalize. It does zero transformation of them – they have been carefully chosen to be directly useful for everything the program needs them for, in-place, as-is.

Instead, the common pattern is that everything is code, which, thanks to Forth's flexible non-syntax, can be made to look a lot like data. Often data is compiled directly into the code that uses it – instead of naming a constant that's passed into a function to do a particular thing, you name a function that takes no arguments that just does the thing. (There are lots of flexible ways to make this sort of thing easy and inexpensive in Forth.) Forth is hyper-imperative to a degree that, as modern programmers, we've largely forgotten is even possible. Even, say, the number 4 is arguably a word executed for its side effects (push the value 4 onto the current stack). Of course, this is how CPUs work, too – you don't have a concept of “4” on its own in assembly, you have the concept of moving “4” into a register, or into memory. The only thing you can tell a CPU is to do things. Forth is the same.

Read more...

Forth is perhaps the tiniest possible useful interactive programming language. It is tiny along a number of dimensions:

  • The amount of code required to implement it
  • The size of the code that is generated
  • The amount of memory used
  • The number of features it considers necessary for useful work

It is a language that makes complexity painful, but which reveals that a surprising amount can be accomplished without introducing any. Forth is the opposite of “bloat”. If you've ever been like “Oh my God this Electron-based chat app is taking up 10% of my CPU at idle, what the HELL is it DOING, modern computing has gone MAD”, Forth is there to tell you that computing went mad decades ago, and that programs could be doing SO MUCH MORE with SO MUCH LESS.

WHAT DO YOU MEAN, “FORTH”

There is an expression about Forth: “If you've seen one Forth, you've seen one Forth.” Forth isn't a strictly-defined language, though there is a standardized dialect; it's more a set of ideas that tend to work well together.

In the past month, I wrote a tiny Forth system on a 286 running MS-DOS using Turbo C++ 1.01. It is my first time using Forth in anger, though I read a lot about it 15 years ago. When I refer to my Forth, I am referring to a system literally thrown together in two weeks, written by someone who does not really know Forth that well. It is slow and wildly nonstandard and it doesn't do very much, but I have enjoyed the process of writing it very much. If you are a grizzled old Forth grognard, please let me know if I have misrepresented anything.

Read more...