3 You can run Jonasforth inside QEMU or on real hardware. If you want to run
4 inside QEMU, you should have the following dependencies installed (assuming
7 $ pacman -S fasm qemu edk2-ovmf
9 Then, to run a UEFI shell inside QEMU, run:
13 JONASFORTH will be available as `main` on `FS0:`. Thus, to run it, you can run
14 the following command inside the UEFI shell:
18 : SAY-HELLO S" Hello, World!" TELL NEWLINE ;
22 (Try typing in the code in `lib/example.f` for something a little more
25 ## Running on real hardware
27 Making the program run on physical hardware is pretty easy. Just create a
28 FAT32-formatted USB drive, and copy `out/main` to it. Then, you can execute the
29 program in the same way that you did from inside QEMU, assuming your system
30 comes with a UEFI shell built-in.
32 If your system doesn't have a UEFI shell, then you can copy the executable to
33 `\EFI\BOOT\BOOTx64.EFI` on the USB drive. Then, the system should be able to
34 boot from the USB drive and directly into JONASFORTH. The way to do this is a
35 little bit different depending on the exact firmware, but most firmwares will
36 have some way to enter a boot menu where you can select the USB drive. You may
37 need to disable Secure Boot first.
39 To format a USB drive as FAT32, you can run
41 # mkfs.vfat -F32 /dev/sdx
43 with `/dev/sdx` replaced by the path of your USB drive. Then mount the drive,
44 and copy `out/main` to `\EFI\BOOT\BOOTx64.EFI`:
48 $ mkdir -p mnt/EFI/BOOT
50 $ cp out/main mnt/EFI/BOOT/BOOTx64.EFI
52 Now you should be able to boot directly from the USB drive.
54 # Notes on implementation
56 The implementation is based on
57 [JONESFORTH](https://raw.githubusercontent.com/nornagon/jonesforth/master/jonesforth.S).
58 This is my summary of the most important parts.
62 In Forth, words are stored in a dictionary. The dictionary is a linked list
63 whose entries look like this:
65 +------------------------+--------+---------- - - - - +----------- - - - -
66 | LINK POINTER | LENGTH/| NAME | DEFINITION
68 +--- (4 bytes) ----------+- byte -+- n bytes - - - - +----------- - - - -
70 For example, DOUBLE and QUADRUPLE may be stored like this:
72 pointer to previous word
75 +--|------+---+---+---+---+---+---+---+---+------------- - - - -
76 | LINK | 6 | D | O | U | B | L | E | 0 | (definition ...)
77 +---------+---+---+---+---+---+---+---+---+------------- - - - -
80 +--|------+---+---+---+---+---+---+---+---+---+---+---+---+------------- - - - -
81 | LINK | 9 | Q | U | A | D | R | U | P | L | E | 0 | 0 | (definition ...)
82 +---------+---+---+---+---+---+---+---+---+---+---+---+---+------------- - - - -
88 The Forth variable LATEST contains a pointer to the most recently defined word.
92 In a typical Forth interpreter, code is stored in an unusual format that makes it
93 easy to implement an interpreter, and which was also motivated by space
94 constraints on early systems.
96 The definition of a word is stored as a sequence of memory adresses of each of
97 the words making up that definition. (At the end of a compiled definition, there
98 is also some extra code that causes execution to continue in the correct way.)
100 We use a register (RSI) to store a reference to the next index of the
101 word (inside a definition) that we are executing. Then, in order to execute a
102 word, we just jump to whatever address is pointed to by RSI. The code for
103 updating RSI and continuing execution is stored at the end of each subroutine.
105 Of course, this approach only works if each of the words that we are executing
106 is defined in assembly, but we also want to be able to execute Forth words!
108 We get around this problem by adding a "codeword" to the beginning of any
109 compiled subroutine. This codeword is a pointer to the intrepreter to run the
110 given function. In order to run such functions, we actually need two jumps when
111 executing: In order to execute a word, we jump to the address at the location
112 pointed to by the address in RSI.
116 What does the codeword of a Forth word contain? It needs to save the old value
117 of RSI (so that we can resume execution of whatever outer definition we are
118 executing at the time) and set the new version of RSI to point to the first word
119 in the inner definition.
121 The stack where the values of RSI are stored is called the "return stack". We
122 will use RBP for the return stack.
124 Whenever we finish executing a Forth word, we will need to continue execution
125 somewhere else. When the word being executed is itself written in Forth, we need
126 to pop the old value of RSI that we saved at the beginning of the definition
129 Thus, the actual data for a word in a dictionary will look something like this:
131 pointer to previous word
134 +--|------+---+---+---+---+---+---+---+---+------------+------------+------------+------------+
135 | LINK | 6 | D | O | U | B | L | E | 0 | DOCOL | DUP | + | EXIT |
136 +---------+---+---+---+---+---+---+---+---+------------+--|---------+------------+------------+
139 LINK in next word points to codeword of DUP
141 Here, DOCOL (the codeword) is address of the simple interpreter described above,
142 while EXIT a word (implemented in assembly) that takes care of popping RSI and
143 continuing execution. Note that DOCOL, DUP, + and EXIT are all stored as
144 addresses which point to codewords.
148 Literals are handled in a special way. There is a word in Forth, called LIT,
149 implemented in assembly. When executed, this word looks at the next Forth
150 instruction (i.e. the value of RSI), and places that on the stack as a literal,
151 and then manipulates RSI to skip over the literal value.
153 When compiling a word, we need to handle the input specially such that literals
154 are compiled as `LIT x`.
156 ## Built-in variables
158 - **STATE** -- Is the interpreter executing code (0) or compiling a word (non-zero)?
159 - **LATEST** -- Points to the latest (most recently defined) word in the dictionary.
160 - **HERE** -- Points to the next free byte of memory. When compiling, compiled words go here.
161 - **S0** -- Stores the address of the top of the parameter stack.
162 - **BASE** -- The current base (radix) for printing and reading numbers.
166 `WORD` reads a word from standard input and pushes a string (in the form of an
167 address followed by the length of the string) to the stack. The buffer is only
168 valid until the next call to `WORD`.
170 `FIND` takes a word (as parsed by `WORD`) and looks it up in the dictionary. It
171 returns the address of the dictionary header of that word if it is found.
172 Otherwise, it returns 0.
174 `>CFA` turns a dictionary pointer into a codeword pointer. This is used when
179 The Forth word `INTERPRET` runs in a loop, reading in words (with `WORD`),
180 looking them up (with `FIND`), turning them into codeword pointers (with `>CFA`)
181 and then deciding what to do with them.
183 In immediate mode (when `STATE` is zero), the word is simply executed
186 In compilation mode, `INTERPRET` appends the codeword pointer to user memory
187 (which is at HERE). However, if a word has the immediate flag set, then it is run
188 immediately, even in compile mode.
190 ### Definition of `:` and `;`
192 The word `:` starts by reading in the new word. Then it creates a new entry for
193 that word in the dictoinary, updating the contents of `LATEST`, to which it
194 appends the word `DOCOL`. Then, it switches to compile mode.
196 The word `;` simply appends `EXIT` to the currently compiling definition and
197 then switches back to immediate mode.
199 These words rely on `,` to append words to the currently compiling definition.
200 This word simply appends some literal value to `HERE` and moves the `HERE`
205 Jonasforth runs without an underlying operating system, instead using the
206 facilities provided by UEFI by running as a UEFI application. This section
207 contains some notes about how this functionality is implemented.
209 I also wrote an entire tutorial that descirbes how to write and compile a
210 "Hello, World!" UEFI application, including how to run it on real hardware,
211 which you can find here: [Getting started with bare-metal
212 assembly](https://johv.dk/blog/bare-metal-assembly-tutorial.html).
214 ## Packaging and testing the image
216 UEFI expects a UEFI application to be stored in a FAT32 file system on a
217 GPT-partitioned disk.
219 Luckily, QEMU has a convenient way of making a subdirectory availabe as a
220 FAT-formatted disk (see [the relevant section in the QEMU User
221 Documentation](https://qemu.weilnetz.de/doc/qemu-doc.html#disk_005fimages_005ffat_005fimages)
222 for more information):
224 $ qemu-sytem-x86_64 ... -hda fat:/some/directory
226 We use this to easily test the image in QEMU; see the Makefile for more
227 information, or just run the `qemu` target to run the program inside of QEMU
228 (of course, you must have QEMU installed for this to work):
232 ## Interfacing with UEFI
234 From [OSDev Wiki](https://wiki.osdev.org/UEFI#How_to_use_UEFI):
236 > Traditional operating systems like Windows and Linux have an existing software
237 > architecture and a large code base to perform system configuration and device
238 > discovery. With their sophisticated layers of abstraction they don't directly
239 > benefit from UEFI. As a result, their UEFI bootloaders do little but prepare
240 > the environment for them to run.
242 > An independent developer may find more value in using UEFI to write
243 > feature-full UEFI applications, rather than viewing UEFI as a temporary
244 > start-up environment to be jettisoned during the boot process. Unlike legacy
245 > bootloaders, which typically interact with BIOS only enough to bring up the OS,
246 > a UEFI application can implement sophisticated behavior with the help of UEFI.
247 > In other words, an independent developer shouldn't be in a rush to leave
250 For `JONASFORTH`, I have decided to run as a UEFI application, taking advantage
251 of UEFI's features, including its text I/O features and general graphical device
252 drivers. Eventually, we would like to add some basic graphical drawing
253 capabilities to `JONASFORTH`, and it's my impression that this would be possible
254 using what is provided to us by UEFI.
256 A UEFI images is basically a windows EXE without symbol tables. There are three
257 types of UEFI images; we use the EFI application, which has subsystem `10`. It
258 is an x68-64 image, which has value `0x8664`.
260 UEFI applications use [Microsoft's 64-bit calling
261 convention](https://en.wikipedia.org/wiki/X86_calling_conventions#Microsoft_x64_calling_convention)
262 for x68-64 functions. See the linked article for a full description. Here is
265 - Integer or pointer arguments are given in RCX, RDX, R8 and R9.
266 - Additional arguments are pushed onto the stack from right to left.
267 - Integer or pointer values are returned in RAX.
268 - An integer-sized struct is passed directly; non-integer-sized structs are passed as pointers.
269 - The caller must allocate 32 bytes of "shadow space" on the stack immediately
270 before calling the function, regardless of the number of parameters used, and
271 the caller is responsible for popping the stack afterwards.
272 - The following registers are volatile (caller-saved): RAX, RCX, RDX, R8, R9, R10, R11
273 - The following registers are nonvolatile (callee-saved): RBX, RBP, RDI, RSI, RSP, R12, R13, R14, R15
275 When the application is loaded, RCX contains a firmware allocated `EFI_HANDLE`
276 for the UEFI image, RDX contains a `EFI_SYSTEM_TABLE*` pointer to the EFI system
277 table and RSP contains the return address. For more infromation about how a UEFI
278 application is entered, see "4 - EFI System Table" in [the latest UEFI
279 specification as of March 2020 (PDF)](https://uefi.org/sites/default/files/resources/UEFI_Spec_2_8_A_Feb14.pdf).
283 - [UEFI applications in detail - OSDev Wiki](https://wiki.osdev.org/UEFI#UEFI_applications_in_detail)
284 - [Microsoft x64 calling convention](https://en.wikipedia.org/wiki/X86_calling_conventions#Microsoft_x64_calling_convention)
285 - [UEFI Specifications](https://uefi.org/specifications)
289 We might want to consider using something like this: https://wiki.osdev.org/Uefi.inc)
291 FASM can generate UEFI application binaries by default. Use the following
292 template to output a 64-bit UEFI application:
297 section '.text' code executable readable
303 section '.data' data readable writable
307 Use `objdump -x` to inspect the assembled application binary.
309 ### UEFI documentation
311 - [Latest specification as of March 2020 (PDF)](https://uefi.org/sites/default/files/resources/UEFI_Spec_2_8_A_Feb14.pdf)
316 - 4\. EFI System Table (89)
317 - 7\. Services - Boot Services (140)
318 - 8\. Services - Runtime Services (228)
319 - 12\. Protocols - Console Support (429)
320 - 13\. Protocols - Media Access (493)
321 - Appendix B - Console (2201)
322 - Appendix D - Status Codes (2211)
326 - [UEFI - OSDev Wiki](https://wiki.osdev.org/UEFI)
327 - [Unified Extensible Firmware Interface (Wikipedia)](https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface)
328 - [UEFI Specifications](https://uefi.org/specifications)