Feasibility of running tilck on i386DX for Micropython

[peterzieba]

Hi,

I'm interested in running Micropython on bare-metal i386DX and it sounded like Tilck might be a way of doing that, but I wanted to ask a little bit about feasibility. My goal would be to implement the bare necessities (so, probably an 8259 PIC, say 8MB SRAM, a serial port at the usual 0x3f8, an 82C54 PIT, and some ROM for the kernel).

Provided I could get such hardware to work, does something like this sound within the realm of feasibility? Would it be easier to try and get Micropython to run directly on a 386 without Tilck if my initial goals are limited to little more than a repl on a serial console?

I'm very interested in embedded x86 and it looks like there are ports of Micropython to just about anything but not quite something like this. I believe the hardware is within my ability to create and get running but I doubt my ability to handle the software components necessary to pull this off.

Obviously getting the Tilck kernel working would open up many more possibilities than Micropython alone but want to set a goal that is at least achievable in the initial as quite a lot of things have to be functional for anything to work at all.

Because there are projects like tinytapeout and Pistorm (for 68k) I have a feeling that 32-bit x86 may find a way to continue to exist in some way shape or form (even if it is a RPi emulating one over its GPIO pins) and would love to see x86-32 get the same level of enthusiasm as other processors get.

Any thoughts would be greatly appreciated.

Cheers!

[vvaltchev]

Hello @peterzieba,
this looks like an interesting idea. Tilck already has support for the 8259 PIC, actually that's the only interrupt controller it supports. Same thing for the PIT, it supports the 8254, and for the UART controller as well. It can run with 8 MB of RAM and even with less, depends on what you put in the initial ramdisk. Micropython already runs of Tilck as well. [ Note that Tilck has no storage support (yet), so all the apps need to stay in-memory all the time. ]

However, Tilck does not run on i386DX. It runs on i686 for sure and probably on i586 too, if all the BIOS requests by the bootloader can be serviced by the firmware. Beyond that, making Tilck to work on older CPUs will require removing many assumptions that made my life easier, both in the bootloader and in the kernel itself. That's going to require a lot of work and hardcore low-level skills on your side.

Starting from the legacy bootloader: I'm not sure if all the int 0x13 and int 0x10 traps will be serviced as expected by such an old machine. I really have no idea about what does such a machine support exactly. After that, the bootloader will enter in the stage3 which is written in C and runs in protected mode 32. From there, stage3 will do more BIOS calls by going back and forth to real mode, using a special sub-mode called "unreal" mode, which works only on some of the newer 32-bit machines. Probably not on i386DX. Also, there, the C code will use even more BIOS traps to gather information about the system's memory layout and available graphic modes. None of them will probably work on i386DX. See:

1. https://github.com/vvaltchev/tilck/blob/master/boot/legacy/boot.S
2. https://github.com/vvaltchev/tilck/blob/master/boot/legacy/stage3/rm_calls.S
3. https://github.com/vvaltchev/tilck/blob/master/boot/legacy/stage3/realmode_call.c
4. https://github.com/vvaltchev/tilck/blob/master/boot/legacy/stage3/mm.c
5. https://github.com/vvaltchev/tilck/blob/master/boot/legacy/stage3/vbe.c

After that, assuming that you fixed completely the bootloader by adding #ifdefs everywhere with major simplifications and by hardcoding stuff here and there, you'll end up in the kernel, here:

https://github.com/vvaltchev/tilck/blob/master/kernel/arch/i386/start.S

The first and probably the biggest problem is that I use the big 4MB pages (Page Size Extension, PSE), introduced in Pentium Pro. Without that feature, you'll need to allocate several page tables in order to do the proper mappings. The problem gets worse after the "early start" because more and more pages are needed for the page tables themselves. To get some memory, we need to map pages, but to map pages, we need page tables, which require memory. This problem is tricky to solve for larger amounts of memory. Fortunately, for smaller amounts of memory like 8 MB or 16 MB you just need 2 or 4 page tables, which can be statically allocated in the kernel's BSS section.

After that, you need to disable the INVLPG instruction at compile time because it cannot be used on i386DX as it doesn't exist. Also CPUID doesn't exist on i386DX as it has been introduced with 486. Not so sure about the framebuffer. It should work even without PAT being supported, but you would still need to emulate CPUID and hardcode the feature booleans somehow, for the rest of the kernel. Currently Tilck assumes CPUID to exist.

Also, the whole project (both kernel and userspace) is compiled with a i386 toolchain with -march=i686. You'd need to carefully recompile everything using -march=i386 and introduce an option in the CMake build and the toolchain to make that happen. You'd also need to disable the sysenter support and make all the tests using sysenter to forcibly use int 0x80 on that architecture, because sysenter is a new thing and doesn't exist.

Also, you need to make sure that libmusl doesn't use sysenter directly. I'm not even sure that libmusl supports the i386 target at all, given that not even Linux supports that nowadays. That means that even if you make Tilck to boot and run on bare-metal i386, you'll have troubles making the libc used by Tilck to work on that. Therefore, you'd need to either compile the userspace apps in a different way, by using a custom libc or using a custom-patched libmusl, which is probably the easier of the two.

Even after you'd done all of that, you'd need to build Micropython with your custom-patched libmusl or other libc and make it all work. And, Micropython's build isn't one of the easiest to work with, unfortunately. You’ll also need to make Micropython to use soft math as the i386dx does not have an FPU.

Running Micropython on bare-metal without Tilck will be even harder because you'll need to implement all the rest of the things that Tilck already does for you, so that's at another level entirely. Tilck seems like the easier of the two. To be completely fair, probably it will be even easier to pick up an older version of Linux, from the period where i386 was still supported and work with that directly but.. what's the fun in doing that? :D

In summary, it's possible to achieve that with Tilck, but it would require a significant amount of work, like several months full-time, plus a very good technical background to begin with (i386 assembly, mode switching, real mode, paging, virtual mem, etc.). Without a solid technical background, it will take much longer. I have no idea about your technical background, but that's what mostly will impact the time and effort to achieve something like this.

Maybe take a look at the assembly files I've mentioned above and see how they look like. Does everything or most of it make sense quickly or it looks insanely hard? Or, given the start.S file in the kernel, how hard does it feels like to replace all the 4-MB mappings using 2-levels of paging and regular 4 KB pages ?

Cheers,
Vlad

[peterzieba]

Unreal mode is actually supported by a 80286 with LOADALL, but with the 80386 and above it sounds like it can be entered through protected mode. I was interested in this myself as it looks like it can simplify a lot of things.

I was initially hoping to get Micropython on metal to work without need for a bios as there would not be a need for complex hardware initialization (no DRAM controllers, no chipset, etc). I understand now that you're relying on bios routines however, so that makes things a little more difficult in that a BIOS has to be present.

The reason I'm centering around the 386 is because it represents a reasonably capable embedded x86 processor but still simple enough to be achievable from a hardware perspective -- there are only so many pins and the bus can downshift to 16-bit if need be without any other compromises (other than slower access of course). This could in theory lend itself to a cheap 2-layer PCB and have the rest of what is needed be handled by an FPGA or chips for various needs (16C550 for serial, etc.) resulting in a tiny little USB programmable embedded x86 board.

The Pentium on the other hand is rather difficult from a hardware perspective. It has quite a lot of pins, two power rails, and a 64-bit data bus that always has to be filled. Once you head down this path far enough it quickly spirals into a very serious project. While not impossible, it is probably more trouble than it is worth and using a full off-the-shelf machine or some x86 SBC might be a better route, sort of defeating the whole point.

From a hardware perspective, what could be possible is a 486DX, and in fact, there is one reason this could make things easier: The 486 will happily allow for an 8-bit, 16-bit, or 32-bit bus width, something not possible with any other x86 processor.

My goal is to make embedded x86 be something that is approachable by creating a low-cost board and I think I can pull it off, but the software is what concerns me. Without an application like Micropython or Tilck/Busybox on a platform, the ability to do interesting things with it drops off quickly.

Initially I was hoping to toss the processor into unreal mode and just have Micropython be the scope as it sounded like I'd just need a stub of code for sending console I/O to the serial port along with a timer and not having any BIOS of any sort. I have seen quite a few ports to other significantly less capable CPUs, so I assumed this wouldn't be too much trouble. I guess now I'm reconsidering simply trying to get SeaBIOS to work as the first thing I will attempt to run. It sounds like SeaBIOS will happily use a serial port as a console, and with the SGABIOS option ROM, it will operate without a dedicated video card by forwarding most of the VGA BIOS and keyboard events to the serial port. In theory SeaBIOS sounds like it can be targeted for a 386.

Recreating a PC is not my intention, but rather keeping the bare minimum of what would be necessary in an embedded system (serial port, PIT, PIC) and keeping these at same PC I/O locations so as not to create serious incompatibilities in the event that one would want to do more with such a board.

Provided unreal mode is accessible on a 486DX in the same manner as currently implemented in Tilck, SGABIOS handles int 10, and an FPU is present, would this sound any more feasible without it being a massive undertaking?

Peter

[vvaltchev]

Hello @peterzieba,
I'm so glad to see that you already have experience in the field. That changes things significantly. If you already have the necessary technical background, it won't be that hard and also, there are several shortcuts you can take.

So, you're trying to re-create a motherboard for the i386DX CPU using PCB + FPGA for some components? But where are you going to take the i386DX itself from? i386DXs aren't sold nowadays, so the only source are old CPUs, unless to plan to also re-create the whole CPU too :-) In any case, your idea looks super cool to me. I've always being sad about x86 being non-existent in the embedded world. Given today's amount of retro computing and fans of the Commodore 64, I don't see why we can't have something like that for x86 as well.

Back to the topic. Unreal mode: the need for unreal mode can be totally eliminated by just reading up to 64 KB of data using the BIOS I/O and by placing the destination in some lower address, accessible from real mode. That can be done and even easily tested using QEMU. So, while it will require some work, it's probably not the biggest limitation.

BIOS I/O: I'm using that because I made Tilck to work on PCs and VMs, so the kernel and the initial ramdisk had to be loaded from disk somehow by a bootloader. Most of the fancy calls like VBE and E820 can totally avoided, if you hard-code the expectations. However, if you want to completely avoid also int 0x10 and int 0x13 you should basically totally skip the bootloader. If do things the "embedded way" using some sort of ROM (EEPROM/FPGA/whatever), you don't need anything of the bootloader, so the whole conversation about "unreal mode" can be discarded. The Tilck kernel itself never goes to RM back and forth like the bootloader does.

Therefore, if I assume that you have figured out a way to get Tilck's kernel in memory along with its initial ramdisk, you just need to have all of that somewhere in the physical memory and correctly setup the MULTIBOOT 1.0 data structures for Tilck using some minimal bootstrap code, instead of a whole bootloader. Then, enter in PM32 and jump to the entry address (or just jump if you're in PM32 already). With that approach, we already skipped a significant part of the complexity.

From there, the next challenge would be to make Tilck to not use PSE because even i486 doesn't have it. Probably it still has value having an i486dx over an i386dx because of INVLPG, CMPXCHG, CPUID, plus the built-in FPU which will make your life much easier for the userspace apps.

Avoiding PSE will simply mean (hopefully) automatically detecting if PSE is present or not and re-writing part of start.S using 2-level paging and, to make things simpler, just statically alloc a few pages for the page tables and put a hard-coded cap to the max ram supported like 16 MB for example. It won't be that hard, if you're already familiar with x86 assembly, paging etc. After that, in kernel/arch/i386/paging.c you'd need to check the PSE support flag and always force big_pages_allowed to false in map_pages_int(). If necessary, you can also introduce a "CPU_i486" option in CMake and through a series of defines end up knowing exactly what to do everywhere. We can talk about this later.

Sysenter: The problem with sysenter and libmusl will remain if you are looking for a fully-integrated solution. A temporary solution for that is to:

• Check for the CPU_i486 option from the tests or check for that using CPUID and replace sysenter with int 0x80. Yeah, you could completely ignore the tests in theory, but it's better to have them working to make sure the kernel works as expected on the "new" arch.

• Compile init, devshell, busybox and Micropython yourself aside from Tilck and statically link them against another libc or a patched libmusl. Then, build the "fatpart" with a script of your own. That will spare you initially a lot of troubles to integrate with Tilck's build system and toolchain. It will be good enough for experiments. When you get there, we'll figure out the best way to put it all back into the main source tree.

Serial port: In addition to that, I have to admit that I never used Tilck's serial console on real HW because it's hard to do that with modern PCs and I don't have a PC that's old enough to have an actual classic serial port accessible using the ISA I/O ports. For the moderns ones requiring MMIO, the problem is figuring out its physical address and that's not easy, even with a decent ACPI support.

Therefore, the serial port support might require some debugging/fixes to work properly on real HW, but it's certainly doable and probably not too hard.

How does that plan sound now? :-)

Vlad