Export to .img for embedded computers

[faddat]

Hi, I just used buildx to replace a copy-binfmt system found at https://github.com/pikvm/os.

https://github.com/faddat/os/tree/real-starport/starport

And at this point, all I'm really doing is building a wrapper around buildx.

It may make sense to investigate a direct "export to .img" feature that initially just targets arm64 pi 3 and 4 and branches outward from there into more devices-- I reckon this would be great for ecosystem growth.

Needs more info

Is it a raw image format?

Yes, I think so. Here is how we are creating the image now:

https://github.com/faddat/spos/blob/76f8d720e0b67412814f7f9f7e9590b5ee30eb48/build.sh#L60-L108

What's the file system?

200 MB boot partition (FAT)
Rest of image is / (EXT4)

parted --script /dev/loop0 mklabel msdos
parted --script /dev/loop0 mkpart primary fat32 0% 200M
parted --script /dev/loop0 mkpart primary ext4 200M 100%

Does it have a partition table? If so, is it GPT? Or MBR?

I think mklabel msdos means MBR, right? Not 100% sure.

Does it contain kernel? What about boot loader?

bootloader
kernel: Arch linux arm's aarch64 build pretty much runs mainline.

I copy arch linux arm into the image like:

https://github.com/faddat/spos/blob/76f8d720e0b67412814f7f9f7e9590b5ee30eb48/Dockerfile#L12-L13

Does that help? Making these images is pretty tough now, but I reckon some automation could make it very easy and expand what can be done with device/application specific images.

[AkihiroSuda]

Needs more info

• Is it a raw image format?
• What's the file system?
• Does it have a partition table? If so, is it GPT? Or MBR?
• Does it contain kernel? What about boot loader?

[faddat]

More info you shall have! Put it above for easy reference.

Also, here is another account of some folks pursuing the same thing:

https://www.boulderes.com/resource-library/building-raspberry-pi-disk-images-with-docker-a-case-study-in-software-automation

[tonistiigi]

This should be moved to Buildkit repo.

We are open to adding more exporters if they don't add significant maintenance cost and the contributor is willing to fix the bugs later.

Another possibility is to define API so that container images can be used as exporters so they can define a custom behavior, similar to frontends support. Some of it you can already do today, eg you can create img file inside your build and export with a local exporter. But with a custom exporter, this could be done more efficiently.

[faddat]

If you want a fairly clear and detailed look at how I'm doing it currently, here is a link to my build script. Please feel free to move the issue, I will of course follow it.

https://github.com/faddat/sos/blob/no-kernel/build.sh

Because I do want this to be a binary free affair, I will be looking into what the Docker - exporter binary in the toolbox folder does. Other than that, it is a somewhat straightforward although complex process of:

• Making an image file
• Making the image file a loop mount
• Partitioning and formatting the loop mounted image file
• Mounting the partitions in the image file
• Rsync /boot
• Rsync /
• Unmount
• Disconnect the loop mount

And after that you have a bootable raspberry pi image. It's pretty cool. Not only is it pretty cool, it's actually a great deal faster than the current standard for building these images.

Recently, I decomposed this process into a couple of images

docker.io/faddat/sos-base

• All this does is add a root file system for Arch Linux Arm

docker.io/faddat/sos

• This adds device-specific stuff so for example with the raspberry Pi, it adds a raspberry pie optimized kernel. It also adds some basic stuff missing from the root file system. It can be considered an OS distribution.

docker.io/faddat/starport

• This is actually The end destination, an image containing complete peer-to-peer development tooling for people building application specific blockchains.

All right so that is my roundup. I broke it up so that it could be faster and also so that I can build images for many devices in SOS and then also build images for many devices in Starport.

[faddat]

Moving to buildkit

[faddat]

pikvm/pi-builder#5