Alternate disk I/o behavior for picky hosts

[user340q]

When fighting with the notoriously picky Korg Triton I found inconsistent behavior between the 415 and 435 gotek, where the 415 would only work with v3 but the 435 could work with v3 or v4.6a. I was still encountering errors saving files with the 415 unit. Please disregard this inaccurate intro because it was really intended for a separate discussion in the context of whether a Triton would load a disk image at all and thus was a poor choice of intro to this issue.

The reason for making this discussion is that the Triton is fully working with stock firmware gotek and also not working with HXC

At the present time this is the status of Korg Triton compatibility with gotek models

• Common issues on all firmwares are that the initial save to a blank disk image works if the file fits on one disk image, but any further attempts to save will fail, the current workaround is to use one disk image per saved file
• Saves which require to be split across multiple disk images fail to save at the beginning of the save, common issue between FF/HXC, only Gotek can do this correctly
• v3.x firmware can load from any disk image and format any disk image without any issue on the 415's or 435's
• v4.x firmware can load from any disk image and format but not save whatsoever on the 435's, and is not working on the 415 at all for some reason

Because my ability to save from the Triton remained inconsistent between different models of USB drives, I built the v3 firmware with the btw value hard coded to 512 bytes, as that matched the sector size of the USB drive I was using. With this firmware I was able to get consistent successful saves to my USB drives with the 415 based gotek. I only have Sandisk USB 2.0 16GB which are pretty reliable for Goteks and a few miscellaneous USB 3.0 sticks which are really slow to write to test with.

I have not tested this build on any other host device and I also do not guarantee data loss is not a possibility, mainly because I am still attempting to resolve a separate issue on the Triton where saving to anything but a blank IBM formatted 1.44MB image will always fail.
Please consult this comment for better context https://github.com/keirf/flashfloppy/discussions#discussioncomment-3668803

[keirf]

Does this similarly improve behaviour on the 435 unit?

[user340q]

Yes..... when it isn't corrupting disk images 🥲
With the 435 unit it surpasses the standard I/O speed when formatting a disk image and stays very consistent the whole time. With the standard behavior it is prone to lag here and there. The big issue is that it doesn't seem to work on formats that write a little extra data after the format, because the host device will throw an error at that point, and disk images already containing information are definitely getting corrupted.

So for example, on the Triton which wipes the disk image blank and then checks it many times, this behavior is working fine and the files produced by both firmwares are identical. On other host devices, or even just with disk images which are not totally blank, this seems to mess things up.

I would need to try more options for the btw value and make sure it's the best way to control how data is written to the USB stick.

[keirf]

Okay, so the btw value should not really be modified. That is the actual number of bytes being supplied by the caller, to be written to a file. If you shrink btw, data will be dropped. If you increase btw, extra garbage will be written.

Attach a patch here of exactly what change you made, if you like, or push your updates to a fork of flashfloppy onto github, and post a link here.

My guess is that forcing btw to 512 bytes means you are dropping data. But the writes complete faster so the write process appears to proceed okay. Until the time you come to actually read and use that data!

I would also concentrate on 435. It has to be no harder than 415 to get working. The lessons learned on 435 may apply to 415. Or if not, it doesn't really matter if you can't sell your existing 415 stock for this one specific host type.

[user340q]

For clarification, the issue I am referring to which is present on both the Triton and A4000 is: you can load or format without issue. If you save to an formatted disk image, the save will succeed. Attempting to save another file to the disk image will fail, without hesitation, stating there is not enough space on the disk. The workaround is to switch to a new blank image, but this isn't needed on a real disk or factory firmware Gotek.

[keirf]

The explanation is probably that stock and HxC firmwares buffer a track in RAM, and do not write-back from that buffer until track is changed (eg. side change, or possibly even not until heads are stepped). Some hosts are sensitive to the end-of-write to start-of-read latency. FlashFloppy v3.x has worse latency than other firmwares here because it eagerly and synchronously writes back to USB.

v4.x is supposed to fix this by buffering tracks. The async I/O is the most complex part of v4.x but almost an incidental in this scenario. But v4.x was designed with at least 64k RAM in mind. The 32k models may not work great. Especially at HD data rates.

I don't think you ever mentioned how well 435 works with v4.x firmware? Is that working okay? Because, if so, I would say case closed. I don't have much interest in tweaking FlashFloppy for 32k Goteks for those few problematic hosts -- just use the new 435 model.

[keirf]

I maybe also could implement a delayed-writeback for 435 models for v3.x firmware. It gets a little bit tricky to do timeout-based writeback (eg. write back after 2 seconds, even if track hasn't changed) to avoid losing data on power-off. This is because writes are synchronous in v3.x and an unexpected long write on timeout could interrupt the ongoing read stream. Either:
(a) no timeout based writeback
(b) put up with possibly interrupted read stream, on assumption disk is probably not being accessed if no track change after a few seconds
(c) implement some async I/O or move more read path to be interrupt driven (neither is very attractive in v3.x branch really)

Of course, if v4.x is working anyway okay for 435 model, then the effort of any of the above perhaps doesn't make sense for v3.x. Unless part of a larger effort to make a new v5.x taking the best of v3.x and v4.x and combining them, and specifically with 435 in mind.

[user340q]

Thank you for the detailed info!

"I don't think you ever mentioned how well 435 works with v4.x firmware? Is that working okay?"
I apologize for not making this adequately clear earlier. 4.x can load but not save or format at all on 435. 3.35 can load, save and format with the caveat that one must use a blank image file each time on 415 or 435. This is the only host device where 4.x doesn't save fine or quicker than v3 that I have tried.

For example on the Triton and A4000 both I am wondering if buffering the very first track is the key to successful saving, or if it should be optional to respond to all step commands etc. I say this for two reasons

1. each time before saving it will go check T:00.0 and T:00.1 blazingly fast and then go back to T03.0/T03.1 to start saving (if the image is blank). As soon as the "W" [Write] indictor appears, the host unit gives up and write fails. This happens on 4.x on any image file and it happens on 3.x if I have already saved a file and try to save a second file..
2. when you are in the save menu, every single button press on the host unit sends some very quick pulse to the floppy emulator because the green activity led will flicker only for a brief instant, not even long enough to illuminate the LED fully. I should probably add the speaker mod to my 435 to see if it is the step command.

I do feel I need to use the oscilloscope to find out what is going on at a deeper level unless this gives you enough of an idea of what could be the matter? I think maybe the best thing for me to do at this point is get the scope out and see what signal is actually being sent at this instance.

Again thank you for the help!!!

[keirf]

Maybe you do. It may not be the issue I expected, if 4.x doesn't help.