[DO NOT MERGE] Sim changes for forthcoming beta MicroPython release #113
Conversation
On the basis that's as close as we can get to the beta for now. Still has HAL issues. One local change to MicroPython was needed to include mphal.h in modspeech - need to review how this works for the non-sim build as it seems like it should be included.
|
Preview build will be at |
✅ Deploy Preview for distracted-dubinsky-fd8a42 ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Audio format needs some investigation - MediaRecorder might not be the right approach.
We'll still need to resample them and write them to the buffer.
Closes #101
Needs testing but likely closes #110
Edge cases need checking. Requires a higher AUDIO_CHUNK_SIZE which has other consequences but 4ms is too short.
src/board/audio/index.ts
Outdated
| // TODO: consider AudioWorklet - worth it? Browser support? | ||
| // consider alternative resampling approaches | ||
| // what sample rates are actually supported this way? |
There was a problem hiding this comment.
ScriptProcessorNode support vs AudioWorklet support so we'd lose early Safari 14.x and Safari 13 (for 13 I think limited support is OK).
See also the techniques used in this project.
| @@ -9,6 +9,8 @@ | |||
|
|
|||
| extern ringbuf_t stdin_ringbuf; | |||
|
|
|||
| #define mp_hal_ticks_cpu() (0) | |||
There was a problem hiding this comment.
Need to understand this / investigate new time API.
We need to do the stubs for that too.
There was a problem hiding this comment.
Some discussion on the stubs PR too
- Fix missing typescript types - Use Safari 13 compatible OfflineAudioContext constructor - this may prove pointless due to lack of < 44100 sample rate support, but is an easier starting point to try things out.
|
Having used this a little, I've noticed the microphone LED isn't coming on when using the computer mic. |
by stopping micStream tracks
Passing this.microphone.microphoneOn/Off functions as BoardAudio constructors in board/index.ts didn't work for some reasons. The microphone element was flagged as being undefined. So instead I passed the element directly into the BoardAudio constructor, which worked.
Deploying micropython-microbit-v2-simulator with
|
| Latest commit: |
ab07528
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://81027e5a.micropython-microbit-v2-simulator.pages.dev |
| Branch Preview URL: | https://beta-updates.micropython-microbit-v2-simulator.pages.dev |
Enable a larger audio buffer
- Use smaller buffer to avoid too much audio after wait=True - Don't reset nextStartTime to avoid overlapping audio - Add indirection to callback so we can clear it properly to avoid calling it when audio finishes after MicroPython has terminated
|
Re the Firefox regression the sample rate seems to have changed. Main, working: Branch, not working: This 4x reduction in sample rate is due to the removal of buffer expansion in 2c67ed68a15bb64949e7825eaba46acb57f680b2 which drops the sample rate out of Firefox's supported range (older Safari will have issues too as we saw previously for speech). I guess we need to look at resampling in the sim for multiple reasons:
|
Remove our workaround
|
Picking this up again. MicroPython changes since the last update for reference: |
Adjust build and samples for record/playback No HAL changes but the API has changed
We need to do this for recording as we can't specify a rate. We need to do this for playback for sample rates outside of the browser's supported range. That includes the current default rate for Firefox and older Safari. Uses a resampler build that doesn't support medium/best to save on bundle size. I've also increased the buffer size so I can play a 44k sample in an OK-ish way. Maybe 256 will be needed. We know Firefox on Windows still performs poorly. It did badly with audio before these changes. Another significant issue is that it can take 3 seconds to call getUserMedia on Desktop Safari.
See microbit-foundation/python-editor-v3#1155
Closes microbit-foundation/python-editor-v3#1082
Closes #110
Closes #101
Browser support issues
Firefox
Regression playing AudioFrame even at default rate
There's a 4x reduction in sample rate is due to the removal of buffer expansion in 2c67ed68a15bb64949e7825eaba46acb57f680b2 which drops the sample rate out of Firefox's supported range (older Safari will likely have issues too as we saw previously for speech).
Presumably user supplied rates can cause issues even if we fix this with interpolation, so we need to determine the range we must support and write resampling code, even if we can skip it on some browsers/at some rates.
Recording sample rate issue
Seems that Firefox just doesn't support using web audio to resample. We've also found you can't ask media recorder for arbitrary sample rates. So that leaves us with manually resampling, at least for Firefox. We could use something like https://github.com/rochars/wave-resampler (or at least the same techniques, not clear how maintained that is).
Safari
Perhaps we can accept some limitations here. Need to check whether the Firefox playback issue above applies here.
Misc quality issues
Work to do if/when implemented in MicroPython
4da3f33
microphone.sound_level_db()function. bbcmicrobit/micropython#803 (stubs too, what about the sim?) Stub WIP [DO NOT MERGE] Update for forthcoming beta MicroPython micropython-microbit-stubs#97 (comment)SoundEvent.CLAP. bbcmicrobit/micropython#793 (stubs, how do you trigger in the sim? current UX is loud/quiet oriented)