Let users access timestamp of reception in promiscuous mode and TX timestamp of sent packets (802.11mc FTM) (IDFGH-8374)

[dalibor-drgon]

Is your feature request related to a problem?

I am writing bachelor thesis and doing research with focus on realtime tracking technologies and there are some limitations in the provided API regarding the 802.11mc functionality.

To allow users write their own more complicated and resilient multilateration algorithms on top of existing
FTM functionality, it would be nice to have access to some lower level functionality such as obtaining the
timestamps for when a packet was received and when a packet was sent. Even though the expected error of FTM
timestamping is around 2 meters in 80 MHz mode which is far more than regular ultrawideband transcievers
such as DWM1000, it still would be useful to have a functionality that allows to extract these timestamps so
that they can be used not only in custom ToF models, but also potentionally more complex custom TDoA multilateration
algorithms with time synchronization. I would like to pinpoint that TDoA is the main thing that
I am focusing here on, because for larger spaces that have dozen and more meters on width, it should outperform
RSSI based localization while not requiring any additional hardware for the device that is being tracked
and while not requiring any additional software / models for RSSI correction and filtering.

The TDoA works in a way that anchors (static devices with known position) communicate between themselves by
regularly (~10Hz to ~100Hz) sending a beacon for which the TX timestamp of anchor #1, RX timestamp of anchor #2 (and their addresses)
are needed to calculate synchronization function that is used to convert from clocks of one anchor
to the clock of another one. All that is needed is then some data / beacon from the tracked device where
all anchors save the device's address, RX timestamp and their own address and send all of these measurements
to a server (or some other computer) where synchronization functions are calculated and differences of arrivals
are calculated using these functions and plugged into regular TDoA multilateration algorithm. In the
end you end with few linear regression models that can be solved to obtain relatively precise measurement,
given the error in timestamp calculation is few meters and that the distance between receiving anchors is
few times bigger than the error.

Describe the solution you'd like.

1. (required) Add FTM RX timestamp to wifi_pkt_rx_ctrl_t, and
2. (optional) either:

• a) somehow let the user see when a packet was sent (e.g. by an event, by returning from the esp_wifi_80211_tx()-style like function, etc...)
• b) let the user read the current timer value and choose when the packet should be sent in esp_wifi_80211_tx()-style like function

Describe alternatives you've considered.

No response

Additional context.

The values of these timestamps needs to be more accessible to the user so that precision of using various methods such as ToF and TDoA can be assessed and verified. ESP32 is currently one of the cheapest transceiver SoCs and it would be nice to have more control of the 802.11mc functionality.

[nachiketkukade]

@dalibor-drgon , in FTM example you can try enabling 'Example Configuration -> FTM Report logging' in the menuconfig to display per packet timestamps. Also for comparing with current time you can use API esp_system_get_time. Please note that timestamps in FTM Report are in picoseconds while the API gets the system time in microseconds. Hope this is helpful for you.

[dalibor-drgon]

@dalibor-drgon , in FTM example you can try enabling 'Example Configuration -> FTM Report logging' in the menuconfig to display per packet timestamps. Also for comparing with current time you can use API esp_system_get_time. Please note that timestamps in FTM Report are in picoseconds while the API gets the system time in microseconds. Hope this is helpful for you.

Sorry for not being enough clear. I need to know the exact value from just one of the counters, it's not possible to read one precise timestamp and try to adjust it to other counter with relatively very low rate (that would ruin precision). The point is these precise picosecond (or nanosecond) timestamps are needed both when a packet was received and when a packet was sent - from the same counter - as anything other is useless in terms of accuracy/precision at least. That is the point of TDoA, you need precision (in which case 80MHz bandwidth meant around 2-4 meters precision, but since current esp32s have 40mhz max, its around twice that amount). We are measuring electromagnetic waves here, and 1us * speed of light = 300m, so microsecond timestamping is not precise enough and is not actually useful as I need to know the value inside the WiFi counter that timestamps these packets and not something else and less precise. It's very likely the hardware supports timestamping both incoming and outcoming traffic, as these timestamps must come from inside the radio hardware given their precision.

[dalibor-drgon]

Just a quick note: It is possible to do synchronized TDoA just with the RX timestamping, the TX timestamping isn't that horribly needed and probably a lot harder to do. So if you are thinking about implementing this, the RX timestamping is probably the easier and more important thing to implement. At least in the promiscuous mode, all that is needed to do is add the RX timestamp from the WiFi hardware (in picoseconds or whatever are the units of choice) to the wifi_pkt_rx_ctrl_t struct (and of course modify the lib80211 library).

[nachiketkukade]

the RX timestamping is probably the easier and more important thing to implement. At least in the promiscuous mode, all that is needed to do is add the RX timestamp from the WiFi hardware

This is possible with newer chips since the Rx Timestamp already exists in wifi_pkt_rx_ctrl_t, we're reviewing if we can expose the timestamp variable after internal testing

[Jeija]

Hi @nachiketkukade, thanks so much for considering this request! I don't want to nag you, but I was wondering if you have any update on the topic of picosecond-precision RX timestamps in wifi_pkt_rx_ctrl_t that you hinted at in your previous comment?

I'm developing a system where I clock/phase-synchronize multiple ESP32 chips for sensing applications (https://espargos.net/, still WIP), and it is working really well already (for example, I can do AoA estimation in promiscuous mode with the CSI API), but I'm having issues with unknown sampling time offsets. My guess is that the ESP32 has some internal sampling time offset compensation, and that if I have multiple clock-synchronized ESP32 chips in an array, they will end up with slightly different time offset estimates.

So basically, the CSI in wifi_csi_info_t has already undergone time offset compensation, but every ESP32 in my array applied a slightly different offset compensation. My guess is that if I had access to precise packet reception timestamps, those that are used for FTM, I would be able un-do the time offset compensation. So any progress on this topic would be super helpful for me 😄

[ProfFan]

Hi @nachiketkukade Is there any update on this?

[nachiketkukade]

This is possible with newer chips since the Rx Timestamp already exists in wifi_pkt_rx_ctrl_t, we're reviewing if we can expose the timestamp variable after internal testing

@ProfFan , above changes are currently under internal review

[Jeija]

Hi @nachiketkukade , sorry to nag you about this... Is there any estimate on when and if access to precise RX timestamps will be made available? This issue is very important to me, it is crucial for improving my work on ESP32-based WiFi sensing (https://espargos.net/). If possible, I'd also be happy to assist in testing / reviewing the required changes, I have the necessary hardware setup to validate if the feature is working as expected. Thanks for taking our feedback / feature requests into account in the first place 😄

[nachiketkukade]

@Jeija , apology for the delay. Sometimes the review takes longer due to other prioritised tasks. To give you a headstart for testing, I'm attaching the IDF side change here, please note there may be some more modifications during the internal review.
0001-fix-wifi-Expose-Rx-pkt-timstamp-related-calculations.patch
You can use WIFI_PKT_RX_TIMESTAMP_NSEC to get a packet Rx timestamp of nano-second resolution. I'd be interested to know the testing results from your side!

[Jeija]

Thanks for the patch, very useful!

To quickly test if this is working, I built the following test setup:

• 2 ESPARGOS antenna arrays with 8 ESP32-S2s each, all connected to the same 40MHz clock source. The two antenna arrays were placed at a distance of around 5m to each other.
• I collected nanosecond receive timestamps from all ESP32s and streamed them to a central computer. Since all receivers are connected to the same 40MHz clock source, they should all remain synchronous and the timestamps should not drift. All timestamps are calibrated to a reference WiFi packet.
• I have a WiFi transmitter (another ESP32) in my hand, and I am walking around in between the two ESP32 antenna arrays.

And it's actually working 😄 !

tdoa-demo2.mp4

As you can see in the video, as I am walking around between the two antenna arrays, the time differences of arrival split into two "groups" of 8 antennas each, corresponding to the two antenna arrays, depending on where I'm standing. The range of the values (around 30-40ns) more or less matches the TDoA values one expect for a distance of 5m (2 * 5m / 300000m/s = 33.3ns). It's still a bit flaky, especially for the antenna array that is further away from me, probably due to significant multipath propagation in the room (metallic ceiling, ferro-concrete, ...). But I found it pretty cool to get any results at all for such "cheap" hardware.

I will keep working on this and keep you posted. My next step is to try and see if I can use the nanosecond timestamps to get rid of the sampling time offset-induced phase shift in the complex-valued CSI channel coefficients in wifi_csi_info_t.

[Jeija]

I made some further improvements so that the time of arrival estimates are a bit less noisy now.

tdoa-demo3.mp4

Source code for the demo and timestamp processing is here, to provide some context (firmware is not open source yet): https://github.com/ESPARGOS/pyespargos