W600 and ESP8266 MemoryError

[sabeehalam]

Description

So I am using Thonny and installed this library and ran the slave code with Wemos d1 mini esp8266, Wemos W600-pico and the Raspberry Pi PICO. The code ran fine with the Raspberry Pi Pico. However, both the Wemos W600-PICO and ESP8266 ran into trouble with memory allocation.

Reproduction steps

1.Install the umodbus library on Micropython via the manage packages in Thonny
2. Copy/ paste the RTU slave code in it.
3.Run the code
...

MicroPython version

v1.19.1-870-gb9300ac5b-dirty

MicroPython board

other

Relevant log output

For W600:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/flash/lib/umodbus/serial.py", line 23, in <module>
MemoryError: memory allocation failed, allocating 704 bytes

User code

No response

Additional informations

No response

[beyonlo]

Hello @sabeehalam

Could you please to add this code below before you run the RTU Slave example (before start to import modbus modules), and paste here the output?

import gc
print('run gc.collec now.')
gc.collect()
print('Mem info before load ModBus lib: mem_alloc: {} | mem_free: {}'.format(gc.mem_alloc(), gc.mem_free()))

@brainelectronics I did a test with my ESP32-S3 to check how much memory the RTU example use, and it used ~108KB. Is a hudge usage for the start. But after gc.collect() mem usage is ~31KB. Details below:

RTU Slave example (just part of print mem info):

print('run gc.collec now.')
gc.collect()
print('Mem info before load ModBus lib: mem_alloc: {} | mem_free: {}'.format(gc.mem_alloc(), gc.mem_free()))

from umodbus import version
print('Running ModBus version: {}'.format(version.__version__))

# import modbus client classes
from umodbus.serial import ModbusRTU

# All RTU example code here

print('Mem info after ModBus lib loaded: mem_alloc: {} | mem_free: {}'.format(gc.mem_alloc(), gc.mem_free()))
print('run gc.collec now.')
gc.collect()
print('Mem info after gc.collect(): mem_alloc: {} | mem_free: {}'.format(gc.mem_alloc(), gc.mem_free()))

while True:
    result = client.process()

Output:

$ mpremote run rtu_client_example_with_callback.py
Clock running at: 240000000
run gc.collec now.
Mem info before load ModBus lib: mem_alloc: 4944 | mem_free: 163760
Running ModBus version: 2.3.1-rc26.dev53
Mem info after ModBus lib loaded: mem_alloc: 113904 | mem_free: 54800
run gc.collec now.
Mem info after gc.collect(): mem_alloc: 36640 | mem_free: 132064

Some points:

1. Maybe this hudge memory usage to load the modbus lib can be the problem with microcontrollers with less than 100KB RAM, and maybe this is @sabeehalam scenario. Is a good idea while ModBus lib is loading, in some strategy parts of ModBus lib source code, run the gc.collect()? I think that in this case the total start memory allocated will not so big.

2. After ModBus lib is loaded, and gc.collect() run, we can see the real memory used by the lib: ~31KB. I'm not specialist, but I think as this lib is very complete, this memory used is very acceptable. My question is: I want to run together the RTU Slave, the TCP Slave too - will that increase so much memory usage, or just part of the TCP Server (bind)? I see that there is a common.py that share the same ModBus protocol between TCP and RTU.

3. In my case I still not running the ModBus RTU Slave and TCP Slave lib together my application, because I'm waiting for the uasyncio support PR to be merged, because my entire application use uasyncio. So, I'm a bit worried when I will run ModBus RTU Slave + TCP Slave + my uasyncio application, if the memory will be enough :)

Thank you in advance!

[GimmickNG]

2. After `ModBus` lib is loaded, and `gc.collect()` run, we can see the real memory used by the lib: `~31KB.` I'm not specialist, but I think as this lib is very complete, this memory used is very acceptable. My question is: I want to run together the `RTU Slave`, the `TCP Slave` too - will that increase so much memory usage, or just part of the `TCP Server` (bind)? I see that there is a `common.py` that share the same `ModBus` protocol between `TCP` and `RTU.`

I think it definitely would result in a slight memory increase since the TCP and RTU classes are separated, so importing one should not also load the other. They use common parts so the increase should be very little, but it will most likely increase nonetheless.

3. In my case I still not running the `ModBus RTU Slave and TCP Slave` lib together my application, because I'm waiting for the `uasyncio` support `PR` to be merged, because my entire application use `uasyncio`. So, I'm a bit worried when I will run `ModBus` `RTU Slave` + `TCP Slave` + `my uasyncio application`, if the memory will be enough :)

Since the uasyncio version extends the synchronous versions of the library to avoid code duplication, the additional raw code itself should be very minimal. I don't know how much memory the entire thing consumes when running, though, since I don't have a board to test on myself. In the Ubuntu version it results in an increase from the base 2 MB on startup to 6 MB, but I think this is because it's built for a different runtime, and also because I monkey-patch custom parts into the library, which are not included in the PR. Even so, no better way to find out than to try it - if I had to guess, I think it would probably take around 40-50 KB, not including the size of the uasyncio library itself.

[beyonlo]

@GimmickNG Thanks for reply. I have board with RS485, so as soon that PR will be merged I can to test it (TCP and RTU) to we know how much memory will increase 😄

[sabeehalam]

@beyonlo Here is the output you asked for

[sabeehalam]

When I tried the RTU Slave example, I go this:

[beyonlo]

@sabeehalam The problem is that ModBus lib are allocating more memory than your microcontroller has free memory.

Please, try do call the gc.threshold(4000) before all that code.

import gc
gc.threshold(4000)

That will trigger the gc to collect always that amount that bytes will be allocated. I think that will solve your problem. Let us know if that works for you.

[GimmickNG]

You could also try recompiling along with package freezing to reduce the memory footprint of the library - see this link

[brainelectronics]

You could also try recompiling along with package freezing to reduce the memory footprint of the library - see this link

relates to brainelectronics/micropython-nextion#18

@sabeehalam I've ordered the boards you have to be able to test and improve the code. I also plan a always on hardware test framework to verify all changes on PRs so everybody can test and verify their changes like the async implementation #56 by @GimmickNG

[thalesmaoa]

Can you provide the mpy file for the esp8266?

[brainelectronics]

@brainelectronics add instructions for using mpy-cross to README to resolve this issue.