Is there a way to limit the API request rate?

[ShahanaFarooqui]

Discussed in #1066

^{Originally posted by DerEwige August 16, 2022}
My RTL freezes (overloads) my eclair node when it tries to load the invoices.
(When I go to lightning -> transactions)

The node CPU goes up to 100% (4 cores) and I get time out errors in the logs for http://localhost:8080/listinvoices

I asume that RTL has problem handling over a million of invoices.
Is there an option in the config to tell RTL to load the data slower?

[DerEwige]

Some additional info regarding this problem.

The issue happens because I have several hundred thousend invoices in my database.
RTL tries to fetch them all at once. This results in a timout of the API call and then RTL tries again.

Solution: use the from and to fields of the listinvoice API point and if it gives a timeout reduce the time to get smaller junks

[ShahanaFarooqui]

@DerEwige Thank you for the issue.

After analysing it more closely, we believe that it will be better handled if pagination option is added for the API. I opened two issues requesting pagination on listinvoices and audit APIs as below:
ACINQ/eclair#2450
ACINQ/eclair#2451

I also opened another issue to delete any unpaid expired invoice in order to remove unnecessary records from the node:
ACINQ/eclair#2449

I am putting this issue on hold until at least 2450 is addressed.

[ShahanaFarooqui]

@DerEwige Eclair will provide an option to delete all unpaid expired invoices in their next release.

Please feel free to configure if you have a lot of expired invoices.

We will improve both API request as soon as pagination is available.

[DerEwige]

Thanks @ShahanaFarooqui for sharing
Once the new release drops. I will defenitly use this

[ShahanaFarooqui]

Unblocked with PR ACINQ/eclair#2474

[mviswanathsai]

Hey! I am looking to contribute, can someone guide me on where I can start?

[ShahanaFarooqui]

Hey, you can start by following https://github.com/Ride-The-Lightning/RTL/blob/master/.github/docs/Contributing.md and start a new thread in discussions board if more information is needed after that.

[mviswanathsai]

I have cloned the repository and tried running the server, all is fine. But I am just lost as to where to start since I have not tried meddling with a code base as big. I would be very helpful if you could guide me around what kind of beginner issues I can tackle first to get a feel for the system. That way I can overcome my inertia and contribute, since I plan on being regular. Thanks for reaching out!

…

On Thu, Dec 15, 2022 at 12:08 AM ShahanaFarooqui ***@***.***> wrote: Hey, you can start by following https://github.com/Ride-The-Lightning/RTL/blob/master/.github/docs/Contributing.md and start a new thread in discussions board if more information is needed after that. — Reply to this email directly, view it on GitHub <#1067 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2MJRF7WQWYSGB75TMJ337LWNIHZPANCNFSM56WZK4LQ> . You are receiving this because you commented.Message ID: ***@***.***>

[ShahanaFarooqui]

Pagination cannot be implemented without API responding with total number of records which is missing currently.
I will wait for the issue 2855 to add this in the response before implementing it in the UI.

[ShahanaFarooqui]

Updated code to enable pagination in future (whenever APIs are fixed) with PR #1393.