Technical question about Mosparo projects usage

[nadorator]

Hello, I would like to know If there is a limitation for project creation on mosparo server.

My client have installed Mosparo on a dedicated debian 12 server. He use a custom CMS made internally with many modules, newsletter subscription, contact form, subscription to e-shop form, reservations form and many more. Right now he have around 60K website using the CMS and I have replaced all google recpatcha calls to Mosparo calls using the dedicated server and PHP API.

He would like to know if he can create one project per client so between 60K and 300K projects. He want to know if there will be performance problem or not do you have an idea? did you do some testing about it?

Also he would like to know if you already have common ruleset he could import for each project or import globaly?

Right now there is 22 servers runing all the 60K website perhaps we also could one project per server. We have one project for all server/clients for the moment.

What would be better?

Also he wants to give client some mosparo stats about the validation of the submission so I use the mosparo API for that. If I use only one project or 22 projects could we have detailled stats for each project based on the domain used or something else? Would it be better to have 60K project instead and use differents keys for each website?

Thanks you again for your time also we are very pleased with the Mosparo project wich is a great solution to prevent spam and bots.

[zepich]

Hi @nadorator

I will reverse your message and go from bottom to top. I think it's easier to answer your questions this way.

Thanks you again for your time also we are very pleased with the Mosparo project wich is a great solution to prevent spam and bots.

Thank you very much for your feedback. It's always great to read such feedback, and it helps us continue the project! Also, and that is the biggest motivation, it's fantastic to hear that you use mosparo for so many websites and that you replaced this old-school spam protection with a modern one... :)

Your questions

Projects

The idea we had in mind when we developed was simple: One project is a thing that shares the same things between multiple websites. The same things in this idea are rules, designs, settings, and - especially - data privacy.

This means, for example, If you're a company with five websites that share the design, it makes sense to have one project since the mosparo box will look the same for all five websites. Data privacy is also okay since an employee of this company can view the user data for all five websites. But if five different teams will work with the user data, or if these five websites have various designs, the idea, in general, is to use one project per page since, with a project, all data in mosparo is separated from other projects.

Now, in your case, with 60k websites, I estimate that you have more or less 60k different customers (yes, some customers may have more than one website, but you'll probably have 50k+ customers for these 60k+ websites).

So, from the perspective of our initial idea, you should use 60k projects in mosparo (or maybe a bit less if multiple websites belong to the same customer and share the same design and so on).

On the other hand, we didn't think about such a setup with 60k projects. I will talk about performance in a second, but first, the main question is: How do you create 60k projects? Since there is no API for creating and managing projects, you would have to do it by hand (in my view, that's more or less impossible) or access the database directly and create those projects.

You could, of course, adjust the mosparo code and add a new API controller to create and manage projects. But with that change, it's more complicated to update mosparo.

From this perspective, creating 60k projects is only possible by directly accessing the database or creating additional API methods.

Creating a project per server or using one project for all websites is possible, and from our perspective, no problem, as long as the customers do not have access to the mosparo project directly. Otherwise, they could see the other websites' user data, which would be a data privacy issue. It's not what we had in mind when we built it, but there is no reason why you couldn't or shouldn't do that.

Statistics

mosparo collects the statistics for each project independently. The statistics are stored without the info from where the submission is coming from, so you would need one project per website to get the statistics for a website.

Since the CMS is custom-made, you could also use the JavaScript event handler to log the spam submission from the frontend (when mosparo detects a spam submission) and use the backend verification process to count valid and invalid submissions independently from mosparo (in your own database/table).

Rulesets

We do not have any general rulesets available. Honestly, all of my productive mosparo installations and projects contain zero rules, mainly because we haven't received any spam since we added mosparo. At one point in the future, we will receive spam on these websites, and we must add rules then.

In the future, we know that a service provider has the idea of offering rulesets as a service. But as far as I know, that's still in the works and not yet ready.

Performance

Now, some details about performance...

General performance

We haven't done performance testing for such high numbers as you have. We performed general performance tests before releasing mosparo to determine how much traffic mosparo could handle. We used a setup of three virtual servers from DigitalOcean (1GB, 1vCPU, 25GiB). One of the servers was a load balancer with nginx, and the other two were PHP-FPM servers. We used the smallest instance of a hosted database from DigitalOcean for the database.

With this setup, we could run 1.4m API requests per day. Since one submission requires 3 API requests (request submit token, validate form data, verify form data), this means that with such a setup, you can handle 460k submissions per day. I have to admit that we haven't made a technically correct test procedure (no repeated tests, no additional load on the servers) and can't guarantee these numbers. We did it to understand whether mosparo is performing as expected or if we have a significant bottleneck somewhere. Additionally, the tests were made with an (almost ready) alpha version of mosparo but not a version released since the start of public beta in January 2023. I write this to clarify that while some things were optimized since this alpha version, we also added new features that could make mosparo slower. If we repeat this test today, it may not be as good as we had back then.

But overall, since you're using a dedicated server (in my understanding, that's not a virtual server, right?), you have much more power, and because of that, the number of handled submissions per day should be much higher.

Additionally, we received an issue last year from a user regarding the cleanup logic, which could not handle the 500k submissions the user had in its database. We optimized the cleanup logic, and I can confirm that it can handle way more than 500k submissions.

That said, I think it's no problem for mosparo to handle the submissions of 60k websites in one or 22 mosparo projects.

Performance regarding 60k projects

The next part is simply speculation since I haven't tested that.

I think that mosparo, in general, should be able to handle 60k projects if there is some API to handle the gigantic amount of projects (additional API controller or directly accessing the database). This means that mosparo should be okay with processing the submissions for so many projects since all project-related entities are linked correctly to the project, and the other entities have no direct connection to the projects.

But, here is the biggest issue of this idea, and I know already that we will have this problem: you, as an admin, cannot access mosparo anymore if there are 60k projects in mosparo. This is because of the project switcher in the top left corner. This dropdown lists all 60k projects. This is first a problem for the webserver and PHP to get and add all the projects to the dropdown. However, I think your browser and the JavaScript functionality will also not be happy if you open the dropdown and search for a project.

So, yes, I can guarantee you already that 60k projects will not work in mosparo in the version right now.

Summary

While mosparo can handle large amounts of submissions and do spam checking for so many websites, I think the way to go right now is to use one or 22 projects. I think that technically, mosparo can handle 60k projects, but these projects are not easy to create, and the UI will stop working with such a high number of projects.

Future

Since we want to make mosparo available for everybody and every possible situation, I will add the idea of creating 60k projects to the list of future developments. I cannot guarantee that we will optimize mosparo in this direction, and I cannot tell you when that will happen, but I know already that it will not be before the second half of 2024, not before October 2024.

This is because the next bigger (minor) version, v1.2, is already developed and in testing. We could start with these optimizations sometime in June, and since the adjustments will require a lot of time (with additional API methods and so on), it will take a moment to develop it.

I hope I answered all of your questions. Please let me know if I missed one or if you have more questions.

Thank you for being so helpful in making mosparo better!

Kind regards,

zepich

[nadorator]

Hello,

thanks for the quick responses and the reacitivty we apreciate that a lot.

You did answer all my questions (for the moment) It is totally what I have expected. At first I was also thinking about 1 project for all because indeed right now we don't use any design for mosparo as we use the invisible version.

Also, how we will use the admin interface, in fact, we will have one administrator and some manager whom will configure the projects and add new projects for clients if needed eventually so only a few users.

The project(s) will be used by 60K website with only one common CMS code. What differ for each website is the database. One database = one client and their specific hosting. One hosting per client as well but using the same base code for the CMS on each server. All servers code is synchronized throught rsync and git. That's why the code base is the same for all.

So, I like your idea to use the javascript event to build the stats instead of using mosparo stat API. It will reduce the code of the network communications and liberate some resources on the Mosparo server. I will try to use that way probably. I will have to discuss of this with the project manager/product owner.

In the history of the project we have in fact around 300K websites created but not all of them are still online.

Still I think the project manager is interested to build 60K project so if this is still the case, the project will probably be created by the managers in a first time manually and then configured in the admin of each client or perhaps we will create a custom function to do that automatically in the mosparo server through the client admin. We will have to think about it.

I have understood the problem with the select with many projects if we choose to create one project per client then we will also work on that on make some proposal of updates.

Also notice that our clients are not only using english language but also french, spanish and netherland so we will also probably help working on the NL translations for mosparo as well.

Regards.
Nadir.

[zepich]

Hi @nadorator

Thank you very much for your answer and the details.

I've created #226 and #227 to describe these two things which would be required to have an unlimited number of projects (well, probably not unlimited since at one point we will always have a problem, but the idea is to push the limit as high as possible). Regarding #227: You wrote that the manager will create the projects manually. Still, for me, it's unimaginable to create even 1,000 projects manually, so I think the API is a must-have for managing many projects.

I will investigate these two things when I have more time. Your contribution is always welcome! Thank you very much!

If you plan to work on these two tasks, please give me a heads-up so we can discuss their current state.

Kind regards,

zepich

[nadorator]

Hello @zepich, thanks for the investigations. Right now, we have discussed and agreed to use only 4 projects to ease the process and to not use a direct access or api update to create the projects automatically yet.

If the option become available in the future it would be great.

The 4 projects will be per countries. So Netherlands, Belgium/Luxemburg, France and Spain.
The statistics will be in the database of each client at runtime using the returns form the js events and the actual usage of the php api sdk in our php controllers.

The updates for the stats will be done later, right now we focus on the update to choose the correct project for each clients of each countries so we will be able to put online the replacement of recaptcha by mosparo very soon.

Regards,
Nadir.

[zepich]

Hi @nadorator

Thank you very much for your feedback.

I think using a project per country is a good idea. You can also adjust the rules between the projects regarding the countries and (more or less) the languages.

Please let me know if you need any help with the integration.

Kind regards,

zepich