condition on fields within a complex collections cause major UI lag

[klapec]

Bug Report

I'm building a new CMS for my client and I've noticed that once I put condition on some of the fields (mainly array-s and row-s) within a complex collection (~16KB of response when querying a single document with the REST api, over 3k lines of prettified json) the UI initially becomes laggy. Looking at the chrome devtools' performance tab I noticed that the browser spends most of the time on the 'MODIFY_CONDITION' case within the field reducer.

Here's the saved performance profiles

• with-conditionals.json
• without-conditionals.json

The difference is staggering - browser spends approximately 5 seconds before the relationship field is filled with the correct value and during that time the UI is completely unresponsive. And that's on my mbpro m1, this would've been even worse on my client's old pc laptops.

I'll share more data (like the collection file or a sample document) privately if needed.

About the use case - I've got Pages and Templates collections. The "raw" template data is contained within Templates documents (slots, components info, meta tags etc.) and a Page has a hasMany: false relation to Templates. So, for example, there could be a couple of "search results"-type pages, each having a different URL value but they all point to the same Template. This is done to reduce the data footprint on the mongodb weight - we've got lots of pages (it's an ecommerce site).
But here's the twist though - each "section" of a template (there are three: searchState, slots, metaProperties) can be overrode within a page, so that for example many pages can point to the same template BUT each of them can override particular meta properties.
And so we've thought that we'd slap a override checkbox field within each section in the Pages collection and once user clicks it the section expands - and we've done it with the condition option.

condition: (_, siblingData) => siblingData.override === true,

We've got beforeRead hooks that deal with responding with the correct data and that works beautifully.
The only thing that's left is UI responsiveness.

[jmikrut]

Hey @klapec — I see what you're going after here.

Yes, our current implementation of conditions means that each time a condition is used, we need to re-run the condition each time a field value changes, because you may need access to any number of fields within the condition itself.

We do use conditions quite frequently in large projects and have not seen this level of UI lag, so I'm thinking there may be some more optimizations you could do to the way that you're using them. Could you utilize a tabs component, to avoid rendering certain components until the tab is active? Or could you split out your bigger collections into more individual collections?

we'd slap a override checkbox field within each section in the Pages collection and once user clicks it the section expands - and we've done it with the condition option.

Could you, instead of having template overrides exist on the top-level Pages collection, maybe have the overrides themselves exist in a different collection?

There may be some performance optimizations to make with the frequency that MODIFY_CONDITION runs, but this might only help a little, rather than completely remove the lag, because of the way that conditions need to work.

What are your thoughts? I'd like to suggest that this is moved to a Discussion for now, because at some point having too many top-level conditions will inevitably create performance issues. I'm not sure off the top of my head how we can rectify this 100%.

However, if you can hook me up with a reproducible repo, and some seeded data maybe, I think that would be great. Maybe we could batch together the MODIFY_CONDITION action so that fields like arrays and blocks automatically handle all of their own children in one state update, rather than each field doing it individually.

Let me know!

[klapec]

Thanks for the reply.

Could you, instead of having template overrides exist on the top-level Pages collection, maybe have the overrides themselves exist in a different collection?

Not really - a template provides sane defaults that a page "proxies" by default with an option to override particular sections. As I've mentioned before, they way it's modelled is that there could be multiple pages referencing the same template but a page might decide to override particular fields (e.g. title, description within meta properties) while keeping the defaults provided by the template (e.g. slots and components configuration).

I've tried your suggestion with the tabs field but it didn't make any significant difference. Here's the performance profile:
https://gist.github.com/klapec/7bb89fddc211a646a69dee6f0e44cfd5/raw/f703c8cc264e15d1cad6f3dac52396f63b5a05d7/tabs.json

Could you give me a way to contact you directly so that I could share more concrete data - the collection file and example data? I'd rather not post it publicly here. I'm sure we'll post our findings here, though! :)

[jmikrut]

Sure thing! Wanna reach out to me on our Discord? My username is jmikrut there. I'd be up for helping diagnose this in further depth for sure.

You can also send over an email to info@payloadcms.com if you'd rather do that. I can respond from there!

[thgh]

Vue builds an internal dependency graph for computed properties. It only observes dependencies that are actually accessed. For this usecase, where most state changes do not affect most conditions, it could save many calculations.

I could not find an off-the-shelf library to handle this. Payload is in control of the arguments that are passed to conditions, so it seems pretty doable to build such a system.

[vladvlasov256]

I want to revisit this old topic. We've encountered a performance issue with ~400 elements and ~10 nested fields.

First off, what do you think about this hack in fieldReducer:

case 'MODIFY_CONDITION':
    {
        const { path, result, user } = action;
        const resultState = {...state}

        function binarySearchEntries(entries, path) {
            let low = 0;
            let high = entries.length - 1;
            const prefix = `${path}.`;
        
            while (low <= high) {
                const mid = Math.floor((low + high) / 2);
                const [midKey] = entries[mid];
        
                if (midKey === path || midKey.startsWith(prefix)) {
                    high = mid - 1;
                } else if (midKey < path) {
                    low = mid + 1;
                } else {
                    high = mid - 1;
                }
            }
            return low; 
        }

        const entries = Object.entries(state);
        const startIndex = binarySearchEntries(entries, path);

        for (let i = startIndex; i < entries.length; i++) {
            const [fieldPath, field] = entries[i];
            if (fieldPath !== path && !fieldPath.startsWith(`${path}.`)) {
                break;
            }

            let passesCondition = result;
            // If a condition is being set to true,
            // Set all conditions to true
            // Besides those who still fail their own conditions
            if (passesCondition && field.condition) {
                passesCondition = Boolean(field.condition((0, _reduceFieldsToValues.default)(state, true), (0, _getSiblingData.default)(state, path), {
                    user
                }));
            }
            resultState[fieldPath] = {
                ...field,
                passesCondition
            }
        }

        return resultState
    }

The two main optimizations are:

• Eliminating shallow copies in each iteration
• Leveraging the sorted Object.entries() result with a binary search.

Thoughts, @jmikrut?

---

More importantly, I can't find this block in the beta branch. MODIFY_CONDITION is still present in the enum, but it’s not being used. Is it planned for removal?