Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design a new memory allocation scheme for derived variables #4283

Open
anagainaru opened this issue Aug 1, 2024 · 0 comments
Open

Design a new memory allocation scheme for derived variables #4283

anagainaru opened this issue Aug 1, 2024 · 0 comments
Assignees

Comments

@anagainaru
Copy link
Contributor

Current implementation

In the current release (https://github.com/ornladios/ADIOS2/releases/tag/v2.10.1) memory for a derived variable is allocated each time the variable values are computed.

template <class T>
T *ApplyOneToOne(T *data, size_t dataSize, size_t nVariables, std::function<T(T, T)> compFct)
{
    T *outValues = (T *)malloc(dataSize * sizeof(T));
}

For a complex math expression, each intermediate data is stored in a newly allocated buffer and freed at the end.

E.g.

            +
          /   \
         a    *
             /   \
            b   c

will allocate memory for temp = b*c and will allocate memory for a + temp

Future optimizations

We need to use a pre-allocated buffer for derived variables in the same way as we do with normal variable buffers and use parts of it for temporary data.

#4253 is removing complex numbers from derived variables because of memory allocation issues (this needs to be reintroduced once we implement a more elegant solution of dealing with memory)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants