Million Nodes creation

[jigar88]

Hello,

I am developing OPC server using this library. However my use case requires a lot of variable nodes to create. Is it possible to create Million nodes quickly. Currently even with threading server response is very slow and it takes forever to create that many nodes. it creates roughly 4000 nodes per minute which is quite slow for my use case. Any suggestion ?

[swamper123]

I guess the fastest way to get nodes is loading a pickle file, you dumped before.

python-opcua/opcua/server/internal_server.py

Line 159 in 8582747

def load_address_space(self, path):

Remember: Never touch pickles from strangers, unless you realy trust the file.

Less faster would be creating on you own in Python (which you may have done) and the least is via xml. Perhaps a bit execution boost may happen if you compile your Pythoninterpreter with --enable-optimisations...but if you realy need it even more faster, Python may not be your programming language of choice.

[starturtle]

Please calculate your required resources.
A million nodes, if created at 1ms per node (roughly 20 times faster than what you perceived), still take over 15 min to create.
Maybe some very custom trickery (that emulates rather than creates nodes) is closer to your use case than the default implementation. Also, like swamper mentioned, python may not be for you.
Good luck!

[zerox1212]

Python won't be great for this. Even OPCUA isn't good for this because it's a very heavy protocol.

You could see if PyPy is faster.

Otherwise you probably need to rethink your implementation... do you really need one million nodes up front? Why not create as needed or some such thing.

[jigar88]

@zerox1212 . I am creating these nodes as they come or require to create. So I have my application running on aws ec2 instance. The problem is when we test fail over or when I restarted the ec2 server it will take way long to recreate all those nodes. Is there a way to export all nodes and import after server come back online?

[zerox1212]

You can try dumping the address space, but honestly I have no idea how it will handle dumping so many nodes.