Suggest me a better approach for indexing in janus.

[MounikaMandi]

Hi,

We have cassandra as our storage backend. We have nealry 600 key spaces. Each key space contains 3.5M nodes and 40M relations on an average. We are using elastic search as our mixed index backend.
Let X be my keyspace,Indexing is failing for X ,when ever any read/write operation is getting triggered for that X while it is getting indexed on any new prop.
Currently we are pausing all the ongoing ingestions while indexing and writing it to some intermediate inmemory db . After indexing is done we are reprocessing the stored data.
Some keyspaces with huge data are taking hours of time to get indexed.This is causing much data lag and memory issues at our end. This has been a major challenge at our end.
I wanted to know how to handle this type of scenario?
How to speed up indexing procedure?
Can we configure janus to handle ingestions while getting indexed? Like background indexing?
Any relavant documentation / research paper / resources / suggestions are appreciated to tackle this kind scenario.

Thanks in advance.

[FlorianHockmann]

Can you explain how you perform the indexing? Do you have some automated process in place which dynamically creates new indexes?
It sounds like you are just getting a new kind of property for which you then create a mixed index but you also directly want to insert data with this new property. Is that correct?

The usual approach is to first create the index and only afterwards insert new data for that index. If the index uses a property key that wasn't in the graph before, then it will be directly enabled. So, you don't have a delay afterwards and can directly insert data with this property which will be also be added to the new index.
It sounds like this approach doesn't work in your use case, but I'm also not sure that I completely understood your use case.

[MounikaMandi]

@FlorianHockmann
So I wanted to know, can we do indexing in background without holding the current ongoing ingestions?

[FlorianHockmann]

Thanks for the explanation. So, your use case is simply that you later on notice that you need to add an index for a property which already exists. This of course requires a reindex operation as JanusGraph needs to add already existing vertices to the new index if you don't want the index to only work for newly created vertices. In general, I would recommend to already create such indexes when you initially add these new property keys or vertex labels to the schema, even if you don't have a concrete use case already, but might reasonably get one in the future. With your example of student vertices, it's sounds very likely to me that you want to be able to search for them based on their first and / or last name.
But of course, you cannot foresee all future use cases and therefore will sometimes come into such a situation where you need to add an index for already existing data.

Unfortunately, I don't know whether it's possible to let JanusGraph add new data to an index while it performs a reindex operation on that index at the same time. Maybe @porunov or @rngcntr can say something about that?

[rngcntr]

I never knew JanusGraph is unable to perform other index operations during a reindex, so I don't think I can provide much help here other than what @FlorianHockmann already suggested.
One (rather hacky) approach could be to create a new vertex label with an enabled (empty) index and run a job that removes all vertices of the old label and inserts them with the new label. That would basically perform a reindex without index downtime, but you would have to handle the duplicate label in your business logic outside of JanusGraph.

[MounikaMandi]

Hi @rngcntr ,
Thanks for the reply.
But if We see the data size, we have nealry 600 key spaces. Each key space contains 3.5M nodes .
Will the approach suggested by you is efficeint?

[rngcntr]

No, it is definitely everything but efficient. As I stated, that would rather be a last resort solution

[jianhuang1648678534]

@MounikaMandi how do you manage those 600 keyspaces? From what I understand, Janusgraph can only support one keyspace.