Could not find a suitable index to answer graph query and graph scans are disabled

[sharma-kunal]

I am using JanusGraph with Cassandra as the back-end and Elasticsearch for indexing. When I try to send a query from my Python code to JanusGraph, it fails with timeout error and "Warning: please use indexing".

So I enabled this flag, query.force-index=true to check if the request is going to index or not and it throws the error

Could not find a suitable index to answer graph query and graph scans
are disabled:

This is my Janusgraph config file,

storage.backend=cql
storage.cql.keyspace=education
storage.hostname=<host IP>
storage.port=9042
index.education.backend=elasticsearch
index.education.index-name=education
index.education.hostname=<elastic ip>
index.education.port=9200
index.education.elasticsearch.http.auth.type=BASIC
index.education.elasticsearch.http.auth.basic.username=<username>
index.education.elasticsearch.http.auth.basic.password=<password>

I define my schema for vertex and edge like this,

management.buildIndex("vertex", Vertex.class).addKey(management.getPropertyKey("profile_id"))
                .addKey(management.getPropertyKey("institute_name"))
                .addKey(management.getPropertyKey("company"))
                .buildMixedIndex("education");
        management.buildIndex("edge", Edge.class).addKey(management.getPropertyKey("passing_year"))
                .buildMixedIndex("education");

which makes my indexes,

education_vertex and education_edge

Then I insert my data, it is correctly inserted in Cassandra and Elastic.

When I query it using my Python script:

connection = DriverRemoteConnection(url='ws://<IP>:8182/gremlin', traversal_source='educationGraph')
g = traversal().withRemote(connection)
vertex = g.V().has("institute_name", "Saline Water Conversion Corporation").next()
print(vertex)

It fails with the error,

Could not find a suitable index to answer graph query and graph scans are disabled:

I am not sure what I am missing here.

[FlorianHockmann]

I see two issues here:

1. You seem to add an index on property keys that were already present in the graph before. In this case, JanusGraph doesn't know whether the graph already contains data which needs to be added to the new index and it will therefore not enable the index automatically. You then either need to perform a reindex operation to add existing data to the index or enable the index manually in case you don't want to add existing data to the new index.
   This is described here in the docs.
2. You only create mixed indexes but then perform equality checks (simple has() steps). If you just want to find vertices by their exact property values, then you should use a composite index that is backed by the storage backend, not the index backend.

[sharma-kunal]

I am sorry, I didn't completely get your first point. See I have to create a fresh keyspace and a fresh index, so I don't have any existing data to index, but still I'll have to ENABLE_INDEX.

package org.example;

import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.janusgraph.core.JanusGraph;
import org.janusgraph.core.JanusGraphFactory;
import org.janusgraph.core.schema.JanusGraphManagement;
import org.janusgraph.core.schema.SchemaAction;
import org.janusgraph.graphdb.database.management.ManagementSystem;

public class Test {
    static JanusGraph graph;
    static GraphTraversalSource g;

    static {
        graph = JanusGraphFactory.build()
                .set("query.force-index", true)
                .set("storage.backend", "cql")
                .set("storage.cql.keyspace", "experience")
                .set("storage.hostname", "<Host IP>")
                .set("storage.port", "9042")
                .set("index.experience.backend", "elasticsearch")
                .set("index.experience.index-name", "experience")
                .set("index.experience.hostname", "<Elastic IP>")
                .set("index.experience.port", "9200")
                .set("index.experience.elasticsearch.http.auth.type", "BASIC")
                .set("index.experience.elasticsearch.http.auth.basic.username", <Username>)
                .set("index.experience.elasticsearch.http.auth.basic.password", <Password>)
                .open();
        g = graph.traversal();
    }
    private static void create_schema() {
        graph.tx().rollback();
        JanusGraphManagement management = graph.openManagement();
        try {

            management.makePropertyKey("name").dataType(String.class).make();
            management.makePropertyKey("p_uuid").dataType(String.class).make();
            management.makePropertyKey("site_context").dataType(String.class).make();
            management.makePropertyKey("company_name").dataType(String.class).make();
            management.makePropertyKey("start_date").dataType(String.class).make();
            management.makePropertyKey("end_date").dataType(String.class).make();
            management.makePropertyKey("current").dataType(Long.class).make();

            management.makeVertexLabel("seeker").make();
            management.makeVertexLabel("company").make();

            management.makeEdgeLabel("worked").make();

            management.buildIndex("vertex", Vertex.class).addKey(management.getPropertyKey("p_uuid"))
                    .addKey(management.getPropertyKey("company_name"))
                    .buildMixedIndex("experience");
            management.buildIndex("edge", Edge.class).addKey(management.getPropertyKey("start_date"))
                    .addKey(management.getPropertyKey("end_date"))
                    .buildMixedIndex("experience");

            management.commit();
        } catch (Exception e) {
            management.rollback();
            System.out.println("Error: " + e);
        }
    }

    private static void read_data() {
        Vertex v = g.V().has("p_uuid", "abcd1234").next();
        System.out.println(v.id());
    }

    private static void insert_data() {
        Vertex seeker = g.addV("seeker").property("name", "seeker")
                .property("p_uuid", "abcd1234")
                .property("site_context", "india")
                .next();
        Vertex company = g.addV("company").property("company_name", "company").next();

        g.V(seeker).addE("worked").to(company).property("start_date", "2010-02-28")
                .property("end_date", "2012-01-01").iterate();
        g.tx().commit();
    }

    public static void main(String[] args) {
        create_schema();
        insert_data();
//        read_data();
    }
}

If we look at this code, it fails with the same error,

Could not find a suitable index to answer graph query and graph scans are disabled:

Now should I do this after committing the schema changes,

management = graph.openManagement();
management.updateIndex(management.getGraphIndex("vertex"), SchemaAction.ENABLE_INDEX);
management.updateIndex(management.getGraphIndex("edge"), SchemaAction.ENABLE_INDEX);
management.commit();

or something else, can you please give an example ?

One more thing, after committing the schema changes, I tried

ManagementSystem.awaitGraphIndexStatus(graph, "edge").call();

but this keeps on showing

Some key(s) on index vertex do not currently have status(es) [REGISTERED]: company_name=ENABLED,p_uuid=ENABLED

till it exits with a timeout.

Timed out (PT1M) while waiting for index vertex to converge on status(es) [REGISTERED]

[FlorianHockmann]

The error messages sound to me simply like the index is already ENABLED so there is no need to call awaitGraphIndexStatus on it. You can verify this by executing management.printIndexes().

Could not find a suitable index to answer graph query and graph scans are disabled:

You are getting this error for the traversal in read_data(), right? I just tried to reproduce your scenario and ran into the same problem. It looks like the mixed index is not used when it is created for multiple property keys (p_uuid and company_name in your case), but only one property key is used in the traversal and no search predicate is used.
So, the index is used if both property keys are used or if a search predicate like textContains() is used.

I created an issue for this: #3977 as this seems to be a bug to me.

[sharma-kunal]

Ok, I understand that I can only use search predicate or both fields, only then it would work, so now I tried using composite index.

management.buildIndex("vertex", Vertex.class).addKey(management.getPropertyKey("p_uuid"))
                      .addKey(management.getPropertyKey("company_name"))
                      .buildCompositeIndex();

like this, but now it is still showing the same error

Could not find a suitable index to answer graph query and graph scans are disabled: [(p_uuid = abcd1234)]:VERTEX

I checked the status of index, and it is showing

------------------------------------------------------------------------------------------------
Graph Index (Vertex)           | Type        | Unique    | Backing        | Key:           Status |
---------------------------------------------------------------------------------------------------
vertex                         | Composite   | false     | internalindex  | p_uuid:       ENABLED |
                               |             |           |                | company_name:    ENABLED |
---------------------------------------------------------------------------------------------------
Graph Index (Edge)             | Type        | Unique    | Backing        | Key:           Status |
---------------------------------------------------------------------------------------------------
edge                           | Mixed       | false     | experience | start_date:    ENABLED |
                               |             |           |                | end_date:     ENABLED |
---------------------------------------------------------------------------------------------------
Relation Index (VCI)           | Type        | Direction | Sort Key       | Order    |     Status |
---------------------------------------------------------------------------------------------------

Am I missing something with the composite index use ?

[FlorianHockmann]

With a composite index, you actually have to use all indexed property keys in the traversal. Since you added p_uuid and company_name, you now have to use both in your traversal.
If you want to be able to search also by just p_uuid, then you need to create another composite index with just that property key.

[sharma-kunal]

Oh Ok !! I tried creating separate indexes and it seems to be working fine now, Thanks for the help.