Progress listener for SHACL validation?

[fkleedorfer]

Hi,

We're using a ShaclSail for validating data, which works as expected. For long-running tasks (resulting in large commits), we show users a progress bar, so they don't get the impression that something went wrong. However, such large commits can take a long time in SHACL-validation (30 seconds or longer), so the impression for users might still be that the process is stuck at the end when actually everything is fine. If we could show them the progress of shacl validation, that problem would be solved for us.

Is there a way to obtain feedback from the validation process (ideally: expected steps and steps done - even if the expected steps increase over time)?

Thanks!

[hmottestad]

At the moment there isn't any functionality for this. You could enable performance logging, that way you would see what is currently being executed.

Would be useful to be able to attach a listener to a connection to see progress for validation as well as queries and reasoning. Something generic that could be used for many things.

[fkleedorfer]

It could be done without having to change the connection interface using a ThreadLocal, like so:

public abstract class ConnectionListenerContext {

  private static final ThreadLocal<ShaclValidationListener> shaclValidationListenerThreadLocal = new ThreadLocal<>();

   public static void setShaclValidationListener(ShaclValidationListener listener){
      shaclValidationListenerThreadLocal.set(listener);
   }

 public static Optional<ShaclValidationListener> getShaclValidationListener(){
      return Optional.ofNullable(shaclValidationListenerThreadLocal.get());
   }

   public static void clearShaclValidationListener(ShaclValidationListener listener){
      shaclValidationListenerThreadLocal.set(null);
   }
}

before calling commit() on the connection, the client uses setShaclValidationListener(..) to set the listener, the shacl validator obtains the listener if there is one using getShaclValidationListener, and updates it as needed, and at the end of processing the commit, the listener ThreadLocal is cleared using clearShaclValidationListener.

Just a thought.

[fkleedorfer]

As an additional complication, for this to work in our setup, the update would have to be propagated from the RDF4J database via a HTTPRepositoryConnection. Not sure if such communication is possible at all