Getting error while sending a message to another stateless service

[thakursagar]

Hi,

Trying to send a message using Service Fabric Remoting Services (ServiceProxy.Create) and getting the following error:

[{"parsedStack":[{"assembly":"Microsoft.ServiceFabric.Services, Version=6.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1+<InvokeWithRetryAsync>d__23`1.MoveNext","level":0,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw","level":1,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification","level":2,"line":0},{"assembly":"Microsoft.ServiceFabric.Services.Remoting, Version=6.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.ServiceFabric.Services.Remoting.V1.Client.ServiceRemotingPartitionClient+<InvokeAsync>d__2.MoveNext","level":3,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw","level":4,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification","level":5,"line":0},{"assembly":"Microsoft.ServiceFabric.Services.Remoting, Version=6.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase+<InvokeAsync>d__15.MoveNext","level":6,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw","level":7,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification","level":8,"line":0},{"assembly":"Microsoft.ServiceFabric.Services.Remoting, Version=6.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase+<ContinueWithResult>d__16`1.MoveNext","level":9,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw","level":10,"line":0},{"assembly":"mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089","method":"System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification","level":11,"line":0},{"assembly":"Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null","method":"Common.SignalR.SignalRBroadcastClient+<Broadcast>d__1.MoveNext","level":12,"line":0}],"outerId":"0","message":"One or more errors occurred.","severityLevel":"Error","type":"System.AggregateException","id":"65652211"},{"outerId":"65652211","message":"The stream has been closed.","severityLevel":"Error","type":"System.InvalidOperationException","id":"48378117"}]

Service Proxy code:

 IHandleSignalRBroadcast webServiceClient = ServiceProxy.Create<IHandleSignalRBroadcast>(_internalSignalRBroadcastUri);
                var serializedMessage = JsonConvert.SerializeObject(message,
                    new JsonSerializerSettings
                    {
                        ContractResolver = new CamelCasePropertyNamesContractResolver()
                    });
                var response = await webServiceClient.HandleBroadcast(groupName, methodName, serializedMessage).ConfigureAwait(false);

[ameyanator]

Hi @thakursagar, there doesn't seem to be a problem with ServiceProxy.Create

I tried to send a similar serialized message using it and it worked. Could you confirm that your Service is all correct and HandleBroadcast() works correctly.

You could also try and create the proxy using ServiceProxyFactory() if you'd want to.

var proxyFactory = new ServiceProxyFactory(c =>new FabricTransportServiceRemotingClientFactory());
var proxy= proxyFactory.CreateServiceProxy<ISimpleInterface>(new Uri("fabric:/SimpleListener/Stateless1"));
var response = await proxy.SomeFunction(message);

[thakursagar]

@ameyanator yes the HandleBroadcast() works correctly and the service is also fine. This has been happening intermittently in our environment. So you don't think this is a Service Fabric issue and it is coming from the HandleBroadcast() ?

[ameyanator]

Just a few thoughts

The official documentation says "The remoting framework propagates exceptions thrown by the service to the client. As a result, when ServiceProxyis used, the client is responsible for handling the exceptions thrown by the service.". So could you verify that this is not the case.

Could you also give a bit more insight into when you are facing this in your environment. I'll try and look more into it if its a Service Fabric issue.

Could you also specify which remoting framework you're using? V1 or V2