Mock Batch Request

[minimalisticMe]

Hi, I would like to mock batch requests.
There is an old Github Issue that is not up to date anymore linking to a way of mocking batch requests to Graph SDK prior to v5 (https://dev.to/kenakamu/c-how-to-unit-test-graph-sdk-batch-request-38j2).

I have the following code that I want to mock with Moq in Graph SDK 5.27.0.

var batchRequestContentCollection = new BatchRequestContentCollection(client);

var request = client.Users.Me.Messages[mailId].Content.ToGetRequestInformation();
await batchRequestContentCollection.AddBatchRequestStepAsync(request);

var response = await client.Batch.PostAsync(batchRequestContentCollection, cancellationToken);

var mailStream = await response.GetResponseStreamByIdAsync(request.Key);

The test code looks like the following:

var mockRequestAdapter = RequestAdapterMockFactory.Create();
mockRequestAdapter.Setup(adapter => adapter.SendPrimitiveAsync<Stream>( // Download mail stream
    It.Is<RequestInformation>(info => info.HttpMethod == Method.GET),
    It.IsAny<Dictionary<string, ParsableFactory<IParsable>>>(),
    It.IsAny<CancellationToken>()))
    .ReturnsAsync(EmlTestMailStream());
var mockRequestAdapter = DownloadMailBatchResponseMock();
var mockGraphServiceClient = new Mock<GraphServiceClient>(mockRequestAdapter.Object);
mockGraphServiceClient.Setup(x => x.Users[It.IsAny<string>()].Messages[It.IsAny<string>()].Content.GetAsync(
        It.IsAny<Action<Microsoft.Graph.Users.Item.Messages.Item.Value.ContentRequestBuilder.ContentRequestBuilderGetRequestConfiguration>>(), 
        It.IsAny<CancellationToken>()))
    .ReturnsAsync(EmlTestMailStream());

With mockGraphServiceClient being heavily inspired by the outdated link above.
However I feel like this is not the correct way to go to mock mockRequestAdapter instead of mockRequestAdapter only, as there is an error upon executing the test stating

System.NotSupportedException : Unsupported expression: ... => ....GetAsync(It.IsAny<Action<ContentRequestBuilder.ContentRequestBuilderGetRequestConfiguration>>(), It.IsAny<CancellationToken>())
Non-overridable members (here: ContentRequestBuilder.GetAsync) may not be used in setup / verification expressions.

Keeping out the sketchy mockGraphServiceClient part leaves me to another issue:

IRequestAdapter.ConvertToNativeRequestAsync<HttpRequestMessage>(RequestInformation, CancellationToken) invocation failed with mock behavior Strict.\r\nAll invocations on the mock must have a corresponding setup.

and I do not know how to mock ConvertToNativeRequestAsync as extending mockRequestAdapter the following way

mockRequestAdapter.Setup(adapter => adapter.ConvertToNativeRequestAsync<HttpRequestMessage>(
    It.IsAny<RequestInformation>(),
    It.IsAny<CancellationToken>()))
    .ReturnsAsync((RequestInformation requestInformation, CancellationToken token) =>
    {
        var message = new HttpRequestMessage();
        var mailId = (string)requestInformation.PathParameters["message%2Did"];
        message.Content = new StringContent(mailId);
        return message;
    });

just throws Value cannot be null. (Parameter 'requestUri').

How can I mock batch requests for mails receiving a mail stream?

[amhokies]

Hi @andrueastman is there any best practices guidance for this?

[olivermue]

I'm currently also stuck on mocking batch requests. So far I already made some effort, but unfortunately I'm still not up with a working solution. Here is my progress so far:

At first we need a mocked RequestAdapter we can inject into the GraphServiceClient which I already described here:

    public static class RequestAdapterMockFactory
    {
        public static Mock<IRequestAdapter> Create(MockBehavior mockBehavior = MockBehavior.Strict)
        {
            var mockSerializationWriterFactory = new Mock<ISerializationWriterFactory>();
            mockSerializationWriterFactory.Setup(factory => factory.GetSerializationWriter(It.IsAny<string>()))
                .Returns((string _) => new JsonSerializationWriter());

            var mockRequestAdapter = new Mock<IRequestAdapter>(mockBehavior);
            // The first path element must have four characters to mimic v1.0 or beta
            // This is especially needed to mock batch requests.
            mockRequestAdapter.SetupGet(adapter => adapter.BaseUrl).Returns("http://graph.test.internal/mock");
            mockRequestAdapter.SetupSet(adapter => adapter.BaseUrl = It.IsAny<string>());
            mockRequestAdapter.Setup(adapter => adapter.EnableBackingStore(It.IsAny<IBackingStoreFactory>()));
            mockRequestAdapter.SetupGet(adapter => adapter.SerializationWriterFactory).Returns(mockSerializationWriterFactory.Object);

            return mockRequestAdapter;
        }
    }

With this in place I wrote a first test that tries to mimic a batch request and response. The biggest problem is, that the individual requests within a batch are written as JSON strings into the body of the batch request.
When we receive this JSON batch string within our mock we have to somehow deserialize it (there is no corresponding class in the Graph SDK) and create an answer that matches the desired steps of the batch, which also has to be done as JSON string and which also isn't available as class that could be used somehow.

Here what I achieved so far:

[Fact]
public async Task TryMockingBatchRequestAndAnswer()
{
    var requestAdapter = RequestAdapterMockFactory.Create();
    var graphServiceClient = new GraphServiceClient(requestAdapter.Object);

    // Will be called via BatchRequestContentCollection.AddBatchRequestStepAsync()
    // for each request that is added to the batch.
    requestAdapter.Setup(adapter => adapter.ConvertToNativeRequestAsync<HttpRequestMessage>(
        It.IsAny<RequestInformation>(), It.IsAny<CancellationToken>()))
        .ReturnsAsync((RequestInformation info, CancellationToken _) =>
        {
            var message = new HttpRequestMessage
            {
                Method = new HttpMethod(info.HttpMethod.ToString()),
                RequestUri = info.URI,
            };

            if (info.Content?.Length > 0)
            {
                message.Content = new StreamContent(info.Content);
                // If we provide a different content type, the body will be
                // serialized as base64 string instead of JSON.
                message.Content.Headers.ContentType = new MediaTypeHeaderValue(MediaTypeNames.Application.Json);
            }

            // Some approach to pipe the original request through the batch wrapper.
            //    (Doesn't work, see comments below)
            message.Options.TryAdd(nameof(RequestInformation), info);

            // If we need additional information, we can add them as string to message.Headers
            message.Headers.Add("FooBar", "Baz");

            return message;
        });

    requestAdapter.Setup(adapter => adapter.SendNoContentAsync(
        It.Is<RequestInformation>(info => info.HttpMethod == Method.POST),
        It.IsAny<Dictionary<string, ParsableFactory<IParsable>>>(), It.IsAny<CancellationToken>()))
        .Returns((RequestInformation info, Dictionary<string, ParsableFactory<IParsable>> errorMappings, CancellationToken _) =>
        {
            // The incoming request information contains the serialized batched requests.
            // It converted the above provided HttpRequestMessage
            //    by calling BatchRequestContent.GetBatchRequestContentAsync()
            // This calls BatchRequestContent.WriteBatchRequestStepAsync()
            // This accesses our HttpRequestMessage from ConvertToNativeRequestAsync()
            //    to form a manually crafted json object for the batch request.
            // The properties it will access are
            // - HttpRequestMessage.Method.Method
            // - HttpRequestMessage.Headers
            // - HttpRequestMessage.Content.Headers
            // - HttpRequestMessage.Content.Headers.ContentType.MediaType
            // - HttpRequestMessage.Content stream

            // Depending on the value within each HttpRequestMessage.Content.Headers.ContentType.MediaType
            // the Content stream of each request will either be serialized as a JSON object
            // or as base64 encoded string into the body property within the batched JSON.
            var jsonBody = Encoding.UTF8.GetString(((MemoryStream)info.Content).ToArray());

            var option = info.RequestOptions.First() as ResponseHandlerOption;
            var responseMessage = new HttpResponseMessage
            {
                StatusCode = HttpStatusCode.OK,
                // There is no class in the Graph SDK that represents the response of a batch request.
                // We have to manually craft the response or create a class that can be serialized
                //    to the expected JSON format.
                Content = new StringContent(""),
            };
            option.ResponseHandler.HandleResponseAsync<HttpResponseMessage, object>(responseMessage, errorMappings);
            return Task.CompletedTask;
        });

    var batch = new BatchRequestContentCollection(graphServiceClient);

    var requestGetUser = graphServiceClient.Users["abc"].ToGetRequestInformation();
    // Calls ConvertToNativeRequestAsync to convert RequestInformation to HttpRequestMessage
    await batch.AddBatchRequestStepAsync(requestGetUser);

    var requestPostGroup = graphServiceClient.Groups.ToPostRequestInformation(new Group { Mail = "group@test.internal" });
    await batch.AddBatchRequestStepAsync(requestPostGroup);

    // Calls SendNoContentAsync to send the batched requests
    var response = await graphServiceClient.Batch.PostAsync(batch);

}

[pitbra]

Based on the answer of @olivermue I added some stuff for the response handling.

1st I added the classes for mocking the request and the response.

public class GraphBatchRequestMock
{
    public GraphRequestMock[] Requests { get; set; }
    public GraphBatchResponseMock CreateResponse()
    {
        var responses = Requests.Select(req => new GraphResponseMock
        {
            Id = req.Id,
            Status = (int)System.Net.HttpStatusCode.OK
        });

        return new GraphBatchResponseMock
        {
            Responses = responses.ToArray(),
        };
    }
}

public class GraphRequestMock
{
    public string Id { get; set; }
    public string Url { get; set; }
    public Method Method { get; set; }
    public Dictionary<string, object> Headers { get; set; }
    public JsonObject Body { get; set; }

    public T? BodyTo<T>()
        where T : IParsable
    {
        return KiotaSerializer.Deserialize<T>("application/json", Body.ToString());
    }
}

public class GraphBatchResponseMock
{
    public GraphResponseMock[] Responses { get; set; }
}

public class GraphResponseMock
{
    public string Id { get; set; }
    public int Status { get; set; }
    public Dictionary<string, object> Headers { get; set; }
    public object Body { get; set; }
}

After that I added some extension methods so we can build the correct response we needed

public static class GrapBatchResponseMockExtensions
{
    public static GraphBatchResponseMock ForResponse(this GraphBatchResponseMock self, 
        int index, Action<GraphResponseMock> responseBuildAction)
    {
        if (self.Responses.Count() < index)
            throw new IndexOutOfRangeException();

        responseBuildAction(self.Responses[index]);

        return self;
    }

    public static GraphResponseMock WithStatusCode(this GraphResponseMock self, 
        HttpStatusCode status)
    {
        self.Status = (int)status;
        return self;
    }

    public static GraphResponseMock WithHeader(this GraphResponseMock self, string key, object value)
    {
        if (self.Headers == null)
            self.Headers = new Dictionary<string, object>();

        self.Headers[key] = value;
        return self;
    }

    public static GraphResponseMock WithContent(this GraphResponseMock self, object body)
    {
        self.Body = body;
        return self;
    }
}

Now we can update the Mock for the RequestAdapter of @olivermue for the SendNoContentAsync

 requestAdapter.Setup(adapter => adapter.SendNoContentAsync(
    It.Is<RequestInformation>(info => info.HttpMethod == Method.POST),
    It.IsAny<Dictionary<string, ParsableFactory<IParsable>>>(), It.IsAny<CancellationToken>()))
    .Returns((RequestInformation info, Dictionary<string, ParsableFactory<IParsable>> errorMappings, CancellationToken _) =>
    {
        // The incoming request information contains the serialized batched requests.
        // It converted the above provided HttpRequestMessage
        //    by calling BatchRequestContent.GetBatchRequestContentAsync()
        // This calls BatchRequestContent.WriteBatchRequestStepAsync()
        // This accesses our HttpRequestMessage from ConvertToNativeRequestAsync()
        //    to form a manually crafted json object for the batch request.
        // The properties it will access are
        // - HttpRequestMessage.Method.Method
        // - HttpRequestMessage.Headers
        // - HttpRequestMessage.Content.Headers
        // - HttpRequestMessage.Content.Headers.ContentType.MediaType
        // - HttpRequestMessage.Content stream

        // Depending on the value within each HttpRequestMessage.Content.Headers.ContentType.MediaType
        // the Content stream of each request will either be serialized as a JSON object
        // or as base64 encoded string into the body property within the batched JSON.
        var jsonBody = Encoding.UTF8.GetString(((MemoryStream)info.Content).ToArray());

        var request = JsonSerializer.Deserialize<GraphBatchRequestMock>(jsonBody, 
            new JsonSerializerOptions
            {
                PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
                Converters = { new JsonStringEnumMemberConverter() },
            });

        var response = request.CreateResponse()
            .ForResponse(0, res => res.WithStatusCode(HttpStatusCode.OK).WithContent("").WithHeader("random", "header"));

        var option = info.RequestOptions.First() as ResponseHandlerOption;
        var responseMessage = new HttpResponseMessage
        {
            StatusCode = HttpStatusCode.OK,
            // There is no class in the Graph SDK that represents the response of a batch request.
            // We have to manually craft the response or create a class that can be serialized
            //    to the expected JSON format.
            Content = new StringContent(""),
        };
        option.ResponseHandler.HandleResponseAsync<HttpResponseMessage, object>(responseMessage, errorMappings);
        return Task.CompletedTask;
    });

Now we have the ability to mock each response of the batch request and will get the customized result.

[minimalisticMe]

@pitbra Your addition is great.
I am now able to return a value.

However I have troble returning a stream. For single mail donwload I can just stream a MemoryStream and everything is fine.
I create my request like this:

var requestMap = new Dictionary<string, string>();
foreach (var mailInfo in mailInfoList)
{
	var request = _client.Users[mailbox].Messages[mailInfo.MailId].Content.ToGetRequestInformation();
	var requestId = await batchRequestContentCollection.AddBatchRequestStepAsync(request);
	requestMap.Add(requestId, mailInfo.MailId);
}
var response = await _client.Batch.PostAsync(batchRequestContentCollection, cancellationToken);

And catch the response stream with this line:

await response.GetResponseStreamByIdAsync(request.Key);

When I take your code and modify responseMessage like this

var content = // mail file encoded as Base64 MemoryStream
var responseMessage = new HttpResponseMessage
{
    StatusCode = HttpStatusCode.OK,
    Content = new StreamContent(content),
};

I get an exception Microsoft.Graph.ClientException: Unable to deserialize content..
In your code you comment that the response is a JSON. Is it possible to return a Stream as well? The error message reads that the program wants to deserialize JSON (which it ist not).

[pitbra]

Maybe you have to switch the message.Content.Headers.ContentType in the Mock of ConvertToNaiveRequestAsync to a more suitable for streams (e.g. application/octet-stream), but this is just a guess, i didn't test this one.

requestAdapter.Setup(adapter => adapter.ConvertToNativeRequestAsync<HttpRequestMessage>(
        It.IsAny<RequestInformation>(), It.IsAny<CancellationToken>()))
        .ReturnsAsync((RequestInformation info, CancellationToken _) =>
        {
            var message = new HttpRequestMessage
            {
                Method = new HttpMethod(info.HttpMethod.ToString()),
                RequestUri = info.URI,
            };

            if (info.Content?.Length > 0)
            {
                message.Content = new StreamContent(info.Content);
                // If we provide a different content type, the body will be
                // serialized as base64 string instead of JSON.
                message.Content.Headers.ContentType = new MediaTypeHeaderValue(MediaTypeNames.Application.Octet);
            }

            // Some approach to pipe the original request through the batch wrapper.
            //    (Doesn't work, see comments below)
            message.Options.TryAdd(nameof(RequestInformation), info);

            // If we need additional information, we can add them as string to message.Headers
            message.Headers.Add("FooBar", "Baz");

            return message;
        });

[obsad1an]

Thanks @pitbra , your example got my unit test running. I just had to refactor the mocking a bit because I couldn't find the reference for the method JsonStringEnumMemberConverter().

[minimalisticMe]

I was able to correct my mistake, as I was not serializing the response-object correctly. Changing the code the following way now does not throw errors but returns empty messages still:

var response = request.CreateResponse();
for (int i = 0; i < response.Responses.Length; i++)
{
    response.ForResponse(i, res => res.WithStatusCode(HttpStatusCode.OK)
        .WithContent(RealReponseBody())
        .WithHeader("Cache-Control", "private")
        .WithHeader("Content-Type", "text/plain"));
}

var option = info.RequestOptions.First() as ResponseHandlerOption;
var responseMessage = new HttpResponseMessage
{
    StatusCode = HttpStatusCode.OK,
    Content = new StringContent(JsonSerializer.Serialize(response,
    new JsonSerializerOptions
    {
        Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
    }))
};

In order to get the correct format I traced my connection with Fiddler. I then used a single response body I got from Fiddler as a response for all mocked responses (RealReponseBody() in the code above). Fiddler also showed the two headers Cache-Control and Content-Type, so I added them both as well.

Now I face the issue that

var mailStream = await response.GetResponseStreamByIdAsync(request.Key);

returns null values only and I do not know why, as the mocked response has the same format as the real request, which I observed and compared with Fiddler. I cannot explain why I cannot get the correct response.