Specific PDF file causing internal server error.

[Minisaints]

Hello, (Me again!)

During our internal testing, we seem to have identified a PDF file that is causing our API endpoint to throw an internal server error (500) when calling "ParseInboundEmailWebhookAsync".

We have sent hundreds of emails with a plethora of attachments to test our email server over the past week and it works great especially with all PDF attachments, except this one. The file isn't corrupt and opens as normal in many PDF viewers too.

This is the current API code running when attempting to parse inbound web responses. I can confirm the code executes up till the yellow marker but never hits the red marker nor the exception block.

I can't attachment this PDF publicly as it is confidential but I can provide in private.

Am I missing something? Has this been reported before?

[Jericho]

I'm a little confused by your description of the problem. On one hand you say that this problem occurs when calling "ParseInboundEmailWebhookAsync" but on the other hand you say that your code never hits the red marker (on the screenshot you provided, the red marker precedes the line where you invoke ParseInboundEmailWebhookAsync). So, I'm not sure whether the problem occurs when you invoke ParseInboundEmailWebhookAsync or prior to it.

Right above the red marker is the line where you attempt to copy the content of the request to a memory stream. Isn't that the most likely culprit? Could it be that the PDF is exceedingly large and somehow it's causing an issue with copying the content? In addition to the PDF file being very large, could there be an issue where your VB.NET code is reading the content of the request faster than it's being streamed? I know this sounds far fetched, but I have heard of this situation before: here and here. In both of these cases, the developers were using Tcp NetworkStream so they probably don't directly apply to your situation.

[Minisaints]

I'm a little confused by your description of the problem. On one hand you say that this problem occurs when calling "ParseInboundEmailWebhookAsync" but on the other hand you say that your code never hits the red marker (on the screenshot you provided, the red marker precedes the line where you invoke ParseInboundEmailWebhookAsync). So, I'm not sure whether the problem occurs when you invoke ParseInboundEmailWebhookAsync or prior to it.

Right above the red marker is the line where you attempt to copy the content of the request to a memory stream. Isn't that the most likely culprit? Could it be that the PDF is exceedingly large and somehow it's causing an issue with copying the content? In addition to the PDF file being very large, could there be an issue where your VB.NET code is reading the content of the request faster than it's being streamed? I know this sounds far fetched, but I have heard of this situation before: here and here. In both of these cases, the developers were using Tcp NetworkStream so they probably don't directly apply to your situation.

Sorry, the red marker was supposed to be "after" the call to "ParseInboundEmailWebhookAsync".

The PDF file itself is only 500KB and as I noted before, every other PDF file larger and smaller we've tested along with numerous other files all work perfect.

[Jericho]

Can you create a private GitHub repo, upload this PDF to the private repo and invite me so I can run some tests and attempt to reproduce the problem?

[Minisaints]

Sure, I have just invited you.

Thank you.

[Jericho]

I don't see anything wrong with this PDF. In fact, I wasn't even able to reproduce the problem you are experiencing. I was able to successfully parse an inbound email with this PDF attached.

[Jericho]

To help us investigate further, I suggest you save the content of the inbound email to a file and upload to the private repo and that will allow me to run a unit test with the actual data that is posted to your controller.

Here's a handy method I wrote for debugging purposes. It writes the HTTP headers and the payload of the request to text files. you invoke this method like so:

await Utils.SaveRequest(Request, "DebuggingWeirdPdf").ConfigureAwait(false);

public static async Task SaveRequest(HttpRequest request, string name)
{
	try
	{
		// Check if file(s) already exist
		var previousHeaderFiles = Directory.EnumerateFiles("C:\\Temp\\", $"{name}InboundHeaders*.txt").ToArray();
		var previousPayloadFiles = Directory.EnumerateFiles("C:\\Temp\\", $"{name}InboundPayload*.txt").ToArray();

		// Save the headers to a file
		await File
			.WriteAllTextAsync(
				$"C:\\Temp\\{name}InboundHeaders{(previousHeaderFiles.Length + 1):00}.txt",
				JsonConvert.SerializeObject(request.Headers, Formatting.Indented))
			.ConfigureAwait(false);

		// Save the payload to a file
		await using var body = new MemoryStream();
		await request.Body.CopyToAsync(body).ConfigureAwait(false);
		body.Position = 0;

		using var reader = new StreamReader(body);
		var postedData = await reader.ReadToEndAsync().ConfigureAwait(false);

		await File
			.WriteAllTextAsync(
				$"C:\\Temp\\{name}InboundPayload{(previousPayloadFiles.Length + 1):00}.txt",
				postedData)
			.ConfigureAwait(false);
	}
	catch (Exception e)
	{
		Debug.WriteLine(e.Message);
	}
}

[Minisaints]

Hey,

I have committed those 2 files that are returned from the code you provided.

Thanks.

[Jericho]

Thanks for the payload file. I wrote the following unit test:

[Fact]
public async Task InboundEmailWithWeirdPDFAsync()
{
	using (var fileStream = File.OpenRead(@"C:\Temp\DebuggingWeirdPdf_InboundPayload1.txt"))
	{
		var parser = new WebhookParser();
		var inboundEmail = await parser.ParseInboundEmailWebhookAsync(fileStream).ConfigureAwait(false);

		// Assert the payload was parsed correctly (including one attachment)
		inboundEmail.ShouldNotBeNull();
		inboundEmail.Attachments.ShouldNotBeNull();
		inboundEmail.Attachments.Length.ShouldBe(1);
		inboundEmail.Attachments[0].ContentType.ShouldBe("application/pdf");

		// Write the content of the attachment to disk
		using (var destination = File.OpenWrite(@"C:\Temp\TheFileWeReceivedWithTheInboundEmail.pdf"))
		{
			inboundEmail.Attachments[0].Data.CopyTo(destination);
		}
	}
}

This unit test successfully parses the payload you provided which means that I still can't reproduce the problem you originally experienced. Having said that, I noticed a few things:

• the PDF written to disk is 861 kb, not 500 kb as you mentioned previously (also I checked the PDF you uploaded in the private repo and it's indeed 510 KB). This is probably a red flag. Is there some sort of compression at some point in the process that could be corrupting the content of this file? Or something of that nature?
• I noticed the email originally came from a protonmail server. I'm not super familiar with how protonmail works but don't they encrypt the content of emails? Could this encryption be causing the problem?
• Maybe the content of the PDF needs to be decrypted before StrongGrid can parse the inbound email? Just a guess on my part. Again, I am not familiar with protonmail so maybe I'm completely off-base here.
• Have you tried sending this same email (including the PDF attachment) from a non-protonmail server? Let's say, from your personal Gmail or Hotmail address in order to check whether the encryption and/or compression is causing this problem or not?

[Minisaints]

Hello,

Indeed, the filesize is a red flag. The example email I provided was from one of my personal ProtonMail aliases but previous emails sent were all from Outlook.

I'll go back over the points you've recognised, run some more checks and comment back if I discover anything.

Thanks again for your help.

[Minisaints]

I believe I've found the issue. Even though I'm not 100% confident on why the issue exists, the above PDF now parses correctly along with all the other files previously tested.

For context, this is the first time we're bolting on a few APIs in VB on an older webform project (legacy) so I expected it to get "ugly".

It appears the WebAPI framework differs slightly across VB/C#. Normally in C# I can directly write the byte contents from a memory stream to another memory stream using "copyto" and all is well. However, in VB, it must first be read in using "readasstreamasync" then it can be copied.

Thanks for help with this though. (should be the last you hear from me!) 😇