Add progress to file uploading

[garrettjstevens]

In the CLI code, when we upload a file, the request body is a stream, and so we're able to listen in on the stream and output progress. We should do the same where possible in the UI. Currently in the UI, the request body is FormData.

Reference code for the CLI can be found here:

Apollo3/packages/apollo-cli/src/utils.ts

Lines 453 to 505 in 2698455

	export async function uploadFile(
	address: string,
	accessToken: string,
	file: string,
	type: string,
	) {
	const filehandle = await fs.promises.open(file)
	const { size } = await filehandle.stat()
	const stream = filehandle.createReadStream()
	const progressBar = new SingleBar({ etaBuffer: 100_000_000 })
	const progressTransform = new ProgressTransform({ progressBar })
	const body = pipeline(stream, progressTransform, (error) => {
	if (error) {
	progressBar.stop()
	console.error('Error processing file.', error)
	throw error
	}
	})
	const init: RequestInit = {
	method: 'POST',
	body,
	duplex: 'half',
	headers: {
	Authorization: `Bearer ${accessToken}`,
	'Content-Type': type,
	'Content-Length': String(size),
	},
	dispatcher: new Agent({ headersTimeout: 60 * 60 * 1000 }),
	}
	const fileName = path.basename(file)
	const url = new URL(localhostToAddress(`${address}/files`))
	url.searchParams.set('name', fileName)
	url.searchParams.set('type', type)
	progressBar.start(size, 0)
	try {
	const response = await fetch(url, init)
	if (!response.ok) {
	const errorMessage = await createFetchErrorMessage(
	response,
	'uploadFile failed',
	)
	throw new ConfigError(errorMessage)
	}
	const json = (await response.json()) as object
	return json['_id' as keyof typeof json]
	} catch (error) {
	console.error(error)
	throw error
	} finally {
	progressBar.stop()
	}
	}

This change in the UI will need to happen in packages/jbrowse-plugin-apollo/src/components/AddAssembly.tsx and packages/jbrowse-plugin-apollo/src/components/ImportFeatures.tsx.

To update the progress percentage, call jobsManager.update(job.name, 'message', percent). jobsManager and job already exist in those files, so you won't need to create them.

Unfortunately, currently only Chrome (and other Chromium-based browsers) allow streams in request bodies. So we'll need to do feature detection and fall back to the current method if the browser doesn't support it.

I found some code here that should work to detect if the browser supports streams in request bodies:
https://github.com/web-platform-tests/wpt/blob/d90f52b5215dca9fd334314b7353a4b57b4719c9/fetch/api/basic/request-upload.h2.any.js#L130-L142
If supported is true after running that code, it should use streams, otherwise fall back to FormData.

[garrettjstevens]

More clear example of feature detection: https://developer.chrome.com/docs/capabilities/web-apis/fetch-streaming-requests#feature_detection

[garrettjstevens]

Since this can only be done on HTTP2, we'll need to see if it's possible for our server to support HTTP2. We have a couple different options to try this.

One option is to use the spdy package to add HTTP2 support to express, as is described here.

The other option is to use fastify instead of express. This is supported natively by NestJS as is described here, but with the http2 option set as new FastifyAdapter({http2: true}).

In either case, we should test this locally to see if it works in a development environment without a key and cert file.