Skip to content
forked from basisai/remote-zip

Fetch file listings and individual files from a remote ZIP file

License

Notifications You must be signed in to change notification settings

gyng/remote-zip

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

remote-zip

API documentation

Fetch file listings and individual files from a remote ZIP file.

Features

Without downloading the entire ZIP:

  • Fetch individual files in a remote ZIP
  • Fetch file listings

The gist of what the library does is:

  1. Get the content size of the ZIP file using a HTTP HEAD request.
  2. Get the last 100 bytes using HTTP Range queries (hoping it's enough) to read the end-of-central-directory (EOCD) section.
  3. Using the EOCD, read the central directory (CD) section with a Range request. This section contains a complete file listing and file byte offsets in the ZIP.
  4. To get individual files, use another Range request with an offset to get the local file header + compressed data.

Limitations

  • No ZIP64 support
  • No encrypted ZIP support
  • No stream support via ReadableStream due to testing/dev difficulties
  • Long comments in a ZIP file might cause populate() to fail

Install

yarn add @gyng/remote-zip
npm install --save @gyng/remote-zip

Usage

See the generated API documentation.

If using in the browser, the server will need to whitelist CORS for GET, HEAD, and the Range header.

Basic

const url = new URL("http://www.example.com/test.zip");
const remoteZip = await new RemoteZipPointer({ url }).populate();
const fileListing = remoteZip.files(); // RemoteZipFile[]
const uncompressedBytes = await remoteZip.fetch("test.txt"); // ArrayBuffer

With more features

const method = "POST";
const additonalHeaders = new Headers();
additonalHeaders.append("X-Example", "foobar");
const url = new URL("http://www.example.com/test.zip");
const remoteZip = await new RemoteZipPointer({
  url,
  additionalHeaders,
  method,
  credentials: "include",
}).populate();
const uncompressedBytes = await remoteZip.fetch("test.txt", additionalHeaders);

Dev

Dev instructions See `scripts` in `package.json` for more scripts.
  • yarn d watch and build
  • yarn t:watch watch and test
  • yarn lint
  • yarn build

Run tests and checks with Docker

docker-compose -f docker-compose.test.yml up --build

Publish

Setup

  1. Get an automation token from npm under settings

    https://www.npmjs.com/settings/aicadium/tokens/
    
  2. Add the token to your repository secrets.

    https://github.com/$YOUR_USERNAME/$YOUR_REPO_NAME/settings/secrets/actions/new
    
    • Name: NPM_TOKEN
    • Value: The automation token you got from the previous step

Run

  1. Create a new release.

    https://github.com/$YOUR_USERNAME/$YOUR_REPO_NAME/releases
    

    The workflow at ./github/workflows/publish.yml should run and publish your packages to both NPM and GitHub Packages.

    Don't forget to bump your version number in package.json before this.

About

Fetch file listings and individual files from a remote ZIP file

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Languages

  • TypeScript 94.3%
  • JavaScript 5.1%
  • Shell 0.6%