Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement fast NPY upload #48

Open
spopham opened this issue Aug 2, 2017 · 2 comments
Open

Implement fast NPY upload #48

spopham opened this issue Aug 2, 2017 · 2 comments

Comments

@spopham
Copy link
Contributor

spopham commented Aug 2, 2017

Rewrite upload_npy_array and download_npy_array to do in-memory and have a npz compressed option

Reference: https://github.com/numpy/numpy/blob/master/numpy/lib/format.py

@anwarnunez
Copy link
Contributor

👍

@anwarnunez
Copy link
Contributor

Just had a quick look b/c it was on my mind. This should be pretty straightforward (see below). This should really only be allowed for uncompressed arrays and GZIP compressed arrays.

https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L326
https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L430

Using these those two functions, I think one just needs to dump that at the beginning of the StringIO (

zipdata = StringIO()
and
filestream = StringIO(array.data)
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants