Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The with-cache macro is marked private #175

Open
scarytom opened this issue Jun 6, 2024 · 4 comments
Open

The with-cache macro is marked private #175

scarytom opened this issue Jun 6, 2024 · 4 comments

Comments

@scarytom
Copy link

scarytom commented Jun 6, 2024

The with-cache macro is marked as private, but, as the docstring says, users may wish to call it if they are using freeze-to-out! or thaw-from-in!. Is there any harm in making this macro public?

(defmacro ^:private with-cache
"Executes body with support for freezing/thawing cached values.
This is a low-level util: you won't need to use this yourself unless
you're using `freeze-to-out!` or `thaw-from-in!` (also low-level utils).

@ptaoussanis
Copy link
Member

@scarytom Hi Tom, I'm happy to make this public on the next release - just want to clean up some of the docs, and consider final changes before potentially marking the feature as non-experimental.

I'm curious, could you share a little about your use case for the cache feature?

@scarytom
Copy link
Author

scarytom commented Jun 6, 2024

No problem,

I'm hoping to use nippy to deserialise input-streams directly from S3, so that I don't have to hold all the bytes in memory before deserialisation. I'm using thaw-from-in! for this and, taking the code in the fast-thaw fn as an example, I'm wrapping my input-stream in a new DataInputStream and then wrapping the call to thaw-from-in! in with-cache. If you don't think the cache call is necessary, then I guess there is no issue.

nippy/src/taoensso/nippy.clj

Lines 1780 to 1781 in 5a9a391

(let [dis (DataInputStream. (ByteArrayInputStream. ba))]
(with-cache (thaw-from-in! dis))))

I'm slightly curious why there isn't a thaw function that accepts an input-stream (and a corresponding freeze function that takes an output-stream).

@ptaoussanis
Copy link
Member

ptaoussanis commented Jun 6, 2024

If you don't think the cache call is necessary, then I guess there is no issue.

The call to with-cache is only necessary if you're using the cache feature - otherwise it's presence or absence won't have any affect.

I'm slightly curious why there isn't a thaw function that accepts an input-stream (and a corresponding freeze function that takes an output-stream).

No particular reason, happy to look at a PR if you'd like to add something. Would just note that in general I'd recommend using the standard freeze (with header) unless you have a strong reason not to. The header can be important for portability, esp. if you're dealing with long-lived data.

@scarytom
Copy link
Author

scarytom commented Jun 6, 2024

Well, we aren't using the cache feature yet, so this issue might be moot.

I couldn't immediately see how to call the standard freeze (with header) with an output-stream (nor standard thaw on an input-stream).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants