Introduce an option to not use any cache

[victornoel]

Do you want to request a feature or report a bug?
feature

What is the current behavior?
When calling yarn, packages are first downloaded to the global cache before being put in node_modules

What is the expected behavior?
With an option, e.g., --no-cache, calling yarn --no-cache will skip the use of a global cache and directly work at the node_modules level.

Why?
In a CI runner, where only the package used by the project being tested/integrated are of interest, it would make sense to simply populate the node_modules directory instead of the yarn global cache and then copy the dependencies to node_modules.
Coupled with CI caching, one would just need to cache the node_modules directory and call yarn --no-cache --pure-lockfile to get the correct node_modules in a minimum of time.
Currently, the only solution is either to cache the global yarn cache and rebuild node_modules on every build, or to cache both but then we cache twice every dependency!

Please mention your node.js, yarn and operating system version.
yarn v0.15.1, node v6.7.0 and linux

[olingern]

You can actually change your global and cache folders folder, but there seems to be some linking errors at the moment.

yarn --global-folder ./node_modules/ --cache-folder ./node_modules/

[victornoel]

@olingern do you think it will prevent the double actions of downloading and then copying to node_modules? I feel like yarn is not able to realise this specific situation…
The later action is quite slow on CI systems usually, hence the need for not using any cache at all :)

[jdno]

I am also very interested in such a feature, with the same requirements and use case as in #1178. In a Docker image, the cache will never be used again, so it should be possible to just skip it altogether to improve the final disk size.

[deployable]

RUN set -ex; yarn install --prod --cache-folder ./ycache; rm -rf ./ycache

@victornoel it looks like the cache is stored in a different format so I don't think setting the cache to ./node_modules will help. I can't think of a simple way to inject a null or tmpfs into a docker build so I think you are stuck with writing and removing until this is added to yarn.

[victornoel]

@deployable I don't think the solution is via a hack, this should be handled by yarn itself.

[deployable]

@victornoel Agreed, I wasn't suggesting otherwise. Just providing the workaround "until this is added to yarn"

[victornoel]

by the way @deployable, I just realised I was misinterpreting your comment: my need is not to remove after a yarn install (what your workaround does), my need is not to use any cache at all and not download dependencies in the cache before copying them to nodes_modules, that's why I was saying a hack will never do, not even as a workaround.

[deployable]

@victornoel oic. It actually is possible to access an in memory tmpfs during a build, if you don't mind overloading the /dev directory and have enough memory to fit your dependencies:

RUN set -ex; yarn install --prod --cache-folder /dev/.ycache; rm -rf /dev/.ycache

There's no simple /dev/null type construct for directories to use for a work around though.

[victornoel]

@deployable I don't know why you insist on having me using a cache folder :) I don't want to!
In memory or not, there will be download and copy and processing from the cache to the nodes_modules directory. My need is NOT to build docker images, but to run CI builds, I don't care about removing the cache or not, I simply want to speed up my CI builds and looking for ways to do that :)

Still, these are good workaround for those who are interested by the use case of #1178, so I'm pretty sure @jdno and @andrewmclagan are happy for your share :)

[deployable]

@victornoel I'm not insisting, it's yarn insisting! :) at least until a PR.

[andrewmclagan]

how about running yarn cache clean after an installation? If its in the same RUN command it should not create a layer.

This is what i do:

RUN cd /tmp && \
    yarn install && \
    mv /tmp/node_modules /var/www/html && \
    yarn cache clean

[victornoel]

@andrewmclagan yes, I think this is exactly the same as what other people are proposing (it is equivalent to removing the cache folder), but that's not the subject of the issue.

[dealloc]

I second @victornoel, the ability to just install straight into node_modules without having an intermediate cache has it's use cases.

(assuming I understand the issue of @victornoel correctly that is)

[felixfbecker]

Our cache directories keep growing in size on the server. We would like to simply disable it

[dguo]

I've created yarnpkg/rfcs#53 for a RFC for this issue. I tried to summarize the discussion here. Please comment on it if there is anything about it that should be changed.

[victornoel]

@dguo thanks, good job!

[bestander]

Discussion moved to RFC

[shengbeiniao]

I think disable cache will add to package server load~
disk is cheap, bandwidth is expensive

[kirill-konshin]

In docker it's an extra command to clean cache to make sure image is lean.

[vincentkedison]

I need this feature also. kind of a global config to disable yarn cache completely. My case is my development Macbook storage is quite small and yarn cache use around 1 GB of storage and internet bandwidth is free. Thanks

[zjye-idealhub]

In docker it's an extra command to clean cache to make sure image is lean.

What command is that?

[kirill-konshin]

@zjye-idealhub yarn cache clean: https://classic.yarnpkg.com/en/docs/cli/cache/

[Prozi]

@victornoel oic. It actually is possible to access an in memory tmpfs during a build, if you don't mind overloading the /dev directory and have enough memory to fit your dependencies:

RUN set -ex; yarn install --prod --cache-folder /dev/.ycache; rm -rf /dev/.ycache

There's no simple /dev/null type construct for directories to use for a work around though.

yarn install --cache-folder /tmp/.junk; rm -rf /tmp/.junk

[suzil]

This feature would really help me for my use case. I'm trying to install a package inside of AWS Lambda, which has a 512 MB disk limit and only allows me to write to /tmp. If I just install my package, I'm well under the disk limit, but the yarn cache is making me exceed the disk limit by doubling the amount of disk used. I can't just remove the cache afterwards as it causes the yarn install to fail.

[nicholasruunu]

This feature would really help me for my use case. I'm trying to install a package inside of AWS Lambda, which has a 512 MB disk limit and only allows me to write to /tmp. If I just install my package, I'm well under the disk limit, but the yarn cache is making me exceed the disk limit by doubling the amount of disk used. I can't just remove the cache afterwards as it causes the yarn install to fail.

Haven't tried this, but shouldn't you be able to point the cache to /dev/null?
yarn install --cache-folder /dev/null

edit Nevermind, doesn't seem to work with folders.

[ghost]

Right now I'm trying to run a docker image using --read-only, I want to use to restrict permissions on it after creating a multi stage release build. However yarn breaks the build because of not being able to disable the cache with error Yarn hasn't been able to find a cache folder it can use. Please use the explicit --cache-folder option to tell it what location to use, or make one of the preferred locations writable since /tmp isn't writable. Because yarn is forcing a cache, it means I can't run that image with --read-only. I also just don't want it caching anything in my production build.

[fedulovivan]

In my case, with yarn v1, adding yarn cache clean step, allowed to reduce final image size from 280mb down to 135mb (!) with no impact on functionality.
This is with 15 production dependencies (101 deps total in node_modules) in package.json.
From my dockerfile:
RUN apk add --no-cache tzdata && yarn install --production=true && yarn cache clean

[daveisfera]

I've found that using a cache mount is an effective solution. It prevents the data from being added to the layer and has the added benefit of still allowing the cache to be used between builds.

[AmrSaber]

Thank you @daveisfera for mentioning cache mounts. For anyone wondering about the exact command, it should be something like:

RUN --mount=type=cache,target=/usr/local/share/.cache yarn install

Where /usr/local/share/.cache is where the cache was saved in my case (alpine image).

In my case, this reduced the image size from 1.1Gb to 294Mb!

[ComfyDust]

@daveisfera HUGE help. Thanks so much! I've been poking around for days trying to solve this issue and your solution reduced my size from 2.47GB to 926MB. (I still want to whittle it down further but this at least gets me past my issues building/scanning the image in a GitHub action.)

[Prozi]

@daveisfera HUGE help. Thanks so much! I've been poking around for days trying to solve this issue and your solution reduced my size from 2.47GB to 926MB. (I still want to whittle it down further but this at least gets me past my issues building/scanning the image in a GitHub action.)

Might be offtopic but also be sure to rm -rf .git