-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fallbacks to /tmp if the preferred cache folder isn't writable #4143
Changes from all commits
a54db55
5a3f279
38679b6
eaaeeef
80504bc
3324612
eec7201
1a0791b
6c522e8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -55,7 +55,7 @@ export const {run, setFlags, examples} = buildSubCommands('cache', { | |
}, | ||
|
||
dir(config: Config, reporter: Reporter) { | ||
reporter.log(config.cacheFolder); | ||
reporter.log(config.cacheFolder, {force: true}); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am not happy with this solution since I think it needs a better overall solution as we have discussed. Let's have a |
||
}, | ||
|
||
async clean(config: Config, reporter: Reporter, flags: Object, args: Array<string>): Promise<void> { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -60,7 +60,8 @@ export function main({ | |
'--modules-folder <path>', | ||
'rather than installing modules into the node_modules folder relative to the cwd, output them here', | ||
); | ||
commander.option('--cache-folder <path>', 'specify a custom folder to store the yarn cache'); | ||
commander.option('--preferred-cache-folder <path>', 'specify a custom folder to store the yarn cache if possible'); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Better to say "prefer a custom cache folder when it is available". We can make this change in a follow-up. |
||
commander.option('--cache-folder <path>', 'specify a custom folder that must be used to store the yarn cache'); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A+ with the added emphasis with "must be". 👍 |
||
commander.option('--mutex <type>[:specifier]', 'use a mutex to ensure only one yarn instance is executing'); | ||
commander.option('--emoji [bool]', 'enable emoji in output', process.platform === 'darwin'); | ||
commander.option('-s, --silent', 'skip Yarn console logs, other types of logs (script output) will be printed'); | ||
|
@@ -324,6 +325,7 @@ export function main({ | |
binLinks: commander.binLinks, | ||
modulesFolder: commander.modulesFolder, | ||
globalFolder: commander.globalFolder, | ||
preferredCacheFolder: commander.preferredCacheFolder, | ||
cacheFolder: commander.cacheFolder, | ||
preferOffline: commander.preferOffline, | ||
captureHar: commander.har, | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -276,9 +276,47 @@ export default class Config { | |
networkConcurrency: this.networkConcurrency, | ||
networkTimeout: this.networkTimeout, | ||
}); | ||
this._cacheRootFolder = String( | ||
opts.cacheFolder || this.getOption('cache-folder', true) || constants.MODULE_CACHE_DIRECTORY, | ||
); | ||
|
||
let cacheRootFolder = opts.cacheFolder || this.getOption('cache-folder', true); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should be pushed to the top of the list we try and we should fall back to There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I disagree - if the explicitely specified cache folder cannot be used, that's an hard error. This option contract forces us to use this folder. By (almost) silently using a different directory, we might end up with subtle bugs. For example, two Yarn process might run concurrently with different caches. In such a scenario, if for some reason their respective cache could not be reached, they would end up corrupting themselves. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For the readers of this thread, if there are any: we have talked about getting a unique folder once we fallback to We should still probably try to ensure a unique folder in |
||
|
||
if (!cacheRootFolder) { | ||
let preferredCacheFolders = constants.PREFERRED_MODULE_CACHE_DIRECTORIES; | ||
const preferredCacheFolder = opts.preferredCacheFolder || this.getOption('preferred-cache-folder', true); | ||
|
||
if (preferredCacheFolder) { | ||
preferredCacheFolders = [preferredCacheFolder].concat(preferredCacheFolders); | ||
} | ||
|
||
for (let t = 0; t < preferredCacheFolders.length && !cacheRootFolder; ++t) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's just use |
||
const tentativeCacheFolder = String(preferredCacheFolders[t]); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm guessing this is to silence Flow. Note to self: look into this since it may be pointing to a possibly |
||
|
||
try { | ||
await fs.mkdirp(tentativeCacheFolder); | ||
|
||
const testFile = path.join(tentativeCacheFolder, 'testfile'); | ||
|
||
// fs.access is not enough, because the cache folder could actually be a file. | ||
await fs.writeFile(testFile, 'content'); | ||
await fs.readFile(testFile); | ||
await fs.unlink(testFile); | ||
|
||
cacheRootFolder = tentativeCacheFolder; | ||
} catch (error) { | ||
this.reporter.warn(this.reporter.lang('cacheFolderSkipped', tentativeCacheFolder)); | ||
} | ||
|
||
if (cacheRootFolder && t > 0) { | ||
this.reporter.warn(this.reporter.lang('cacheFolderSelected', cacheRootFolder)); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note to self: make this information an info or debug level message for all cases to signal which cache folder is being used for each run. |
||
} | ||
} | ||
} | ||
|
||
if (!cacheRootFolder) { | ||
throw new MessageError(this.reporter.lang('cacheFolderMissing')); | ||
} else { | ||
this._cacheRootFolder = String(cacheRootFolder); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do you need the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Was there before. Probably because of Flow, I'd guess :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. May be we can remove it now? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just tried, still fails. |
||
} | ||
|
||
this.workspacesEnabled = Boolean(this.getOption('workspaces-experimental')); | ||
|
||
this.pruneOfflineMirror = Boolean(this.getOption('yarn-offline-mirror-pruning')); | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
/* @flow */ | ||
|
||
const os = require('os'); | ||
const path = require('path'); | ||
const userHome = require('./util/user-home-dir').default; | ||
|
||
|
@@ -46,15 +47,21 @@ function getDirectory(category: string): string { | |
return path.join(userHome, `.${category}`, 'yarn'); | ||
} | ||
|
||
function getCacheDirectory(): string { | ||
function getPreferredCacheDirectories(): Array<string> { | ||
const preferredCacheDirectories = []; | ||
|
||
if (process.platform === 'darwin') { | ||
return path.join(userHome, 'Library', 'Caches', 'Yarn'); | ||
preferredCacheDirectories.push(path.join(userHome, 'Library', 'Caches', 'Yarn')); | ||
} else { | ||
preferredCacheDirectories.push(getDirectory('cache')); | ||
} | ||
|
||
return getDirectory('cache'); | ||
preferredCacheDirectories.push(path.join(os.tmpdir(), '.yarn-cache')); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should probably append a timestamp or something to the |
||
|
||
return preferredCacheDirectories; | ||
} | ||
|
||
export const MODULE_CACHE_DIRECTORY = getCacheDirectory(); | ||
export const PREFERRED_MODULE_CACHE_DIRECTORIES = getPreferredCacheDirectories(); | ||
export const CONFIG_DIRECTORY = getDirectory('config'); | ||
export const LINK_REGISTRY_DIRECTORY = path.join(CONFIG_DIRECTORY, 'link'); | ||
export const GLOBAL_MODULE_DIRECTORY = path.join(CONFIG_DIRECTORY, 'global'); | ||
|
@@ -70,6 +77,7 @@ export const LOCKFILE_FILENAME = 'yarn.lock'; | |
export const METADATA_FILENAME = '.yarn-metadata.json'; | ||
export const TARBALL_FILENAME = '.yarn-tarball.tgz'; | ||
export const CLEAN_FILENAME = '.yarnclean'; | ||
export const ACCESS_FILENAME = '.yarn-access'; | ||
|
||
export const DEFAULT_INDENT = ' '; | ||
export const SINGLE_INSTANCE_PORT = 31997; | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,6 +13,15 @@ const globModule = require('glob'); | |
const os = require('os'); | ||
const path = require('path'); | ||
|
||
export const constants = | ||
typeof fs.constants !== 'undefined' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Comment about Node 4 compatibility here would be great so in the future we don't get confused about this code. |
||
? fs.constants | ||
: { | ||
R_OK: fs.R_OK, | ||
W_OK: fs.W_OK, | ||
X_OK: fs.X_OK, | ||
}; | ||
|
||
export const lockQueue = new BlockingQueue('fs lock'); | ||
|
||
export const readFileBuffer = promisify(fs.readFile); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are no
expect
statements here and it is not clear what this test is testing. My guess is "don't fail when preferred cache folder cannot be used" but it'd be nice to make that more visible. We should also, probably make sure the warning is printed.