Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: symlinks handling #3298

Open
wants to merge 60 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
0189750
don't warn on ignored directory symlinks
2ZeroSix Jan 18, 2022
7320081
Don't crash on directory symlinks (default behavior is copy-paste of …
2ZeroSix Jan 18, 2022
963b645
add missing await
2ZeroSix Feb 1, 2022
b87e533
remove unnecessary brackets in string interpolation
2ZeroSix Feb 1, 2022
c087295
codestyle
2ZeroSix Feb 1, 2022
6e23dcc
add symlink cycle test
2ZeroSix Feb 1, 2022
b9b9314
use ignore rule without delimiter in checked-in but ignored warning test
2ZeroSix Feb 1, 2022
4486f00
probable fix for cycles on windows
2ZeroSix Feb 3, 2022
e49c45b
don't follow links
2ZeroSix Feb 3, 2022
7e1cb20
use appropriate error message (wrongly cherry-picked)
2ZeroSix Feb 3, 2022
05f6cd9
double check for directory symlinks
2ZeroSix Feb 3, 2022
0a6ffaa
replace "throws" => "not throws on valid directory symlinks"
2ZeroSix Feb 3, 2022
10b8e3e
Revert "double check for directory symlinks"
2ZeroSix Feb 12, 2022
f6bd8d2
Revert "don't follow links"
2ZeroSix Feb 12, 2022
bfc30e5
Revert "probable fix for cycles on windows"
2ZeroSix Feb 12, 2022
b1b69bc
try to resolve symbolic link manually before listSync invocation
2ZeroSix Feb 12, 2022
d97bfe2
Revert "Revert "probable fix for cycles on windows""
2ZeroSix Feb 12, 2022
add6e5c
Revert "Revert "don't follow links""
2ZeroSix Feb 12, 2022
380a1e0
Revert "Revert "double check for directory symlinks""
2ZeroSix Feb 12, 2022
6623ea2
add additional cycles tests
2ZeroSix Feb 12, 2022
8e6aa02
try to see detailed log on windows
2ZeroSix Feb 13, 2022
aa391dc
maintain list of visited symlink dirs
2ZeroSix Jun 3, 2022
dfb47e5
use resolvedLinkTargets
2ZeroSix Jun 3, 2022
2705410
fix assertLinksResolvable
2ZeroSix Jun 3, 2022
f3aa7c3
update message
2ZeroSix Jun 3, 2022
ad092c5
add a bit more tests
2ZeroSix Jun 3, 2022
003a59d
try to resolve or else count occurrences
2ZeroSix Jun 3, 2022
73455c6
typo: symlinks => symlink
2ZeroSix Jun 3, 2022
fa09a85
rewrite symlinks detection
2ZeroSix Jun 4, 2022
1a72dc3
create and append in single assertion
2ZeroSix Jun 4, 2022
3a58e92
posix dirname for internal representation
2ZeroSix Jun 4, 2022
ce5544a
fix copy on write solution
2ZeroSix Jun 5, 2022
d5b595b
code style
2ZeroSix Jun 5, 2022
7745fc7
explicitly say that error happened because of loop
2ZeroSix Jun 5, 2022
af93eb1
not throws on ignored broken directory symlinks
2ZeroSix Jun 5, 2022
8aa575d
not throws on valid links to the same directory
2ZeroSix Jun 5, 2022
9700d92
instant loop is actually non-resolving
2ZeroSix Jun 5, 2022
0dcf557
add detailed comment with explanation
2ZeroSix Jun 5, 2022
9c2c78d
make assert methods static
2ZeroSix Jun 5, 2022
42b0821
add test for nested loop
2ZeroSix Jun 5, 2022
020f465
rename: posixDir => internalDir
2ZeroSix Jun 5, 2022
07b8cf9
make assertions private
2ZeroSix Jun 5, 2022
68831dc
use join instead of raw posix path
2ZeroSix Jun 5, 2022
cdb66d3
Use FileSystemEntity.resolveSymbolicLinks
sigurdm Jun 7, 2022
f98d77d
Merge
sigurdm Oct 21, 2024
599a42f
publish integration test
sigurdm Oct 21, 2024
4ed382a
Rely on system detecting cycles
sigurdm Oct 22, 2024
0c56b15
fmt
sigurdm Oct 22, 2024
f620e18
Remove deprecated lint
sigurdm Oct 22, 2024
7679721
Detect cycles
sigurdm Oct 25, 2024
f8ee397
canonicalize before examining parents (gets rid of /.)
sigurdm Oct 25, 2024
6d2ef92
Update test/package_list_files_test.dart
sigurdm Oct 29, 2024
3bd1765
Address review comments
sigurdm Oct 31, 2024
3837226
Merge remote-tracking branch '2ZeroSix/fix/symlinks' into fix/symlinks
sigurdm Oct 31, 2024
66b1ae3
Add missing file
sigurdm Oct 31, 2024
d4a89e0
attempt at windows fix
sigurdm Oct 31, 2024
8909acf
Use link descriptors and prefix in lish/symlinks_test
sigurdm Oct 31, 2024
778e35c
Move link() -> descriptor.dart
sigurdm Oct 31, 2024
b179ccf
move test/link_descriptor.dart -> test/descriptor/linkdescriptor.dart
sigurdm Oct 31, 2024
2d8ec9b
attempt at windows fix2
sigurdm Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion analysis_options.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ linter:
- missing_whitespace_between_adjacent_strings
- no_adjacent_strings_in_list
- no_runtimeType_toString
- package_api_docs
- prefer_const_declarations
- prefer_final_locals
- require_trailing_commas
Expand Down
69 changes: 54 additions & 15 deletions lib/src/package.dart
Original file line number Diff line number Diff line change
Expand Up @@ -278,27 +278,62 @@ See $workspacesDocUrl for more information.
return p.join(root, path);
}

return Ignore.listFiles(
/// Throws if [path] is a link that cannot resolve.
///
/// Circular links will fail to resolve at some depth defined by the os.
void verifyLink(String path) {
final link = Link(path);
if (link.existsSync()) {
try {
link.resolveSymbolicLinksSync();
} on FileSystemException catch (e) {
if (!link.existsSync()) {
return;
}
throw DataException(
'Could not resolve symbolic link $path. $e',
);
}
}
}

/// We check each directory that it doesn't symlink-resolve to the
/// symlink-resolution of any parent directory of itself. This avoids
/// cycles.
///
/// Cache the symlink resolutions here.
final symlinkResolvedDirs = <String, String>{};
jonasfj marked this conversation as resolved.
Show resolved Hide resolved
String resolveDirSymlinks(String path) {
return symlinkResolvedDirs[path] ??=
Directory(path).resolveSymbolicLinksSync();
}

final result = Ignore.listFiles(
beneath: beneath,
listDir: (dir) {
var contents = Directory(resolve(dir)).listSync();
final resolvedDir = p.normalize(resolve(dir));
verifyLink(resolvedDir);

{
final canonicalized = p.canonicalize(resolvedDir);
final symlinkResolvedDir = resolveDirSymlinks(canonicalized);
for (final parent in parentDirs(p.dirname(canonicalized))) {
final symlinkResolvedParent = resolveDirSymlinks(parent);
if (p.equals(symlinkResolvedDir, symlinkResolvedParent)) {
dataError('''
Pub does not support symlink cycles.

$symlinkResolvedDir => ${p.canonicalize(symlinkResolvedParent)}
''');
}
}
}
var contents = Directory(resolvedDir).listSync(followLinks: false);

if (!recursive) {
contents = contents.where((entity) => entity is! Directory).toList();
}
return contents.map((entity) {
if (linkExists(entity.path)) {
final target = Link(entity.path).targetSync();
if (dirExists(entity.path)) {
throw DataException(
'''Pub does not support publishing packages with directory symlinks: `${entity.path}`.''',
);
}
if (!fileExists(entity.path)) {
throw DataException(
'''Pub does not support publishing packages with non-resolving symlink: `${entity.path}` => `$target`.''',
);
}
}
final relative = p.relative(entity.path, from: root);
if (Platform.isWindows) {
return p.posix.joinAll(p.split(relative));
Expand Down Expand Up @@ -367,6 +402,10 @@ See $workspacesDocUrl for more information.
isDir: (dir) => dirExists(resolve(dir)),
includeDirs: includeDirs,
).map(resolve).toList();
for (final f in result) {
verifyLink(f);
}
return result;
}

/// Applies [transform] to each package in the workspace and returns a derived
Expand Down
17 changes: 9 additions & 8 deletions lib/src/validator/gitignore.dart
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,11 @@ class GitignoreValidator extends Validator {
final unignoredByGitignore = Ignore.listFiles(
beneath: beneath,
listDir: (dir) {
final contents = Directory(resolve(dir)).listSync();
return contents
.where((e) => !(linkExists(e.path) && dirExists(e.path)))
.map(
(entity) => p.posix
.joinAll(p.split(p.relative(entity.path, from: root))),
);
final contents = Directory(resolve(dir)).listSync(followLinks: false);
return contents.map(
(entity) =>
p.posix.joinAll(p.split(p.relative(entity.path, from: root))),
);
},
ignoreForDir: (dir) {
final gitIgnore = resolve('$dir/.gitignore');
Expand All @@ -86,7 +84,10 @@ class GitignoreValidator extends Validator {
];
return rules.isEmpty ? null : Ignore(rules);
},
isDir: (dir) => dirExists(resolve(dir)),
isDir: (dir) {
final resolved = resolve(dir);
return dirExists(resolved) && !linkExists(resolved);
},
).map((file) {
final relative = p.relative(resolve(file), from: package.dir);
return Platform.isWindows
Expand Down
20 changes: 10 additions & 10 deletions pubspec.lock
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,23 @@ packages:
dependency: transitive
description:
name: _fe_analyzer_shared
sha256: c57b02f47e021c9d7ced6d2e28824b315e0fd585578274bc4c2a5db0626f154a
sha256: "45cfa8471b89fb6643fe9bf51bd7931a76b8f5ec2d65de4fb176dba8d4f22c77"
url: "https://pub.dev"
source: hosted
version: "75.0.0"
version: "73.0.0"
_macros:
dependency: transitive
description: dart
source: sdk
version: "0.3.3"
version: "0.3.2"
analyzer:
dependency: "direct main"
description:
name: analyzer
sha256: ef226c581b7cd875f734125b1b9928df3db08cc85ff87ce7d9be89a677aaee23
sha256: "4959fec185fe70cce007c57e9ab6983101dbe593d2bf8bbfb4453aaec0cf470a"
url: "https://pub.dev"
source: hosted
version: "6.10.0"
version: "6.8.0"
args:
dependency: "direct main"
description:
Expand Down Expand Up @@ -178,10 +178,10 @@ packages:
dependency: transitive
description:
name: lints
sha256: "4a16b3f03741e1252fda5de3ce712666d010ba2122f8e912c94f9f7b90e1a4c3"
sha256: "3315600f3fb3b135be672bf4a178c55f274bebe368325ae18462c89ac1e3b413"
url: "https://pub.dev"
source: hosted
version: "5.1.0"
version: "5.0.0"
logging:
dependency: transitive
description:
Expand All @@ -194,10 +194,10 @@ packages:
dependency: transitive
description:
name: macros
sha256: "1d9e801cd66f7ea3663c45fc708450db1fa57f988142c64289142c9b7ee80656"
sha256: "0acaed5d6b7eab89f63350bccd82119e6c602df0f391260d0e32b5e23db79536"
url: "https://pub.dev"
source: hosted
version: "0.1.3-main.0"
version: "0.1.2-main.4"
matcher:
dependency: transitive
description:
Expand Down Expand Up @@ -479,4 +479,4 @@ packages:
source: hosted
version: "2.2.1"
sdks:
dart: ">=3.6.0-0 <4.0.0"
dart: ">=3.5.0 <4.0.0"
5 changes: 5 additions & 0 deletions test/descriptor.dart
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import 'package:pub/src/sdk/sdk_package_config.dart';
import 'package:test_descriptor/test_descriptor.dart';

import 'descriptor/git.dart';
import 'descriptor/link_descriptor.dart';
import 'descriptor/package_config.dart';
import 'descriptor/tar.dart';
import 'descriptor/yaml.dart';
Expand Down Expand Up @@ -403,3 +404,7 @@ Descriptor flutterVersion(String version) {
FileDescriptor sdkPackagesConfig(SdkPackageConfig sdkPackageConfig) {
return YamlDescriptor('sdk_packages.yaml', yaml(sdkPackageConfig.toMap()));
}

Descriptor link(String name, String target, {bool forceDirectory = false}) {
return LinkDescriptor(name, target, forceDirectory: forceDirectory);
}
52 changes: 52 additions & 0 deletions test/descriptor/link_descriptor.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:io';

import 'package:path/path.dart' as p;
import 'package:test/test.dart';

import '../descriptor.dart' as d;

/// Describes a symlink.
class LinkDescriptor extends d.Descriptor {
/// On windows symlinks to directories are distinct from symlinks to files.
final bool forceDirectory;
final String target;
LinkDescriptor(super.name, this.target, {this.forceDirectory = false});

@override
Future<void> create([String? parent]) async {
final path = p.join(parent ?? d.sandbox, name);
if (forceDirectory) {
if (Platform.isWindows) {
Process.runSync('cmd', ['/c', 'mklink', '/D', path, target]);
} else {
Link(path).createSync(target);
}
} else {
Link(path).createSync(target);
}
}

@override
String describe() {
return 'symlink at $name targeting $target';
}

@override
Future<void> validate([String? parent]) async {
final link = Link(p.join(parent ?? d.sandbox, name));
try {
final actualTarget = link.targetSync();
expect(
actualTarget,
target,
reason: 'Link doesn\'t point where expected.',
);
} on FileSystemException catch (e) {
fail('Could not read link at $name $e');
}
}
}
80 changes: 80 additions & 0 deletions test/lish/symlinks_test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:io';

import 'package:path/path.dart' as p;
import 'package:tar/tar.dart';
import 'package:test/test.dart';

import '../descriptor.dart' as d;
import '../test_pub.dart';

Future<void> main() async {
test('symlink directories are replaced by their targets', () async {
await d.validPackage().create();
await d.dir('a', [d.file('aa', 'aaa')]).create();
await d.file('t', 'ttt').create();

await d.dir(appPath, [
d.dir('b', [d.file('bb', 'bbb'), d.link('l', p.join(d.sandbox, 't'))]),
d.link(
'symlink_to_dir_outside_package',
p.join(d.sandbox, 'a'),
forceDirectory: true,
),
d.link(
'symlink_to_dir_outside_package_relative',
p.join('..', 'a'),
forceDirectory: true,
),
d.link(
'symlink_to_dir_inside_package',
p.join(d.sandbox, appPath, 'b'),
forceDirectory: true,
),
d.link(
'symlink_to_dir_inside_package_relative',
'b',
forceDirectory: true,
),
]).create();

await runPub(args: ['publish', '--to-archive=archive.tar.gz']);

final reader = TarReader(
File(p.join(d.sandbox, appPath, 'archive.tar.gz'))
.openRead()
.transform(GZipCodec().decoder),
);

while (await reader.moveNext()) {
final current = reader.current;
expect(current.type, isNot(TypeFlag.symlink));
}

await runPub(args: ['cache', 'preload', 'archive.tar.gz']);

await d.dir('test_pkg-1.0.0', [
...d.validPackage().contents,
d.dir('symlink_to_dir_outside_package', [
d.file('aa', 'aaa'),
]),
d.dir('symlink_to_dir_outside_package_relative', [
d.file('aa', 'aaa'),
]),
d.dir('b', [d.file('bb', 'bbb')]),
d.dir('symlink_to_dir_inside_package', [
d.file('bb', 'bbb'),
d.file('l', 'ttt'),
]),
d.dir('symlink_to_dir_inside_package_relative', [
d.file('bb', 'bbb'),
d.file('l', 'ttt'),
]),
]).validate(
p.join(d.sandbox, cachePath, 'hosted', 'pub.dev'),
);
});
}
Loading