Skip to content

Commit

Permalink
docs: more pre-release edits
Browse files Browse the repository at this point in the history 
Signed-off-by: Alex Aizman <alex.aizman@gmail.com>
  • Loading branch information
@alex-aizman
alex-aizman committed Sep 27, 2024
1 parent 84d11a3 commit 7e4e03a
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 21 deletions.
4 changes: 2 additions & 2 deletions docs/changelog/v3.24.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -786,7 +786,7 @@ Version 3.24 arrives nearly 4 months after the previous one and contains more th
| - github-CI python
| - tests: skip ec-destroy-bucket
| - fs-path-error: empty path is now treated as not
* "user-friendly "d[id you mean" message" d015ea5e6](https://github.com/NVIDIA/aistore/commit/id)
* "user-friendly "did you mean" message" [d015ea5e6](https://github.com/NVIDIA/aistore/commit/d015ea5e6)
| * erroneous 'ais show gs://abc[/object]' will now produce the right hint
| * up cli
* "fix rendering issues for EC xactions output" [94578e9f0](https://github.com/NVIDIA/aistore/commit/94578e9f0)
Expand Down Expand Up @@ -842,7 +842,7 @@ Version 3.24 arrives nearly 4 months after the previous one and contains more th
| * do not build disk metric names at runtime
| * CLI: skip internal (`lcache`, `stream`) counters unless verbose
| * CLI: version check vs. nodes in maintenance
* "support per-backend cumulative ("t[otal") latencies" 988051c28](https://github.com/NVIDIA/aistore/commit/otal)
* "support per-backend cumulative ["total" latencies](https://github.com/NVIDIA/aistore/commit/988051c28)
| * revise 'ais performance latency'
| * use `.total.` latencies and their respective counters
| * related commit: 5fd101589c7
Expand Down
31 changes: 17 additions & 14 deletions docs/cli/bucket.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -183,13 +183,15 @@ NAME:
* ais ls s3 - list all s3 buckets that are present in the cluster;
* ais ls s3 --all - list all s3 buckets, both present and remote;
with template, regex, and/or prefix:
* ais ls gs: --regex "^abc" --all - list all accessible GCP buckets with names starting with "abc";
* ais ls ais://abc --regex ".md" --props size,checksum - list *.md objects with their respective sizes and checksums;
* ais ls gs://abc --template images/ - list all objects from the virtual subdirectory called "images";
* ais ls gs://abc --prefix images/ - same as above (for more examples, see '--template' below);
* ais ls gs: --regex "^abc" --all - list all accessible GCP buckets with names starting with "abc";
* ais ls ais://abc --regex ".md" --props size,checksum - list *.md objects with their respective sizes and checksums;
* ais ls gs://abc --template images/ - list all objects from the virtual subdirectory called "images";
* ais ls gs://abc --prefix images/ - same as above (for more examples, see '--template' below);
* ais ls gs://abc/images/ - same as above.
and summary (stats):
* ais ls s3 --summary - for each s3 bucket in the cluster: print object numbers and total size(s);
* ais ls s3 --summary --all - generate summary report for all s3 buckets; include remote objects and buckets that are _not present_
* ais ls s3 --summary - for each s3 bucket in the cluster: print object numbers and total size(s);
* ais ls s3 --summary --all - generate summary report for all s3 buckets; include remote objects and buckets that are _not present_;
* ais ls s3 --summary --all --dont-add - same as above but without adding _non-present_ remote buckets to cluster's BMD.
```

## Assorted options
Expand Down Expand Up @@ -274,19 +276,20 @@ NAME:
* ais ls s3 - list all s3 buckets that are present in the cluster;
* ais ls s3 --all - list all s3 buckets, both in-cluster and remote;
with template, regex, and/or prefix:
* ais ls gs: --regex "^abc" --all - list all accessible GCP buckets with names starting with "abc";
* ais ls ais://abc --regex ".md" --props size,checksum - list *.md objects with their respective sizes and checksums;
* ais ls gs://abc --template images/ - list all objects from the virtual subdirectory called "images";
* ais ls gs://abc --prefix images/ - same as above (for more examples, see '--template' below);
* ais ls gs: --regex "^abc" --all - list all accessible GCP buckets with names starting with "abc";
* ais ls ais://abc --regex ".md" --props size,checksum - list *.md objects with their respective sizes and checksums;
* ais ls gs://abc --template images/ - list all objects from the virtual subdirectory called "images";
* ais ls gs://abc --prefix images/ - same as above (for more examples, see '--template' below);
* ais ls gs://abc/images/ - same as above.
with in-cluster vs remote content comparison (diff):
* ais ls s3://abc --check-versions - for each remote object in s3://abc: check whether it has identical in-cluster copy
and show missing objects
and show missing objects;
* ais ls s3://abc --check-versions --cached - for each in-cluster object in s3://abc: check whether it has identical remote copy
and show deleted objects
and show deleted objects.
with summary (stats):
* ais ls s3 --summary - for each s3 bucket in the cluster: print object numbers and total size(s);
* ais ls s3 --summary --all - generate summary report for all s3 buckets; include remote objects and buckets that are _not present_
* ais ls s3 --summary --all --dont-add - same as above but without adding _non-present_ remote buckets to cluster's BMD
* ais ls s3 --summary --all - generate summary report for all s3 buckets; include remote objects and buckets that are _not present_;
* ais ls s3 --summary --all --dont-add - same as above but without adding _non-present_ remote buckets to cluster's BMD.

USAGE:
ais ls [command options] PROVIDER:[//BUCKET_NAME]
Expand Down
16 changes: 11 additions & 5 deletions docs/howto_virt_dirs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

Unlike hierarchical POSIX, object storage is flat, treating forward slash ('/') in object names as simply another symbol.

But that's not the entire truth. The other part of it is that user may want to operate (list, train, copy, transform, etc.)
on a subset of objects in a dataset that, for the lack of better word, looks _exactly_ like a directory.
But that's not the entire truth. The other part of it is that user may want to operate on (ie., list, load, shuffle, copy, transform, etc.)
subset of objects in a dataset that, for the lack of better word, looks _exactly_ like a directory.

In fact, user often wants to do exactly that.
Train, for instance, on all audio files under `en_es_synthetic/v1/train/`, or similar.
Expand All @@ -27,7 +27,7 @@ Needless to say, aistore provides for all of that and more. There is a certainty

## Examples

### Show everything that has a certain prefix
### 1. Show everything that has a certain prefix

```console
$ ais ls s3://speech --prefix .inventory
Expand All @@ -41,7 +41,13 @@ NAME SIZE
.inventory/speech/hive/dt=2024-05-31-01-00/symlink.txt 85B no
```

### Same as above, without directories
### 2. Same as above using familiar `*nix` notation

```console
$ ais ls s3://speech/.inventory
```

### 3. Same as above, without directories

```console
$ ais ls s3://speech --prefix .inventory --no-dirs
Expand All @@ -54,7 +60,7 @@ NAME SIZE
.inventory/speech/hive/dt=2024-05-31-01-00/symlink.txt 85B no
```

### Show dataset structure at a certain nested depth
### 4. Show dataset structure at a certain nested depth

```console
$ ais ls s3://speech --prefix .inventory/speech/ --nr
Expand Down

0 comments on commit 7e4e03a

Please sign in to comment.