Skip to content

Commit

Permalink
Merge pull request #7815 from shirady/nsfs-nc-update-doc
Browse files Browse the repository at this point in the history 
NSFS | NC | Update Docs Related to Last Changes in Manage NSFS
  • Loading branch information
@shirady
shirady authored Feb 14, 2024
2 parents cada310 + 697b7ec commit 075a950
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 36 deletions.
38 changes: 21 additions & 17 deletions docs/design/NonContainerizedNSFSDesign.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ node src/cmd/nsfs ../standalon/nsfs_root --config_dir ../standalon/fs_config

---

## Compments
## Components

### 1. Accounts
### 1. Account

- Account will have a directory and in that, it will map 1 file per account, and the file name will be {account_name}.json
- Account will have a directory and in that, it will map 1 file per account
- Account file name will be {account_name}.json
- Secrets will be saved as plain text
- Note: We had the option to save it as plain text or Encrypted strings.
- JSON Schema - `src/server/object_services/schemas/nsfs_account_schema.js`.
- JSON Schema - `src/server/system_services/schemas/nsfs_account_schema.js`.
- Note: It is based on $ref: account_api#/definitions/account_info.
- `new_buckets_path` in the account schema will be used to populate the namespace store (read_resources, write_resource).
- `BucketSpace` interface is updated with the new method `read_account_by_access_key()`
Expand All @@ -27,7 +28,7 @@ node src/cmd/nsfs ../standalon/nsfs_root --config_dir ../standalon/fs_config
```json
{
"name": "user1",
"email": "user1@noobaa.io", // this just an example - the email will be internally (the account name), email will not be set by user
"email": "user1", // the email will be internally (the account name), email will not be set by user
"creation_date": "2024-01-11T08:24:14.937Z",
"access_keys": [
{
Expand All @@ -40,28 +41,31 @@ node src/cmd/nsfs ../standalon/nsfs_root --config_dir ../standalon/fs_config
"gid": 1001, //
"new_buckets_path": "/",
},
"allow_bucket_creation": true
"allow_bucket_creation": true,
"_id": "65cb1e7c9e6ae40d499c0ae3" // _id automatically generated
}
```

### 2. Bucket

- Bucket will have a directory and in that, it will map 1 file per Bucket
- bucket files name will be same as bucket name; eg: {bucket_name}.json
- Bucket will have a directory and in that, it will map 1 file per bucket
- Bucket file name will be {bucket_name}.json
- Bucket schema should have a name, bucket_owner, path, etc. The are optional properties that can be added like bucket policy.
- JSON Schema - src/server/object_services/schemas/nsfs_bucket_schema.js
- JSON Schema - `src/server/system_services/schemas/nsfs_bucket_schema.js`.
- It is based on $ref: 'bucket_api#/definitions/bucket_info'
- `BucketSpace`` interface is updated with the new method `read_bucket_sdk_info()`
- `BucketSpace` interface is updated with the new method `read_bucket_sdk_info()`
- Method will read bucket schema referring to bucket name and return the bucket to `bucket_namespace_cache`.

```json
{
"name": "mybucke-test1",
"system_owner": "user1@noobaa.io",
"bucket_owner": "user1@noobaa.io",
"creation_date": "2024-01-11T08:26:16.9731",
"_id": "65cb1efcbec92b33220112d6",
"name": "mybucket-test1",
"owner_account": "65cb1e7c9e6ae40d499c0ae3",
"system_owner": "user1",
"bucket_owner": "user1",
"versioning": "DISABLED",
"path": "mybucke-test1",
"creation_date": "2024-01-11T08:26:16.9731",
"path": "mybucket-test1",
"should_create_underlying_storage": true,
"s3_policy": {
"Version": "2012-10-17",
Expand All @@ -77,8 +81,8 @@ node src/cmd/nsfs ../standalon/nsfs_root --config_dir ../standalon/fs_config
"s3:*"
],
"Resource": [
"arn:aws:s3:::mybucke-test1/*",
"arn:aws:s3:::mybucke-test1"
"arn:aws:s3:::mybucket-test1/*",
"arn:aws:s3:::mybucket-test1"
]
}
]
Expand Down
40 changes: 21 additions & 19 deletions docs/non_containerized_NSFS.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ make_bucket: s3bucket
cat /tmp/noobaa_config_dir/buckets/s3bucket.json

Output -
{"name":"s3bucket","tag":"","system_owner":"account1@noobaa.io","bucket_owner":"account1@noobaa.io","versioning":"DISABLED","path":"/tmp/fs1/s3bucket","should_create_underlying_storage":true,"creation_date":"2023-09-26T05:56:16.252Z"}
{"_id":"65cb1efcbec92b33220112d7","name":"s3bucket","owner_account":"65cb1e7c9e6ae40d499c0ae4","system_owner":"account1","bucket_owner":"account1","versioning":"DISABLED","creation_date":"2023-09-26T05:56:16.252Z","path":"/tmp/fs1/s3bucket","should_create_underlying_storage":true}
```

4.3. Check that the file system bucket directory was created successfully -
Expand Down Expand Up @@ -204,7 +204,7 @@ echo "This is the content of object1" | s3-account1 cp - s3://s3bucket/object1.
6.2. Check the object was created on the file system -

```sh
cat /tmp/fs1/s3bucket/object1.txt
sudo cat /tmp/fs1/s3bucket/object1.txt

Output -
This is the content of object1
Expand Down Expand Up @@ -431,41 +431,43 @@ These error codes will get attached with a specific Bucket or Account schema ins
- Check for schema config file in respective Accounts or Buckets dir.

## Bucket and Account Manage CLI
Users can create, update, delete, and list buckets and accounts using CLI. If the config directory is missing CLI will create one and also create accounts and buckets sub-directories in it and default config directory is `${config.NSFS_NC_DEFAULT_CONF_DIR}`.
Users can create, get status, update, delete, and list buckets and accounts using CLI. If the config directory is missing CLI will create one and also create accounts and buckets sub-directories in it and default config directory is `${config.NSFS_NC_DEFAULT_CONF_DIR}`.

CLI will never create or delete a bucket directory for the user if a bucket directory is missing CLI will return with error.

NOTES -
1. manage_nsfs execution requires root permissions.
2. While path specifies a GPFS path, it's recommended to add fs_backend=GPFS in order to increase performance by ordering NooBaa to use GPFS library.

Bucket Commands
```
sudo node src/cmd/manage_nsfs bucket add --config_root ../standalon/config_root --name bucket1 --owner noobaa --path ../standalon/nsfs_root/1 --fs_backend GPFS 2>/dev/null
sudo node src/cmd/manage_nsfs bucket update --config_root ../standalon/config_root --name bucket1 --owner noobaa --fs_backend GPFS 2>/dev/null
sudo node src/cmd/manage_nsfs bucket list --config_root ../standalon/config_root 2>/dev/null
sudo node src/cmd/manage_nsfs bucket delete --config_root ../standalon/config_root --name bucket1 2>/dev/null
```

Account Commands
```
sudo node src/cmd/manage_nsfs account add --config_root ../standalon/config_root --name noobaa --new_buckets_path ../standalon/nsfs_root/ --fs_backend GPFS 2>/dev/null
sudo node src/cmd/manage_nsfs account update --config_root ../standalon/config_root --name noobaa --fs_backend GPFS 2>/dev/null
sudo node src/cmd/manage_nsfs account delete --config_root ../standalon/config_root --access_key abc 2>/dev/null
sudo node src/cmd/manage_nsfs account status --config_root ../standalon/config_root --name noobaa 2>/dev/null
sudo node src/cmd/manage_nsfs account delete --config_root ../standalon/config_root --name noobaa 2>/dev/null
sudo node src/cmd/manage_nsfs account list --config_root ../standalon/config_root 2>/dev/null
```

`Important`: All the Account/Bucket commands end with `2>/dev/null` to make sure there are no unwanted logs.
Bucket Commands
Note: before creating a bucket, a user must create an account.
```
sudo node src/cmd/manage_nsfs bucket add --config_root ../standalon/config_root --name bucket1 --owner noobaa --path ../standalon/nsfs_root/1 --fs_backend GPFS 2>/dev/null
sudo node src/cmd/manage_nsfs bucket update --config_root ../standalon/config_root --name bucket1 --owner noobaa --fs_backend GPFS 2>/dev/null
sudo node src/cmd/manage_nsfs bucket status --config_root ../standalon/config_root --name bucket1 2>/dev/null
sudo node src/cmd/manage_nsfs bucket delete --config_root ../standalon/config_root --name bucket1 2>/dev/null
sudo node src/cmd/manage_nsfs bucket list --config_root ../standalon/config_root 2>/dev/null
Users can also pass account and bucket/account values in JSON file instead of passing them in cli as arguments.
```
**Important**: All the Account/Bucket commands end with `2>/dev/null` to make sure there are no unwanted logs.


```
Expand All @@ -484,7 +486,7 @@ Non containerized NSFS restrict insecure HTTP connections when `ALLOW_HTTP` is s
### Setting Up Self signed SSL/TLS Certificates for Secure Communication Between S3 Client and NooBaa NSFS Service -

#### 1. Creating a SAN (Subject Alternative Name) Config File -
`Important`: This step is needed only if S3 Client and NooBaa Service Running on different nodes.
**Important**: This step is needed only if S3 Client and NooBaa Service Running on different nodes.

To accommodate S3 requests originating from a different node than the one running the NooBaa NSFS service, it is recommended to create a Subject Alternative Name (SAN) configuration file. <br />
This file specifies the domain names or IP addresses that will be included in the SSL certificate.<br />
Expand Down

0 comments on commit 075a950

Please sign in to comment.