Error creating training and validation datasets

[kibangadas]

While running ex1, I encountered this following error:

This is my 2d.yaml file:

trajectory_dim: 2
dataset_file: /home/kibangadas/AE4353-Y24/data/2D_QUAD_HOVER.npz

batchsize_train: 1024
batchsize_val: 4096
shuffle_train: True
shuffle_val: False
num_workers_train_loader: 0
num_workers_val_loader: 0

model_hidden_dims: 120
model_num_layers: 4

epochs: 2
validate_per_steps: 500
lr: 1.0 # to be tuned

This is my .json file:

{
    "name": "ae4353",
    "build": {
        "dockerfile": "./Dockerfile",
        "context": ".."
    },
    "mounts": [
        // Bind mount data here if necessary e.g.
        "source=/home/kibangadas/ae4353_dataset,target=${containerWorkspaceFolder}/data,type=bind" 
    ],
    "features": {
        "ghcr.io/devcontainers/features/common-utils:2": {
            "installZsh": "true",
            "username": "vscode",
            "userUid": "1000",
            "userGid": "1000",
            "upgradePackages": "true"
        },
        "ghcr.io/devcontainers/features/git:1": {
            "version": "latest",
            "ppa": "false"
        }
    },
    "customizations": {
        // Configure properties specific to VS Code.
        "vscode": {
            "settings": {
                // TODO: change to specific conda env python bin
                "python.defaultInterpreterPath": "/opt/conda/envs/AE4353/bin/python",
                "python.linting.enabled": true,
                "jupyter.widgetScriptSources": [
                    "jsdelivr.com",
                    "unpkg.com"
                ],
                "notebook.defaultFormatter": "ms-python.black-formatter",
                "notebook.formatOnSave.enabled": true,
                "python.terminal.activateEnvironment": true,
                "[python]": {
                    "editor.defaultFormatter": "ms-python.black-formatter",
                    "editor.formatOnSave": true
                }
            },
            "extensions": [
                "ms-python.python",
                "ms-python.vscode-pylance",
                "ms-python.black-formatter",
                "ms-toolsai.jupyter",
                "github.copilot",
                "github.copilot-chat"
            ]
        }
    },
    "hostRequirements": {
        "gpu": "optional"
    },
    // Uncomment the below "runArgs" section if you intend to use a GPU.
     "runArgs": [
         "--gpus",
         "all"
     ],
    "postCreateCommand": "./.devcontainer/postCreateCommand.sh",
    // Use 'forwardPorts' to make a list of ports inside the container available locally.
    // "forwardPorts": [],
    // Use 'postCreateCommand' to run commands after the container is created.
    // "postCreateCommand": "uname -a",
    // Set `remoteUser` to `root` to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
    "remoteUser": "vscode",
    "remoteEnv": {
        "SERVICE_HOST": "host.docker.internal",
        "SERVICE_PORT": "9876"
    }
}

This is proof that the file is in the path I'm giving:

What should I do to fix this? Thanks in advance

[yilun-wu]

You need to make sure the dataset files are located in /home/kibangadas/ae4353_dataset as this is the path which we mount to the devcontainer workspace.

Please move everything from /home/kibangadas/AE4353-Y24/data/ to /home/kibangadas/ae4353_dataset/. After that, you should be able to see the dataset files under /workspaces/AE4353-Y24/data in the devcontainer. That's where the code is looking for dataset files.