Nimbo
Search…
Useful commands
Here is a short list of the most common commands. To see all the commands from the terminal use nimbo --help or nimbo <command> --help.

nimbo run

Runs a command on an AWS instance, according to the parameters defined in the nimbo-config.yml file. This command does the following:
  • launch your instance (on-demand or spot instance)
  • setup your environment
  • sync your code
  • pull your datasets from S3
  • run the job
  • when the job is done
    • save the results and logs back to S3 (this also happens during the job and on error)
    • delete the instance
    You can then do nimbo pull results to get the job's results into your computer, or nimbo pull logs to just see the run logs. The latter option is useful when there's an error, as the logs will record all standard output.

nimbo notebook

Launches jupyter lab on an AWS instance, with the same setup as nimbo run.
The notebook becomes available in your localhost at http://localhost:57467/lab. The data written to the results folder is synced to S3 in real-time (according to the local and S3 results paths on your nimbo-config.yml), so you can import the results to your local machine with nimbo pull results.
Make sure to run nimbo sync-notebooks <instance-id> frequently to sync the notebooks in your instances to your local folder, as the remote notebooks will be lost once the instance is terminated.

nimbo spending <quantity> days/months

Shows EC2 and S3 billing in the past <quantity> days or months (before credits or refunds applied).
1
$ nimbo spending 4 months
2
3
Spending for region eu-west-1:
4
5
EC2 S3
6
Jan 2021 0 0
7
Feb 2021 0.32 0.01
8
Mar 2021 75.44 0.21
9
Apr 2021 87.81 0.11
10
May 2021 3.42 0.00
11
--------------------------------
12
Total 166.99 0.33
Copied!

nimbo ls-prices

Shows a list of prices and specs for the GPU-enabled instances in your region. The command nimbo ls-spot-prices is also available for spot instances.
1
$ nimbo ls-prices
2
3
InstanceType Price ($/hour) GPUs CPUs Mem (Gb)
4
g4dn.12xlarge 4.36 4 x T4 48 192
5
g4dn.16xlarge 4.85 1 x T4 64 256
6
g4dn.2xlarge 0.84 1 x T4 8 32
7
g4dn.4xlarge 1.34 1 x T4 16 64
8
g4dn.8xlarge 2.43 1 x T4 32 128
9
g4dn.metal 8.72 8 x T4 96 384
10
g4dn.xlarge 0.59 1 x T4 4 16
11
p2.16xlarge 15.55 16 x K80 64 732
12
p2.8xlarge 7.78 8 x K80 32 488
13
p2.xlarge 0.97 1 x K80 4 61
14
p3.16xlarge 26.44 8 x V100 64 488
15
p3.2xlarge 3.31 1 x V100 8 61
16
p3.8xlarge 13.22 4 x V100 32 244
17
p3dn.24xlarge 33.71 8 x V100 96 768
18
p4d.24xlarge 35.4 8 x A100 96 1152
Copied!

nimbo push datasets/results/logs

Syncs the data in your S3 datasets or results folder with your local datasets or results folder.
1
$ cat nimbo-config.yml
2
...
3
local_datasets_path=data/datasets
4
S3_datasets_path=s3://my-bucket/data/datasets
5
...
6
7
$ nimbo push datasets
8
Running command: aws s3 sync data/datasets s3://my-bucket/data/datasets --profile my_aws_profile --delete
9
upload: data/datasets/MNIST/raw/t10k-images.gz to s3://my-bucket/data/datasets/MNIST/raw/t10k-images.gz
10
upload: data/datasets/MNIST/raw/t10k-labels-idx1-ubyte.gz to s3://my-bucket/data/datasets/MNIST/raw/t10k-labels-idx1-ubyte.gz
11
upload: data/datasets/MNIST/raw/train-labels.gz to s3://my-bucket/data/datasets/MNIST/raw/train-labels.gz
12
upload: data/datasets/MNIST/raw/train-images.gz to s3://my-bucket/data/datasets/MNIST/raw/train-images.gz
Copied!
If you want to delete remote files, you can delete them locally and then use nimbo push datasets/results/logs --delete, to delete any files that no longer exist locally.

nimbo pull datasets/results/logs

Syncs the data in your local datasets or results folder with your S3 datasets or results folder.
1
$ nimbo pull results
2
3
4
Running command: aws s3 sync s3://my-bucket/data/results data/results --profile my_aws_profile --delete
5
6
download: s3://my-bucket/data/results/model.ckpt to data/results/model.ckpt
7
download: s3://my-bucket/data/results/eval.json to data/results/eval.json
8
download: s3://my-bucket/data/results/logs.txt to data/results/logs.txt
Copied!

nimbo ls-active

Show a summarized list of your running instances.
1
$ nimbo ls-active
2
3
ID: i-0be1989edd819b442
4
Launch Time: 2021-03-25 22:27:37+00:00
5
InstanceType: g4dn.2xlarge
6
Public IP: 3-249-8-237
7
8
ID: i-03d8d97e0850743a2
9
Launch Time: 2021-03-25 22:29:36+00:00
10
InstanceType: p2.xlarge
11
Public IP: 34-244-47-53
Copied!

nimbo ssh <instance_id>

SSH into an instance by instance ID.
1
$ nimbo ssh i-0be1989edd819b442
Copied!

nimbo rm-instance <instance_id>

nimbo rm-all-instances is also available.
1
$ nimbo rm-all-instances
2
3
Instance i-0be1989edd819b442: shutting-down
4
Instance i-03d8d97e0850743a2: shutting-down
Copied!

nimbo launch-and-setup

This command does the same steps as nimbo run , but does not run any job, and keeps the instance alive. This option is useful for ssh'ing into the instance to do your own debugging or run any code by hand directly from within the instance.
1
$ nimbo launch-and-setup
2
3
Job command: _nimbo_launch_and_setup
4
Launching instance... Instance running. (19.05s)
5
InstanceId: i-0c4473b5c6cb5b56a
6
7
Waiting for instance to be ready for ssh at 54.72.144.90... Ready.
8
9
Syncing conda, config, and setup files...
10
...
11
...
12
=================================================
13
14
Setup complete. You can now use 'nimbo ssh i-0c4473b5c6cb5b56a' to ssh into this instance.
Copied!
Last modified 5mo ago