Nimbo
Search…
What is Nimbo?
Nimbo is a dead-simple wrapper for the AWS CLI that allows you to run jobs on AWS with a single command. All you need is a tiny config file, and Nimbo handles all the instance, environment, data, and IAM management. It also provides many useful commands to make it easier than ever to work with AWS from your terminal.
Since Nimbo is independent of your code, you can run any type of job you want - train a neural network, run quantum computing simulations, or model biological processes.
Read the Getting Started, Running Jobs and Examples sections for a complete guide on how to set up and use Nimbo. We give a brief overview below.

Quick overview

Let's say you have a local script to train a neural network that you usually run like this:
1
python train-mnist.py --epochs=10 --lr=3e-4
Copied!
With Nimbo, you can run it on an EC2 instance on your AWS account by simply running:
1
nimbo run "python -u train-mnist.py --epochs=10 --lr=3e-4"
Copied!
In order to run this job on Nimbo, all you need is one tiny config file and a Conda environment file (to set the remote environment), and Nimbo does the following for you:
  • launch your instance (on-demand or spot instance)
  • setup your environment
  • sync your code
  • pull your datasets from S3
  • run the job
  • when the job is done
    • save the results and logs back to S3
    • delete the instance
    You can then do nimbo pull results or nimbo pull logs to get the job's results/nimbo-logs onto your computer.
And of course, no changes to your codebase are needed.
A minimal Nimbo configuration file looks like this:
1
$ cat nimbo-config.yml
2
3
cloud_provider: AWS
4
5
# Data paths
6
local_datasets_path: your-datasets-folder
7
local_results_path: your-results-folder
8
s3_datasets_path: s3://your-bucket/your-project/example-datasets-folder
9
s3_results_path: s3://your-bucket/your-project/example-results-folder
10
11
# Device, environment and regions
12
aws_profile: your-aws-profile
13
region_name: eu-west-1
14
instance_type: p2.xlarge
15
spot: no # to use a spot instance simply use 'yes' here
16
17
image: ubuntu18-latest-drivers
18
disk_size: 128 # in GB
19
conda_env: your-env-file.yml
20
21
# Permissions and credentials
22
security-group: default
23
instance_key: your-ec2-key-pair.pem
24
role: NimboUserGroup
Copied!
To make sure the system-specific dependencies are correctly resolved, you need to have a manually created Conda environment file. For example, if your code uses Tensorflow 2.2.0, it should look something like this:
1
$ cat my-env-file.yml
2
3
name: tensorflow2.2.0
4
channels:
5
- defaults
6
dependencies:
7
- python=3.6
8
- pip
9
- tensorflow-gpu=2.2.0
10
- pip:
11
- requests
Copied!
Once you have these two files, you are ready to roll. Now you can execute nimbo run command above in your command line.
Read the Getting Started and Running Jobs sections for a complete guide on how to set up your AWS permissions and credentials to use Nimbo.
Last modified 5mo ago
Copy link