nimbo-config.ymlin the current directory. You can choose a different path by setting the
project/data/datasetsfolder, you should set
local_datasets_pathbut for the results folder. This will typically correspond to the folder where you write outputs of experiments, logs, models, checkpoints, etc.
nimbo push datasetscommand.
s3_datasets_pathbut for results. This is the remote folder where the outputs of your job and Nimbo logs will be stored. After and during a job (every 5 seconds), Nimbo backs-up your results to this S3 folder, which you can sync into your computer (at
local_results_path) by using
nimbo pull resultsor
nimbo pull logs.
p3.xlarge, etc. Any instance type that exists in your region is supported (not limited to GPU instances).
spot: no, an on-demand instance is used. Specs and prices for on-demand GPU instances can be easily checked with the command
nimbo list-gpu-prices. If
spot: yes, a spot instance is used (assuming your account has the necessary permissions and quotas). Specs and prices for spot GPU instances can be checked with the command
image: ubuntu18-latest-drivers. You can find more details in the Manged Images section.
conda_env: environment.yml. Note: This environment will be used only in the remote instance. You do not need to use it to launch Nimbo. You can install and run
nimbofrom any local Conda/pip/venv environment you want.
run_in_background: yes, the job will run on AWS without any process running on your computer (you only need to keep an internet connection while the instance is being setup and code/environment files are synced). This option is useful if you are confident a job will run correctly and want to be able to turn off your internet/computer while jobs are running, or to run many jobs are once.
runinbackground: no, the job logs will be streamed to your terminal session. Clicking
ctrl-cat any point will store any results/logs produced so far and shut-down the instance depending on the value of the
persistfield (see below).
s3_result_spath/nimbo-logs/, which can be synced locally using
nimbo pull logs.
persist: no, the instance will be automatically terminated if there is an error at any point during instance setup or job running, or when the job finished successfully.
persist: yes, the instance will remain active in those cases. This is useful if you want to be able to log onto the instance (using
nimbo ssh <instance-id>) to perform some debugging if there's an error. If you are confident the jobs will run correctly and if you are using
run_in_background: yes, we recommend using
persist: no, to avoid more EC2 billing than necessary.
defaultsecurity group. Make sure the security group used has the necessary inbound address permissions to allow SSH from your computer into the instance.
example-keyin your EC2 dashboard, you should set
run_jobwas executed per AWS user. This is used by us to gauge how widely Nimbo is used. By default telemetry is set to