python>=3.6. Only Linux/MacOS are supported at the moment.
~/.aws/credentialsor generate any necessary temporary tokens as per AWS or your admin's instructions.
~/.aws/credentials) you can skip this step. Otherwise, get your AWS credentials from the AWS dashboard, from this page. Alternatively, if you have an IAM account and you are not the administrator, you can ask your administrator for further help. You can find more information on AWS credentials here.
~/.aws/credentialsfile should look something like this:
[my-profile]is not present or the line is
[default]instead, leave the
aws_profilefield in the Nimbo config file as
default. Otherwise set
aws_profileto your chosen AWS profile name, in this case
NimboUserGroupthat gives minimal EC2 access and full S3 access to users in that group. It also creates an instance role
NimboFullS3AccessRole(used in the config file below), which gives your instances full access to your S3 buckets. If you are an admin and don't want to give full S3 access to a user group and/or instance role, please see the Advanced setup tab above.
NimboUserGroup, simply run:
NimboUserGroupthat gives minimal EC2 access and no S3 access to users in that group. Make sure to create a role/instance profile that allows the instance to access the necessary S3 buckets. For example, if you create a role called
SpecialProjectRolefor users in your AWS org to use with Nimbo, tell them to set
roleparameter in their
NimboUserGroup, simply run:
Pinkmanto use the
SpecialProjectRole, you can add the following policy to the user:
nimbo-config.ymlfile contains all the information needed for running jobs with Nimbo. In the config file you can specify data, access, instance, and job details for tailoring your experience. You can create a default config file with the command:
nimbo-config.ymlin current directory.This file contains several default parameters you should customise. For more info on what each parameter does, read the Nimbo config options page.
your-ec2-key-pair.pemfile. Place this file in your project's folder, run
chmod 400 your-ec2-key-pair.pem, and set the
instance_keyin the Nimbo config file to the name of your key pair.
nimbo-config.ymlfile, and skip to the “Test your setup” section.
local_datasets_paththat you want to use in your jobs, you can upload them from your local computer to S3 using:
s3_datasets_pathas set in your Nimbo config file. If you want to delete some content, you can delete the files locally and then run
nimbo push datasets --delete, which will delete any files that don't exist in the source folder.
s3_datasets_pathto the instance's
local_datasets_path, mimicking your local project's structure. If the datasets are correctly loaded when you run the job on your computer, it should run correctly on the instance.