# Public subnet details. After that, open a command prompt or terminal and enter the following command: When doing so, the CLI will ask for your access key and secret. username: MyUsername Using the command above, I will be creating the EKS cluster itself and three t3a.small EC2 nodes. # Change to be the subnet region and subnet id. You can now use Helm to finish the setup and get the cluster ready to be used for deploying the cnvrg app or running cnvrg jobs. , "arn:aws:ssm:*:
groups: Before you can complete the installation, you must install and prepare the following dependencies on your local machine: ::: NOTE To create the cluster based on your prepared cluster.yaml file use the following command: Congratulations, you have now set up a working EKS cluster in AWS! However, excluding the lines commented as Required., you can edit the rest of the configuration to your needs. To use eksctl, you will need to create a YAML file that provides the necessary configuration for the cluster. This can be obtained by using the following command: After that, you can scale your nodegroup by using the following command: Be aware that its not possible to downscale the nodegroup or to replace the instance type of the nodes. If you wish to use an existing VPC, add the following section to the YAML with the required details. If you want to do this, youll have to delete the old nodegroup before creating a new one. In this tutorial I will be using eksctl since I found it the easiest tool to work with. This service account will have the cluster-admin role, so it can access pretty much everything. To calculate the maximum amount, you have to apply the following formula: For example, for t3.small there is a maximum of 3 network interfaces and 4 IP addresses per interface. Before you execute this, make sure to go over to the AWS estimate calculator to see what this will cost. Click on that role and copy the ARN of that role. We will use eksctl to create the cluster in AWS. - system:nodes, # This should contain the ARN of the user you want to give access, - userarn: arn:aws:iam::1234567890:user/MyUsername This can be done by executing the following command: Now you can go back to the login screen, select token and paste the bearer token.
Head over to their website and download and run the installer. When estimating the size of your cluster, also take into account that Kubernetes uses rolling updates. The rest of the configuration is already set within the file to the default recommended values for running cnvrg. To create a command, we can use the eksctl create cluster command. Architecture Overview To create an AWS user account with the necessary permissions, first create a new Access Policy in your AWS Console. You can skip this step if you have already logged in to the AWS CLI. On AWS, we can set up a Kubernetes cluster using the Amazons managed Kubernetes service. In addition to ekctl, we also want to install kubectl to work with our Kubernetes cluster. aws-auth.yaml) and add the following: After that, deploy the configuration by using the following command: After that, the users will be able to see the workloads and nodes on the AWS console. Copy the following JSONs and use it to create the new policy: Now, add the policy to the user who will be creating the cluster. There are several ways to set up a Kubernetes cluster on Amazon. By default, only the user who created the cluster has access to it. The name of the role were looking for contains the name of the nodegroup and the name of the cluster, for example eksctl-my-cluster-nodegroup-ng-1234ab-NodeInstanceRole. Once signed in, you should be presented with the Kubernetes dashboard. If not, then make sure that their user has access to view the EKS resources. Now that were completely set up, we can start using eksctl. You could also change the lower and upper bounds of the node pools.
- system:bootstrappers For example, the Kubernetes dashboard can be accessed from this link. After completing the aws configure prompt, youll be able to find a folder called .aws within your home directory, which contains a credentials file. If you do need to rescale your nodegroup, you first have to find out what the name of your nodegroup is. Similar to eksctl, we can install it by using Homebrew: For alternative ways to install kubectl, you can check the Kubernetes documentation. To delete the nodegroup, you can use the following command: After that, you can create a new nodegroup for your existing cluster by using the following command: Once your cluster is created, you can hook up kubectl to your EKS cluster by executing the following command: After that, you should be able to see all running pods by using the following command: If you dont like to use the kubectl command for monitoring your cluster, you can also use the Kubernetes dashboard. Be aware that the amount of pods you can run on a node cannot exceed the amount of IPv4 addresses there are available for that instance. Creating an EKS cluster requires certain permissions within AWS. To deploy it, you first have to install the metrics server. cnvrg can natively run using an EKS cluster hosted on your own AWS account and an EKS cluster can also be used to run all your cnvrg job. Of course, any permission level higher than contributor should also be suitable. This can be done by applying the following configuration: Once thats done, you should be able to verify if its working by using the following command: If the metrics-server is running, you can deploy the Kubernetes dashboard in a similar fashion: Now, to access the Kubernetes dashboard, we have to create a service account. Each of these nodes will have a disk space of 8GB and will allow 11 Kubernetes pods to be deployed on the server. If youre on macOS, the easiest way to install it is by using Homebrew: Alternative ways to install the eksctl can be found within the documentation on AWS. The final thing you have to do is to create a proper credentials file so eksctl can communicate with AWS on your behalf. # In this section, include all the subnets of your AWS VPC.
Add an entry for each region of your VPC. Add an entry for each region of your VPC. Create a User in AWS with the Correct Permissions, Prerequisites: Prepare Your Local Environment, Use the AWS CLI to Connect to your Account, additional setup is required to configure the VPC correctly, Create a EKS cluster using eksctl and the AWS CLI. Over there, you can generate a new access key and secret (or revoke the ones you dont need). Follow the example format below. To find out what their user ARN is, log into the AWS console, go to IAM and click on the user you want to add. # Private subnet details. For example, you could add new node pools or change the instance types for the node pools. This means that when updating it first tries to create a new (updated) pod, and then it destroys the old pod. To create a cluster, we first have to install eksctl. This allows you to leverage cnvrg and Kubernetes to create one flexible compute and DS environment. This means that the total amount of pods on that node will be limited to 11 (3 x (4 - 1) + 2). To do so, we create a new YAML file (for example dashboard-admin.yaml) and add the following content: If you deploy this configuration on Kubernetes, a new service account called dashboard-admin will be created. The amount of available IP addresses per network interface per instance is listed within the documentation. Please note, additional setup is required to configure the VPC correctly . username: system:node:{{EC2PrivateDNSName}}
- system:masters. To give access to other people, we have to add their Amazon Resource Name or ARN. After that, you can proxy the Kubernetes cluster by using the following command: By doing so, you can now connect to the cluster by going to localhost. Copy the following text into a new file called cluster.yaml and then edit it accordingly: When using the above YAML, eksctl will automatically configure a new VPC in your AWS account with all of the configurations and setup required for a EKS VPC. If youre planning on using containers, a platform like Kubernetes can be handy. With that, weve deployed a complete EKS cluster with dashboard on AWS! At the top of the page, it should say User ARN, followed by something like this: In addition, select Roles and search for a role starting with eksctl. However, when you do so, you will be presented with a login screen: To log in, we have to obtain a token for our dashboard-admin user. This service is called the Elastic Kubernetes Service or EKS. So, if youre planning to run exactly 11 pods onto a t3.small node, there wont be enough room for updates to happen. It provides several things out of the box such as service discovery, load balancing, self healing, configuration management and many more.
