Skip to content

Deploy Pachyderm on Amazon EKS

Amazon EKS provides an easy way to deploy, configure, and manage Kubernetes clusters. If you want to avoid managing your Kubernetes infrastructure, EKS might be the right choice for your organization. Pachyderm seamlessly deploys on Amazon EKS.

Prerequisites

Before you can deploy Pachyderm on an EKS cluster, verify that you have the following prerequisites installed and configured:

Deploy an EKS cluster by using eksctl

Use the eksctl tool to deploy an EKS cluster in your Amazon AWS environment. The eksctl create cluster command creates a virtual private cloud (VPC), a security group, and an IAM role for Kubernetes to create resources. For detailed instructions, see Amazon documentation.

To deploy an EKS cluster, complete the following steps:

  1. Deploy an EKS cluster:
eksctl create cluster --name <name> --version <version> \
--nodegroup-name <name> --node-type <vm-flavor> \
--nodes <number-of-nodes> --nodes-min <min-number-nodes> \
--nodes-max <max-number-nodes> --node-ami auto

Example output:

[ℹ]  using region us-east-1
[ℹ]  setting availability zones to [us-east-1a us-east-1f]
[ℹ]  subnets for us-east-1a - public:192.168.0.0/19 private:192.168.64.0/19
[ℹ]  subnets for us-east-1f - public:192.168.32.0/19 private:192.168.96.0/19
[ℹ]  nodegroup "pachyderm-test-workers" will use "ami-0f2e8e5663e16b436" [AmazonLinux2/1.13]
[ℹ]  using Kubernetes version 1.13
[ℹ]  creating EKS cluster "pachyderm-test-eks" in "us-east-1" region
[ℹ]  will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
[ℹ]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-east-1 --name=pachyderm-test-eks'
[ℹ]  2 sequential tasks: { create cluster control plane "svetkars-eks", create nodegroup "pachyderm-test-workers" }
[ℹ]  building cluster stack "eksctl-pachyderm-test-eks-cluster"
[ℹ]  deploying stack "eksctl-pachyderm-test-eks-cluster"

...
[✔]  EKS cluster "pachyderm-test" in "us-east-1" region is ready
  1. Verify the deployment:
kubectl get all

System Response:

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.100.0.1   <none>        443/TCP   7m9s
  1. Deploy Pachyderm as described in Deploy Pachyderm on AWS.
Last update: January 5, 2021
Does this page need fixing? Edit me on GitHub