GCP + Pachyderm

Learn how to deploy to Pachyderm to the cloud with GCP.

December 5, 2022

Before You Start #

This guide assumes that:

⚠️

This is not a production-level setup guide; see the Google Cloud Platform deploy guide for in-depth setup.


1. Create a New Project #

  1. Log in to Google Cloud Console.
  2. Create a new project (e.g.,pachyderm-quickstart-project).
  3. Enable the Compute Engine API.

You are now ready to create a GKE Cluster.

2. Run Setup Script #

You can run this setup script either through the Cloud Shell or in a local terminal via the gcloud cli. Running this script creates all of the following:

  • One GKE cluster
  • Workload identity service accounts
  • Permissions
  • A static IP address
  • The cloud SQL instance and databases
  • One cloud storage bucket
  • One file called ${NAME}.values.yaml in the current directory

It also installs Pachyderm into the cluster.

3. Connect to Cluster #

  1. Run the following:
pachctl config import-kube local --overwrite
pachctl config set active-context local
pachctl port-forward
  1. Open your browser at localhost:4000.
💡

You can also connect to Console via Google’s Cloud Shell:

console-in-browser