Upgrade Pachyderm¶
Upgrades between minor releases or point releases, such as 2.1.0
to version 2.2.0
, should be seamless. Therefore, the upgrade procedure is simple and requires little to no downtime. As a good practice, we recommend that you check the release notes before an upgrade to get an understanding of the changes introduced between your current version and your target.
Warning
Do not use these steps to upgrade between major versions as it might result in data corruption.
Complete the following steps to upgrade Pachyderm from one minor release to another.
Backup your cluster¶
As a general good practice, start with the backup of your cluster as described in the Backup and Restore section of this documentation.
Update your helm values¶
When left blank, some values are automatically generated during the installation. If you do not retrieve, then include these values back into your original values.yaml, the upgrade will likely fail.
Warning
You must add a secret to your values.yaml file or supply the generated values back via --set
on the command line during the upgrade. Check the troubleshooting section at the end of this page for details on how to retrieve those generated values.
The settings that are autogenerated if they are left blank during an initial installation are:
global:
postgresql:
postgresqlPassword
pachd:
clusterDeploymentID
rootToken
enterpriseSecret
oauthClientSecret
CARRY YOUR CLUSTERID OVER
Aside from retrieving any generated values and inserting them back as credentials or as secrets in your values.yaml, you will need to retrieve your clusterID and carry it over to your new deployment by setting the field pachd.clusterDeploymentID
with your ID. Note that your Cluster ID is located in your ~/.pachyderm/config.json
. You can also find it by running pachctl config get context <your-context-name>' (run
pachctl list context` to list out all contexts).
Upgrade pachctl
version¶
-
To update to the latest version of Pachyderm, run the steps below depending on your operating system:
- For macOS, run:
brew tap pachyderm/tap && brew install pachyderm/tap/pachctl@2.2
- For a Debian-based Linux 64-bit or Windows 10 or later running on
WSL:
curl -o /tmp/pachctl.deb -L https://github.com/pachyderm/pachyderm/releases/download/v2.2.0/pachctl_2.2.0_amd64.deb && sudo dpkg -i /tmp/pachctl.deb
- For all other Linux flavors:
curl -o /tmp/pachctl.tar.gz -L https://github.com/pachyderm/pachyderm/releases/download/v2.2.0/pachctl_2.2.0_linux_amd64.tar.gz && tar -xvf /tmp/pachctl.tar.gz -C /tmp && sudo cp /tmp/pachctl_2.2.0_linux_amd64/pachctl /usr/local/bin
Note
For a specific target release, specify the targeted major/minor version of pachctl
for brew and major/minor/point release for curl in the commands above.
-
Verify that the installation was successful by running
pachctl version --client-only
:pachctl version --client-only
System Response:
COMPONENT VERSION pachctl <This should display the version you installed>
Helm upgrade¶
-
Redeploy Pachyderm by running the helm upgrade command with your updated values.yaml:
helm repo add pach https://helm.pachyderm.com helm repo update helm upgrade pachd -f my_pachyderm_values.yaml pach/pachyderm --version <your_chart_version>
Note
Each chart version is associated with a given version of Pachyderm. You will find the list of all available chart versions and their associated version of Pachyderm on Artifacthub.
-
The upgrade can take some time. You can run
kubectl get pods
periodically to check the status of the deployment. When Pachyderm is deployed, the command shows all pods asREADY
:Once the pods are up, you should see a pod forkubectl get pods
pachd
running (alongside etcd, pg-bouncer, postgres, console etc... depending on your installation).System response:
NAME READY STATUS RESTARTS AGE pachd-3677268306-9sqm0 1/1 Running 0 4m ...
-
Verify that the new version has been deployed:
pachctl version
System response:
COMPONENT VERSION pachctl 2.2.0 pachd 2.2.0
The
pachd
andpachctl
versions must both match the new version.
Troubleshoot upgrades¶
Most of the issues you might run into when upgrading Pachyderm are related to the failure to retrieve then re-set default values that were generated during the first installation.
Here is a list of all the values used by Pachyderm when installing/upgrading a cluster. Some are optional and dependent on your installation choices.
In general, these values can be provided in three different ways:
-
A - (RECOMMENDED in production) Create your secret(s) ahead of your cluster creation
Create your secret(s) ahead of your cluster creation and provide each secret name in your values.yaml at the time of the first installation.
Find the list of Secret Keys for each setting and their corresponding secret name field in your values.yaml (Column A) in the table below.
-
B - Provide credentials directly in your values.yaml
You do not want to create your secrets right away, but provide specific secret credentials directly in your values.yaml or by setting them with a
--set
argument during the upgrade.In this case, (see column B), we will read then insert those values in a default secret
pachyderm-bootstrap-config
for you. The associated keys inpachyderm-bootstrap-config
are listed in column C of the following table. -
C - Neither A nor B
You have not created secrets ahead of your installation, nor did you provide values directly in the values.yaml; we created default values for you in the default secret
pachyderm-bootstrap-config
(See column C below for their key).
Important
It is important to note that if no secret name is provided for the fields mentioned in A, Pachyderm will retrieve the dedicated plain-text secret values in the helm values (B) and populate a generic, default, auto-generated secret (pachyderm-bootstrap-config) at the time of the installation (see keys in C). If no value is found in either one of those two cases, default values are used in pachyderm-bootstrap-config
. This default secret is reset at each upgrade, and new default values are created causing the upgrade to fail unless they are retrieved and set back into their appropriate fields.
Secret KEY name | Description | A - Create your secrets ahead of your cluster creation | B - Pass credentials in values.yaml | C - Neither A nor B - KEY name in default pachyderm-bootstrap-config secret |
---|---|---|---|---|
root-token | Root clusterAdmin | pachd.rootTokenSecretName | pachd.rootToken | rootToken |
root-token | Root clusterAdmin of the enterprise server | pachd.enterpriseRootTokenSecretName | pachd.enterpriseRootToken | enterpriseRootToken |
postgresql-password | Password to your database | global.postgresql.postgresqlExistingSecretName global.postgresql.postgresqlExistingSecretKey | global.postgresql.postgresqlPassword | postgresql-password * in separate secret called postgres |
OAUTH_CLIENT_SECRET | Oauth client secret for Console Required if you set Console | console.config.oauthClientSecretSecretName | console.config.oauthClientSecret | oidcClients[1].secret |
enterprise-license-key | Your enterprise license | pachd.enterpriseLicenseKeySecretName | pachd.enterpriseLicenseKey | license |
pachd-oauth-client-secret | Oauth client secret for pachd | pachd.oauthClientSecretSecretName | pachd.oauthClientSecret | oidcClients[0].secret |
enterprise-secret | Needed if you connect to an enterprise server | pachd.enterpriseSecretSecretName | pachd.enterpriseSecret | enterpriseSecret |
upstream-idps | The list of dex connectors, each containing Oauth client info connecting to an upstream IDP | oidc.upstreamIDPsSecretName | oidc.upstreamIDPs | idps |
Tip
Run kubectl get secret pachyderm-bootstrap-config -o go-template='{{.data.rootToken | base64decode }}'
to retrieve the value corresponding to the key rootToken
in the default secret pachyderm-bootstrap-config
.
Attention users of Pachyderm bundled PostgreSQL
If you are using the built in postgreSQL, you cannot use an existing secret to pass postgreSQL credentials. You need to retrieve then pass your auto-generated credentials in your values (global.postgresql.postgresqlPassword
).