Skip to content

Enterprise Server Setup

The Enterprise Server is a component in Pachyderm which manages Enterprise Licensing and the integration with a company's Identity Providers (IDPs).

An organization can have many Pachyderm clusters registered with one single Enterprise Server. Administrators activate the Enterprise Server with an Enterprise License Key from Pachyderm sales, and optionally configure authentication with their IDP via SAML, OIDC, LDAP, etc...

The following diagram gives you a quick overview of an organization with multiple Pachyderm clusters behind a single Enterprise Server. Enterprise Server General Deployment

Note

For POCs and smaller organizations with one single Pachyderm cluster, the Enterprise Server services can be run embedded in pachd. A separate deployment is not necessary. An organization with a single Pachyderm cluster can run the Enterprise Server services embedded within pachd.

The setup of an Enterprise Server requires to:

  1. Deploy it.
  2. Activate your Enterprise Key and enable Auth.
  3. Register your newly created or existing Pachyderm clusters with your enterprise server.
  4. Optional: Enable Auth on each cluster.

1 - Deploy An Enterprise Server

Deploying and configuring an enterprise server can be done in one of two flavors:

  1. Provide all licensing and authentication configurations as a part of the Helm deployment.
  2. Or, install a bare-bones version of Pachyderm with Helm, then use pachctl commands to set up licensing and authentication.

As Part Of A Regular Pachyderm Helm Deployment

Update your values.yaml with your enterprise license key and auth configurations (for an example on localhost, see the example values.yaml here) or insert our minimal example below to your values.yaml.

pachd:
    enterpriseLicenseKey: "<ENTERPRISE-LICENSE-KEY>"
    oauthClientID: "pachd"
    oauthRedirectURI: "http://<PACHD-IP>:30657/authorization-code/callback"
    ## if a secret is not provided, a secret will be autogenerated on install and stored in the k8s secret 'pachyderm-bootstrap-config.authConfig.clientSecret'
    oauthClientSecret: ""
    ## if a secret is not provided, a secret will be autogenerated on install and stored in the k8s secret 'pachyderm-bootstrap-config.enterpriseSecret'
    enterpriseSecret: ""
    ## if a token is not provided, a secret will be autogenerated on install and stored in the k8s secret 'pachyderm-bootstrap-config.rootToken'
    rootToken: ""
externalService:
  enabled: true
oidc:
    issuerURI: "http://<PACHD-IP>:30658/"
    ## userAccessibleOauthIssuerHost is necessary in localhost settings or anytime the registered Issuer address isn't accessible outside the cluster
    # userAccessibleOauthIssuerHost: "localhost:30658"
    ## if `mockIDP` is set to true, `pachd.upstreamIDPs` will be ignored in favor of a testing placeholder IDP with username/password: admin/password
    mockIDP: false
    ## to set up upstream IDPs, set pachd.mockIDP to false,
    ## and populate the pachd.upstreamIDPs with an array of Dex Connector configurations.
    ## See the example below or https://dexidp.io/docs/connectors/
    upstreamIDPs:
      - id: idpConnector
        jsonConfig: >-
          {
              "issuer": "<ISSUER>",
              "clientID": "<CLIENT-ID>",
              "clientSecret": "<CLIENT-SECRET>",
              "redirectURI": "http://<PACHD-IP>:30658/callback",
              "insecureEnableGroups": true,
              "insecureSkipEmailVerified": true,
              "insecureSkipIssuerCallbackDomainCheck": true,
              "forwardedLoginParams": ["login_hint"]
          }
        name: idpConnector
        type: oidc
This results in a single pachd pod, with authentication enabled, and an IDP integration configured.

Deploying a stand-alone enterprise server requires setting the helm parameter enterpriseServer.enabled to true and the pachd.enabled to false.

enterpriseServer:
    enabled: true
pachd:
    enabled: false
    enterpriseLicenseKey: "<ENTERPRISE-LICENSE-KEY>"
    oauthClientID: "pachd"
    oauthRedirectURI: "http://<PACHD-IP>:30657/authorization-code/callback"
    ## if a secret is not provided, a secret will be autogenerated on install and stored in the k8s secret 'pachyderm-bootstrap-config.authConfig.clientSecret'
    oauthClientSecret: ""
    ## if a secret is not provided, a secret will be autogenerated on install and stored in the k8s secret 'pachyderm-bootstrap-config.enterpriseSecret'
    enterpriseSecret: ""
    ## if a token is not provided, a secret will be autogenerated on install and stored in the k8s secret 'pachyderm-bootstrap-config.rootToken'
    rootToken: ""
externalService:
  enabled: true
oidc:
    issuerURI: "http://<PACHD-IP>:30658/"
    ## userAccessibleOauthIssuerHost is necessary in localhost settings or anytime the registered Issuer address isn't accessible outside the cluster
    # userAccessibleOauthIssuerHost: "localhost:30658"
    ## if `mockIDP` is set to true, `pachd.upstreamIDPs` will be ignored in favor of a testing placeholder IDP with username/password: admin/password
    mockIDP: false
    ## to set up upstream IDPs, set pachd.mockIDP to false,
    ## and populate the pachd.upstreamIDPs with an array of Dex Connector configurations.
    ## See the example below or https://dexidp.io/docs/connectors/
    upstreamIDPs:
      - id: idpConnector
        jsonConfig: >-
          {
              "issuer": "<ISSUER>",
              "clientID": "<CLIENT-ID>",
              "clientSecret": "<CLIENT-SECRET>",
              "redirectURI": "http://<PACHD-IP>:30658/callback",
              "insecureEnableGroups": true,
              "insecureSkipEmailVerified": true,
              "insecureSkipIssuerCallbackDomainCheck": true,
              "forwardedLoginParams": ["login_hint"]
          }
        name: idpConnector
        type: oidc

Note

Update the following values as follow:

  • PACHD-IP: The address of Pachyderm's IP. Retrieve Pachyderm external IP address if necessary.
  • ISSUER, CLIENT-ID, CLIENT-SECRET: Refer to our Identity Provider Configuration page.

Check the list of all available helm values at your disposal in our reference documentation or on Github.

  • If a pachyderm cluster will also be installed in the same kubernetes cluster, they should be installed in different namespaces:

    kubectl create namespace enterprise
    helm install ... --set enterpriseServer.enabled=true  --namespace enterprise
    

    This command deploys postgres, etcd and a deployment and service called pach-enterprise. pach-enterprise uses the same docker image and pachd binary, but it listens on a different set of ports (31650, 31657, 31658) to avoid conflicts with pachd.

  • Check the state of your deployment by running:

    kubectl get all --namespace enterprise
    
    System Response
    NAME                                   READY   STATUS    RESTARTS   AGE
    pod/etcd-5fd7c675b6-46kz7              1/1     Running   0          113m
    pod/pach-enterprise-6dc9cb8f66-rs44t   1/1     Running   0          105m
    pod/postgres-6bfd7bfc47-9mz28          1/1     Running   0          113m
    

Warning

  • When enterprise is enabled through Helm, auth is automatically activated (i.e., you do not need to run pachctl auth activate) and a pachyderm-bootstrap-config k8s secret is created containing an entry for your rootToken. Use kubectl get secret pachyderm-bootstrap-config -o go-template='{{.data.rootToken | base64decode }}' to retrieve it and save it where you see fit. In such a case, skip step 2.

However, this secret is only used when configuring through helm:

  • If you run pachctl auth activate, the secret is not updated. Instead, the rootToken is printed in your STDOUT for you to save.
  • Same behavior if you activate enterprise manually (pachctl license activate) then activate authentication (pachctl auth activate).

On An Existing Pachyderm Cluster

To enable the Enterprise Server on an existing cluster:

2- Activate Enterprise Licensing And Enable Authentication

  • Use your enterprise key to activate your enterprise server:
    echo <your-activation-token> | pachctl license activate
    
  • Then enable Authentication at the Enterprise Server level:

    pachctl auth activate --enterprise
    

    Warning

    Enabling Auth will return a root token for the enterprise server. This is separate from the root tokens for each pachd (cluster). They should all be stored securely.

Once the enterprise server is deployed, deploy your cluster(s) (helm install...) and register it(them) with the enterprise server. Note that you have the option to register your clusters directly in your values.yaml when deploying or after its deployment, using pachctl.

You migh want to expose your cluster(s) to the internet. Check the setup of a Load Balancer in our deployment section.

3- Register Your Cluster With The Enterprise Server

Similarly to the enterprise server, we can configure our pachyderm clusters to leverage Helm for licensing and authentication in one of two flavors:

  1. Provide enterprise registration information as a part of the Helm deployment of a cluster.
  2. Register a cluster with the Enterprise Server using pachctl commands.

Register Clusters With Helm

Add the enterprise server's root token, and network addresses to the values.yaml of each cluster you plan to deploy and register, for the cluster and enterprise server to communicate (for an example on localhost, see the example values.yaml here), or insert our minimal example below to your values.yaml.

pachd:
    activateEnterpriseMember: true
    enterpriseServerAddress: "grpc://<ENTERPRISE_SERVER_ADDRESS>"
    enterpriseCallbackAddress: "grpc://<PACHD_ADDRESS>"
    enterpriseRootToken: "<ENTERPRISE-ROOT-TOKEN>" # the same root token of the enterprise cluster

Warning

When setting your enterprise server info as part of the Helm deployment of a cluster, auth is automatically activated (i.e., you can skip step 4).

In this case, a pachyderm-bootstrap-config k8s secret is automatically created on the cluster. It contains an entry for your clusters' rootToken. This is separate from the enterprise server root token. Use kubectl get secret pachyderm-bootstrap-config -o go-template='{{.data.rootToken | base64decode }}' to retrieve it and save it where you see fit.

This secret is only used when configuring through helm

Register Clusters With pachctl

  • Run this command for each of the clusters you wish to register using pachctl:

    pachctl enterprise register --id <my-pachd-config-name> --enterprise-server-address <pach-enterprise-IP>:650 --pachd-address <pachd-IP>:650
    
    • --id is the name of the context pointing to your cluster in ~/.pachyderm/config.json.

    • --enterprise-server-address is the host and port where pachd can reach the enterprise server. In production, the enterprise server may be exposed on the internet.

    • --pachd-address is the host and port where the enterprise server can reach pachd. This may be internal to the kubernetes cluster, or over the internet.

  • Display the list of all registered clusters with your enterprise server:

    pachctl license list-clusters
    

    Using enterprise context: my-enterprise-context-name
    id: john
    address: ae1ba915f8b5b477c98cd26c67d7563b-66539067.us-west-2.elb.amazonaws.com:650
    version: 2.0.0
    auth_enabled: true
    last_heartbeat: 2021-05-21 18:37:36.072156 +0000 UTC
    
    ---
    id: doe
    address: 34.71.247.191:650
    version: 2.0.0
    auth_enabled: true
    last_heartbeat: 2021-05-21 18:43:42.157027 +0000 UTC
    ---
    

4- Enable Auth On Each Cluster

Finally, if your clusters were registered with the Enterprise Server using pachctl, you might choose to activate auth on each (or some) of them. This is an optional step. Clusters can be registered with the enterprise server without authentication being enabled.

  • Before enabling authentication, set up the issuer in the idp config between the enterprise server and your cluster:

    echo "issuer: http://<enterprise-server-IP>:658" | pachctl idp set-config --config -
    
    Check that your config has been updated properly: pachctl idp get-config

  • For each registered cluster you want to enable auth on:

    pachctl auth activate --client-id <my-pachd-config-name> --redirect http://<pachd-IP>:657/authorization-code/callback 
    

Note

  • Note the /authorization-code/callback appended after <pachd-IP>:657 in --redirect.
  • --client-id is to pachctl auth activate what --id is to pachctl enterprise register: In both cases, enter <my-pachd-config-name>.
  • Make sure than your enterprise context is set up properly:
    pachctl config get active-enterprise-context
    
    If not:
    pachctl config set active-enterprise-context <my-enterprise-context-name>
    

To manage you server, its context, or connect your IdP, visit the Manage your Enterprise Server page.


Last update: November 29, 2021
Does this page need fixing? Edit me on GitHub