Skip to content

Features Overview

Note

To get more information about Pachyderm Enterprise Edition, to ask questions, or to get access for evaluation, don't hesitate to get in touch with us at sales@pachyderm.io or on our Slack.

Enterprise Features List

Pachyderm Enterprise Edition helps you scale and manage Pachyderm data pipelines in an enterprise setting.

It delivers the most recent version of the Community Edition of Pachyderm along with additional features and a UI (Console) for visualizing pipelines and exploring data.

THE ENTERPRISE EDITION LIFTS ALL SCALING LIMITATIONS

Note that the activation of the Enterprise Edition lifts all scaling limits of the Community Edition. You can run as many pipelines as you need and parallelize your jobs without constraints.

Additional Features

Pachyderm Enterprise unlocks a series of additional administrative and security features needed for enterprise-scale deployments of Pachyderm, namely:

  • Authentication: Pachyderm allows for authentication against any OIDC provider. Users can authenticate to Pachyderm by logging into their favorite Identity Provider.
  • Role-Based Access Control - RBAC: Enterprise-scale deployments require access control. Pachyderm Enterprise Edition gives teams the ability to control access to production pipelines and data. Administrators can silo data, prevent unintended modifications to production pipelines, and support multiple data scientists or even multiple data science groups by controlling users' access to Pachyderm resources.
  • Enterprise Server: An organization can have many Pachyderm clusters registered with one single Enterprise Server that manages the Enterprise licensing and the integration with a company's Identity Provider.
  • Additionally, you have access to a pachctl command that pauses (pachctl enterprise pause) and unpauses (pachctl enterprise unpause) your cluster for a backup and restore.

Tooling

Pachyderm Enterprise comes with a complementary tool that will quickly become indispensable when designing and debugging pipelines: Pachyderm Console, a visual interface for pipeline visualization and data exploration.

Pachyderm Enterprise Edition includes a full Web UI for visualizing pipelines and exploring data. It automatically infers the structure of data scientists' DAGs and displays them visually. Data scientists and cluster admins can click on individual segments of pipelines and repos to see how many jobs have run, explore commits and data, or access Pachyderm logs. Console is an indispensable tool when designing and troubleshooting your data workflow. Console also supports file ingress from your local computer via a drag-and-drop file upload feature.

Console Pipeline

You can deploy Console with Pachyderm by adding the relevant fields to your Helm values. A production environment requires setting up Authentication, an Ingress Controller, and a DNS. You can also choose to deploy Console locally to experiment with the product.


Last update: May 19, 2022
Does this page need fixing? Edit me on GitHub