Setup for contributors¶
First, go through the general local installation instructions here. Additionally, make sure you have the following installed:
To stay up to date, we recommend doing the following.
First clone the code: (Note, as of 07/11/19 pachyderm is using go modules and recommends cloning the code outside of the $GOPATH, we use the location ~/workspace as an example, but the code can live anywhere)
cd ~/workspace git clone firstname.lastname@example.org:pachyderm/pachyderm
Then update your
~/.bash_profile by adding the line:
And you'll stay up to date!
Special macOS configuration¶
File descriptor limit¶
If you're running tests locally, you'll need to up your file descriptor limit. To do this, first setup a LaunchDaemon to up the limit with sudo privileges:
sudo cp ~/workspace/pachyderm/etc/contributing/com.apple.launchd.limit.plist /Library/LaunchDaemons/
Once you restart, this will take effect. To see the limits, run:
launchctl limit maxfiles
Before the change is in place you'll see something like
256 unlimited. After the change you'll see a much bigger number in the first field. This ups the system wide limit, but you'll also need to set a per-process limit.
Second, up the per process limit by adding something like this to your
ulimit -n 12288
Unfortunately, even after setting that limit it never seems to report the updated version. So if you try
And just see
unlimited, don't worry, it took effect.
To make sure all of these settings are working, you can test that you have the proper setup by running:
If this fails with a timeout, you'll probably also see 'too many files' type of errors. If that test passes, you're all good!
You'll need the
timeout utility to run the
make launch task. To install on mac, do:
brew install coreutils
And then make sure to prepend the following to your path:
Now launch the dev cluster:
And check it's status:
kubectl get all.
This will install the dev version of
cd ~/workspace/pachyderm make install pachctl version
And make sure that
$GOPATH/bin is on your
Instead of running the makefile targets to re-compile
pachctl and redeploy a dev cluster, we have a script that you can use to fully reset your pachyderm environment:
1) All existing cluster data is deleted 2) If possible, the virtual machine that the cluster is running on is wiped out 3)
pachctl is recompiled 4) The dev cluster is re-deployed
This reset is a bit more time consuming than running one-off Makefile targets, but comprehensively ensures that the cluster is in its expected state, and is especially helpful when you're first getting started with contributions and don't yet have a complete intuition on the various ways a cluster may get in an unexpected state. It's been tested on docker for mac and minikube, but likely works in other kubernetes environments as well.
To run it, simply call
./etc/reset.py from the pachyderm repo root.