Setting up a development environment

Required dependencies

pip will not handle installation of system dependencies

pip will not install system dependencies needed for some packages such as psycopg2 and kerberos.

For a better development experience see the conda or nix setup instructions.

  1. Install gh

  2. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis
  3. Change directory into ibis:

    cd ibis
  4. Install development dependencies

    pip install 'poetry==1.7.1'
    pip install -r requirements-dev.txt
  5. Install ibis in development mode

    pip install -e '.[all]'

Support matrix 1

Python Version Python 3.9 Python 3.10 Python 3.11
Operating System
Linux 2

macOS (x86_64)

macOS (aarch64)

Windows

  1. Install Miniconda

  2. Install gh

    conda install -c conda-forge gh
  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis
  4. Create a Conda environment using environment.yml

    cd ibis
    conda env create --file conda/environment.yml

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

  5. Activate the environment

    conda activate ibis-dev
  6. Install your local copy of ibis into the Conda environment

    pip install -e '.[all]'
  1. Install Mamba

  2. Install gh

    mamba install -c conda-forge gh
  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis
  4. Create a Conda environment using environment.yml

    cd ibis
    mamba env create --file conda/environment.yml
  5. Activate the environment

    mamba activate ibis-dev
  6. Install your local copy of ibis into the Mamba environment

    pip install -e '.[all]'

Support matrix

Python Version Python 3.9 Python 3.10 Python 3.11
Operating System
Linux 3

macOS (x86_64)

macOS (aarch64) 4

Windows 5

  1. Install nix

  2. Configure nix

    Edit/create your nix.conf file ($XDG_CONFIG_HOME/nix/nix.conf) and add the line

    experimental-features = nix-command flakes
  3. Install gh:

    nix-shell -p gh
    nix-env -iA gh
  4. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis
  5. Set up the public ibis Cachix cache to pull pre-built dependencies:

    nix-shell -p cachix --run 'cachix use ibis'
  6. Run nix develop in the checkout directory:

    cd ibis
    nix develop

    This will launch a bash shell with all of the required dependencies installed. This may take a while due to artifact download from the cache.

Building the docs

Install just and run

just docs-preview

to build and serve the documentation.

Back to top

Footnotes

  1. Some optional dependencies for Windows and Mac OS are not available through conda/mamba↩︎

  2. Tested in CI. If this doesn’t work for you, please file an issue.↩︎

  3. Tested in CI. If this doesn’t work for you, please file an issue.↩︎

  4. Not tested in CI. Unknown if this setup works.↩︎

  5. Unlikely to ever be supported or no upstream support.↩︎