Setting up a development environment

Required dependencies

Support matrix 1

Python 3.10 Python 3.11 Python 3.12
Linux 2

macOS (x86_64)

macOS (aarch64)

Windows

  1. Install Miniconda

  2. Install gh

    conda install -c conda-forge gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Create a Miniconda environment using environment.yml

    cd ibis
conda env create --file conda/environment.yml

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

  5. Activate the environment

    conda activate ibis-dev

  6. Install your local copy of ibis into the Conda environment

    uv pip install -e .

  1. Install Mamba

  2. Install gh

    mamba install -c conda-forge gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Create a Mamba environment using environment.yml

    cd ibis
mamba env create --file conda/environment.yml

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

  5. Activate the environment

    mamba activate ibis-dev

  6. Install your local copy of ibis into the Mamba environment

    uv pip install -e .

  1. Install Pixi

  2. Install gh

    pixi global install gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Create a Pixi environment using environment.yml

    cd ibis
pixi init --import conda/environment.yml

    This command will initialize a pixi.toml file and also modify the .gitignore to prevent Pixi configuration from being added to git.

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

  5. Activate the environment

    pixi shell

  6. Install your local copy of ibis into the Pixi environment

    uv pip install -e .

Support matrix

Python 3.10 Python 3.11 Python 3.12 Python 3.13
Linux (x86_64) 3

Linux (arm64)

macOS (x86_64)

macOS (arm64/M1/M2)

Windows 4

  1. Install nix

  2. Configure nix

    Edit/create your nix.conf file ($XDG_CONFIG_HOME/nix/nix.conf) and add the line

    experimental-features = nix-command flakes

  3. Install gh:

    nix-shell -p gh
    nix-env -iA gh

  4. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  5. Set up the public ibis Cachix cache to pull pre-built dependencies:

    nix-shell -p cachix --run 'cachix use ibis'

  6. Run nix develop in the checkout directory:

    cd ibis
nix develop

    This will launch a bash shell with all of the required dependencies installed. This may take a while due to artifact download from the cache.

Support matrix

Python 3.10 Python 3.11 Python 3.12 Python 3.13
Linux

macOS (x86_64)

macOS (arm64/M1/M2)

Windows

  1. Git clone the project repository.

  2. Install Docker Desktop for your platform.

  3. Install VS Code

  4. Install VS Code Docker Extension

  5. Install VS Code Dev Containers Extension

  6. If using an Apple Silicon Mac, virtualization may be fastest with Colima.

    1. Install Colima
    2. Verify that the disk allocation to Colima is satisfactory with colima template --editor code.
    3. To use Colima for virtualization, docker context use colima or export DOCKER_CONTEXT=colima.
    4. Verify that the Colima context is in effect with docker context ls (look for ’*’).
    5. Start the Colima VM: start colima.
    6. If you encounter disk resource issues after building images, colima prune or colima delete may be needed.

    As an alternative to Colima, install Rosetta 2.

    softwareupdate --install-rosetta

  7. In VS Code, open the project directory.

  8. Menu options for working with devcontainers are available through the blue >< button, at the lower left corner of the project window.

    • Use Reopen the container to build an image and launch a container.
    • Press any button to close the automatically launched terminal.
    • Launch a new VS Code terminal from the main menu.
    • The project will be in the container as an editable install with Ibis library, dev and test dependencies installed, and with the working directory /app.

  9. Use uv commands such uv pip list to show the installed packages in the uv .venv.

  10. Run non-uv commands in the virtual environment using uv run, for example uv run pytest -m core. Standard git commands are available without uv run because they do not need packages in the .venv to work.

  11. To exit a container, click the Dev Container button on the lower left of the window and select the last menu option, Close Remote Connection.

  12. To ensure you have the latest dependencies from the main upstream branch based on pyproject.toml:

    • Exit any running container.
    • Sync your fork.
    • From your local Git repo, git pull origin main.
    • Reopen the project in a new container.
    • Rebuild Container to copy files from the local Git repo and have the build run uv sync.
uv will not handle installation of system dependencies

uv will not install system dependencies needed for some packages such as psycopg2 and kerberos.

For a better development experience see the conda/mamba or nix setup instructions.

  1. Install uv

  2. Install gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Change directory into ibis:

    cd ibis

  5. Install development dependencies

    This will create a virtual environment at .venv and install all dependencies inside. It will also install Ibis in development mode as ibis-framework.

    just sync

  6. Activate the virtual environment

    source .venv/bin/activate
pip will not handle installation of system dependencies

pip will not install system dependencies needed for some packages such as psycopg2 and kerberos.

For a better development experience see the conda/mamba or nix setup instructions.

  1. Install gh

  2. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  3. Change directory into ibis:

    cd ibis

  4. Install development dependencies

    This will also install Ibis in development mode as ibis-framework.

    pip install 'uv>=0.4.29'
pip install -r requirements-dev.txt

Code style and formatting

Ibis uses several code linters and has a style guide for documentation, please checkout the style and formatting guide for instructions on how to set things up.

Building the docs

Install just (if you installed via conda/mamba you are covered) and run

just docs-preview

to build and serve the documentation.

Back to top

Footnotes

  1. Some optional dependencies for Windows and Mac OS are not available through conda/mamba↩︎

  2. Tested in CI. If this doesn’t work for you, please file an issue.↩︎

  3. Tested in CI. If this doesn’t work for you, please file an issue.↩︎

  4. Unlikely to ever be supported or no upstream support.↩︎