import ibis
from ibis import _
ibis.options.interactive = TrueSummary
This notebook takes you through an analysis of Ibis’s CI data using ibis on top of Google BigQuery.
- First, we load some data and poke around at it to see what’s what.
- Second, we figure out some useful things to calculate based on our poking.
- Third, we’ll visualize the results of calculations to showcase what changed and how.
Imports
Let’s start out by importing ibis and turning on interactive mode.
Connect to BigQuery
We connect to BigQuery using the ibis.connect API, which accepts a URL string indicating the backend and various bit of information needed to connect to the backend. Here we’re using BigQuery, so we need the project id (ibis-gbq) and the dataset id (workflows).
Datasets are analogous to schemas in other systems.
url = "bigquery://ibis-gbq/workflows"
con = ibis.connect(url)Let’s see what tables are available.
con.list_tables()['analysis', 'jobs', 'workflows']Analysis
Here we’ve got our first bit of interesting information: the jobs and workflows tables.
Terminology
Before we jump in, it helps to lay down some terminology.
- A workflow corresponds to an individual GitHub Actions YAML file in a GitHub repository under the
.github/workflowsdirectory. - A job is a named set of steps to run inside a workflow file.
What’s in the workflows table?
Each row in the workflows table corresponds to a workflow run.
- A workflow run is an instance of a workflow that was triggered by some entity: a GitHub user, bot, or other entity. Each row of the
workflowstable is a workflow run.
What’s in the jobs table?
Similarly, each row in the jobs table is a job run. That is, for a given workflow run there are a set of jobs run with it.
- A job run is an instance of a job in a workflow. It is associated with a single workflow run.
Rationale
The goal of this analysis is to try to understand ibis’s CI performance, and whether the amount of time we spent waiting on CI has decreased, stayed the same or increased. Ideally, we can understand the pieces that contribute to the change or lack thereof.
Metrics
To that end there are a few interesting metrics to look at:
- job run duration: this is the amount of time it takes for a given job to complete
- workflow run duration: the amount of time it takes for all job runs in a workflow run to complete.
- queueing duration: the amount time time spent waiting for the first job run to commence.
Mitigating Factors
- Around October 2021, we changed our CI infrastructure to use Poetry instead of Conda. The goal there was to see if we could cache dependencies using the lock file generated by poetry. We should see whether that had any effect.
- At the end of November 2022, we switch to the Team Plan (a paid GitHub plan) for the Ibis organzation. This tripled the amount of job runs that could execute in parallel. We should see if that helped anything.
Alright, let’s jump into some data!
jobs = con.tables.jobs
jobs┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ url ┃ steps ┃ status ┃ started_at ┃ runner_group_name ┃ run_attempt ┃ name ┃ labels ┃ node_id ┃ id ┃ runner_id ┃ run_url ┃ run_id ┃ check_run_url ┃ html_url ┃ runner_name ┃ runner_group_id ┃ head_sha ┃ conclusion ┃ completed_at ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ string │ array<!struct<status: string, conclusion: string, started_at: timestamp('UTC'),… │ string │ timestamp('UTC') │ string │ int64 │ string │ array<!string> │ string │ int64 │ int64 │ string │ int64 │ string │ string │ string │ int64 │ string │ string │ timestamp('UTC') │ ├────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┼───────────┼───────────────────────────┼───────────────────┼─────────────┼───────────────────────────────────────────┼────────────────┼──────────────────────────────┼────────────┼───────────┼───────────────────────────────────────────────────────────────────────┼───────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┼─────────────┼─────────────────┼──────────────────────────────────────────┼────────────┼───────────────────────────┤ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214556 │ [{...}, {...}, ... +5] │ completed │ 2020-09-05 19:52:40+00:00 │ NULL │ 1 │ Tests OmniSciDB/Spark (3.7) │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NTU2 │ 1076214556 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214556 │ https://github.com/ibis-project/ibis/runs/1076214556?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ success │ 2020-09-05 20:08:23+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214567 │ [{...}, {...}, ... +5] │ completed │ 2020-09-05 19:52:41+00:00 │ NULL │ 1 │ Tests SQL (3.7) │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NTY3 │ 1076214567 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214567 │ https://github.com/ibis-project/ibis/runs/1076214567?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ success │ 2020-09-05 20:03:29+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214573 │ [{...}, {...}, ... +5] │ completed │ 2020-09-05 19:52:42+00:00 │ NULL │ 1 │ Tests SQL (3.8) │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NTcz │ 1076214573 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214573 │ https://github.com/ibis-project/ibis/runs/1076214573?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ success │ 2020-09-05 20:02:04+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214584 │ [{...}, {...}, ... +3] │ completed │ 2020-09-05 19:52:40+00:00 │ NULL │ 1 │ Tests pandas / files (ubuntu-latest, 3.7) │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NTg0 │ 1076214584 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214584 │ https://github.com/ibis-project/ibis/runs/1076214584?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ success │ 2020-09-05 19:59:41+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214594 │ [{...}, {...}, ... +3] │ completed │ 2020-09-05 19:52:41+00:00 │ NULL │ 1 │ Tests pandas / files (ubuntu-latest, 3.8) │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NTk0 │ 1076214594 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214594 │ https://github.com/ibis-project/ibis/runs/1076214594?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ success │ 2020-09-05 20:00:30+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214604 │ [{...}, {...}, ... +9] │ completed │ 2020-09-05 19:52:41+00:00 │ NULL │ 1 │ Lint, package and benckmark │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NjA0 │ 1076214604 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214604 │ https://github.com/ibis-project/ibis/runs/1076214604?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ success │ 2020-09-05 20:46:45+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076214621 │ [] │ completed │ 2020-09-05 20:52:43+00:00 │ NULL │ 1 │ Tests Impala / Clickhouse │ [] │ MDg6Q2hlY2tSdW4xMDc2MjE0NjIx │ 1076214621 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240931982 │ 240931982 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076214621 │ https://github.com/ibis-project/ibis/runs/1076214621?check_suite_focus=true │ NULL │ NULL │ 0b720cbe9a6ab62d2402d5b400ca3f6f3f480855 │ skipped │ 2020-09-05 20:52:43+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076164518 │ [{...}, {...}, ... +5] │ completed │ 2020-09-05 19:20:53+00:00 │ NULL │ 1 │ Tests OmniSciDB/Spark (3.7) │ [] │ MDg6Q2hlY2tSdW4xMDc2MTY0NTE4 │ 1076164518 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240911764 │ 240911764 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076164518 │ https://github.com/ibis-project/ibis/runs/1076164518?check_suite_focus=true │ NULL │ NULL │ 39794be13e92913c753b37fb20bab70523444c6d │ success │ 2020-09-05 19:36:44+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076164569 │ [{...}, {...}, ... +5] │ completed │ 2020-09-05 19:20:54+00:00 │ NULL │ 1 │ Tests SQL (3.7) │ [] │ MDg6Q2hlY2tSdW4xMDc2MTY0NTY5 │ 1076164569 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240911764 │ 240911764 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076164569 │ https://github.com/ibis-project/ibis/runs/1076164569?check_suite_focus=true │ NULL │ NULL │ 39794be13e92913c753b37fb20bab70523444c6d │ success │ 2020-09-05 19:29:55+00:00 │ │ https://api.github.com/repos/ibis-project/ibis/actions/jobs/1076164581 │ [{...}, {...}, ... +5] │ completed │ 2020-09-05 19:20:55+00:00 │ NULL │ 1 │ Tests SQL (3.8) │ [] │ MDg6Q2hlY2tSdW4xMDc2MTY0NTgx │ 1076164581 │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/240911764 │ 240911764 │ https://api.github.com/repos/ibis-project/ibis/check-runs/1076164581 │ https://github.com/ibis-project/ibis/runs/1076164581?check_suite_focus=true │ NULL │ NULL │ 39794be13e92913c753b37fb20bab70523444c6d │ success │ 2020-09-05 19:29:53+00:00 │ │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ └────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────┴───────────┴───────────────────────────┴───────────────────┴─────────────┴───────────────────────────────────────────┴────────────────┴──────────────────────────────┴────────────┴───────────┴───────────────────────────────────────────────────────────────────────┴───────────┴──────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┴─────────────┴─────────────────┴──────────────────────────────────────────┴────────────┴───────────────────────────┘
These first few columns in the jobs table aren’t that interesting so we should look at what else is there
jobs.columns('url',
'steps',
'status',
'started_at',
'runner_group_name',
'run_attempt',
'name',
'labels',
'node_id',
'id',
'runner_id',
'run_url',
'run_id',
'check_run_url',
'html_url',
'runner_name',
'runner_group_id',
'head_sha',
'conclusion',
'completed_at')A bunch of these aren’t that useful for our purposes. However, run_id, started_at, completed_at are useful for us. The GitHub documentation for job information provides useful detail about the meaning of these fields.
run_id: the workflow run associated with this job runstarted_at: when the job startedcompleted_at: when the job completed
What we’re interested in to a first degree is the job duration, so let’s compute that.
We also need to compute when the last job for a given run_id started and when it completed. We’ll use the former to compute the queueing duration, and the latter to compute the total time it took for a given workflow run to complete.
run_id_win = ibis.window(group_by=_.run_id)
jobs = jobs.select(
_.run_id,
job_duration=_.completed_at.delta(_.started_at, "microsecond"),
last_job_started_at=_.started_at.max().over(run_id_win),
last_job_completed_at=_.completed_at.max().over(run_id_win),
)
jobs┏━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ run_id ┃ job_duration ┃ last_job_started_at ┃ last_job_completed_at ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ int64 │ int64 │ timestamp('UTC') │ timestamp('UTC') │ ├───────────┼──────────────┼───────────────────────────┼───────────────────────────┤ │ 206299115 │ 393000000 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 206299115 │ 408000000 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 206299115 │ 355000000 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 206299115 │ 333000000 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 206299115 │ 3167000000 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 206299115 │ 0 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 206299115 │ 1105000000 │ 2020-08-13 00:55:18+00:00 │ 2020-08-13 00:55:18+00:00 │ │ 207470553 │ 477000000 │ 2020-08-13 18:55:30+00:00 │ 2020-08-13 18:55:30+00:00 │ │ 207470553 │ 350000000 │ 2020-08-13 18:55:30+00:00 │ 2020-08-13 18:55:30+00:00 │ │ 207470553 │ 1217000000 │ 2020-08-13 18:55:30+00:00 │ 2020-08-13 18:55:30+00:00 │ │ … │ … │ … │ … │ └───────────┴──────────────┴───────────────────────────┴───────────────────────────┘
Let’s take a look at workflows
workflows = con.tables.workflows
workflows┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ workflow_url ┃ workflow_id ┃ triggering_actor ┃ run_number ┃ run_attempt ┃ updated_at ┃ cancel_url ┃ rerun_url ┃ check_suite_node_id ┃ pull_requests ┃ id ┃ node_id ┃ status ┃ repository ┃ jobs_url ┃ previous_attempt_url ┃ artifacts_url ┃ html_url ┃ head_sha ┃ head_repository ┃ run_started_at ┃ head_branch ┃ url ┃ event ┃ name ┃ actor ┃ created_at ┃ check_suite_url ┃ check_suite_id ┃ conclusion ┃ head_commit ┃ logs_url ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ string │ int64 │ struct<subscrip… │ int64 │ int64 │ timestamp('UTC') │ string │ string │ string │ array<!struct<number: int64, url: string, id: int64, head: struct<sha: string, … │ int64 │ string │ string │ struct<trees_url: string, teams_url: string, statuses_url: string, subscribers_… │ string │ string │ string │ string │ string │ struct<trees_url: string, teams_url: string, statuses_url: string, subscribers_… │ timestamp('UTC') │ string │ string │ string │ string │ stru… │ timestamp('UTC') │ string │ int64 │ string │ struct<tree_id: string, timestamp: timestamp('UTC'), message: string, id: strin… │ string │ ├──────────────────────────────────────────────────────────────────────────┼─────────────┼──────────────────┼────────────┼─────────────┼───────────────────────────┼──────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────────────────────────────┼───────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────┼──────────────────────┼─────────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────┼───────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┼──────────────┼──────────┼───────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────┼────────────────┼────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2170517 │ 2170517 │ NULL │ 31 │ 1 │ 2020-09-07 19:17:53+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465015/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465015/rerun │ MDEwOkNoZWNrU3VpdGUxMTU2NDgxNzIw │ [{...}, {...}] │ 243465015 │ MDExOldvcmtmbG93UnVuMjQzNDY1MDE1 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465015/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465015/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243465015 │ e7ac01853b5534a3378f78ebff25c861bc9209e8 │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ 2020-09-07 18:57:15+00:00 │ master │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465015 │ push │ BigQuery │ NULL │ 2020-09-07 18:57:15+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1156481720 │ 1156481720 │ failure │ {'tree_id': 'a9497cb44b4aa63f304f69505e596a4446f22883', 'timestamp': datetime.datetime(2020, 9, 7, 18, 57, 13, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465015/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 298 │ 1 │ 2020-09-07 19:57:27+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465016/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465016/rerun │ MDEwOkNoZWNrU3VpdGUxMTU2NDgxNzIy │ [{...}, {...}] │ 243465016 │ MDExOldvcmtmbG93UnVuMjQzNDY1MDE2 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465016/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465016/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243465016 │ e7ac01853b5534a3378f78ebff25c861bc9209e8 │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ 2020-09-07 18:57:15+00:00 │ master │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465016 │ push │ Main │ NULL │ 2020-09-07 18:57:15+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1156481722 │ 1156481722 │ failure │ {'tree_id': 'a9497cb44b4aa63f304f69505e596a4446f22883', 'timestamp': datetime.datetime(2020, 9, 7, 18, 57, 13, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243465016/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 297 │ 1 │ 2020-09-07 19:47:34+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243457947/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243457947/rerun │ MDEwOkNoZWNrU3VpdGUxMTU2NDU2ODE1 │ [] │ 243457947 │ MDExOldvcmtmbG93UnVuMjQzNDU3OTQ3 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243457947/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243457947/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243457947 │ 66463beac16e48b12f001637791d966786539047 │ {'trees_url': 'https://api.github.com/repos/zbrookle/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/zbrookle/ibis/teams', ... +44} │ 2020-09-07 18:47:22+00:00 │ fix_slowdown_caused_by_fixing_aliases │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243457947 │ pull_request │ Main │ NULL │ 2020-09-07 18:47:22+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1156456815 │ 1156456815 │ success │ {'tree_id': '0af846ddd6161bfae9fcd558d58fa6026ebb1ff0', 'timestamp': datetime.datetime(2020, 9, 7, 18, 47, 11, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243457947/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 296 │ 1 │ 2020-09-07 19:44:06+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243454838/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243454838/rerun │ MDEwOkNoZWNrU3VpdGUxMTU2NDQ3MTM5 │ [] │ 243454838 │ MDExOldvcmtmbG93UnVuMjQzNDU0ODM4 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243454838/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243454838/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243454838 │ 19966825608bc00e2dc2a17fc8f3285fb83a5a9d │ {'trees_url': 'https://api.github.com/repos/zbrookle/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/zbrookle/ibis/teams', ... +44} │ 2020-09-07 18:43:53+00:00 │ fix_slowdown_caused_by_fixing_aliases │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243454838 │ pull_request │ Main │ NULL │ 2020-09-07 18:43:53+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1156447139 │ 1156447139 │ success │ {'tree_id': '409b991e08567a5dce9e2325265f3d9660acdf8e', 'timestamp': datetime.datetime(2020, 9, 7, 18, 43, 40, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243454838/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 295 │ 1 │ 2020-09-07 16:35:46+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262051/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262051/rerun │ MDEwOkNoZWNrU3VpdGUxMTU1Nzk3Nzk5 │ [{...}, {...}] │ 243262051 │ MDExOldvcmtmbG93UnVuMjQzMjYyMDUx │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262051/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262051/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243262051 │ 9f44fd7fd2cd9f333a9d4f646a96a75002fa6a08 │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ 2020-09-07 15:35:31+00:00 │ master │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262051 │ push │ Main │ NULL │ 2020-09-07 15:35:31+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1155797799 │ 1155797799 │ success │ {'tree_id': '4ba3fc9dc3c1d72d10dc726580e5b1661f0c6a49', 'timestamp': datetime.datetime(2020, 9, 7, 15, 35, 28, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262051/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2170517 │ 2170517 │ NULL │ 30 │ 1 │ 2020-09-07 15:56:15+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262053/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262053/rerun │ MDEwOkNoZWNrU3VpdGUxMTU1Nzk3ODA0 │ [{...}, {...}] │ 243262053 │ MDExOldvcmtmbG93UnVuMjQzMjYyMDUz │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262053/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262053/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243262053 │ 9f44fd7fd2cd9f333a9d4f646a96a75002fa6a08 │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ 2020-09-07 15:35:31+00:00 │ master │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262053 │ push │ BigQuery │ NULL │ 2020-09-07 15:35:31+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1155797804 │ 1155797804 │ failure │ {'tree_id': '4ba3fc9dc3c1d72d10dc726580e5b1661f0c6a49', 'timestamp': datetime.datetime(2020, 9, 7, 15, 35, 28, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243262053/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 294 │ 1 │ 2020-09-07 13:34:27+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243022374/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243022374/rerun │ MDEwOkNoZWNrU3VpdGUxMTU0OTE2NzEw │ [] │ 243022374 │ MDExOldvcmtmbG93UnVuMjQzMDIyMzc0 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243022374/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243022374/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243022374 │ 6b086d1c7d2a66535aa7d2416b7700b44bffc23b │ {'trees_url': 'https://api.github.com/repos/datapythonista/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/datapythonista/ibis/teams', ... +44} │ 2020-09-07 12:34:14+00:00 │ pyspark2 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243022374 │ pull_request │ Main │ NULL │ 2020-09-07 12:34:14+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1154916710 │ 1154916710 │ success │ {'tree_id': '986fe0ce796e5e5f271660e61a8e34ac8b3aad83', 'timestamp': datetime.datetime(2020, 9, 7, 12, 33, 54, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243022374/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 293 │ 1 │ 2020-09-07 13:16:27+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243013375/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243013375/rerun │ MDEwOkNoZWNrU3VpdGUxMTU0ODg2NDQ3 │ [] │ 243013375 │ MDExOldvcmtmbG93UnVuMjQzMDEzMzc1 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243013375/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243013375/artifacts │ https://github.com/ibis-project/ibis/actions/runs/243013375 │ cd4b02f33abd85dc543a24f28d4f2160a4e37be1 │ {'trees_url': 'https://api.github.com/repos/datapythonista/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/datapythonista/ibis/teams', ... +44} │ 2020-09-07 12:27:54+00:00 │ backends_toc │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243013375 │ pull_request │ Main │ NULL │ 2020-09-07 12:27:54+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1154886447 │ 1154886447 │ success │ {'tree_id': '72f78d3b17b37cd795b0ad3e4206aae06e5b0f5f', 'timestamp': datetime.datetime(2020, 9, 7, 12, 25, 33, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/243013375/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2100986 │ 2100986 │ NULL │ 292 │ 1 │ 2020-09-07 13:09:18+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985647/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985647/rerun │ MDEwOkNoZWNrU3VpdGUxMTU0Nzk5ODYy │ [{...}, {...}] │ 242985647 │ MDExOldvcmtmbG93UnVuMjQyOTg1NjQ3 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985647/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985647/artifacts │ https://github.com/ibis-project/ibis/actions/runs/242985647 │ a37c24cdf213c67ce844e27279f4fceb46358c80 │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ 2020-09-07 12:09:06+00:00 │ master │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985647 │ push │ Main │ NULL │ 2020-09-07 12:09:06+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1154799862 │ 1154799862 │ success │ {'tree_id': '01da912d363a08018b3dc4b5ed61d89ee5b964ac', 'timestamp': datetime.datetime(2020, 9, 7, 12, 9, 3, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985647/logs │ │ https://api.github.com/repos/ibis-project/ibis/actions/workflows/2170517 │ 2170517 │ NULL │ 29 │ 1 │ 2020-09-07 12:28:42+00:00 │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985648/cancel │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985648/rerun │ MDEwOkNoZWNrU3VpdGUxMTU0Nzk5ODY1 │ [{...}, {...}] │ 242985648 │ MDExOldvcmtmbG93UnVuMjQyOTg1NjQ4 │ completed │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985648/jobs │ NULL │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985648/artifacts │ https://github.com/ibis-project/ibis/actions/runs/242985648 │ a37c24cdf213c67ce844e27279f4fceb46358c80 │ {'trees_url': 'https://api.github.com/repos/ibis-project/ibis/git/trees{/sha}', 'teams_url': 'https://api.github.com/repos/ibis-project/ibis/teams', ... +44} │ 2020-09-07 12:09:06+00:00 │ master │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985648 │ push │ BigQuery │ NULL │ 2020-09-07 12:09:06+00:00 │ https://api.github.com/repos/ibis-project/ibis/check-suites/1154799865 │ 1154799865 │ failure │ {'tree_id': '01da912d363a08018b3dc4b5ed61d89ee5b964ac', 'timestamp': datetime.datetime(2020, 9, 7, 12, 9, 3, tzinfo=<UTC>), ... +4} │ https://api.github.com/repos/ibis-project/ibis/actions/runs/242985648/logs │ │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ … │ └──────────────────────────────────────────────────────────────────────────┴─────────────┴──────────────────┴────────────┴─────────────┴───────────────────────────┴──────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────┴───────────┴──────────────────────────────────┴───────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────┴──────────────────────┴─────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────┴──────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────┴───────────────────────────────────────┴───────────────────────────────────────────────────────────────────────┴──────────────┴──────────┴───────┴───────────────────────────┴────────────────────────────────────────────────────────────────────────┴────────────────┴────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────┘
Again we have a bunch of columns that aren’t so useful to us, so let’s see what else is there.
workflows.columns('workflow_url',
'workflow_id',
'triggering_actor',
'run_number',
'run_attempt',
'updated_at',
'cancel_url',
'rerun_url',
'check_suite_node_id',
'pull_requests',
'id',
'node_id',
'status',
'repository',
'jobs_url',
'previous_attempt_url',
'artifacts_url',
'html_url',
'head_sha',
'head_repository',
'run_started_at',
'head_branch',
'url',
'event',
'name',
'actor',
'created_at',
'check_suite_url',
'check_suite_id',
'conclusion',
'head_commit',
'logs_url')We don’t care about many of these for the purposes of this analysis, however we need the id and a few values derived from the run_started_at column.
id: the unique identifier of the workflow runrun_started_at: the time the workflow run started
We compute the date the run started at so we can later compare it to the dates where we added poetry and switched to the team plan.
workflows = workflows.select(
_.id, _.run_started_at, started_date=_.run_started_at.date()
)
workflows┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓ ┃ id ┃ run_started_at ┃ started_date ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩ │ int64 │ timestamp('UTC') │ date │ ├───────────┼───────────────────────────┼──────────────┤ │ 287271818 │ 2020-10-04 01:41:55+00:00 │ 2020-10-04 │ │ 298246779 │ 2020-10-09 21:10:32+00:00 │ 2020-10-09 │ │ 298245816 │ 2020-10-09 21:09:44+00:00 │ 2020-10-09 │ │ 298245817 │ 2020-10-09 21:09:44+00:00 │ 2020-10-09 │ │ 298037703 │ 2020-10-09 19:41:09+00:00 │ 2020-10-09 │ │ 298011263 │ 2020-10-09 18:20:59+00:00 │ 2020-10-09 │ │ 297928727 │ 2020-10-09 17:22:33+00:00 │ 2020-10-09 │ │ 297919985 │ 2020-10-09 17:16:42+00:00 │ 2020-10-09 │ │ 297916050 │ 2020-10-09 17:13:56+00:00 │ 2020-10-09 │ │ 297847529 │ 2020-10-09 16:27:03+00:00 │ 2020-10-09 │ │ … │ … │ … │ └───────────┴───────────────────────────┴──────────────┘
We need to associate jobs and workflows somehow, so let’s join them on the relevant key fields.
joined = jobs.join(workflows, jobs.run_id == workflows.id)
joined┏━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓ ┃ run_id ┃ job_duration ┃ last_job_started_at ┃ last_job_completed_at ┃ id ┃ run_started_at ┃ started_date ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩ │ int64 │ int64 │ timestamp('UTC') │ timestamp('UTC') │ int64 │ timestamp('UTC') │ date │ ├───────────┼──────────────┼───────────────────────────┼───────────────────────────┼───────────┼───────────────────────────┼──────────────┤ │ 240174688 │ 1137000000 │ 2020-09-04 23:20:53+00:00 │ 2020-09-04 23:39:52+00:00 │ 240174688 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174688 │ 1139000000 │ 2020-09-04 23:20:53+00:00 │ 2020-09-04 23:39:52+00:00 │ 240174688 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 3001000000 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 476000000 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 477000000 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 1073000000 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 0 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 633000000 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240174689 │ 579000000 │ 2020-09-05 00:20:53+00:00 │ 2020-09-05 00:20:53+00:00 │ 240174689 │ 2020-09-04 23:20:42+00:00 │ 2020-09-04 │ │ 240099924 │ 1056000000 │ 2020-09-04 22:49:21+00:00 │ 2020-09-04 22:49:21+00:00 │ 240099924 │ 2020-09-04 22:02:51+00:00 │ 2020-09-04 │ │ … │ … │ … │ … │ … │ … │ … │ └───────────┴──────────────┴───────────────────────────┴───────────────────────────┴───────────┴───────────────────────────┴──────────────┘
Sweet! Now we have workflow runs and job runs together in the same table, let’s start exploring summarization.
Let’s encode our knowledge about when the poetry move happened and also when we moved to the team plan.
from datetime import date
POETRY_MERGED_DATE = date(2021, 10, 15)
TEAMIZATION_DATE = date(2022, 11, 28)Let’s compute some indicator variables indicating whether a given row contains data after poetry changes occurred, and do the same for the team plan.
Let’s also compute queueing time and workflow duration.
stats = joined.select(
_.started_date,
_.job_duration,
has_poetry=_.started_date > POETRY_MERGED_DATE,
has_team=_.started_date > TEAMIZATION_DATE,
queueing_time=_.last_job_started_at.delta(_.run_started_at, "microsecond"),
workflow_duration=_.last_job_completed_at.delta(_.run_started_at, "microsecond"),
)
stats┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓ ┃ started_date ┃ job_duration ┃ has_poetry ┃ has_team ┃ queueing_time ┃ workflow_duration ┃ ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩ │ date │ int64 │ boolean │ boolean │ int64 │ int64 │ ├──────────────┼──────────────┼────────────┼──────────┼───────────────┼───────────────────┤ │ 2021-02-18 │ 986000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 1282000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 836000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 974000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 972000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 1258000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 731000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 1047000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 803000000 │ False │ False │ 1432000000 │ 1432000000 │ │ 2021-02-18 │ 0 │ False │ False │ 1432000000 │ 1432000000 │ │ … │ … │ … │ … │ … │ … │ └──────────────┴──────────────┴────────────┴──────────┴───────────────┴───────────────────┘
Let’s create a column ranging from 0 to 2 inclusive where:
- 0: no improvements
- 1: just poetry
- 2: poetry and the team plan
Let’s also give them some names that’ll look nice on our plots.
stats = stats.mutate(
raw_improvements=_.has_poetry.cast("int") + _.has_team.cast("int")
).mutate(
improvements=_.raw_improvements.cases(
(0, "None"),
(1, "Poetry"),
(2, "Poetry + Team Plan"),
else_="NA",
),
team_plan=ibis.ifelse(_.raw_improvements > 1, "Poetry + Team Plan", "None"),
)
stats┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━┓ ┃ started_date ┃ job_duration ┃ has_poetry ┃ has_team ┃ queueing_time ┃ workflow_duration ┃ raw_improvements ┃ improvements ┃ team_plan ┃ ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━┩ │ date │ int64 │ boolean │ boolean │ int64 │ int64 │ int64 │ string │ string │ ├──────────────┼──────────────┼────────────┼──────────┼───────────────┼───────────────────┼──────────────────┼──────────────┼───────────┤ │ 2020-08-05 │ 3013000000 │ False │ False │ 8000000 │ 3021000000 │ 0 │ None │ None │ │ 2020-08-05 │ 2809000000 │ False │ False │ 9000000 │ 2818000000 │ 0 │ None │ None │ │ 2020-08-05 │ 361000000 │ False │ False │ 39000000 │ 400000000 │ 0 │ None │ None │ │ 2020-08-05 │ 1626000000 │ False │ False │ 7000000 │ 1633000000 │ 0 │ None │ None │ │ 2020-08-05 │ 7000000 │ False │ False │ 9000000 │ 16000000 │ 0 │ None │ None │ │ 2020-08-05 │ 2914000000 │ False │ False │ 11000000 │ 2925000000 │ 0 │ None │ None │ │ 2020-08-05 │ 1868000000 │ False │ False │ 10000000 │ 1878000000 │ 0 │ None │ None │ │ 2020-08-05 │ 1999000000 │ False │ False │ 10000000 │ 2009000000 │ 0 │ None │ None │ │ 2020-08-05 │ 1834000000 │ False │ False │ 12000000 │ 1846000000 │ 0 │ None │ None │ │ 2020-08-05 │ 1890000000 │ False │ False │ 9000000 │ 1899000000 │ 0 │ None │ None │ │ … │ … │ … │ … │ … │ … │ … │ … │ … │ └──────────────┴──────────────┴────────────┴──────────┴───────────────┴───────────────────┴──────────────────┴──────────────┴───────────┘
Finally, we can summarize by averaging the different durations, grouping on the variables of interest.
USECS_PER_MIN = 60_000_000
agged = stats.group_by(_.started_date, _.improvements, _.team_plan).agg(
job=_.job_duration.div(USECS_PER_MIN).mean(),
workflow=_.workflow_duration.div(USECS_PER_MIN).mean(),
queueing_time=_.queueing_time.div(USECS_PER_MIN).mean(),
)
agged┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ started_date ┃ improvements ┃ team_plan ┃ job ┃ workflow ┃ queueing_time ┃ ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ date │ string │ string │ float64 │ float64 │ float64 │ ├──────────────┼────────────────────┼────────────────────┼──────────┼───────────┼───────────────┤ │ 2022-03-01 │ Poetry │ None │ 3.542917 │ 17.623869 │ 16.112236 │ │ 2023-07-23 │ Poetry + Team Plan │ Poetry + Team Plan │ 6.377758 │ 23.267700 │ 21.250076 │ │ 2021-03-26 │ None │ None │ 9.974310 │ 17.826115 │ 1.752335 │ │ 2021-10-11 │ None │ None │ 5.243452 │ 12.968254 │ 12.952778 │ │ 2022-01-31 │ Poetry │ None │ 3.613774 │ 27.807325 │ 26.611106 │ │ 2022-07-01 │ Poetry │ None │ 3.049880 │ 12.841667 │ 12.191460 │ │ 2022-09-09 │ Poetry │ None │ 2.687013 │ 35.551232 │ 34.742179 │ │ 2023-06-30 │ Poetry + Team Plan │ Poetry + Team Plan │ 5.946356 │ 22.992439 │ 20.968384 │ │ 2023-10-09 │ Poetry + Team Plan │ Poetry + Team Plan │ 5.977082 │ 30.373458 │ 28.128672 │ │ 2023-11-20 │ Poetry + Team Plan │ Poetry + Team Plan │ 3.469057 │ 10.382100 │ 8.987097 │ │ … │ … │ … │ … │ … │ … │ └──────────────┴────────────────────┴────────────────────┴──────────┴───────────┴───────────────┘
If at any point you want to inspect the SQL you’ll be running, ibis has you covered with ibis.to_sql.
ibis.to_sql(agged)SELECT
`t7`.`started_date`,
`t7`.`improvements`,
`t7`.`team_plan`,
AVG(ieee_divide(`t7`.`job_duration`, 60000000)) AS `job`,
AVG(ieee_divide(`t7`.`workflow_duration`, 60000000)) AS `workflow`,
AVG(ieee_divide(`t7`.`queueing_time`, 60000000)) AS `queueing_time`
FROM (
SELECT
`t6`.`started_date`,
`t6`.`job_duration`,
`t6`.`has_poetry`,
`t6`.`has_team`,
`t6`.`queueing_time`,
`t6`.`workflow_duration`,
CAST(`t6`.`has_poetry` AS INT64) + CAST(`t6`.`has_team` AS INT64) AS `raw_improvements`,
CASE CAST(`t6`.`has_poetry` AS INT64) + CAST(`t6`.`has_team` AS INT64)
WHEN 0
THEN 'None'
WHEN 1
THEN 'Poetry'
WHEN 2
THEN 'Poetry + Team Plan'
ELSE 'NA'
END AS `improvements`,
IF(
(
CAST(`t6`.`has_poetry` AS INT64) + CAST(`t6`.`has_team` AS INT64)
) > 1,
'Poetry + Team Plan',
'None'
) AS `team_plan`
FROM (
SELECT
`t4`.`started_date`,
`t5`.`job_duration`,
`t4`.`started_date` > DATE(2021, 10, 15) AS `has_poetry`,
`t4`.`started_date` > DATE(2022, 11, 28) AS `has_team`,
TIMESTAMP_DIFF(`t5`.`last_job_started_at`, `t4`.`run_started_at`, MICROSECOND) AS `queueing_time`,
TIMESTAMP_DIFF(`t5`.`last_job_completed_at`, `t4`.`run_started_at`, MICROSECOND) AS `workflow_duration`
FROM (
SELECT
`t0`.`run_id`,
TIMESTAMP_DIFF(`t0`.`completed_at`, `t0`.`started_at`, MICROSECOND) AS `job_duration`,
MAX(`t0`.`started_at`) OVER (PARTITION BY `t0`.`run_id` ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS `last_job_started_at`,
MAX(`t0`.`completed_at`) OVER (PARTITION BY `t0`.`run_id` ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS `last_job_completed_at`
FROM `ibis-gbq`.`workflows`.`jobs` AS `t0`
) AS `t5`
INNER JOIN (
SELECT
`t1`.`id`,
`t1`.`run_started_at`,
DATE(`t1`.`run_started_at`) AS `started_date`
FROM `ibis-gbq`.`workflows`.`workflows` AS `t1`
) AS `t4`
ON `t5`.`run_id` = `t4`.`id`
) AS `t6`
) AS `t7`
GROUP BY
1,
2,
3Plot the Results
Ibis doesn’t have builtin plotting support, so we need to pull our results into pandas.
Here I’m using plotnine (a Python port of ggplot2), which has great integration with pandas DataFrames.
Generally, plotnine works with long, tidy data so let’s use Ibis’s pivot_longer to get there.
agged_pivoted = (
agged.pivot_longer(
("job", "workflow", "queueing_time"),
names_to="entity",
values_to="duration",
)
.mutate(started_date=_.started_date.cast("timestamp").truncate("D"))
)
df = agged_pivoted.execute()
df.head()| started_date | improvements | team_plan | entity | duration | |
|---|---|---|---|---|---|
| 0 | 2021-10-14 | None | None | job | 7.072183 |
| 1 | 2021-10-14 | None | None | workflow | 25.551891 |
| 2 | 2021-10-14 | None | None | queueing_time | 23.428684 |
| 3 | 2020-09-11 | None | None | job | 13.146855 |
| 4 | 2020-09-11 | None | None | workflow | 48.506604 |
Let’s make our theme lighthearted by using xkcd-style plots.
from plotnine import *
theme_set(theme_xkcd())Create a few labels for our plot.
poetry_label = f"Poetry\n{POETRY_MERGED_DATE}"
team_label = f"Team Plan\n{TEAMIZATION_DATE}"Without the following line you may see large amount of inconsequential warnings that make the notebook unusable.
import logging
# without this, findfont logging spams the notebook making it unusable
logging.getLogger('matplotlib.font_manager').disabled = True
logging.getLogger('plotnine').disabled = TrueHere we show job durations, coloring the points differently depending on whether they have no improvements, poetry, or poetry + team plan.
import pandas as pd
g = (
ggplot(
df.loc[df.entity == "job"].reset_index(drop=True),
aes(x="started_date", y="duration", color="factor(improvements)"),
)
+ geom_point()
+ geom_vline(
xintercept=[TEAMIZATION_DATE, POETRY_MERGED_DATE],
colour=["blue", "green"],
linetype="dashed",
)
+ scale_color_brewer(
palette=7,
type='qual',
limits=["None", "Poetry", "Poetry + Team Plan"],
)
+ geom_text(aes("x", "y"), label=poetry_label, data=pd.DataFrame({"x": [POETRY_MERGED_DATE], "y": [15]}), color="blue")
+ geom_text(aes("x", "y"), label=team_label, data=pd.DataFrame({"x": [TEAMIZATION_DATE], "y": [10]}), color="blue")
+ stat_smooth(method="lm")
+ labs(x="Date", y="Duration (minutes)")
+ ggtitle("Job Duration")
+ theme(
figure_size=(22, 6),
legend_position=(0.67, 0.65),
legend_direction="vertical",
)
)
g.show()
Result #1: Job Duration
This result is pretty interesting.
A few things pop out to me right away:
- The move to poetry decreased the average job run duration by quite a bit. No, I’m not going to do any statistical tests.
- The variability of job run durations also decreased by quite a bit after introducing poetry.
- Moving to the team plan had little to no effect on job run duration.
g = (
ggplot(
df.loc[df.entity != "job"].reset_index(drop=True),
aes(x="started_date", y="duration", color="factor(improvements)"),
)
+ facet_wrap("entity", ncol=1)
+ geom_point()
+ geom_vline(
xintercept=[TEAMIZATION_DATE, POETRY_MERGED_DATE],
linetype="dashed",
)
+ scale_color_brewer(
palette=7,
type='qual',
limits=["None", "Poetry", "Poetry + Team Plan"],
)
+ geom_text(aes("x", "y"), label=poetry_label, data=pd.DataFrame({"x": [POETRY_MERGED_DATE], "y": [75]}), color="blue")
+ geom_text(aes("x", "y"), label=team_label, data=pd.DataFrame({"x": [TEAMIZATION_DATE], "y": [50]}), color="blue")
+ stat_smooth(method="lm")
+ labs(x="Date", y="Duration (minutes)")
+ ggtitle("Workflow Duration")
+ theme(
figure_size=(22, 13),
legend_position=(0.68, 0.75),
legend_direction="vertical",
)
)
g.show()
Result #2: Workflow Duration and Queueing Time
Another interesting result.
Queueing Time
- It almost looks like moving to poetry made average queueing time worse. This is probably due to our perception that faster jobs means faster ci. As we see here that isn’t the case
- Moving to the team plan cut down the queueing time by quite a bit
Workflow Duration
- Overall workflow duration appears to be strongly influenced by moving to the team plan, which is almost certainly due to the drop in queueing time since we are no longer limited by slow job durations.
- Perhaps it’s obvious, but queueing time and workflow duration appear to be highly correlated.
In the next plot we’ll look at that correlation.
g = (
ggplot(agged.execute(), aes(x="workflow", y="queueing_time"))
+ geom_point()
+ geom_rug()
+ facet_grid(". ~ team_plan")
+ labs(x="Workflow Duration (minutes)", y="Queueing Time (minutes)")
+ ggtitle("Workflow Duration vs. Queueing Time")
+ theme(figure_size=(22, 6))
)
g.show()
Conclusions
It appears that you need both a short queue time and fast individual jobs to minimize time spent in CI.
If you have a short queue time, but long job runs then you’ll be bottlenecked on individual jobs, and if you have more jobs than queue slots then you’ll be blocked on queueing time.
I think we can sum this up nicely:
- slow jobs, slow queue: 🤷 blocked by jobs or queue
- slow jobs, fast queue: ❓ blocked by jobs, if jobs are slow enough
- fast jobs, slow queue: ❗ blocked by queue, with enough jobs
- fast jobs, fast queue: ✅