To use the BigQuery client, you will need a Google Cloud Platform account. Use the BigQuery sandbox to try the service for free.
Install dependencies for Ibis’s BigQuery dialect:
pip install ibis-framework[bigquery]
Create a client by passing in the project id and dataset id you wish to operate with:
>>> con = ibis.bigquery.connect(project_id='ibis-gbq', dataset_id='testing')
By default ibis assumes that the BigQuery project that’s billed for queries is also the project where the data lives.
However, it’s very easy to query data that does not live in the billing project.
When you run queries against data from other projects the billing project will still be billed for any and all queries.
If you want to query data that lives in a different project than the billing
project you can use the
>>> db = con.database('other-data-project.other-dataset') >>> t = db.my_awesome_table >>> t.sweet_column.sum().execute() # runs against the billing project
The BigQuery client is accessible through the
See BigQuery for a tutorial on using this backend.
ibis.bigquery.connect function to create a BigQuery
client. If no
credentials are provided, the
pydata_google_auth.default() function fetches default credentials.
Create a BigQueryClient for use with Ibis.
Create a database object.
Create a table expression.
The BigQuery client object¶
To use Ibis with BigQuery, you first must connect to BigQuery using the
ibis.bigquery.connect() function, optionally supplying Google API
import ibis client = ibis.bigquery.connect( project_id=YOUR_PROJECT_ID, dataset_id='bigquery-public-data.stackoverflow' )
User Defined functions (UDF)¶
BigQuery only supports element-wise UDFs at this time.
The interface is very similar to the pandas UDF API:
import ibis.expr.datatypes as dt from ibis.bigquery import udf @udf([dt.double], dt.double) def my_bigquery_add_one(x): return x + 1.0
When you want to use this function you call it like any other Python function–only it must be called on an ibis expression:
t = ibis.table([('a', 'double')]) expr = my_bigquery_add_one(t.a) print(ibis.bigquery.compile(expr))
Google account and user data¶
Accessing user data¶
connect() function provides access to data
stored in Google BigQuery and other sources such as Google Sheets or Cloud
Storage, via the federated query feature. Your machine communicates directly
with the Google APIs.
Storing user data¶
By default, your credentials are stored to a local file, such as
~/.config/pydata/ibis.json. All user data is stored on
your local machine. Use caution when using this library on a shared