Release Notes¶
Note
These release notes are for versions of ibis 1.0 and later. Release notes for pre-1.0 versions of ibis can be found at Release Notes (pre 1.0)
Next 1.x feature release
[Feature] #2095: Implementations of Log Log2 Log10 for OmniSciDB backend
[Feature] #2173: Create ExtractDayOfYear operation and add its support to Clickhouse, CSV, MySQL, OmniSciDB, Pandas, Parquet, PostgreSQL, PySpark, SQLite and Spark
[Feature] #2086: OmniSciDB - Refactor DDL and Client; Add temporary parameter to create_table and “force” parameter to drop_view
[Feature] #2125: [OmniSciDB] Add support for within, d_fully_within and point
[Feature] #1981: Add load_data to sqlalchemy’s backends and fix database parameter for load/create/drop when database parameter is the same than the current database
[Feature] #2083: fillna and nullif implementations for OmnisciDB
[Feature] #2117: Add non-nullable info to schema output
[Feature] #2165: [OmniSciDB] Add “method” parameter to load_data
[Feature] #2178: Added epoch_seconds extraction operation to Clickhouse, CSV, Impala, MySQL, OmniSciDB, Pandas, Parquet, PostgreSQL, PySpark, SQLite, Spark and BigQuery #2273
[Feature] #2060: Add initial support for ibis.random function
[Feature] #2177: Create ExtractWeekOfYear operation and add its support to Clickhouse, CSV, MySQL, Pandas, Parquet, PostgreSQL, PySpark and Spark
[Feature] #2097: Date, DateDiff and TimestampDiff implementations for OmniSciDB
[Feature] #2171: [OmniSciDB] Add support to week extraction
[Feature] #2062: Implement read_csv for omniscidb backend
[Feature] #2232: Add window operations support to SQLite
[Feature] #2126: Add translation rules for isnull() and notnull() for pyspark backend
[Feature] #2175: Create ExtractQuarter operation and add its support to Clickhouse, CSV, Impala, MySQL, OmniSciDB, Pandas, Parquet, PostgreSQL, PySpark, SQLite and Spark
[Feature] #2094: [OmnisciDB] Support add_columns and drop_columns for OmnisciDB table
[Feature] #2093: IsNan implementation for OmniSciDB
[Feature] #2186: ZeroIfNull and NullIfZero implementation for OmniSciDB
[Feature] #2198: Validate that the output type of a UDF is a single element
[Feature] #2233: Add ibis.pandas.trace module to log time and call stack information.
[Feature] #2246: Implement two level dispatcher for execute_node
[Feature] #2260: Add support for implementign custom window object for pandas backend
[Feature] #2270: Add support for Union in the PySpark backend
[Feature] #2285: Add support for casting category dtype in pandas backend
[Feature] #2303: Add
application_name
argument toibis.bigquery.connect
to allow attributing Google API requests to projects that use Ibis.[Feature] #2230: Add intersection to general ibis api
[Feature] #2251: Add
rowid
expression, supported by SQLite and OmniSciDB[Feature] #2347: Add set difference to general ibis api
[Feature] #2081: Add
start_point
andend_point
to PostGIS backend.[Feature] #2306: Add time context in scope in execution for pandas backend
[Feature] #2366: Add PySpark support for ReductionVectorizedUDF
[Feature] #2402: Use Scope class for scope in pyspark backend
[Feature] #2409: FEAT: Support Ibis interval for window in pyspark backend
[Feature] #2473: Support elementwise UDF to return multiple columns for Pandas and PySpark backend
[Feature] #2487: Support analytic and reduction UDF to return multiple columns for Pandas backend
[Feature] #2511: Support reduction UDF without groupby to return multiple columns for Pandas backend
[Feature] #2310: Add hash and hashbytes support for BigQuery backend
[Feature] #2514: Add Struct.from_dict
[Support] #2113: Enabled cumulative ops support for OmniSciDB
[Support] #2096: Added round() support for OmniSciDB
[Support] #2107: Added fragment_size to table creation for OmniSciDB
[Support] #2244: Use an OmniSciDB image stable on CI
[Support] #2234: Remove “experimental” mentions for OmniSciDB and Pandas backends
[Support] #2304: Update
google-cloud-bigquery
dependency minimum version to 1.12.0[Support] #2351: Simplifying tests directories structure
[Support] #2288: Drop support to Python 3.6
[Support] #2497: Move ibis.HDFS, ibis.WebHDFS and ibis.hdfs_connect to ibis.impala.*
[Support] #2505: Remove deprecated ibis.HDFS, ibis.WebHDFS and ibis.hdfs_connect
Next 1.x bugfix release
[Bug] #2134: [OmniSciDB] Fix TopK when used as filter
[Bug] #2170: Fix millisecond issue for OmniSciDB #2167, MySQL #2169, PostgreSQL #2166, Pandas #2168, BigQuery #2273 backends
[Bug] #2127: Fix PySpark error when doing alias after selection
[Bug] #2157: Fix interactive mode returning a expression instead of the value when used in Jupyter
[Bug] #2223: Fix PySpark compiler error when elementwise UDF output_type is Decimal or Timestamp
[Bug] #2252: Allow group_by and order_by as window operation input in pandas backend
[Bug] #2237: Add missing float types to pandas backend
[Bug] #2256: Fix issue with sql_validate call to OmnisciDB.
[Bug] #1320: Added verbose logging to SQL backends
[Bug] #2342: Update impala kerberos dependencies
[Bug] #2367: Fix the covariance operator in the BigQuery backend.
[Bug] #2376: Fix analytic ops over ungrouped and unordered windows on Pandas backend
[Bug] #2387: Fix equality behavior for Literal ops
[Bug] #2386: Fix scope get to use hashmap lookup instead of list lookup
[Bug] #2386: FEAT: Support rolling window UDF with non numeric inputs for pandas backend.
[Bug] #2395: Fix reduction UDFs over ungrouped, bounded windows on Pandas backend
[Bug] #2229: Fix same column with multiple aliases not showing properly in repr
[Bug] #2414: Fix same window op with different window size on table lead to incorrect results for pyspark backend
[Bug] #2410: Fix
aggcontext.Summarize
not always producing scalar (Pandas backend)[Bug] #2461: Table expressions do not recognize macaddr datatype (Postgres backend)
[Bug] #2462: Table expressions do not recognize inet datatype (Postgres backend)
[Bug] #2577: Make StructValue picklable
1.3.0 2020-02-27
[Feature] #1868: Use pandas rolling apply to implement rows_with_max_lookback
[Feature] #1807: Spark client
[Feature] #1830: Spark tests
[Feature] #1871: Add support for Postgres UDFs
[Feature] #1885: Spark UDFs
[Feature] #1858: Add geopandas as output for omniscidb
[Feature] #1860: Add shapely geometries as input for literals
[Feature] #1923: Support timezone aware arrow timestamps
[Feature] #1908: DDL support for Spark backend
[Feature] #1913: PySpark backend
[Feature] #1942: PySpark backend string and column ops
[Feature] #1951: Implement IntervalSub for pandas backend
[Feature] #1945: window operations for pyspark backend
[Feature] #1943: filter for PySpark backend
[Feature] #1952: Validate AsOfJoin tolerance and attempt interval unit conversion
[Feature] #1967: Implement join for PySpark backend
[Feature] #1973: Add support for params, query_schema, and sql in PySpark backend
[Feature] #1974: Add support for date/time operations in PySpark backend
[Feature] #1978: Implement sort, if_null, null_if and notin for PySpark backend
[Feature] #1983: Add support for array operations in PySpark backend
[Feature] #1969: SQLAlchemy Default precision and scale to decimal types for PostgreSQL and MySQL
[Feature] #1987: Add more POSTGIS operations
[Feature] #2001: Pyspark backend bounded windows
[Feature] #1996: Add docstring check to CI for an initial subset files
[Feature] #2004: Clean up window translation logic in pyspark backend
[Feature] #2035: Add support for multi arguments window UDAF for the pandas backend
[Feature] #2047: Introduce a top level vectorized UDF module (experimental). Implement element-wise UDF for pandas and PySpark backend.
[Feature] #1976: Add DenseRank, RowNumber, MinRank, Count, PercentRank/CumeDist window operations to OmniSciDB
[Feature] #2071: Improve many arguments UDF performance in pandas backend.
[Bug] #1867: Fix return type of Struct.from_tuples
[Bug] #1869: Remove -x from pytest call in linux CI
[Bug] #1872: Fixed issues with geo data
[Bug] #1876: Fix the case where we do not have an index when using preceding with intervals
[Bug] #1883: Fix small formatting buglet in PR merge tool
[Bug] #1893: Fix according to bug in pd.to_datetime when passing the unit flag
[Bug] #1891: Make Nodes enforce the proper signature
[Bug] #1888: Fix various Spark backend issues
[Bug] #1899: Pass scope to grouping execution in the pandas backend
[Bug] #1901: OmniSci/MapD - Fix reduction for bool
[Bug] #1912: Fix failing com imports on master
[Bug] #1910: Fix union implementation
[Bug] #1903: sql method doesn’t work when the query uses LIMIT clause
[Bug] #1938: Upgrade to JDK11
[Bug] #1937: Fix incorrect assumptions about attached SQLite databases
[Bug] #1933: Fixes explain operation
[Bug] #1984: Fix CI log for database
[Bug] #2006: Pinning clickhouse-driver<0.1.2
[Bug] #2010: Fix pydocstyle config
[Bug] #2015: Fix handling of Array types in Postgres UDF
[Bug] #2009: Fix pandas backend to treat trailing_window preceding arg as window bound rather than window size (e.g. preceding=0 now indicates current row rather than window size 0)
[Bug] #2041: Change pymapd connection parameter from “session_id” to “sessionid”
[Bug] #2057: Fix datamgr.py fail if IBIS_TEST_OMNISCIDB_DATABASE=omnisci
[Bug] #2069: Fix load data stage for Linux CI
[Bug] #2089: Pin “clickhouse-driver” to “>=0.1.3”
[Support] #1856: Move CI pipelines to Azure
[Support] #1859: Remove pandas exception for rows_with_max_lookback
[Support] #1882: Added CoC section
[Support] #1884: Fix link to NUMFOCUS CoC
[Support] #1896: Removing linting from windows
[Support] #1895: Update link to Presto website
[Support] #1917: Added missing null literal op
[Support] #1926: Fix failing BigQuery tests
[Support] #1866: Rename mapd to omniscidb backend
[Support] #1925: PostGIS enhancements
[Support] #1928: Improve geospatial literals and smoke tests
[Support] #1949: More informative IntegrityError on duplicate columns
[Support] #1948: Add developer docs to get docker setup
[Support] #1950: Remove pandas deprecation warnings
[Support] #1958: Missing geospatial ops for OmniSciDB
[Support] #1960: Developer doc enhancements
[Support] #1961: Disable codecov report upload during CI builds
[Support] #1963: Re-formatting all files using pre-commit hook
[Support] #1971: Disable BigQuery explicitly in all/test_join.py
[Support] #1972: Fix BigQuery backend fixture so batting and awards_players fixture re…
[Support] #1982: Add log for database services
[Support] #1962: Json UUID any
[Support] #1988: Add black step to CI
[Support] #1999: cache all ops in translate()
[Support] #2012: Fixed impala start up issues
[Support] #2030: Pin “semantic_version” to “<2.7” in the docs build CI, fix “builddoc” and “doc” section inside “Makefile” and skip mysql tzinfo on CI to allow to run MySQL using docker container on a hard disk drive.
[Support] #2031: Change omniscidb image tag from v4.7.0 to v5.0.0 on docker-compose recipe
[Support] #1991: Add geospatial tutorial notebook
[Support] #2023: HTML escape column names and types in png repr.
[Support] #2037: Drop support for Python 3.5
[Support] #2043: Ping dependency of rtree to fix CI failure
[Support] #2046: Add Spark to supported backends list
[Support] #2044: Implement covariance for bigquery backend
[Support] #2034: Add initial documentation for OmniSciDB, MySQL, PySpark and SparkSQL backends, add initial documentation for geospatial methods and add links to Ibis wiki page
[Support] #2050: CI: Drop table only if it exists (OmniSciDB)
[Support] #2055: Fix “cudf” import on OmniSciDB backend
[Support] #2052: Add possibility to run tests for separate backend via make test BACKENDS=[YOUR BACKEND]
[Support] #2056: Fix data map for int8 on OmniSciDB backend
[Support] #2061: CI: Fix CI builds related to new pandas 1.0 compatibility
[Support] #2068: use conda for installation black and isort tools
[Support] #2075: Disable Postgres tests on Windows CI.
[Support] #2074: Link pandas issues with xfail tests in pandas/tests/test_udf.py
[Support] #2078: Unpin rtree version
[Support] #2051: [Omnisci] The same SRIDs for test_geo_spatial_binops
[Support] #2077: Change omniscidb image tag from v5.0.0 to v5.1.0 on docker-compose recipe
[Support] #2082: Use user-defined port variables for Omnisci and PostgreSQL tests
[Support] #2079: Pin back version of isort
[Support] #2066: Add support to Python 3.8
1.2.0 2019-06-24
[Feature] #1787: PostGIS support
[Feature] #1825: Accept rows-with-max-lookback as preceding parameter
[Feature] #1838: allow pandas timedelta in rows_with_max_lookback
[Feature] #1836: Add new geospatial functions to OmniSciDB backend
[Bug] #1837: Partially revert #1758
[Bug] #1843: Add max_lookback to window replace and combine functions
[Bug] #1850: Fix repr of empty schema
[Bug] #1851: Fix nested array literal repr
[Bug] #1855: Fix call to psql causing failing CI
[Support]: No space after :release:
[Support]: Bugs go into feature releases
[Support] #1826: Allow passing a branch to ci/feedstock.py
[Support] #1840: Tighter version spec for pytest
[Support] #1844: Raise exception on backends where rows_with_max_lookback can’t be implemented
[Support] #1845: Generate release notes from commits
[Support] #1848: Validate order_by when using rows_with_max_lookback window
[Support] #1847: Skip SQLAlchemy backend tests in connect method in backends.py
1.1.0 2019-06-09
[Feature] #1753: Added float32 and geospatial types for create table from schema
[Feature] #1743: is_computable_arg dispatcher
[Feature] #1777: Enable window function tests where possible
[Feature] #1778: Add marker for xfailing specific backends
[Feature] #1758: Reimplement the pandas backend using topological sort
[Feature] #1771: Add window operations to the OmniSci backend
[Feature] #1790: Add missing date operations and struct field operation for the pandas backend
[Feature] #1785: Allow strings for types in pandas UDFs
[Feature] #1792: Add min periods parameter to Window
[Feature] #1796: Add session feature to mapd client API
[Feature] #1766: Call to_interval when casting integers to intervals
[Feature] #1809: Conslidate trailing window functions
[Bug] #1661: PostgreSQL interval type not recognized
[Bug] #1765: Fix preceding and following with
None
[Bug] #1775: Fix conda create command in documentation
[Bug] #1782: NullIf should enforce that its arguments are castable to a common type
[Bug] #1772: Fix outer_join generating LEFT join instead of FULL OUTER
[Bug] #1789: Fix compatibility and bugs associated with pandas toposort reimplementation
[Bug] #1798: Fix Python 3.5 dependency versions
[Bug] #1783: Allow projections from joins to attempt fusion
[Bug] #1799: Chained mutate operations are buggy
[Bug] #1757: Remove unnecessary (and erroneous in some cases) frame clauses
[Bug] #1818: Fix failing strftime tests on Python 3.7
[Bug] #1819: Fix group_concat test and implementations
[Support] #1749: Update README reference to MapD to say OmniSci
[Support] #1700: Re-enable CI for building step
[Support] #1750: Add Python 3.5 testing back to CI
[Support] #1759: Add isort pre-commit hooks
[Support] #1638: Add StringToTimestamp as unsupported
[Support] #1763: Remove warnings and clean up some docstrings
[Support] #1776: Constrain PyMapD version to get passing builds
[Support] #1781: Update omnisci container version
[Support] #1779: Clean up most xpassing tests
[Support] #1821: Fix typo in UDF signature specification
[Support] #1793: Add pep8speaks integration
[Support] #1810: Pin sphinx version to 2.0.1
[Support] #1805: Fix design and developer docs
[Support] #1814: Add development deps to setup.py
[Support] #1820: Remove decorator hacks and add custom markers
1.0.0 2019-03-26
[Feature] #1608: Add
range_window
andtrailing_range_window
to docs[Feature] #1611: Make ibis node instances hashable
[Feature] #1604: Implement
approx_median
in BigQuery[Feature] #1618: Publish the ibis roadmap
[Feature] #1627: Support string slicing with other expressions
[Feature] #1637: Fix UDF bugs and add support for non-aggregate analytic functions
[Feature] #1655: Add cross join operator to MapD
[Feature] #1636: Remove implicit ordering requirements in the PostgreSQL backend
[Feature] #1669: Allow operations to hide arguments from display
[Feature] #1675: Support column names with special characters in mapd
[Feature] #1686: Getting started with mapd, mysql and pandas
[Feature] #1685: Add pre commit hook
[Feature] #1666: Implement geospatial types for MapD
[Feature] #1678: Implement geospatial functions for MapD
[Feature] #1716: Implement table distinct for the pandas backend
[Feature] #1714: Remove a bunch of deprecated APIs and clean up warnings
[Feature] #1704: Add
find_nodes
to lineage[Feature] #1725: Remove support for passing integers to
ibis.timestamp
[Feature] #1712: Timestamp literal accepts a timezone parameter
[Feature] #1728: Implement BigQuery auth using
pydata_google_auth
[Feature] #1727: Implement default handler for multiple client
pre_execute
[Feature] #1723: Add cross join to the pandas backend
[Feature] #1744: Clean up merge PR script and use the actual merge feature of GitHub
[Feature] #1731: Add SQL method for the MapD backend
[Feature] #1680: Add support for the arbitrary aggregate in the mapd backend
[Feature] #1735: Add black as a pre-commit hook
[Bug] #1600: Fix equality
[Bug] #1651: Make sure we cast when NULL is else in CASE expressions
[Bug] #1648: Remove parameters from mapd
[Bug] #1653: Fixed where operator for MapD
[Bug] #1643: Fix map get with compatible types
[Bug] #1647: Resolves joining with different column names
[Bug] #1701: Use
==
to compare strings[Bug] #1659: Fix MapD exception for
toDateTime
[Bug] #1705: Fix visualizing an ibis expression when showing a selection after a table join
[Bug] #1639: Add Union as an unsuppoted operation for MapD
[Bug] #1706: Fix BigQuery credentials link
[Bug] #1729: Filter the
raw
warning from newer pandas to support older pandas[Bug] #1732: Fix bug in
TableExpr.drop
[Bug] #1738: Fix skipping for new BigQuery auth flow
[Bug] #1741: Fix testing for BigQuery after auth flow update
[Bug] #1737: Fix casting to float in the MapD backend
[Bug] #1742: Fix
NULLIF
implementation for the pandas backend[Bug] #1745: Make
dev/merge-pr.py
script handle PR branches[Support] #1609: Fix conda builds
[Support] #1640: Set up CI with Azure Pipelines
[Support]: Update README.md
[Support] #1642: Fix Azure deployment step
[Support] #1645: Add TableExpr.drop to API documentation
[Support] #1641: Updated MapD server-CI
[Support] #1646: Fix Azure VM image name
[Support] #1650: Add test for reduction returning lists
[Support] #1657: Unpin clickhouse-driver version
[Support] #1658: Fixes a very typo in the pandas/core.py docstring
[Support] #1660: Remove defaults as a channel from azure pipelines
[Support] #1664: Update README.md for impala and kudu
[Support] #1667: Fix flake8 and many other warnings
[Support] #1670: Remove support for Python 2
[Support]: Update docs to reflect Apache Impala and Kudu as ASF TLPs
[Support] #1672: Enabled mysql CI on azure pipelines
[Support] #1681: Update mapd ci to v4.4.1
[Support] #1696: Fix multiple CI issues
[Support] #1694: Use cudf instead of pygdf
[Support] #1746: Use releases to manage documentation
[Support] #1747: Remove tag prefix from versioneer
[Support] #1748: Do not build universal wheels