- TileDB-Py 0.32.5 includes TileDB Embedded 2.26.2
- Fix GroupMetadata backwards compatibility by @kounelisagis in #2102
- Fix typo in API doc for create_bucket by @nickvigilante in #2101
- Update enumerations exception message by @shaunrd0 in #2096
- Handle removal of 'StorageManager' from stats dumps in 2.27 - Part 2 by @kounelisagis #2098
- Revert "Do not run CI in PR twice (#2082)" by @kounelisagis in #2100
TileDB-Py 0.32.4 was inadvertently released against TileDB 2.26.1. This will be corrected in 0.32.5, but the version delta does not justify yanking 0.32.4
- Add
TILEDB_DATETIME_DAY
type support for Arrow by @kounelisagis in #2002 - Extend the
GroupMetadata
functionality to support NumPy arrays by @kounelisagis in #2085 - Add extra argument to Group::add_member by @kounelisagis in #2093
- Handle removal of 'StorageManager' from stats dumps in 2.27 by @kounelisagis in #2088
- Make default value for
vfs.s3.region
in test conditional by @kounelisagis in #2086 - Make error message for non-existing
Enumeration
in test conditional by @kounelisagis in #2087 - Fix randomly failing test
DenseArrayTest::test_open_with_timestamp[False]
by @kounelisagis in #2090 - Fix skipif condition for test_cloud by @kounelisagis in #2084
- Raise error when sparse=True is passed to
tiledb.from_numpy
by @kounelisagis in #2080 - Documentation and CI updates by @kounelisagis in #2078
- Move
DenseArrayImpl
to pure Python by @kounelisagis and @nguyenv in #2071 - Move
PackedBuffer
andpack_metadata_val
to pure Python, and wraptiledb_datatype_size
with pybind by @kounelisagis in #2072 - Move
SparseArrayImpl
to pure Python by @kounelisagis in #2065
- Add CI to test against the TileDB core library built from source with a custom version by @dudoslav and @kounelisagis in #2091
- Do not run CI in PR twice by @dudoslav in #2082
- Disable release for Python 3.8 by @dudoslav in #2081
- Re-enable automatic upload to pypi by @dudoslav and @kounelisagis in #2034
- Drop Python 3.8 support by @kounelisagis in #2079
- TileDB-Py 0.32.3 includes TileDB Embedded 2.26.2
- Override tag version in manylinux container by @dudoslav in #2077
- Fix object_type return value by @kounelisagis in #2073
- TileDB-Py 0.32.1 includes TileDB Embedded 2.26.1
- Move vacuum to pure Python by @kounelisagis in #2067
- Make dump calls conditional by @kounelisagis in #2062
- Bump libtiledb version to 2.26.0 for daily tests by @kounelisagis in #2063
- TileDB-Py 0.32.0 includes TileDB Embedded 2.26.0
- Replace dump() calls with operator<< by @kounelisagis in #1975
- Convert config values to strings in VFS init by @gspowley and @kounelisagis in #2060
- Move DomainIndexer, Stats, and Array helper functions from Cython to pybind by @kounelisagis in #2057
- Refactor Aggregation from Cython to Pure Python by @kounelisagis in #2056
- Move tiledb_object_* API calls from Cython to pybind by @kounelisagis in #2049
- Don't check dtype if column_types is not set - from_pandas by @kounelisagis in #2055
- Fix serialization test and add command line option by @kounelisagis in #2052
- Remove deprecations by @kounelisagis in #2050
- Make from_pandas respect column_types for index dimensions by @kounelisagis in #2046
- Remove Dask test skips and add Dask as a test dependency by @kounelisagis in #2047
- Fix exception when calling label_index on a Query object by @sric0880 and @kounelisagis in #2044
- Improve Query Condition error messages to align with TileDB-SOMA-Py by @kounelisagis in #2041
- Fix numpy.ma.fix_invalid issue in NumPy 2.1.0 by replacing with numpy.ma.masked_invalid by @kounelisagis in #2042
- Add scope_ctx to doc build by @spencerseale in #2038
- Passing Empty List to multirange_index and df should return Empty Results by @nguyenv and @kounelisagis in #1412
- Get rid all BNF lines in QueryCondition docstring by @kounelisagis in #2028
- Add tiledb_current_domain_t as type by @kounelisagis in #2033
- Add Enumeration in api ref docs by @kounelisagis in #2026
- Hide sensitive params shown in Config::repr by @kounelisagis in #2029
- Wrap current domain API by @kounelisagis in #2015
- Bump libtiledb and numpy versions for daily tests / make CurrentDomain and NDRectangle code conditional by @kounelisagis in #2036
- Fix compilation warnings by @kounelisagis in #1940
- Add packaging as dependency by @dudoslav in #2032
- Expose setup.py flag to enable deprecations by @kounelisagis in #2025
- Update Cython minimum version to 3 by @kounelisagis in #2027
- Fix malformed doc str for tiledb.array_schema.ArraySchema by @kounelisagis in #2007
- Fix deprecation and test by @kounelisagis in #2021
- Add pandas dependency to test group by @dudoslav and @kounelisagis in #2022
- Define TILEDB_REMOVE_DEPRECATIONS macro for cc and remove deprecated code by @kounelisagis and @dudoslav in #2023
- TileDB-Py 0.31.0 includes TileDB Embedded 2.25.0
- Remove deprecated Array.delete_fragments code path by @teo-tsirpanis and @kounelisagis in #2009
- Fix a typo in an error message by @johnkerl in #2004
- Support ctx argument in array_exists by @kounelisagis in #2003
- Move fragment list consolidation API to pybind by @kounelisagis in #1999
- Add version pins for pandas 3.0 by @kounelisagis #2016
- Scikit-build-core build system rework by @dudoslav and @ihnorton in #1988
- Patches for the build system by @dudoslav, @ihnorton and @kounelisagis in #2010, #2011, #2013, #2014, #2018, #2019, #2020
While we currently plan to maintain support for CentOS 7-compatible systems (GLIBC 2.17) through TileDB 2.31, ecosystem and infrastructure updates following the CentOS 7 end-of-life on 30/Jun/2024 may necessitate dropping support earlier. Please contact us if you still use a CentOS 7 (GLIBC 2.17)-like Linux distribution.
- Fix OverflowError: Python int too large to convert to C long by @kounelisagis in #2000
- Wrap as_built function by @kounelisagis in #1994
- Fix array.query() incorrectly handling nullables by @kounelisagis in #1998
- Add offending column when from_pandas -> _get_column_infos fails by @kounelisagis in #1997
- TileDB-Py 0.30.1 includes TileDB Embedded 2.24.1
- Document Azure, GCS and local support for VFS.ls_recursive by @kounelisagis in #1980
- Skip Dask failing test on Windows by @kounelisagis in #1995
- TileDB-Py 0.30.0 includes TileDB Embedded 2.24.0
- Add test for blob attribute by @kounelisagis in #1985
- Deprecate support for [] indexing with floats by @kounelisagis in #1982
- Fix Query constructor to return error for dense arrays with return_incomplete=True by @kounelisagis in #1976
- Expose WebP enums by @kounelisagis in #1974
- Add Array.query in docs and improve docs in general by @kounelisagis in #1965
- Add support for creating WKB/WKT attributes by @jp-dark in #1912
- Add wrapping for ls recursive by @kounelisagis in #1968
- Fix compatibility for delete_fragments by @kounelisagis in #1966
- Fix pinning wrong numpy version by @kounelisagis in #1989
- Remove pin to MSVC toolset version in CI by @teo-tsirpanis in #1991
- Fix ModuleNotFoundError: No module named 'numpy' on build by @kounelisagis and @ihnorton in #1979
- Add support for numpy2 by @kounelisagis in #1969
- Fix syntax error in nightly build workflow by @ihnorton in #1970
- Set an upper bound for numpy to dodge 2.0 by @sgillies in #1963
- Add numpy upper bound to dodge 2.0 by @kounelisagis in #1993
- TileDB-Py 0.29.0 includes TileDB Embedded 2.23.0
- Add wrapping for ls_recursive by @kounelisagis in #1933
- Migrate away from deprecated TileDB C++ APIs by @kounelisagis in #1958
- Pybind11 Config should honor prefix for iter by @Shelnutt2 in #1962
- Fix test skipping by @kounelisagis in #1957
- Make timestamp overrides optional in tests and add faketime test by @kounelisagis in #1953
- Wrap tiledb_array_consolidate_fragments from pybind11 by @kounelisagis in #1948
- TileDB-Py 0.28.0 includes TileDB Embedded 2.22.0
- Update type signature for VFS::readinto by @kounelisagis in #1937
- Show enumerated value-types in enum-printer by @kounelisagis in #1936
- Add wrapping for new consolidation plan API by @kounelisagis in #1935
- Add test for Group constructor invalid uri object type by @kounelisagis in #1941
- Update doc for tiledb.consolidate by @ihnorton in #1946
- Improve documentation of from_numpy function by @kounelisagis in #1942
- Exclude .pytest_cache and .hypothesis files by @kounelisagis in #1932
- Remove modular building option by @kounelisagis in #1938
- Fix wrong version number for Python API docs by @kounelisagis in #1947
- Remove conditional code for TileDB < 2.16 by @kounelisagis in #1949
- Update nightly test target to 2.21 by @ihnorton in #1950
- TileDB-Py 0.27.1 includes TileDB Embedded 2.21.1
- Add tests to ensure empty result on query condition for invalid enum. 1882
- TileDB-Py 0.27.0 includes TileDB Embedded 2.21.0
- Fix VFS
read
,seek
with numpy integer sizes. #1927 - Remove erroneous
_ctx
check for GroupMetadata #1925
- Fix vfs readinto when buff is not bytes. #1915
- Update daily test builds to use single source of truth for libtiledb target versions. 1910
- Remove Python 3.7 CI jobs. 1916
- Added API support for TileDB aggregates. #1889
- For compatibility with fsspec and rasterio,
isdir()
,isfile()
, andsize()
aliases have been added to theVFS
class. #1902.
- TileDB-Py 0.26.1 includes TileDB Embedded 2.20.1
- TileDB-Py 0.26.0 includes TileDB Embedded 2.20.0
- Warn when
os.fork()
is used in the presence of a Tiledb context #1876. - Enable GCS in osx-arm64 wheel builds #1899
- TileDB-Py 0.25.0 includes TileDB Embedded 2.19.0
- Fix fill value for complex attributes 1872
- Update current-release nightly target 1873
- Add full check of attribute properties in eq method 1874
- Add all array properties to ArraySchema.eq 1875
- Error out if query condition given empty set 1877
- TileDB-Py 0.24.0 includes TileDB Embedded 2.18.2
- Release the GIL while consolidation. #1865
- Add Group Metadata Consolidation and Vacuuming. #1833
- Fix domain equality check. #1866
- Fix reading DeltaFilter and DoubleDeltaFilter options for FilterList. #1862
- Do not flush VFS handles on closing. #1863
- Add context manager before enter
.array_exists
. #1868 (thanks, new contributorp4perf4ce
!)
- TileDB-Py 0.23.4 includes TileDB Embedded 2.17.4
- Add
COMPRESSION_REINTERPRET_DATATYPE
to allowedFilterOption
#1855 - Add
filter_name
toFilter
class #1856
- Do not use
dtype.kind
in enumeration extend type checking #1853 - Empty enumerations should be casted to the dtype of the enumeration #1854
- Correct writing nullable string attributes and all nulled data #1848
- Pandas 2+ fix: use
pa.schema.with_metadata
, replacing passing metadata topa.schema
constructor #1858
- Correct
Enumeration.extend
to handle integers, include Booleans, of different sizes #1850
- TileDB-Py 0.23.2 includes TileDB Embedded 2.17.3
- Add support for enumeration extension API. #1847
- Support new set membership query condition. #1837
- Create
ArraySchemaEvolution
for new operation. #1839 - Add sparse dimension label example. #1843
- TileDB-Py 0.23.1 includes TileDB Embedded 2.17.1
- TileDB-Py 0.23.0 includes TileDB Embedded 2.17.0
- Support for "enumerated datatypes" (aka categoricals or factors). #1790
- Introduce
Array.read_subarray
andArray.write_subarray
APIs. #1824 - Avoid importing Pandas until we actually use it. #1825
- Make VFS accept path-like objects to refer to files. #1818
- Use object equality check in buffer conversion, fixes state serialization bug in distributed use-case. #1822
- TileDB-Py 0.22.3 includes TileDB Embedded 2.16.3
- Wrap Delta filter. #1710
- TileDB-Py 0.22.2 includes TileDB Embedded 2.16.2
- TileDB-Py 0.22.1 includes TileDB Embedded 2.16.1
- TileDB-Py 0.22.0 includes TileDB Embedded 2.16.0
- Added support for variable-length dimension label reads #1802
- TileDB-Py 0.21.5 includes TileDB Embedded 2.15.4
- Handle empty string passed to Query condition #1774
- TileDB-Py 0.21.4 includes TileDB Embedded 2.15.3
- TileDB-Py 0.21.3 includes TileDB Embedded 2.15.2
- Add support for Group.delete; fixes sc-28030. #1754
- Fix sc-27374: default order mapping fallback. #1736
- Fix for array_exists hiding errors; fixes SC-27849. #1754
- TileDB-Py 0.21.2 includes TileDB Embedded 2.15.1
- Addition of
Group(config=...)
andGroup.set_config
#1715
0.21.0 tag was invalid and thus deleted before PyPI release.
- TileDB-Py 0.21.0 includes TileDB Embedded 2.15.0
- Move
ArraySchema
from Cython to pure Python #1340
- Correct
Attr.fill
for UTF-8 #1594
- TileDB-Py 0.20.0 includes TileDB Embedded TileDB 2.14.0
- TileDB-Py 0.19.1 includes TileDB Embedded TileDB 2.13.1
- Move
Dim
andDomain
from Cython to pure Python #1327
- Ensure NumPy array matches array schema dimensions for dense writes #1514
- Added support for Python 3.11
- TileDB-Py 0.19.0 includes TileDB Embedded TileDB 2.13.0
FragmentInfoList.non_empty_domain
deprecated forFragmentInfoList.nonempty_domain
FragmentInfoList.to_vacuum_num
deprecated forlen(FragmentInfoList.to_vacuum)
FragmentInfoList.to_vacuum_uri
deprecated forFragmentInfoList.to_vacuum
FragmentInfoList.dense
deprecated fornot FragmentInfoList.dense
FragmentInfo.non_empty_domain
deprecated forFragmentInfo.nonempty_domain
FragmentInfo.to_vacuum_num
deprecated forlen(FragmentInfo.to_vacuum)
FragmentInfo.to_vacuum_uri
deprecated forFragmentInfo.to_vacuum
FragmentInfo.dense
deprecated fornot FragmentInfo.dense
FragmentsInfo
deprecated forFragmentInfoList
tiledb.delete_fragments
deprecated forArray.delete_fragments
Array.timestamp
deprecated forArray.timestamp_range
Array.coords_dtype
deprecated with no replacement; combined coords have been removed from libtiledbArray.timestamp
deprecated forArray.timestamp_range
Array.query(attr_cond=...)
deprecated forArray.query(cond=...)
Array.query(cond=tiledb.QueryCondition('expression'))
deprecated forArray.query(cond='expression')
- Add support for
WebpFilter
#1395 - Support Boolean types for query conditions #1432
- Support for partial consolidation using a list of fragment URIs #1431
- Addition of
ArraySchemaEvolution.timestamp
#1480 - Addition of
ArraySchema.has_dim
#1430 - Addition of
Array.delete_array
#1428
- Fix issue where queries in delete mode error out on arrays with string dimensions #1473
- Fix representation of nullable integers in dataframe when using PyArrow path #1439
- Check for uninitialized query state after submit and error out if uninitialized #1483
- Linux wheels now built on
manylinux2014
; previously built onmanylinux2010
- Windows wheels NOT AVAILABLE for this release
- TileDB-Py 0.18.3 includes TileDB Embedded TileDB 2.12.3
- Move
from_numpy
out of Cython into pure Python #1436
- Fix
.df
and.multi_index
always returning attributes applied in query conditions #1433
- TileDB-Py 0.18.2 includes TileDB Embedded TileDB 2.12.2
- TileDB-Py 0.18.1 includes TileDB Embedded TileDB 2.12.1
- Reintroduce moving
Attr
from Cython to pure Python #1411
- Properly handle whitespaces in a query condition #1398
- TileDB-Py 0.18.0 includes TileDB Embedded TileDB 2.12.0
- Changes to query conditions #1341
- Support query conditions on sparse dimensions
- Deprecate
attr_cond
in favor ofcond
- Deprecate passing
tiledb.QueryCondition
tocond
in favor of passing string directly
- Add support for
XORFilter
#1294 - Addition of
Array.delete_fragments
; deprecatetiledb.delete_fragments
#1329 - Array and Group metadata now store bytes as
TILEDB_BLOB
#1384 - Addition of
{Array,Group}.metadata.dump()
#1384 - Addition of
Group.is_relative
to check if the URI component of a group member is relative #1386 - Addition of query deletes to delete data that satisifies a given query condition #1309
- Addition of
FileIO.readinto
#1389
- Addition of Utility Function
get_last_ctx_err_str()
for C API #1351 - Move
Context
andConfig
from Cython to pure Python #1379
- Correct writing empty/null strings to array.
tiledb.main.array_to_buffer
needs to resize data buffer at the end ofconvert_unicode
; otherwise, last cell will be store with trailing nulls chars #1339 - Revert #1326 due to issues with
Context
lifetime with in multiprocess settings #1372
- Permit true-ASCII attributes in non-from-pandas dataframes #1337
- Addition of
Array.upgrade_version
to upgrade array to latest version #1334 - Attributes in query conditions no longer need to be passed to
Array.query
'sattr
arg #1333 ArraySchemaEvolution
checks context's last error for error message #1335
- TileDB-Py 0.17.4 includes TileDB Embedded TileDB 2.11.3
- Addition of
FloatScaleFilter
#1195 - Addition of
d
mode for arrays to delete data that satisfies a given query condition #1309
- Wheels are minimally supported for macOS 10.15 Catalina #1275
- TileDB-Py 0.17.3 includes TileDB Embedded TileDB 2.11.2
- Add ability to pass shape tuple to empty_like #1316
- Support retrieving MBRs of var-sized dimensions #1311
- Wheels will no longer be supported for macOS 10.15 Catalina; the minimum supported macOS version is now 11 Big Sur #1300
- Wheels will no longer supported for Python 3.6 #1300
- TileDB-Py 0.17.2 includes TileDB Embedded TileDB 2.11.1
- Fix issue where querying an array with a Boolean type when
arrow=True
, but is unselected in.query(attr=...)
, results in an errorpyarrow.lib.ArrowInvalid: Invalid column index to set field.
#1291 - Use Arrow type fixed-width binary ("w:") for non-variable TILEDB_CHAR #1286
- Support
datetime64
forQueryCondition
#1279
- TileDB-Py 0.17.0 includes TileDB Embedded TileDB 2.11.0
- TileDB-Py 0.16.5 includes TileDB Embedded TileDB 2.10.4
- TileDB-Py 0.16.4 includes TileDB Embedded TileDB 2.10.3
setup.py
revert back to retrieving core version by usingctypes
by parsingtiledb_version.h
; the tiledb shared object lib now returns back a full path #1226- Update minimum required cmake version to =>3.23; required for building
libtiledb
#1260
- Addition of
in
operator forQueryCondition
#1214 - Revert the regular indexer
[:]
to return entire array rather than nonempty domain in order to maintain NumPy semantics #1261
- Deprecate
Filestore.import_uri
in lieu ofFilestore.copy_from
#1226
- This removes
import tkinter
fromtest_libtiledb.py
which was preventing the conda package from building properly
- TileDB-Py 0.16.2 includes TileDB Embedded TileDB 2.10.2
setup.py
retrieves core version by usingctypes
to calltiledb_version
rather than parsingtiledb_version.h
#1191
- Set nonempty domain to
(None, None)
for empty string #1182
- Support
QueryCondition
for dense arrays #1198 - Querying dense array with
[:]
returns shape that matches nonempty domain, consistent with.df[:]
and.multi_index[:]
#1199 - Addition of
from_numpy
support formode={ingest,schema_only,append}
#1185
- TileDB-Py 0.16.1 includes TileDB Embedded TileDB 2.10.1
- TileDB-Py 0.16.0 includes TileDB Embedded TileDB 2.10.0
- Addition of
Filestore
API #1070 - Use
bool
instead ofuint8
for Boolean dtype indataframe_.py
#1154 - Support
QueryCondition
OR operator #1146
- TileDB-Py 0.15.6 includes TileDB Embedded TileDB 2.9.5
- TileDB-Py 0.15.5 includes TileDB Embedded TileDB 2.9.4
- Support
TILEDB_BLOB
dtype #1159
- Fix error where passing a
Context
toGroup
would segfault intermittenly #1165 - Correct Boolean values when
use_arrow=True
#1167
- Due to a packaging error, the wheels for 0.15.4 should not be used.
- TileDB-Py 0.15.3 includes TileDB Embedded TileDB 2.9.3
- TileDB-Py 0.15.2 includes TileDB Embedded TileDB 2.9.2
- Refactor
MultiRangeIndexer
&DataFrameIndexer
: addition of ABC_BaseIndexer
with virtual method_run_query
and generator_BaseIndexer.__iter__
; remove_iter_state
; and fix bugs related to incomplete queries #1134
- Fix race condition in
{Dense,Sparse}Array.__new__
#1096 - Correcting
stats_dump
issues: Python stats now also in JSON form ifjson=True
, resolve name mangling ofjson
argument andjson
module, and pulling "timer" and "counter" stats fromstats_json_core
forlibtiledb
>=2.3 #1140
- Addition of
tiledb.DictionaryFilter
#1074 - Add support for
Datatype::TILEDB_BOOL
#1110 - Addition of
Group.__contains__
to check if member with given name is in Group #1125 - Support with-statement for
Group
s #1124 - Addition of
keys
,values
, anditems
toGroup.meta
#1123 Group.member
also returns name if given #1141
- TileDB-Py 0.15.1 includes TileDB Embedded TileDB 2.9.1
- TileDB-Py 0.15.0 includes TileDB Embedded TileDB 2.9.0
- Wheels will no longer be supported for macOS 10.14 Mojave; the minimum supported macOS version is now 10.15 Catalina #1080
- TileDB-Py 0.14.5 includes TileDB Embedded TileDB 2.8.3
- Update
MACOSX_DEPLOYMENT_TARGET
from 10.14 to 10.15 #1080
- Correct handling of Arrow cell count with all empty result #1082
- Refactor display of TileDB objects in Jupyter notebooks to be more readable #1049
- Improve documentation for
Filter
,FilterList
,VFS
,FileIO
,Group
, andQueryCondition
#1043, #1058
Dim.shape
correctly errors out if type is not integer or datetime #1055- Correctly check dtypes in
from_pandas
for supported versions of NumPy <1.20 #1054 - Fix Arrow Table lifetime issues when using
.query(return_arrow=True)
#1056
- TileDB-Py 0.14.2 includes TileDB Embedded TileDB 2.8.2
- Add
Group
andObject
to docs #1040
- Correct
Group.__repr__
to call correct_dump
function #1040 - Check type of
ctx
infrom_pandas
andfrom_csv
#1042 - Only allow use of
.df
indexer for.query(return_arrow=True)
; error out with meaningful error message otherwise #1045
- TileDB-Py 0.14.1 includes TileDB Embedded TileDB 2.8.1
- TileDB-Py 0.14.0 includes TileDB Embedded TileDB 2.8.0
- Addition of
Group
andObject
classes to support improved groups #1022
- The Python 3.10 / manylinux2014 wheels for TileDB-Py 0.13.3 include TileDB Embedded TileDB 2.7.2 built with GCS support
- Move
VFS
,FileIO
, andFileHandle
classes from Cython to Pybind11 #934
- TileDB-Py 0.13.2 includes TileDB Embedded TileDB 2.7.2
- Move
FilterList
andFilter
classes from Cython to Pybind11 #921
- Fix default validity for write to nullable attribute #994
- Reduce query time for dense var-length arrays by including extra offset element in initial buffer allocation #1005
- Fix round-trippable repr for dimension tile #998
- Addition of
ArraySchema.version
to get version of array schema #949 - Deprecate
coords_filters
fromArraySchema
#993 - Allow setting
ascii
incolumn_type
forfrom_pandas
/from_csv
#999
- TileDB-Py 0.13.1 includes TileDB Embedded TileDB 2.7.1
- TileDB-Py 0.13.0 includes TileDB Embedded TileDB 2.7.0
- Deprecate partial vacuuming #930
- Default
from_csv
to usefilter=ZstdFilter()
if not specified forAttr
orDim
#937
- TileDB-Py 0.12.4 includes TileDB Embedded TileDB 2.6.4
- TileDB-Py 0.12.3 includes TileDB Embedded TileDB 2.6.3
- Properly initalize query in order to retrieve estimate results #920
- Enable building with serialization disabled #924
- Do not print out
FragmentInfo_frags
forrepr
#925 - Error out with
IndexError
when attempting to use a step in the regular indexer #911
- TileDB-Py 0.12.2 includes TileDB Embedded TileDB 2.6.2
- Addition of
ArraySchema.validity_filters
#898
- TileDB-Py 0.12.1 includes TileDB Embedded TileDB 2.6.1
- Cast 'dim
's dtype in
Domainto
strprior to applying
html.escape` #883 - Support attributes with spaces in
QueryCondition
by casting with attr(); values may be casted with val() #886
- TileDB-Py 0.12.0 includes TileDB Embedded TileDB 2.6.0
- Allow writing to dimension-only array (zero attributes) by using assignment to
None
, for example:A[coords] = None
(givenA: tiledb.Array
) #854 - Remove repeating header names for
attr
when displayingArraySchema
in Jupyter Notebooks #856 tiledb.VFS.open
returnsFileIO
object; no longer returnsFileHandle
#802- Addition of
tiledb.copy_fragments_to_existing_array
#864
- HTML escape strings for
Dim
andAttr
'sname
anddtype
#856 - Fix attribute view for multi-indexer #866
- Metadata-related API calls are now 'nogil' #867
- TileDB-Py 0.11.4 includes TileDB Embedded TileDB 2.5.3
- Add rich display for TileDB objects in Jupyter notebooks #824
- Support
TILEDB_STRING_ASCII
for array metadata #828
- Support for Python 3.10 #808
- Addition of
tiledb.version()
to return version as a tuple #801 Query.get_stats
andCtx.get_stats
changed function signature; automaticallyprint_out
stats and add option to output asjson
#809
tiledb.delete_fragments
removes unused schemas #813
- TileDB-Py 0.11.2 includes TileDB Embedded TileDB 2.5.2
- Support dict parameter for 'config' argument to VFS constructor #805
- TileDB-Py 0.11.1 includes TileDB Embedded TileDB 2.5.1
- Support dict parameter for 'config' argument to VFS constructor #805
- Correct libtiledb version checking for Fragment Info API getters' MBRs and array schema name #784
- TileDB-Py 0.11.0 includes TileDB Embedded TileDB 2.5.0
- Addition of MBRs to
FragmentInfo
API #760 - Addition of
array_schema_name
toFragmentInfo
API #777 - Addition of
tiledb.create_array_from_fragments
to copy fragments within a given timestamp range to a new array #777
- Addition of
tiledb.delete_fragments
to remove fragments within a given timestamp range #774
- TileDB-Py 0.10.4 includes TileDB Embedded TileDB 2.4.3
- Error out when applying
QueryCondition
to dense arrays; this feature will be implemented in TileDB Embedded 2.5 #753 - Ensure that indexer, multi-indexer, and .df return the same results when applying
QueryCondition
#753 - Fix error when using .df with PyArrow 6 due to incorrect metadata field in exported schema #764
- Fix #755:
from_pandas
to correctly round-trip unnamed Index #761 - Fix .df indexer bug with empty result set #744
- Close the
PyFragmentInfo
object in theFragmentInfoList
constructor to reflect changes in theFragmentInfo
API in TileDB Embedded 2.5 #752 - Make
ctx
argument optional forArraySchemaEvolution
#743 - Remove
coords_filters
fromArraySchema
for dense arrays #762
- TileDB-Py 0.10.3 includes TileDB Embedded TileDB 2.4.2
- Note that 2.4.1 was skipped due to accidental tagging of the 2.4.1 git tag during CI testing
- Pass
Ctx
toArraySchema.load
infrom_pandas
#709 - Give clear error message when attempting to apply
QueryCondition
on dimensions #722 - Do not add string range when querying empty array #721
- String dimension default fix in core #2436 reverts a previous change in which the nonempty domain was passed the to multi-range indexer if unspecified #712
- Deprecate sparse writes to dense arrays #681
- Addition of
Attr.isascii
#681 - Addition of
Ctx.get_stats
andQuery.get_stats
#698
- Added support for
timestamp
argument intiledb.from_numpy
#699
- Do not require
domain=(None, None)
for string dimensions #662
- Print a warning about ContextVar bug when running under ipykernel < 6.0. #665 Please see #667 for more information.
tiledb.Dim
representation now displaysvar=True
for dimensions withbytes
datatype, consistent withtiledb.Attr
#669
- Fix concurrent use of
Array.multi_index
and.df
by using new instance for each invocation #672 - For attributes, if
var=False
but the bytestring is fixed-width or ifvar=True
but the bytestring is variable length, error out #663
- TileDB-Py 0.10.0 includes TileDB Embedded TileDB 2.4.0 with a number of
new features and improvements, including:
- new platform support: Apple M1
- support for ArraySchema evolution (adding and removing attributes)
- support for Azure SAS (shared access signature) tokens
- When using
Array.multi_index
, an empty result is returned if the nonempty domain is empty #656 - Addition of
Array.set_query
to read array using a serialized query #651
- Support numeric column names in
from_pandas
by casting to str dtype #652 - New
tiledb.ArraySchemaEvolution
API to add and drop attributes from an existing array #657
- Correct listing of consolidated fragments to vacuum in the Fragment Info API by deprecating
FragmentInfoList.to_vacuum_uri
,FragmentInfoList.to_vacuum_num
,FragmentInfo.to_vacuum_uri
, andFragmentInfo.to_vacuum_num
and replacing withFragmentInfoList.to_vacuum
#650 - Correct issue where appending
None
toFilterList
causes segfault by checking thefilter
argument #653
- TileDB-Py 0.9.5 includes TileDB Embedded TileDB 2.3.3
- Consolidate
_nonempty_domain_var
intononempty_domain
#632 - Support more valid Python syntax for
QueryCondition
statements #636 - Addition of
ascii
dtype toAttr
allowsQueryCondition
to support var-length strings #637
- Support pickling for arrays in write-mode #626
- Fixed multi-range indexer to default to explicitly pass in the non-empty domain if dimensions are unspecified #630
- Due to a packaging issue released with 0.9.3 (NumPy ABI compatibility with NumPy < 1.20 for Python 3.8), this section is intentionally left blank.
- Fixed release builder ordering issue which led to CRLF line endings in 0.9.1 source distribution.
- Deprecate
Array.timestamp
and replace withArray.timestamp_range
#616
- Set
ArraySchema.tile_order=None
for Hilbert-ordered arrays #609 - Use CIBW to build release wheels on Linux #613
- Addition of Pickling functionality for
SparseArray
#618
- TileDB-Py 0.9.1 includes TileDB Embedded TileDB 2.3.1
- Support passing a timestamp range for consolidation and vacuuming #603
- FragmentInfo API's to_vacuum_uri() function corrected to iterate through
to_vacuum_num
rather thanfragment_num
#603 - Return "NA" For ArraySchema.tile_order if "Hilbert" #605
- TileDB-Py 0.9.0 includes TileDB Embedded TileDB 2.3 with a significant number of new features and improvements.
- Windows wheels are now built with TileDB Cloud REST support enabled #541
- Addition of
QueryCondition
API to filter query on attributes #576
- Fixed
from_pandas
append error for sparse arrayse: no need to specify 'row_start_idx' #593 - Fixed 'index_dims' kwarg handling for
from_pandas
#590
from_dataframe
function has been removed; deprecated in TileDB-Py 0.6 and replaced byfrom_pandas
.
- Fixed incorrect NumPy ABI target in Linux wheels #590
- QueryCondition API will cast condition values to the datatype of the corresponding attribute #589
- QueryCondition API errors out when there are mismatched attributes to
query
'sattr_cond
andattrs
arguments #589 - QueryCondition API can now parse negative numbers #589
- Disabled libtiledb Werror compilation argument for from-source builds via setup.py #574
- Relaxed NumPy version requirements for from-source builds via setup.py #575
- Fixed FragmentInfoList where context was not being passed to ArraySchema #573
- Fixed FragmentInfoList where context was not being passed to ArraySchema #578
- Fixed read bug due to large estimated result size #579
- Fixed bug reading nullable attributes due to missing buffer resize #581
- Fixed Python output for
tiledb.stats_dump
#586
- TileDB-Py 0.8.9 includes TileDB Embedded TileDB 2.2.9
- Support for iterating over incomplete query results #548
- This feature provides the capability to consume partial query results with a fixed maximum buffer size
rather than the the default behavior of resizing buffers and resubmitting to completion.
Usage example:
examples/incomplete_iteration.py
(along with test in:test_libtiledb.py:test_incomplete_return
)
- This feature provides the capability to consume partial query results with a fixed maximum buffer size
rather than the the default behavior of resizing buffers and resubmitting to completion.
Usage example:
- Rename FragmentsInfo to FragmentInfoList #551
- Dataframe creation uses Zstd default compression level (-1) #552
- Rename Fragment Info API's
non_empty_domain
attribute tononempty_domain
#553 - Added configuration option
py.alloc_max_bytes
to control maximum initial buffer allocation #557
- Fixed incorrected error raised in .df[] indexer when pyarrow not installed #554
- Fixed
from_pandas(attr_filters=None, dim_filters=None)
(previously used internal defaults) #564 - Fixed
from_pandas
write bug due to incorrect classification of str/bytes columns #562 - Fix segfault due to mismatched validity num and data buffer sizes #567
- TileDB-Py 0.8.8 includes TileDB Embedded TileDB 2.2.8
- ArraySchema support for
cell_order="hilbert"
#535
- Fixed regression in
from_pandas
with string-valued index dimensions #526 - Fixed GC lifetime bug in string buffer conversion #525
- Fixed
FilterList
'srepr()
method #528
- TileDB-Py 0.8.6 includes TileDB Embedded TileDB 2.2.7
- Addition of
VFS()
functionscopy_file()
andcopy_dir()
#507 - Add support in
from_pandas
for storing Pandas extension types as variable-length attributes #515 - Add support for sparse writes to dense arrays #521
- Multi-length attributes, regardless of fixed or var-length, do not work query properly with PyArrow enabled due to lack of Arrow List support. When using
.df[]
with PyArrow enabled, we are returning a clear message to the user to usequery(use_pyarrow=False)
#513
- TileDB-Py 0.8.5 includes TileDB Embedded TileDB 2.2.6
- Added example reading/writing RGB (multi-component) array #487
- Restore
tiledb.stats_dump
default toverbose=True
#491 - Remove
non_empty_domain_var()
Fragment Info PyBind11 Function and only useget_non_empty_domain()
for both fixed and var-length domains #505
- Addition of high-level function
array_fragments()
that returns aFragmentsInfo
object #488 - Added support for
from_pandas
/df[]
round-trip of Pandas nullable integer and bool types #480 - Fragment info API example usage now provided at
examples/fragment_info.py
#479 - Fragment info API parameters have been rearranged to match the rest of the TileDB Python API such that the
uri
is provided first andcontext
, an optional parameter that defaults totiledb.default_ctx()
, is provided second #479
- Fix bug in
Attr
to ensure that all Unicode strings are automatically set tovar=True
[#495]#495 - Fix bug in Array.multi_index slicing bug for sparse array with dimension range including 0 #482
- TileDB-Py 0.8.3 includes TileDB Embedded TileDB 2.2.4
- Added
nullable
keyword argument toAttr
constructor #474
- Fix bug in Array.multi_index with slice range including 0 (incorrectly used the nonempty domain as endpoint) #473
- This is a version bump to fix numpy compatibility pinning in the wheel build system.
- TileDB-Py 0.8.1 includes TileDB Embedded TileDB 2.2.3
- TileDB-Py 0.8 does not support Python 2.
- TileDB-Py 0.8.0 includes TileDB Embedded 2.2.2 featuring a number of significant improvements in core storage engine functionality. See release notes for TileDB 2.2.1 and TileDB 2.2.2.
- TileDB-Py 0.8 does not support Python 2.
- Add initial
tiledb.from_parquet
functionality (beta) [a90d5d9b1b] - Preload metadata in .df query path to reduce read latency for remote arrays [79ab12fcf0]
- Update py::dtype usage for compatibility with pybind11 2.6.2 [9d3d3d3c43]
- Cherry-pick commit 9d3d3d3c43 to ix runtime bug in conda packages built against pybind11 2.6.2 9d3d3d3c430f
- TileDB-Py 0.7.x will be the last version of TileDB-Py supporting Python 2.
- Fix read compatibility for empty strings written with 2.1 or 2.2 #462
- Fix #457: make sure to fit automatic tile extent to dim range for date type #464
- TileDB-Py 0.7.x will be the last version of TileDB-Py supporting Python 2.
- TileDB-Py 0.7.5 includes TileDB Embedded 2.1.6
- FragmentInfo API by default returns information from all fragments and dimensions #444
- Add integer multi-indexing for NumPy datetime64 dimensions #447
- Add
from_csv/pandas
support fortimestamp
keyword argument to specify write timestamp #450 - Add verbosity option to
stats_dump()
#452 - Add
unique_dim_values()
to return unique dimension values for a givenSparseArray
#454 - Add support to
query()
for returning subsets of specified dimensions #458 - Optimize string array writes #459
- Fix
Dim.shape
for dense array with datetime dimension #448
- Support selecting subset of dimensions in Array.query via new keyword argument
dims: List[String]
. Thecoords=True
kwarg is still supported for compatibility, and continues to return all dimensions #433 - Support Dim(filters=FilterList) keyword argument to set filters on a per-Dim basis #434
- Support tiledb.from_csv setting attribute and dimension filters by dictionary of {name: filter} #434
- Add ArraySchema.check wrapping
tiledb_array_schema_check
#435 - Add support for attribute fill values
tiledb.Attr(fill=...)
andAttr.fill
getter #437
- tiledb.from_csv keyword arg
attrs_filters
renamed toattr_filters
#434
- Fix bug in
multi_index
slicing of dense arrays #438
- The default result layout for indexing/querying sparse arrays is now TILEDB_UNORDERED #428, #431
- Added documentation for all TileDB-Py configuration parameters #430
- Fixed documentation rendering for
Array.query
#430
- Fix sparse dimension type selection when array type is not specified to from_pandas #429
- Don't pass allows_duplicates=True to dense array constructor (tiledb.from_csv) #428
- TileDB-Py 0.7.2 includes TileDB Embedded 2.1.3 Including a fix for issue #409.
- The default array type for
from_pandas
andfrom_csv
is now dense, if unspecified, except when passing a dataframe with string indexes tofrom_pandas
#424
- Automatically determine column to dimension mapping for
tiledb.from_csv
append mode #408
- Fixed
tiledb.from_csv/dataframe
error when ingesting single-row/index datasets #422 - Fixed intermittent
csv_sparse_col_to_dims
failure due to duplicate result ordering #423
- Added support for
df[]
indexing viatiledb.Array.query
#411 - Modified
stats_dump
to return internal stats as string, allowing for output in Jupyter notebooks #403 - Added
__repr__
toArray
andCtx
#413 tiledb.open
now supportstimestamp
keyword argument #419tiledb.Domain
now supports passing a list ofDim
s without unpacking #419
- Fixed PyPI wheels load error on newer macOS due to overlinkage against system libraries in build process (curl -> libintl) #418
- Fixed PyPI wheels load error on Windows due to building against TBB #419
- Fixed indexing of attribute named 'coords' #414
open_dataframe
now uses the underlying Array'snonempty_domain
to avoid errors opening unlimited domain arrays #409
- TileDB-Py 0.7.0 includes TileDB Embedded 2.1.2
featuring a number of significant improvements, with major highlights including:
- no longer uses Intel TBB for parallelization by default. Along with many benefits to TileDB Embedded, this significantly reduces complications and bugs with python multiprocessing fork mode.
- Support coalescing subarray ranges to give major performance boosts.
- TileDB-Py 0.7 packages on PyPI support macOS 10.13+ and manylinux10-compatible Linux distributions only. For now, wheels could be produced supporting older systems but without Google Cloud Support; if needed, please contact us to discuss.
- Added ".df[]" indexer tiledb.Array: directly returns a Pandas dataframe from a query (uses
multi_index
indexing behavior) #390 - Added parallel CSV ingestion example using Python multiprocessing with
tiledb.from_csv
#397 - Added wrapping and support for TileDB checksumming filters:
ChecksumMD5Filter
andChecksumSHA256Filter
#389 - Removed TBB install from default setup.py, corresponding to TileDB Embedded changes #389
- Add support for 'capacity' kwarg to
from_csv
/from_pandas
#391 - Add support for 'tile' kwarg to
from_csv
/from_pandas
to customize Dim tile extent #391 - Added '--release-symbols' option for building in release optimization with debug symbols #402
- Changed
allows_duplicates
default toTrue
forfrom_csv/from_pandas
#394
- Fixed bug indexing anonymous attributes of sparse arrays using
A[]
(did not affect dense or multi_index) #404 - Fixed rendering of column name in mixed dtype exception #382
- Fixed forwarding of 'ctx' kwarg to from_csv/from_pandas #383
- Fixed type of return values for empty results when indexing a sparse array #384
- Added round-trip tests for all filter
repr
objects #389
Note that we will be removing wheel support for macOS 10.9-10.12 in TileDB-Py 0.7 (planned for release in August 2020). This change is due to upstream (AWS SDK) minimum version requirements. The minimum supported version for macOS wheels on PyPI will be macOS 10.13.
Note that we will be removing support for manylinux1 wheels in TileDB-Py 0.7 (planned for release in August 2020). manylinux1 is based on CentOS5, which has been unsupported for several years. We now provide wheels built with manylinux2010, which is based on CentOS6 / glibc 2.12.
- Bump release target to TileDB 2.0.7
We have added manylinux2010 wheels, corresponding to CentOS6 / glibc 2.12.
We are deprecating support for manylinux1 (CentOS5 / glibc 2.0.7), which is not supported by the Google Cloud Storage SDK. We are planning to remove manylinux1 wheel support in the TileDB-Py 0.7 release.
- Enabled Google Cloud Storage support in macOS and linux (manylinux2010) wheels on PyPI (#364)
- Deprecated
initialize_ctx
in favor ofdefault_ctx(config: tiledb.Config)
#351
- Bump release target to TileDB 2.0.6
- Improved error reporting for input data type mismatches #359
- Added
tiledb.VFS.dir_size
#343 - Added read and buffer conversion statistics for python to
tiledb.stats_dump
#354 - Implemented string deduplication to reduce conversion time for string arrays #357
- Fixed argument order for
Array.consolidate
with a Config override parameter #344
- Bump release target to TileDB 2.0.5
- Fix unnecessary implicit ordering requirement for multi-attribute assignment. #328
- Fix
nonempty_domain
with heterogeneous non-string dimensions (#320)
- Add doctest for
tiledb.vacuum
(#319)
- Fix assignment order for
nonempty_domain
with string dimensions (#308) (test in #311) - Fix bug in string attribute handling for var-length attributes (#307)
- Fix regression reading anonymous attributes from TileDB 1.7 arrays (#311)
- Fix incorrect
multi_index
error when string attribute results are empty (#311)
- Target TileDB version 2.0
- Added support for heterogeneous and string-typed dimensions #304
- Added support for
tiledb_array_vacuum
for cleaning up consolidated fragments
- Added Windows wheels for Python 3.7 and 3.8 on PyPI
- Bump release target to TileDB 1.7.7
- Rebuild/release due to wheel build error on linux for 0.5.7.
- Bump release target to TileDB 1.7.6
- Bump release target to TileDB 1.7.5
- Bump release target to TileDB 1.7.4
- Return coordinates by default for dense
multi_index
queries #259
- Bump release target to TileDB 1.7.3
- macOS wheels are now available on PyPI #258
- Delay default ctx initialization, allows per-process global config options to be controlled by user #256
PyPI packages: https://pypi.org/project/tiledb/0.5.3/
- Reduce i/o overhead of
tiledb.open
and array constructors. #239, #240 - Internal support for retrying incomplete queries in all array indexing modes. #238, #252
- Eliminate reference cycles to improve Ctx cleanup. #249
- Support for retrieving compressor level from filter. #234
- Fix variable-length indexing error. #236
- Fix race condition initializing
tiledb.cloud
mixin from thread pool. #246
- Fix bug in multi_index result buffer calculation #232
- add support for multi-range queries #219
- add support for TileDB array metadata #213
- add support for TILEDB_DATETIME_* attributes, domains, and slicing #211
- add support for retrieving list of fragments written by the most recent write to an array #207
- fix read error with multi-attribute sparse arrays #214
- Bump release target to TileDB 1.6.3
- add
dim_type
keyword argument tofrom_numpy
in order to override inferred Dimension dtype #194 - add
Array.domain_index
: slice over any range within the domain bounds, including negative slices #202
- Bump release target to TileDB 1.6.0
- allow
tiledb.open
andArray.create
to take an optional Ctx to override schema #162 - add
tiledb.array_exists
#167
- wrap query_submits into try / finally blocks correctly propagate KeyboardInterrupt errors while cleaning up resources #155
- fixed OOB access in exception handling path #159
- raise an error when trying to consolidate an open readonly array #172
TileDB-Py 0.4.2 contains several improvements as well as bug-fixes associated with the TileDB 1.5.1 release.
- support for NumPy complex types (#142)
- fixed query buffer memory leak (#151)
- fixed segfault during consolidation (TileDB #1213)
- note: to receive this fix, conda and source builds should be updated to TileDB 1.5.1. TileDB-Py 0.4.2 binaries on PyPI bundle the updated TileDB 1.5.1 library.
- fixed indexing with array dtype different from the platform default (#146)
- fixed
VFS.is_bucket
when VFS is initialized with a Ctx object (#148) - fixed
schema_like
to correctly forward a Ctx keyword arg (#148)
- several high-level API additions (tiledb.open, .save, .empty_like, .schema_like), and serialization improvements including pickling support for DenseArray objects (#129)
- manylinux1 wheels for Python 2.7, 3.5, 3.6, and 3.7 are available on PyPI: https://pypi.org/project/tiledb
This release builds TileDB-Py against TileDB 1.5
- support for variable-length arrays (#120)
-
the Ctx argument is now a keyword argument, simplifying API use in the common case (#122)
for example:
tiledb.DenseArray(ctx, uri, ...)
becomes: tiledb.DenseArray(uri, ...) or optionallytiledb.DenseArray(uri, ..., ctx=ctx)