Dgraph v1.0.15

Fixed

• Fix bug that can cause a Dgraph cluster to get stuck in infinite leader election. (#3391)
• Fix bug in bulk loader that prevented loading data from JSON files. (#3464)
• Fix bug with a potential deadlock by breaking circular lock acquisition. (#3393)
• Properly escape strings containing Unicode control characters for data exports. Fixes #3383. (#3429)
• Initialize tablets map when creating a group. (#3360)
• Fix queries with offset not working with multiple orderasc or orderdesc statements. Fixes #3366. (#3455)
• Vendor in bug fixes from badger. (#3348, #3371, #3460)

Changed

• Use Go v1.12.5 to build Dgraph release binaries.
• Truncate Raft logs even when no txn commits are happening. (3be380b)
• Reduce memory usage by setting a limit on the size of committed entries that can be served per Ready. (#3308)
• Reduce memory usage of pending txns by only keeping deltas in memory. (#3349)
• Reduce memory usage by limiting the number of pending proposals in apply channel. (#3340)
• Reduce memory usage when calculating snapshots by retrieving entries in batches. (#3409)
• Allow snapshot calculations during snapshot streaming. (ecb4547)
• Allow quick recovery from partitions by shortening the deadline of sending Raft messages to 10s. (77b52ac)
• Take snapshots less frequently so straggling Alpha followers can catch up to the leader. Snapshot frequency is configurable via a flag (see Added section). (#3367)
• Allow partial snapshot streams to reduce the amount of data needed to be transferred between Alphas. (#3454)
• Use Badger's StreamWriter to improve write speeds during snapshot streaming. (#3457) (#3442)
• Call file sync explicitly at the end of TxnWriter to improve performance. (#3418)
• Optimize mutation and delta application. (#2987)
• Add logs to show Dgraph config options. (#3337)
• Add -v=3 logs for reporting Raft communication for debugging. These logs start with RaftComm:. (9cd628f)

Added

• Add Alpha flag --snapshot_after (default: 10000) to configure the number of Raft entries to keep before taking a snapshot. (#3367)
• Add Alpha flag --abort_older_than (default: 5m) to configure the amount of time since a pending txn's last mutation until it is aborted. (#3367)
• Add Alpha flag --normalize_node_limit (default: 10000) to configure the limit for the maximum number of nodes that can be returned in a query that uses the @normalize directive. Fixes #3335. (#3467)
• Add Prometheus metrics for latest Raft applied index (dgraph_raft_applied_index) and the max assigned txn timestamp (dgraph_max_assigned_ts). These are useful to track cluster progress. (#3338)
• Add Raft checkpoint index to WAL for quicker recovery after restart. (#3444)

Removed

• Remove size calculation in posting list. (0716dc4)
• Remove a -v=2 log which can be too noisy during Raft replay. (2377d9f).
• Remove dgraph_conf from /debug/vars. Dgraph config options are available via logs. (#3337)

Dgraph v1.0.14

Fixed

• Fix bugs related to best-effort queries. (#3125)
• Stream Raft Messages and Fix Check Quorum. (#3138)
• Fix lin reads timeouts and AssignUid recursion in Zero. (#3203)
• Fix panic when running @groupby(uid) which is not allowed and other logic fixes. (#3232)
• Fix a StartTs Mismatch bug which happens when running multiple best effort queries using the same txn. Reuse the same timestamp instead of allocating a new one. (#3187) (#3246)
• Shutdown extra connections. (#3280)
• Fix bug for queries with @recurse and expand(_all_). (#3179)
• Fix assorted cases of goroutine leaks. (#3074)
• Increment tool: Fix best-effort flag name so best-effort queries run as intended from the tool. (d386fa5)

Added

• Add timeout option while running queries over HTTP. Setting the timeout query parameter /query?timeout=60s will timeout queries after 1 minute. (#3238)
• Add badger tool to release binaries and Docker image for inspecting and debugging data directories.

Dgraph v1.0.13

Note: This release supersedes v1.0.12 with bug fixes. If you're running v1.0.12, please upgrade to v1.0.13. It is safe to upgrade in-place without a data export and import. If you're upgrading from earlier versions, you must export and re-import data.

Fixed

• Fix Raft panic. (8cb69ea)
• Log an error instead of an assertion check for SrcUIDs being nil. (691b3b3)

Dgraph v1.0.12

Note: This release requires you to export and re-import data prior to upgrading or rolling back. The underlying data format has been changed.

Added

• Support gzip compression for gRPC and HTTP requests. (#2843)
• Restore is available from a full binary backup. This is an enterprise feature licensed under the Dgraph Community License.
• Strict schema mode via --mutations flag. By default --mutations=allow is set to allow all mutations; --mutations=disallow disables all mutations; --mutations=strict allows mutations only for predicates which are defined in the schema. Fixes #2277.
• Add dgraph increment tool for debugging and testing. The increment tool queries for the specified predicate (default: counter.val), increments its integer counter value, and mutates the result back to Dgraph. Useful for testing end-to-end txns to verify cluster health. (#2955)
• Support best-effort queries. This would relax the requirement of linearizible reads. For best-effort queries, Alpha would request timestamps from memory instead of making an outbound request to Zero. (#3071)

Changed

• Use the new Stream API from Badger instead of Dgraph's Stream framework. (#2852)
• Discard earlier versions of posting lists. (#2859)
• Make HTTP JSON response encoding more efficient by operating on a bytes buffer
  directly. (ae1d9f3)
• Optimize and refactor facet filtering. (#2829)
• Show badger.Item meta information in dgraph debug output.
• Add new option to dgraph debug tool to get a histogram of key and value sizes. (#2844)
• Add new option to dgraph debug tool to get info from a particular read timestamp.
• Refactor rebuild index logic. (#2851, #2866)
• For gRPC clients, schema queries are returned in the Json field. The Schema proto field is deprecated.
• Simplify design and make tablet moves robust. (#2800)
• Switch all node IDs to hex in logs (e.g., ID 0xa instead of ID 10), so they are consistent with Raft logs.
• Refactor reindexing code to only reindex specific tokenizers. (#2948)
• Introduce group checksums. (#2964, #3085)
• Return aborted error if commit ts is 0.
• Reduce number of "ClusterInfoOnly" requests to Zero by making VerifyUid wait for membership information. (#2974)
• Simplify Raft WAL storage caching. (#3102)
• Build release binary with Go version 1.11.5.

Removed

• Remove LRU cache from Alpha for big wins in query latency reduction (5-10x) and mutation throughput (live loading 1.7x faster). Setting --lru_mb is still required but will not have any effect since the cache is removed. The flag will be used later version when LRU cache is introduced within Badger and configurable from Dgraph.
• Remove --nomutations flag. Its functionality has moved into strict schema mode with the --mutations flag (see Added section).

Fixed

• Use json.Marshal for strings and blobs. Fixes #2662.
• Let eq use string "uid" as value. Fixes #2827.
• Skip empty posting lists in has function.
• Fix Rollup to pick max update commit ts.
• Fix a race condition when processing concurrent queries. Fixes #2849.
• Show an error when running multiple mutation blocks. Fixes #2815.
• Bring in optimizations and bug fixes over from Badger.
• Bulk Loader for multi-group (sharded data) clusters writes out per-group
  schema with only the predicates owned by the group instead of all predicates
  in the cluster. This fixes an issue where queries made to one group may not
  return data served by other groups. (#3065)
• Remove the assert failure in raftwal/storage.go.

Dgraph v1.0.11

Note: By downloading these release binaries you accept the terms of the Dgraph Community License.

Added

• Integrate OpenCensus in Dgraph. (#2739) (eccd250, 9e7fa05)
• Add Dgraph Community License for proprietary features.
• Feature: Full binary backups. This is an enterprise feature licensed under the Dgraph Community License. (#2710)
• Add --enterprise_features flag to enable enterprise features. By enabling enterprise features, you accept the terms of the Dgraph Community License.
• Add minio dep and its deps in govendor. (94daeaf, 35a73e8)
• Add network partitioning tests with blockade tool. (fd4e387, dada74f, e59cbfb, 50b0484, 3c4df36, bec18bc, c8ccb94)
• Add Zero endpoints /assign?what=uids&num=10 and /assign?what=timestamps&num=10 to assign UIDs or transaction timestamp leases.
• Adding the acl subcommand to support acl features (still work-in-progress). (#2795)
• Support custom tokenizer in bulk loader (#2820)
• Support JSON data with Dgraph Bulk Loader. (#2799)

Changed

• Make posting list memory rollup happen right after disk. (#2731)
• Do not retry proposal if already found in CommittedEntries. (#2740)
• Remove ExportPayload from protos. Export returns Status and ExportRequest. (#2741)
• Allow more escape runes to be skipped over when parsing string literal. (#2734)
• Clarify message of overloaded pending proposals for live loader. (#2732)
• Posting List Evictions. (e2bcfda)
• Log when removing a tablet. (#2746)
• Deal better with network partitions in leaders. (#2749)
• Keep maxDelay during timestamp req to 1s.
• Updates to the version output info.
  • Print the go version used to build Dgraph when running dgraph version and in the logs when Dgraph runs. (#2768)
  • Print the Dgraph version when running live or bulk loader. (#2736)
• Checking nil values in the equal function (#2769)
• Optimize query: UID expansion. (#2772)
• Split membership sync endpoints and remove PurgeTs endpoint. (#2773)
• Set the Prefix option during iteration. (#2780)
• Replace Zero's /assignIds?num=10 endpoint with /assign?what=uids&num=10 (see Added section).

Removed

• Remove type hinting for JSON and RDF schema-less types. (#2742)
• Remove deprecated logic that was found using vet. (#2758)
• Remove assert for zero-length posting lists. (#2763)

Fixed

• Restore schema states on error. (#2730)
• Refactor bleve tokenizer usage (#2738). Fixes #2622 and #2601.
• Switch to Badger's Watermark library, which has a memory leak fix. (0cd9d82)
• Fix tiny typo. (#2761)
• Fix Test: TestMillion.
• Fix Jepsen bank test. (#2764)
• Fix link to help_wanted. (#2774)
• Fix invalid division by zero error. Fixes #2733.
• Fix missing predicates after export and bulk load. Fixes #2616.
• Handle various edge cases around cluster memberships. (#2791)
• Change Encrypt to not re-encrypt password values. Fixes #2765.
• Correctly parse facet types for both JSON and RDF formats. Previously the
  parsing was handled differently depending on the input format. (#2797)

Dgraph v1.0.10

Note: This release requires you to export and re-import data. We have changed the underlying storage format.

Added

• The Alter endpoint can be protected by an auth token that is set on the Dgraph Alphas via the --auth_token option. This can help prevent accidental schema updates and drop all operations. (#2692)
• Optimize has function (#2724)
• Expose the health check API via gRPC. (#2721)

Changed

• Dgraph is relicensed to Apache 2.0. (#2652)
• Breaking change. Rename Dgraph Server to Dgraph Alpha to clarify discussions of the Dgraph cluster. The top-level command dgraph server is now dgraph alpha. (#2667)
• Prometheus metrics have been renamed for consistency for alpha, memory, and lru cache metrics. (#2636, #2670, #2714)
• The dgraph-converter command is available as the subcommand dgraph conv. (#2635)
• Updating protobuf version. (#2639)
• Allow checkpwd to be aliased (#2641)
• Better control excessive traffic to Dgraph (#2678)
• Export format now exports on the Alpha receiving the export request. The naming scheme of the export files has been simplified.
• Improvements to the dgraph debug tool that can be used to inspect the contents of the posting lists directory.
• Bring in Badger updates (#2697)

Fixed

• Make raft leader resume probing after snapshot crash (#2707)
• Breaking change: Create a lot simpler sorted uint64 codec (#2716)
• Increase the size of applyCh, to give Raft some breathing space. Otherwise, it fails to maintain quorum health.
• Zero should stream last commit update
• Send commit timestamps in order (#2687)
• Query blocks with the same name are no longer allowed.
• Fix out-of-range values in query parser. (#2690)

Dgraph v1.0.9

Changes

• Rename intern pkg to pb (#2608)

Improvements

• Remove LinRead map logic from Dgraph (#2570)
• Sanity length check for facets mostly.
• Make has function correct w.r.t. transactions (#2585)
• Increase the snapshot calculation interval, while decreasing the min number of entries required; so we take snapshots even when there's little activity.
• Convert an assert during DropAll to inf retry. (#2578)
• Fix a bug which caused all transactions to abort if --expand_edge was set to false. Fixes #2547.
• Set the Applied index in Raft directly, so it does not pick up an index older than the snapshot. Ensure that it is in sync with the Applied watermark. Fixes #2581.
• Pull in Badger updates. This also fixes the Unable to find log file, retry error.
• Improve efficiency of readonly transactions by reusing the same read ts (#2604)
• Fix a bug in Raft.Run loop. (#2606)
• Fix a few issues regarding snapshot.Index for raft.Cfg.Applied. Do not overwrite any existing data when apply txn commits. Do not let CreateSnapshot fail.
• Consider all future versions of the key as well, when deciding whether to write a key or not during txn commits. Otherwise, we'll end up in an endless loop of trying to write a stale key but failing to do so.
• When testing inequality value vars with non-matching values, the response was sent as an error although it should return empty result if the query has correct syntax. (#2611)
• Switch traces to glogs in worker/export.go (#2614)
• Improve error handling for dgraph live for errors when processing RDF and schema files. (#2596)
• Fix task conversion from bool to int that used uint32 (#2621)
• Fix expand(_all_) in recurse queries (#2600).
• Add language aliases for broader support for full text indices. (#2602)

Features

• This version switches Badger Options to reasonable settings for p and w directories. This removes the need to expose --badger.options option and removes the none option from --badger.vlog. (#2605)
• Add support for ignoring parse errors in bulk loader with the option --ignore_error. (#2599)
• Introduction of new command dgraph cert to simplify initial TLS setup. See TLS configuration docs for more info. This is also a breaking change that removes functionality (see below).
• The command dgraph server now accepts a --tls_dir option that specifies the directory of certificates to use. This directory can be created with dgraph cert.
• Add expand(_forward_) and expand(_reverse_) to GraphQL+-. If _forward_ is passed as an argument to expand(), all predicates at that level (minus any reverse predicates) are retrieved.
  If _reverse_ is passed as an argument to expand(), only the reverse predicates are retrieved.

Removed

• Password-protected TLS certificates are no longer supported. The flag --tls_cert_key_passphrase has been removed from dgraph server.
• The options to configure TLS between clients/servers has been simplified to point to a directory containing certificates. The following options have been removed: --tls_on, --tls_cert, --tls_cert_key, --tls_min_version, and --tls_max_version .

Dgraph v1.0.8

Changes

• This version introduces tracking of a few anonymous metrics to measure Dgraph adoption (6c89a2e). These metrics do not contain any specifically identifying information about the user, so most users can leave it on. This can be turned off by setting --telemetry=false flag if needed in Dgraph Zero.

Improvements

• Correctly handle a list of type geo in json (#2482 , #2485).
• Fix the graceful shutdown of Dgraph server, so a single Ctrl+C would now suffice to stop it.
• Fix various deadlocks in Dgraph and set ConfState in Raft correctly (#2548, 4b26b6b).
• Significantly decrease the number of transaction aborts by using SPO as key for entity to entity connections. (#2556 ).
• Do not print error while sending Raft message by default. No action needs to be taken by the user, so it is set to V(3) level.

Features

• Introduce a new /assignIds HTTP endpoint in Zero, so users can allocate UIDs to nodes externally.
• Add a new tool which retrieves and increments a counter by 1 transactionally. This can be used to test the sanity of Dgraph cluster.

Dgraph v1.0.7

Improvements:

• All Oracle delta streams are applied via Raft proposals. This deals better with network partition like edge-cases.
• Fix deadlock in 10-node cluster convergence (eb3910c).
• Make ReadIndex work safely (2e044e1).
• Simplify snapshots, leader now calculates and proposes snapshots to the group (4e64b7f).
• Make snapshot streaming more robust (ea1f544).
• Consolidate all txn tracking logic into Oracle, remove inSnapshot logic.
• Bug fix in Badger, to stop panics when exporting (f048c67).
• Use PreVote to avoid leader change on a node join.
• Fix a long-standing bug where raft.Step was being called via goroutines. It is now called serially.
• Fix context deadline issues with proposals (8b0295b).

Changes:

• Set the --conc flag in live loader default to 1, as a temporary fix to avoid tons of aborts.

v1.0.6

We recommend skipping v1.0.6 and going straight to v1.0.7.

Warning: Due to some issues with application of replication logs, we recommend only using --replicas=1 for this release. We're working on solving these issues for the next release.

Warning: Due to a change in Raft storage format, this upgrade would require you to export and re-import your data, clearing out all the WAL (zw in Zero and w in Server) directories.

Added

• Support GraphQL vars as args for Regexp function. #2353
• Support GraphQL vars with filters. #2359
• Add JSON mutations to raw HTTP. #2396

Fixed

• Fix math >= evaluation. #2365
• Avoid race condition between mutation commit and predicate move. #2392
• Ability to correctly distinguish float from int in JSON. #2398
• Remove dummy data key. #2401
• Serialize applying of Raft proposals. Concurrent application was complex and
  cause of multiple bugs. #2428.
• Improve Zero connections.
• Fix bugs in snapshot move, refactor code and improve performance significantly. #2440, #2442
• Add error handling to GetNoStore. Fixes #2373.
• Fix bugs in Bulk loader. #2449
• Posting List and Raft bug fixes. #2457

Changed

• Pull in Badger v1.5.2.
• Raft storage is now done entirely via Badger. This reduces RAM
  consumption by previously used MemoryStorage. #2433
• Trace how node.Run loop performs.
• Allow tweaking Badger options.

Note: This change modifies some flag names. In particular, Badger options
are now exposed via flags named with --badger. prefix.