-
Notifications
You must be signed in to change notification settings - Fork 4
/
CHANGES
352 lines (225 loc) · 9.3 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
0.7.0 2019-05-03
Major features:
* "anonymous users" effectively abolished. Since users without any permanent
record of their existence are useless to test on since one cannot take
statistics on them, they are now just excluded entirely from the buckets
(and therefore experiments) mechanism. There is no more `anonymous`
constraint. This also means that rollouts by tag will no longer unexpectedly
show an experience to logged-out users.
* Bucket numbers are reported in user settings exports, under the key
`__bucket__`. This is for the purposes of tracking buckets for analytics.
Minor features:
* We now verify that all timezones loaded in are timezone-aware. Previously it
was possible to load a timezone-naïve date from a config file which then
caused comparisons to error.
* The error message when using `jacquard conclude` with a branch missing from
the test has been significantly improved.
* Codebase now automatically formatted with `black`.
* Linting is now much better documented. Patch by Pete Owlett.
* A number of testing libraries are now pinned for robustness. Patch by Pete
Owlett.
Bugfixes:
* Error handling improved around the partitioning endpoint: hitting it with a
missing experiment no longer causes a 500.
0.6.2 2018-06-15
Minor features:
* Disjointness proving by dates is now implemented. This means can launch two
50:50 experiments on the same settings if their joined_before/joined_after/
era constraints mean that they apply to disjoint groups of people.
Bugfixes:
* A potentially possible case of an ImportError relating to constraints and
`dateutil` is now fixed.
* Rather than silently accepting bad data and then 500ing on requests,
Jacquard will now validate that all dates specified in an experiment
definition are not naïve (= timezone aware).
0.6.1 2018-05-14
Bugfixes:
* Launching a test with an `era` constraint now properly specialises;
previously was only used in reporting but ignored for setting selection.
0.6.0 2018-02-21
Minor features:
* Improved error message when re-concluding an experiment. Patch by Peter Law.
* Improved error handling when loading config files with recursive structures.
* Improved error messages when not enough buckets are available to launch an
experiment.
* Refactoring work to speed up the tests and remove %-style formatting.
* Internal `bugpoint` command for hunting bugs in Jacquard itself.
* Basic mypy checking on the internals.
Removed:
* `python -m jacquard.cli` no longer works. Use `python -m jacquard` or
`jacquard`: both continue to function as before.
0.5.0 2017-10-19
Major features:
* New command `show experiment` to show extended details of a single
experiment. Patch by Peter Law.
* `jacquard launch` takes a `--relaunch` argument to re-start an experiment as
if for the first time.
Minor features:
* Improved test coverage.
* `storage-flush` is now considered plumbing and needs a `--force` argument.
* `--flush` argument to `storage-import`.
* `jacquard help` command as an alias for using `-h`.
* Storage engines thread-safety refactor.
Experimental features:
* `jacquard settings-under-experiment` shows all settings set by defaults or
experiments and which currently running experiments cover those settings.
Its intended use-case is to discover settings which are 'stable' which users
may want to remove from jacquard and commit to.
Bugfixes:
* Relaunching a test no longer silently corrupts the data store.
0.4.3 2017-09-05
Major features:
* The format of the (temporary) users-by-experiment data has changed.
Bugfixes:
* The dummy directory now always stringifies ID keys.
Removed:
* `jacquard list users`.
* `all_users` in directories.
0.4.2 2017-09-01
Major features:
* Settings can now overlap, as long as the constraints on those settings
constrain to disjoint sets of users.
Bugfixes:
* Fix the Django directory erroring when passed a non-integer ID.
0.4.1 2017-08-31
Bugfixes:
* Upgrade path avoids major breaking from backwards compatibility problems
with storage.
0.4.0 2017-07-05
Major features:
* Major changes to the command structure:
`jacquard show` is now `jacquard show user`,
`jacquard list-users` is now `jacquard list users`,
`jacquard list` is now `jacquard list experiments`.
* `jacquard show defaults` for an explicit output of defaults.
* New command to clear all overrides for a setting or user: `jacquard
clear-overrides`.
* New `union` directory backend, to merge together multiple directories
easily.
* `jacquard rollout` can now roll out to users with or without specific
tags.
Minor features:
* Buckets moved over to the new object-document mapper system, which is a
significant rework of how we store objects.
* `jacquard set-default` now takes an `--add` option, which skips existing
keys.
* More test coverage on some internal utilities.
* Basic test coverage to check layering violations.
* `jacquard set-defaults` now does not do any writes if no changes were made.
0.3.3 2017-03-24
Bugfixes:
* Linting check which caused release to break.
0.3.2 2017-03-23
Minor features:
* Default to more verbose logging from the webserver.
* Add some aggressive debug logging to the cached Redis storage backend.
0.3.1 2017-02-08
Minor features:
* Multiple experiment loading in `jacquard load-experiment`.
* `--skip-launched` argument to `jacquard load-experiment`.
* `jacquard rollout` now has documentation.
* `redis-cloned` storage engine now only uses one background thread.
* Read-only transaction support (minor speed up for `redis`).
* Nicer error messages.
* Storage export can take a `--flush` argument.
* YAML loaded with `safe_load` to avoid accidental code execution.
* Much more extensive test coverage on storage backends.
Bugfixes:
* Package correctly declared zip-unsafe.
* Package supplanted with keyword arguments for PyPI compliance.
0.3.0 2017-01-30
Major features:
* Bucket-based experiment configuration.
* Partial rollout feature.
* `redis-cloned` storage engine no longer experimental.
* Pluggable HTTP endpoints.
* All command-line tools now handle retries gracefully.
Minor features:
* Control-C no longer vomits stack traces in the CLI.
* setuptools plugins are now requirements-checked.
* Integration tests.
* Error message improvements.
* Reworked the constraints system.
* 'named' constraint, the counterpart of 'anonymous'.
Removed:
* etcd storage engine.
0.2.3 2017-01-20
Minor features:
* Improved test coverage over storage engines.
* Explicit dependency on sqlalchemy.
* New experimental "redis-cloned" storage engine.
0.2.2 2017-01-17
Bugfixes:
* Explicitly decode JSON from storage reads as UTF-8. Workaround for a
Python 3.5 compatibility bug.
0.2.1 2017-01-17
Minor features:
* Add an explicit newline to the output of the HTTP API, for ease of
use with curl.
Bugfixes:
* Explicitly encode JSON for storage writes as UTF-8. Workaround for a
Python 3.5 compatibility bug.
0.2.0 2017-01-16
Major features:
* Simplified plugin system, to be able to set up plugins without having
to go through the hassle of using setuptools.
* Values and experiments are now specified as YAML, for ease of reading
and writing.
* `era` constraints for specifying experiments to apply only to users
who registered before or after the start of the test.
* Now packaged for Debian.
* Greatly expanded coverage of documentation.
Minor features:
* Internal plugins API, to unify plugin loading between its different
users.
* Config now supports the `Mapping` protocol for reading other config
file keys.
* User settings from `jacquard show` are now printed as YAML for ease
of reading.
* Experiments system has been refactored to be encapsulated within
`jacquard.experiments` rather than duplicating logic.
* Improvements of test coverage.
* Defaults to loading from /etc/jacquard rather than the current
working directory.
* The config file is now annotated for ease of use.
* Useful error message when a config file is missing.
* Much wider sanity-checking of experiment definitions.
Bugfixes:
* Dropped the old `joined_before`/`joined_after` constraints, which
were horribly broken.
* Various incompatibilities with Python 3.5.
0.1.4 2017-01-13
Major features:
* Beginnings of actual documentation.
Minor features:
* `runserver` is now considered an internal, plumbing command.
* Early infrastructure for building a Debian package.
* Basic test coverage for the constraints system.
Bugfixes:
* Now we always return a list, rather than any iterable, from the WSGI
app for wider compatibility with WSGI servers.
0.1.3 2017-01-13
Major features:
* 100% docstring coverage.
Minor features:
* Various minor refactorings.
* Print out help message when `jacquard` is run with no arguments.
* Hide some internal, plumbing commands.
* Renamed KVStore to StorageEngine.
0.1.2 2017-01-12
Bugfixes:
* Fix the TROVE classifier for the license.
0.1.1 2017-01-12
Major features:
* Directories, a system for querying information about users. Comes
with two built-in directories: one dummy, one for using Django
auth_user tables.
* Constraints, a system for restricting to which users an experiment
applies.
* etcd storage engine.
Minor features:
* Name the package `jacquard-split` so it can be used on PyPI.
* `jacquard list-users`, a command to show all users in the directory.
* config.storage is now thread-safe.
0.1.0 2017-01-12
Initial release.