This repository has been archived by the owner on Jan 25, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
/
ChangeLog
980 lines (717 loc) · 39 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
2013-04-01 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Passing a filename to the -i switch when generating
random function streams now correctly uses data from that file.
2013-03-27 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* src/Makefile.am: Made single library out of the base libraries,
which makes sense since the libraries have some interdependencies.
2013-03-26 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* scripts/randfunc/multifunc_mean.sh: Output combination of
methods into a single large org table for all alpha.
2013-03-25 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* scripts/randfunc: Scripts to calculate values for t-tests on
random functions. Able to aggregate data from all functions for
the same estimators and perform t-tests on those rather than on
individual functions. Improved generation and stuttering functions
by abstracting data out to the top of scripts for easier
modification. Script to calculate the mean of a column of values
from the extracted results. Allows fast calculation of the mean
and standard deviation of different methods for different alpha
values.
2013-03-22 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* scripts/extract_results.sh: Script adds 0. onto the front of the
folder numbers (005 01 03, etc) provided at the top of the
script. This allows the correct values of alpha to be put into
output files.
2013-03-21 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Minor additions to homogeneous Poisson generator to
use for examples later
* estimator_delta.c: Verbose flags on printlines, obsolete code
removed.
2013-03-18 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* math_util.c: Fixed midpoint calculation where abs was being used
instead of fabs, which resulted in incorrect calculations.
* general_util.c: Estimate at point function returns zero if
function is < 0 at the given point to avoid issues later on.
* experimenter.c: Compare stuttered bins function modified
slightly so sums are calculated before entering the pdf
calculation loop.
* generator.c: Gaussian generator takes into account the output
type requested.
2013-03-17 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_IWLS.c: Fixed soul-destroyingly massive bug where beta
estimates were not being done properly due to commented line.
* generator.c: Random generator seeding moved to places where it
does not reseed when multiple functions are generated in the same
call.
* scripts: Improvements to scripts for automatic running of random
function experiments. More use of arrays to make modifications
simpler - only need to modify stuff at the top of the script. More
usage of variables defined at the top of scripts to control
behaviour and specify input and output files.
* scripts/randfunc/extract_results.sh: Fixed issue where output
files were not correctly created based on the function number.
* scripts/get_goodness.sh: Arrays used for storing values, loop
order modified so that data files can be overwritten.
* scripts/randfunc/runexp_batch.sh: Correction of loops, directory
creation, correct launcher call.
* scripts/randfunc/runtd_exp.sh: Modified to match needs of the
time delay experiments, abstraction of variables, directory
creation.
2013-03-13 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* scripts: Added scripts to automatically run experiments, most
are still works in progress but have the basic functionality
required.
2013-03-12 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Minor improvement to file copying.
* math_util.c: Fix to double precision issue with gauss_transform
function which caused incorrect data to be output.
* experimenter.c: Experiment numbers are output based on the
experiment that is running, not the overall count.
2013-03-11 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.c: Added the -u switch, updated information to detail
its usage.
* file_util.c: Function for copying files, and a function
specifically for copying the set of functions generated by the
gaussian random function generator in order to allow for the
generation of multiple sets of streams from a single function.
* generator.c: Now possible to generate multiple streams from the
same random function by using the -u switch.
2013-03-10 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Check that the number of streams sent to the
generator is not greater than the number of timedelta values that
exist in the parameter file. Pre-generating raw files is no longer
necessary. Using a command like ./deltastream -g data/params.txt
-f rand -t 3 -n 2 -c 5 will automatically generate the number of
random functions required and output the data to the current
directory, and then read from the freshly generated files to
generate streams.
* estimator_delta.c: PMF estimator no longer normalises the values
received from combining functions - these values are normalised by
the function combination done prior to the pmf checks.
* math_util.c: Implemented a function for checking that a double
is less than or equal to another with some precision.
* combinefunction.c: Fixed error with combining functions where
the functions would be combined one step shorter than necessary
due to double precision issues. Using the new function in
math_util to make sure this does not happen.
* experimenter.c: Error for each stream is calculated and output
to results file.
2013-03-09 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* experimenter.c: Fixed issue where the last interval to be
stuttered would not be correctly taken into account.
* file_util.c: Fixed issue where parameter lists were not being
output correctly to file due to length being updated at each call.
* generator.c: Generator outputs the parameters used to generate
the streams to the current directory if the number of functions
generated is greater than 1.
* src/Makefile.am: Renamed launcher to deltastream.
* combinefunction.c: Fixed issue where combination was not
correctly using the normaliser when combining gaussian
vectors. Fixed issue where negative memory was being allocated to
double arrays for data storage.
2013-03-08 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_gaussian.c: Both functions take a parameter to
indicate whether to normalise the function being output to
file. This is the only place that it can be done before the file
is output - the normaliser cannot be passed in as a function
estimate is required to compute it.
* experimenter.c: Stuttering function takes into consideration the
-r switch. If on, the random function filename will be used to
attempt to read files to stutter.
* generator.c: Modified old generator code so that it returns a
double_arr struct rather than messing around with
pointers. Function data is output independently of event
generation so that the graph is not jagged. When generating with
the muparser generator, function output step and interval are
defined by output_step and output_interval in the default
parameter file.
* estimator.c: The estimation functions now take a flag which sets
how they construct file names to read from. If the -r switch is
passed, the function_outfile parameter will be used in the
filename. Otherwise, the expression_outfile parameter will be
used. This allows the estimators to read from files generated by
both types of generators. Estimator now takes normalisation
flag. Returned data is normalised if it is nonzero.
2013-03-06 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Muparser generation code can now generate multiple
sets of streams from the same expression by passing a number to
the -c switch. Now uses an updated output file style to match the
one used by the random function generator.
2013-03-03 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* file_util.c: Fixed memory leak caused by lost pointer references
in the get_event_data_interval function. Fixed issue where
experimenter would not be able to open more files because file
descriptors of directories were not being freed correctly.
* experimenter.c: Goodness values averaged over each function in
the experiment to get a representative value for each
experiment. These values are output to file. Function for
stuttering streams modified to read data from the default
parameter file which should be available whenever it is called. It
now uses the interval time for generation to get the actual number
of array locations it needs to store all the intervals that will
be calculated. The best parameter settings for individual
functions are now output to the top level of each experiment
directory.
2013-03-02 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* experimenter.c: Function estimation experiments now output
correctly to files. Stuttering functions simplified so that two
separate versions are not necessary. Intervals for uniform
stuttering are computed and then passed to a function which
operates on double_arr structs. Non-uniform stuttering intervals
are already specified, so no additional computation is
necessary. "Goodness" calculation based on difference between bin
counts from function estimated from stuttered data and the actual
function.
* launcher.c: Fixed some memory leaks caused by changes to return
types.
* estimator.c: _estimate function return value changed to void* so
that gaussian vectors can also be returned. Single stream
estimation when input file is null correctly grabs data from file
with the new naming convention. Fixed some memory leaks resulting
from changes.
2013-03-01 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.c: Added a way to create a default experiment params
file. Use something like ./launcher -x a -d paramfile to
create. -x switch requires a parameter, so pass something random
to it. It will not be used.
* experimenter.c: Added a function for removing certain intervals
from stream data in order to do model selection. Call with the
same parameters as the experimenter, but add the -s switch. This
creates stuttered files in the input directory. Uniform method and
method where the intervals are specified implemented.
2013-02-27 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* math_util.c: RMS and MSE error functions, function to shift an
array so that none of the values it contains are negative.
* combinefunction.c: Gaussian vector combination now normalises
the values calculated based on the normalisation constant which is
passed in.
* estimator.c: Time delay estimates which use gaussians as the
estimation method return normalised combined functions.
2013-02-26 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* math_util.c: Added some functions for computing the RSS,ESS,TSS,
but not sure if they calculate what they are supposed to.
* estimator_gaussian.c: Fixed memory leak due to incorrect free
function usage.
* experimenter.c: Basic analysis of experiment results is
performed, and the results are output to a file in the directory
for that experiment, currently only for the time delay estimation
experiments.
* estimator.c: New struct for storing data output by the
multi_estimate function so that it can be analysed during
experimentation.
2013-02-25 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_gaussian.c: Added output switch parameter to the
estimator functions. Outputting the contribution of each gaussian
takes extremely long, so if it is not necessary it should be
done. The standard output is just the sum of gaussians if using
the non-raw estimator. To output the contributions as well, a
value of 3 should be passed to the -t switch.
* experimenter.c: The number of functions and number of streams to
work on for each function are defined in the experiment file and
passed to the estimators for each experiment. Data for each
experiment is output to a separate directory and each filename can
be used to identify which of the input functions was used to
generate that data. A parameter file is output to each experiment
directory so that the settings used for that experiment can be
checked later. Correct file is now passed to the single stream
estimator.
* combinefunction.c: Fixed memory access issue caused by
allocation of arrays too small to hold data in cases where the
interval length was a round number.
* general_util.c: Finally fixed event subinterval function causing
errors to show up in valgrind.
* estimator_IWLS.c: More fixes to situations which result in -nan
values being produced. Beta estimate function returns zero if the
denominator is zero to prevent nan results.
2013-02-24 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* file_util.c: Functions for creating and checking for existence
of directories.
* general_util.c: Functions for freeing experiment structs.
* struct.h: Experiment structs added.
* estimator_IWLS.c: Lambda estimates can no longer go below
zero. Any value < 0 is replaced with a zero, representing the fact
that it is not possible to have a negative rate. The weight update
also ignores values < 0 as an additional precaution.
* launcher.c: Added -c switch to specify the number of functions
to generate or estimate from. Modified description of -n in the
help information, since it is now only used to specify the number
of streams to generate or estimate.
* estimator.c: Estimating time delays can now be done on multiple
functions in a directory by using the -c switch to pass in the
number of functions to estimate, and providing the directory to
read from using the -i switch. Single function estimates are done
as before by simply ignoring the -c and -i switches. The file
containing the function is assumed to be inside the directory from
which the script is being run.
* generator.c: Generating streams from functions can now be
performed on multiple functions by using the -c switch to specify
a number of functions. The functions must have already been
generated and follow a specific naming pattern - the
function_outfile parameter in the parameter file followed by
"_n.dat", where n refers to the number of the function. The output
filename contains the original filename, which is generated by
truncating the input filename and prepending it to the output file
used for streams. This makes the assumption that the filename ends
with .dat.
* experimenter.c: Now takes input and output directories as
parameters, or reads them from the parameter file if they are not
specified when the script is called. The input directory is
expected to contain stream data for all the streams that are to be
experimented on. Results will be output to the output directory,
with subdirectories for each experiment to reduce clutter. Reduced
code duplication.
2013-02-23 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_piecewise.c: Event file is read before any calls are
made to the IWLS estimator. This significantly reduces the file
accesses that IWLS needs to make during piecewise estimation.
* estimator_IWLS.c: Modified the weight update function to deal
with lambda values of zero. Previously had errors with division by
zero, but modified the function to set weights of the points with
zero value to zero, rather than getting values of
infinity. Estimator can now take an array of doubles containing
event data as a parameter so that it does not have to read files
multiple times when being called more than once.
* file_util.c: Fixed reallocation on incorrect pointer in
get_event_data_interval.
* experimenter.c: Experiment required parameters are generated
from a list of experiment names in the parameter file - this
should make it possible to add new experiments without having to
modify the internals of the system. Can also specify the type of
estimator used by each experiment.
2013-02-21 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* experimenter.c: Added structs for storing experimental data in a
structured way. Data from the experiment parameter file is loaded
into the exp_set struct.
2013-02-20 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* tests: Tests updated to deal with the changes to return types
and parameters of functions caused by the modification of the
event functions.
* file_util.c: Modified the get_event_data functions to return a
double array instead of a double pointer with the length in the
first location. This has changed a few functions in other places
which read event data, but only small changes were required to get
things working.
* estimator_piecewise.c: Fixed bug where when the interval being
extended into had no events in it the estimator would crash.
2013-02-18 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Old generator functions updated to use the output
switch system.
* launcher.h: Modified usage output to detail the -t switch usage
in more depth. Each output type is listed for all things which
use the data passed to the switch.
* estimator_delta.c: The -t switch now affects what files are
output by the delta estimators. 0 outputs nothing, 1 outputs the
value computed by the function for each guess at delta, 2 outputs
lambda and bin data. The hierarchical estimate outputs its data to
files with "hier" in the name.
2013-02-17 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator.c, general_util.c: Removed duplicated parameter
checking function from estimator and made the one in general util
work.
* estimator_gaussian.c: Estimators check that the parameter list
they receive contains the parameters that they require.
* estimator_delta.c: Restructured delay estimators. Each now has a
function which extracts parameters from the parameter file and
then calls the main function which does the calculations. Also
added the ability to searching for an estimate in a hierarchical
fashion, first making a coarse pass over the array and then
refining the estimate. Functions now also check to make sure that
the parameter list contains the parameters they
require. Estimators output relevant data to files with names which
make sense.
* math_util.c: Implemented a function to check whether a double is
less than another with a certain precision value. This is now used
in the array interval summing function, which should work properly
for most cases.
2013-02-16 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_delta.c: pmf estimator now takes two double_arr
structs containing event data as parameters - this means that
files do not have to be read multiple times unnecessarily. Also
fixed a bug which would mean that only the same two files would be
read regardless of the number of streams. Normalisation constant
function split into two functions to make calling it easier. Also
removed the calculation from inside the pmf estimator and made it
a parameter. There is no point calculating it multiple times since
it will be pretty much the same for all functions generated from
gaussians which use the same standard deviation.
* estimator_gaussian.c: Changed structure to reduce code
duplication slightly, output files are now generated depending on
whether a file name is received or not.
* math_util.c: Fixed memory leak from gauss_transform function.
* file_util.c: Fixed memory leak in read_gauss_vector function.
* general_util.c: Fixed freeing string_arr structs.
2013-02-15 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* check_combinefunction.c: Test for combination of gauss vectors.
* check_math_util.c: Tests for new mathematical functions.
* estimator_delta.c: Fixed memory leaks and errors.
* launcher.c: Combined separate multi estimate functions for gauss
and baseline estimates into a single one - lots of unnecessary
code duplication. Fixed memory leaks and errors.
2013-02-14 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimate_delta.c: Fixed issues where baseline pmf calculations
were not being done correctly due to issues with math functions.
* math_util.c: When summing pmfs, points at which lambda is zero
caused -nan to be returned, even if there was only a single point
at which there was a zero. This led to bad behaviour, so the
calculation is now done with a very small lambda value which
should be better than ignoring those values entirely. Array
interval summing now sums values in the right-closed interval
(start,end]
2013-02-13 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* tests: Fixed some tests which were broken by minor modifications
to functions.
* generator.c: Generation stops if the time exceeds the end time
when the homogeneous lambda value is added to the base time. Would
previously have a small number of events that exceeded the
threshold.
* estimate_delta.c: Implemented a pmf estimator which sums the
pmfs from both streams being compared. Previous version used only
one of the streams to check whether the combined function was
representative of the bin counts, which was sub-optimal. pmf
estimation now gives good results. Area estimator uses normalised
start and end points, between which both functions have data at
all of the delays that are being attempted. The same applies for
the pmf estimator as well.
* math_util.c: Fixed array interval summing issue which was
causing pmf delay estimator to work sub-optimally.
2013-02-12 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* general_util.c, defparams.h: Added parameters for delta
estimation to default file. Added comments to some existing
parameters. More headings in the file, as well as some more
separation for delta estimation parameters. Each of the area and
pmf methods has its own start and interval times and
resolutions. Also added separate start and interval times for
estimation and generation. Not sure if all places where this is
used have been found.
* estimate_delta.c: Functions now use parameters from parameter
file.
2013-02-11 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimate_delta.c: More progress on the pmf estimator. Looping
over the different delay values again, as the underlying logic
seems to work reasonably well. Need to work out how to get the
pmfs to come out correctly for different interval lengths, as the
estimates are not as good as expected. Calculating only based on
the interval in which functions overlaps seems like a good idea,
but if this is to be implemented it needs to be normalised
somehow, which is probably quite difficult. Delays with shorter
overlaps always come out with the highest pmf because there are
fewer elements to sum. Maybe incur some fixed penalty for points
at which the combined function is zero? Perhaps using more than 10
intervals would work better.
* combinefunctions.c: Expected structure of time delay matches the
current scheme. Minor fix where the gauss vector function would
not find the correct maximum delay value if it was in the last
array position.
* launcher.c: Fixed time delay arrays to match the scheme. Decided
to use a value to represent the delay of each individual stream
rather than delays between streams. The delays will still be
relative to the first function, but there will be n delay values
rather than n-1 as was the case previously. This makes things more
compatible with the generator setup. The two multi-estimate
functions duplicate a lot of code. Should put them together to
make things cleaner.
2013-02-10 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* math_util.c: Function for finding the sum of the log of
probability mass functions, which takes a normalisation
constant. Takes an array of counts and an array of lambda values
and finds the pmf for each, summing as it goes. The normalisation
constant ensures that the lambda values correspond to the bin
counts so that a true value can be found. Function for summing
values in a specified interval given time values and data
values. Function for summing gaussians at the specified points by
passing an array of desired sample points. Function to find the
largest value in the array, positive or negative.
* general_util.c: Functions for initialising array structs with
specified lengths, function for finding a series of values from an
est_arr struct.
* estimator_delta.c: Prototype of pmf estimator improved and
almost working. Function combination needs to be completed to a
satisfactory level before it can be finished. Implemented a
function to find the normalisation constant required to get
gaussians to correspond to differing numbers of bins when doing
the delta estimation. The normalisation constant required depends
on the standard deviation of the gaussians used to estimate the
function.
* combinefunction.c: Fixed function combination. Required an array
containing a delay for the first function as well, but since that
is taken as a base with no delay, it is unneeded. The function now
takes an array containing delays between the base and subsequent
functions. This will lead to the function combination breaking
when the time delta is read from the parameter file, as there the
base time is specified as zero. Changing the value in the
parameter file also affects the generator. Decide which one is
better and fix this!
2013-02-09 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_delta.c: Untested prototype of the pmf estimator
implemented.
* launcher.c: Function combination for gaussians added to the
multiple estimate process. pmf method now able to be used.
* general_util.c: Function for initialising double_multi_arr
structs so that code is cleaner.
* combinefunction.c: Function for combining gaussians. Takes a set
of gauss vectors and finds their average. This should probably be
done on non-normalised gaussians which can be normalised when
being output to file, or once the normalisation constant has been
estimated.
2013-02-08 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.c: Estimates delta according to parameters in the
paramfile. Perhaps a good idea to add another switch to the
launcher to specify whether or not to estimate it? Seems a little
different to the main estimators, so it's probably fine to keep it
in the parameter file.
* combinefunction.c: Moved functions for finding the value of an
estimate at a given point to general_util as it is useful in other
places as well.
* general_util.c: Parameters for time delta estimation added to
default parameter file. estimate_timedelta specifies whether to
estimate delta or just take it from the
paramfile. timedelta_est_method specifies which method of
estimation to use. Parameters for specifying the step, resolution
and max delta for delta estimation. Modified some of the default
parameters for the baseline estimator so it performs better by
default.
* estimator_delta.c: Function for estimating the time delta based
on the area between two curves. Works for data from both the
gaussian and baseline estimators. Single function for both which
requires the conversion to a void pointer. This is so that there
is less code duplication - there is essentially only one function
that has to change in each of the sub-functions.
* paramlist.c: New function to extract a list of double parameters
separated by commas (with no spaces).
* launcher.c, generator.c: Time delta extraction changed to use
new function in paramlist.
* combinefunction.c: Function now takes a double_arr struct.
* check_math_util.c: Tests modified to use new double checking
function, updated for new gaussian calculation (which was wrong).
* testmain.h: Added function for checking the equivalence of
doubles.
2013-02-06 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* general_util.c: Passing a file which already exists to default
parameter creation asks whether to overwrite.
2013-02-05 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* general_util.c, defparams.h: Added parameters to the files to
allow for dynamic calculation of the standard deviation based on
the step between gaussians and a parameter alpha. Specifying the
standard deviation normally is also possible. The simple_stdev
parameter decides which is used. Parameters for gaussian
generator output modified. One parameter for generating from
stream data, one for generating from a uniform distribution of
gaussians. Files are output with strings appended to show what
they contain.
* generator.c: Improved output file handling. Multiple files are
output depending on the parameter provided to the gaussian
generator. Fixed erroneous time delta extraction.
2013-02-04 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* develop: Merged gaussians branch as it had pretty much all the
intended functionality and was working relatively well. Need to
move on. Also merged the experiment harness branch. Was getting
too far behind and there were structural changes in both that and
the gaussian branch that had some minor conflicts - better to
merge now than to risk things getting even more disparate and
difficult to merge later on. Development of the experiment harness
put on hold for now.
2013-02-03 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_baseline.c: Discovered crashes, but not sure of the
cause. Mostly related to estimation of functions generated by the
random generator, but this implies that there is something to do
with the shape of the function that causes the crashes.
* math_util.c: Functions to create gaussian vectors take a
multiplier which modifies the weights of gaussians. Standard
weights are ~ N(0,1). This allows for functions to have larger
ranges than just -5 to 5 or so.
* estimator_gaussian.c: Reverted to using the standard system in
use for everything else rather than kernel density.
* printf: Some print statements depend on the VERBOSE flag being
defined. can do this at compile time by passing -DVERBOSE to
make.
* output: Modifications to output filenames for the various
generators and estimators.
2013-02-02 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.h: Updated help info for switches, -R replaced with -t,
specify output type.
* general_util.c: Function for freeing memory allocated for
double_multi_arr structs. Default parameter file has more files to
output gaussian data to. Separate ones for raw data and that
generated from an event file.
* math_util.c: GSL random number generator is deallocated when the
program exits using an atexit function. Function for shifting a
gauss transform so that all points are non-negative
* generator.c: Generation of event streams from
gaussians. Generation of functions simplified to a single
function. Output of contribution of gaussians if the -t 2 switch
is used.
* launcher.c: Generation of event streams from gaussians possible
by passing the -f rand switch to the launcher. If no parameter is
provided then the default muparser version is used which takes the
function in the paramfile. Function generation is done using the
-r switch. Adding -R outputs gaussians in their raw forms, and
including an input file with one-dimensional data has the program
centre a gaussian on each point.
2013-02-01 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.c: Added struct to make adding switches easier. Split
multi-estimate function into gaussian and other estimators since
return types are different.
* generator.c: Random functions can now be generated with the -r
switch. Still untested, but appears to work as intended. Another
function for gaussians also implemented, but not sure of what
functionality it should have. One function is required to generate
random functions for experimenting on, the other is used to
generate gaussians from event data to use in gaussian estimates?
* file_util.c: Function for outputting double_multi_arr struct.
* math_util.c: Fixed cause of crash on fclose when calling the
gauss_transform function - data was being written to un-malloced
locations. Changed return type of gaussian transform to struct to
make it easier to work with. Gaussian transform needs some kind of
normalisation parameter so that it outputs data int the correct
range for the function it is being used to estimate. Attempted to
implement kernel density estimation, but still unsure of how to
proceed.
* estimator.c: New file can be run by calling the estimator with
the -a gauss switch. Multiple stream estimates are also possible.
* estimator_gaussian.c: New estimator which uses kernel density to
estimate functions. Prototype functions implemented.
* makefile.am: Dependencies for new file added.
* defparams.h: Added gaussian estimation and generation
parameters.
* general_util.c: Added writing of gaussian parameters to default
paramfile.
* params.txt: Gaussian estimation and generation parameters split.
2013-01-31 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* file_util.c: Extremely weird bug where crash occurs on fclose
discovered. Not sure how to solve yet. Functions to read and
write gaussians with mean, stdev and weight representation, along
with tests.
* struct.h: Header file for structs to prevent interdependencies
causing compilation issues.
* defparams.h: Default gaussian parameters
* general_util.c: Gaussian parameters added to default paramfile
generation. Functions for printing and freeing gaussian vectors.
* params.txt: Added gaussian parameters
2013-01-30 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* check_math_util.c: tests for the new functions
* math_util.c: Function for generating a vector initialised with
the same values, finding the minimum value in an array, adding a
constant to an array.
* launcher.c: Can generate gaussians from the launcher into output
files.
* generator.c: Generation functions for gaussians. Output data to
files.
* general_util.c: Parameters for gaussians added to default file
generation.
* file_util.c: Gauss transform output can be shifted along the y
axis.
2013-01-29 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* file_util.c: Wondering whether to allow parameters without
values specified. Makes having to specify loads of parameters in
the experiment paramfile unnecessary if you just want to run
experiments on one part of the system.
* experimenter.c: Function to parse parameters from a string of
comma separated values. If the string contains a range indicator
(...), then it will revert to the function to calculate the range.
Experimenter structure changed. Requires parameters to be defined
for all the parts of the system you want to experiment
with. Checks whether these parameters exist in the
paramfile. Still a work in progress.
* general_util.c: Function to check whether parameters in a string
array are present in a paramlist.
2013-01-28 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* general_util.c: Modified split_string function and its tests.
* experimenter.c: Completed range parsing functions, with tests.
Experimenter now takes two parameter files to read - one to read
default parameters, and one for its own use. Checks that some
required parameters are present.
* exp_param.txt: Added some parameters that make sense to use.
* check_file_util.c: Test for create file function.
* file_util.c: Added function for creating files within
directories. Fixed crash when parameter in paramfile does not
have a value.
2013-01-27 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* math_util.c: Error checking on parameters for some gaussian
functions
* check_math_util.c: Added tests for gaussian-related code
* combinefunction.c: Basic error checking added.
* tests: Tests for combining functions.
2013-01-26 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* GMP: Removed use of GMP in math library - necessary
functionality available in GSL.
* estimator_piecewise.c: Replaced custom probability mass function
with the function from the GSL.
* math_util.h: Fixed bug in gaussian transform calculation, split
some functions to get more control over what goes on.
* file_util.h: Added functions for outputting gaussians and
discrete Gaussian transforms to file. Moved includes into header
file. Function for writing one of the new structs to file.
* launcher.c: multi-estimate data should go to specified output
file.
* general_util.c: Parameter for step used when going through data
added
* general_util.h: Added some structs to make array length storage
easier.
* combinefunction.c: Function used to combine functions now works
as it should, returning an array containing data to use.
2013-01-25 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* generator.c: Changed random number function usage to echo below
changes.
* configure.ac: Added checks for GSL library
* math_util.c: Switched to using GSL to generate random numbers.
Implemented discrete Gaussian transform with some structs to store
Gaussians in a clean way. Various functions for initialising
vectors of gaussians.
2013-01-23 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.c: Updated to reflect changes to functions in
combinefunction.c
* combinefunction.c: Implemented a basic function
combinator. Current implementation is rough and outputs to a file
specified within the code.
2013-01-20 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* launcher.c: Estimation of multiple streams at once by the
launcher. Updated switches to allow for this. Some commenting and
more checking for null values.
* combinefunction.c: Function skeletons for combining functions.
* makefile.am: Added new file and dependencies generated by it to
launcher.
* scripts: Removed unused plotting scripts. Obsoleted by
plotmaster.
* estimators: All now return an est_arr rather than some returning
est_data. Easier to integrate with the launcher and other parts
of the system.
* memory: Fixed some leaks caused by swath of changes, and some
old ones.
2013-01-19 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* headers: Moved headers around in some files in src. Launcher
strings moved into header file.
* src/makefile.am: No longer includes muparser. Correctly added by
configure.ac.
* configure.ac: Checks for correct version of muparser.
* plotmaster.sh: Plots data from the new plot
* plots: Gnuplot code for plotting two estimates added
2013-01-13 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* paramlist.c: Added functions for setting the value of parameters
in the list.
* file_util.c: Modified file generator to take parameters.
* generator.c: Added helper function to extract parameters from a
parameter file.
* estimator_piecewise.c: Fixed bug with estimation data from the
first subinterval not being output to file. Minimum length of
interval now in place for piecewise estimate. Affects the baseline
estimator as well.
* defparams.h: New header file containing default parameters for
estimators and generator.
* general_util.c: Added a function for generating a default
parameter file, moved in function for checking parameters in a
list.
2013-01-11 Michal Staniaszek <mxs968@cs.bham.ac.uk>
* estimator_piecewise.c: Removed preprocessor variables and
modified functions to take additional parameters instead.
* estimators: Added a helper function to the estimators so that
you do not have to call them with all the parameters specified if
it is not necessary. You can now pass in a parameter list
containing the relevant parameters, which will be extracted and
passed into the function.
* tests/Makefile.am: Tests can now be compiled using `make check`
* tests/check_estimator.c: Added a basic test for estimator.c
* tests/testmain.c: Added estimator test suite.
* AUTHORS: Added name
* ChangeLog: Put some actual stuff in it.
* NEWS: Very brief information about releases.
* README: Added basic information.