Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R Client - UpdateBy #4412

Merged
merged 55 commits into from
Sep 29, 2023
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
0dc69ea
updateBy - untested
alexpeters1208 Aug 24, 2023
c9e25e4
Merge branch 'deephaven:main' into update_by
alexpeters1208 Aug 24, 2023
065fd4f
More updateBy
alexpeters1208 Aug 25, 2023
e49e3eb
Merge remote-tracking branch 'origin/update_by' into update_by
alexpeters1208 Aug 25, 2023
375df6d
Update_by unit testing
alexpeters1208 Aug 25, 2023
9c919c9
More unit testing
alexpeters1208 Aug 25, 2023
cd798f6
Even more unit testing
alexpeters1208 Aug 28, 2023
68fdea9
Merge branch 'deephaven:main' into update_by
alexpeters1208 Aug 28, 2023
d1f684c
Unit testing
alexpeters1208 Aug 29, 2023
9e77196
Wow, more unit testing
alexpeters1208 Aug 29, 2023
65d65b2
Unit testing
alexpeters1208 Aug 30, 2023
19e0aef
Merge branch 'deephaven:main' into update_by
alexpeters1208 Aug 30, 2023
945aa02
More unit testing
alexpeters1208 Aug 30, 2023
425fa9d
Merge remote-tracking branch 'origin/update_by' into update_by
alexpeters1208 Aug 30, 2023
1bf5230
Unit testing
alexpeters1208 Aug 30, 2023
962925f
That's all folks
alexpeters1208 Aug 30, 2023
4316919
Merge branch 'deephaven:main' into update_by
alexpeters1208 Aug 30, 2023
46efa10
Update pure R emstd calculation
alexpeters1208 Aug 31, 2023
b26a8ea
Merge remote-tracking branch 'origin/update_by' into update_by
alexpeters1208 Aug 31, 2023
3961f57
Merge branch 'deephaven:main' into update_by
alexpeters1208 Aug 31, 2023
fd2c646
Support agg_all_by at the R level, provide a stable API that allows t…
alexpeters1208 Sep 7, 2023
f12a940
Merge branch 'deephaven:main' into add-agg-all-by
alexpeters1208 Sep 7, 2023
6acdeb9
Merge remote-tracking branch 'origin/main' into update_by
alexpeters1208 Sep 7, 2023
41f15ad
Fix unit tests
alexpeters1208 Sep 8, 2023
d037dc1
Merge remote-tracking branch 'origin/add-agg-all-by' into update_by
alexpeters1208 Sep 8, 2023
c7af4bd
Merge remote-tracking branch 'origin/main' into update_by
alexpeters1208 Sep 8, 2023
6fdba6b
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 8, 2023
0d2a9d9
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 8, 2023
942929f
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 11, 2023
b25b983
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 13, 2023
8e09578
Merge remote-tracking branch 'origin/main' into update_by
alexpeters1208 Sep 18, 2023
8b7f0de
Merge remote-tracking branch 'origin/update_by' into update_by
alexpeters1208 Sep 18, 2023
9ed5f61
Add docs to update_by (wip)
alexpeters1208 Sep 18, 2023
76d6d01
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 19, 2023
e77971e
Wip
alexpeters1208 Sep 19, 2023
de31b3e
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 19, 2023
3c726a3
More docs
alexpeters1208 Sep 19, 2023
96f4493
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 20, 2023
4ac9d27
More docs
alexpeters1208 Sep 20, 2023
92d47de
Even more docs
alexpeters1208 Sep 20, 2023
0e60509
Clean up error messages
alexpeters1208 Sep 20, 2023
2eead7f
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 20, 2023
a10e0fd
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 21, 2023
cb093da
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 22, 2023
f6098ae
Refactor manual emstd_time calculation
alexpeters1208 Sep 22, 2023
11f504c
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 25, 2023
8ab29b6
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 26, 2023
714ba05
Code review suggestions
alexpeters1208 Sep 26, 2023
bd754ff
Update docs
alexpeters1208 Sep 26, 2023
9a09ad1
Small change in updateBy docs
alexpeters1208 Sep 27, 2023
2a19a2d
Code review suggestions
alexpeters1208 Sep 28, 2023
38d3fe1
Merge branch 'deephaven:main' into update_by
alexpeters1208 Sep 29, 2023
a3c6b0f
Update docs with window examples
alexpeters1208 Sep 29, 2023
0c9125d
Update docs
alexpeters1208 Sep 29, 2023
da15b90
Update docs again
alexpeters1208 Sep 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion R/rdeephaven/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ License: Apache License (== 2.0)
Depends: R (>= 3.5.3), Rcpp (>= 1.0.10), arrow (>= 12.0.0), R6 (>= 2.5.0), dplyr (>= 1.1.0)
Imports: Rcpp (>= 1.0.10), R6 (>= 2.5.0), dplyr (>= 1.1.0)
LinkingTo: Rcpp
Suggests: testthat (>= 3.0.0)
Suggests: testthat (>= 3.0.0), lubridate (>= 1.9.0), zoo (>= 1.8-0)
Config/testthat/edition: 3
RoxygenNote: 7.2.3
35 changes: 35 additions & 0 deletions R/rdeephaven/NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,41 @@ export(agg_w_avg)
export(merge_tables)
export(ncol.TableHandle)
export(nrow.TableHandle)
export(op_control)
export(uby_cum_max)
export(uby_cum_min)
export(uby_cum_prod)
export(uby_cum_sum)
export(uby_delta)
export(uby_ema_tick)
export(uby_ema_time)
export(uby_emmax_tick)
export(uby_emmax_time)
export(uby_emmin_tick)
export(uby_emmin_time)
export(uby_ems_tick)
export(uby_ems_time)
export(uby_emstd_tick)
export(uby_emstd_time)
export(uby_forward_fill)
export(uby_rolling_avg_tick)
export(uby_rolling_avg_time)
export(uby_rolling_count_tick)
export(uby_rolling_count_time)
export(uby_rolling_group_tick)
export(uby_rolling_group_time)
export(uby_rolling_max_tick)
export(uby_rolling_max_time)
export(uby_rolling_min_tick)
export(uby_rolling_min_time)
export(uby_rolling_prod_tick)
export(uby_rolling_prod_time)
export(uby_rolling_std_tick)
export(uby_rolling_std_time)
export(uby_rolling_sum_tick)
export(uby_rolling_sum_time)
export(uby_rolling_wavg_tick)
export(uby_rolling_wavg_time)
import(Rcpp)
importFrom(Rcpp,evalCpp)
importFrom(arrow,arrow_table)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' @description
#' An Aggregation represents an aggregation operation that can be passed to `agg_by()` or `agg_all_by()`.
#' Note that Aggregations should not be instantiated directly by user code, but rather by provided agg_* functions.
Aggregation <- R6Class("Aggregation",
#' An AggOp represents an aggregation operation that can be passed to `agg_by()` or `agg_all_by()`.
#' Note that AggOps should not be instantiated directly by user code, but rather by provided agg_* functions.
AggOp <- R6Class("AggOp",
cloneable = FALSE,
public = list(
.internal_rcpp_object = NULL,
Expand All @@ -18,7 +18,7 @@ Aggregation <- R6Class("Aggregation",
)
)

### All of the functions below return an instance of the above class
### All of the functions below return an instance of an 'AggOp' object

#' @description
#' Creates a First aggregation that computes the first value of each column in `cols` for each aggregation group.
Expand All @@ -28,7 +28,7 @@ Aggregation <- R6Class("Aggregation",
#' @export
agg_first <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_first, "agg_first", cols=cols))
return(AggOp$new(INTERNAL_agg_first, "agg_first", cols=cols))
}

#' @description
Expand All @@ -39,7 +39,7 @@ agg_first <- function(cols = character()) {
#' @export
agg_last <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_last, "agg_last", cols=cols))
return(AggOp$new(INTERNAL_agg_last, "agg_last", cols=cols))
}

#' @description
Expand All @@ -50,7 +50,7 @@ agg_last <- function(cols = character()) {
#' @export
agg_min <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_min, "agg_min", cols=cols))
return(AggOp$new(INTERNAL_agg_min, "agg_min", cols=cols))
}

#' @description
Expand All @@ -61,7 +61,7 @@ agg_min <- function(cols = character()) {
#' @export
agg_max <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_max, "agg_max", cols=cols))
return(AggOp$new(INTERNAL_agg_max, "agg_max", cols=cols))
}

#' @description
Expand All @@ -72,7 +72,7 @@ agg_max <- function(cols = character()) {
#' @export
agg_sum <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_sum, "agg_sum", cols=cols))
return(AggOp$new(INTERNAL_agg_sum, "agg_sum", cols=cols))
}

#' @description
Expand All @@ -83,7 +83,7 @@ agg_sum <- function(cols = character()) {
#' @export
agg_abs_sum <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_abs_sum, "agg_abs_sum", cols=cols))
return(AggOp$new(INTERNAL_agg_abs_sum, "agg_abs_sum", cols=cols))
}

#' @description
Expand All @@ -94,7 +94,7 @@ agg_abs_sum <- function(cols = character()) {
#' @export
agg_avg <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_avg, "agg_avg", cols=cols))
return(AggOp$new(INTERNAL_agg_avg, "agg_avg", cols=cols))
}

#' @description
Expand All @@ -107,7 +107,7 @@ agg_avg <- function(cols = character()) {
agg_w_avg <- function(wcol, cols = character()) {
verify_string("wcol", wcol, TRUE)
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_w_avg, "agg_w_avg", wcol=wcol, cols=cols))
return(AggOp$new(INTERNAL_agg_w_avg, "agg_w_avg", wcol=wcol, cols=cols))
}

#' @description
Expand All @@ -118,7 +118,7 @@ agg_w_avg <- function(wcol, cols = character()) {
#' @export
agg_median <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_median, "agg_median", cols=cols))
return(AggOp$new(INTERNAL_agg_median, "agg_median", cols=cols))
}

#' @description
Expand All @@ -129,7 +129,7 @@ agg_median <- function(cols = character()) {
#' @export
agg_var <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_var, "agg_var", cols=cols))
return(AggOp$new(INTERNAL_agg_var, "agg_var", cols=cols))
}

#' @description
Expand All @@ -140,7 +140,7 @@ agg_var <- function(cols = character()) {
#' @export
agg_std <- function(cols = character()) {
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_std, "agg_std", cols=cols))
return(AggOp$new(INTERNAL_agg_std, "agg_std", cols=cols))
}

#' @description
Expand All @@ -153,7 +153,7 @@ agg_std <- function(cols = character()) {
agg_percentile <- function(percentile, cols = character()) {
verify_in_unit_interval("percentile", percentile, TRUE)
verify_string("cols", cols, FALSE)
return(Aggregation$new(INTERNAL_agg_percentile, "agg_percentile", percentile=percentile, cols=cols))
return(AggOp$new(INTERNAL_agg_percentile, "agg_percentile", percentile=percentile, cols=cols))
}

#' @description
Expand All @@ -164,5 +164,5 @@ agg_percentile <- function(percentile, cols = character()) {
#' @export
agg_count <- function(col) {
verify_string("col", col, TRUE)
return(Aggregation$new(INTERNAL_agg_count, "agg_count", col=col))
return(AggOp$new(INTERNAL_agg_count, "agg_count", col=col))
}
2 changes: 1 addition & 1 deletion R/rdeephaven/R/client_wrapper.R
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Client <- R6Class("Client",
#' Initializes a Client object using a pointer to an existing client connection.
#' @param xptr External pointer to an existing client connection.
initialize_for_xptr = function(xptr) {
verify_type("xptr", xptr, "externalptr", "XPtr", TRUE)
verify_type("xptr", xptr, TRUE, "externalptr", "an XPtr")
self$.internal_rcpp_object = new(INTERNAL_Client, xptr)
},

Expand Down
95 changes: 0 additions & 95 deletions R/rdeephaven/R/helper_functions.R

This file was deleted.

48 changes: 48 additions & 0 deletions R/rdeephaven/R/operation_control.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#' @description
#' An OperationControl defines the control parameters of some UpdateByOps used in an update_by table operation.
#' The UpdateByOps can use OperationControl to handle erroneous data are `ema_tick`, `ema_time`, `emmax_tick`,
#' `emmax_time`, `emmin_tick`, `emmin_time`, and `ems_time`.
#' Note that OperationControl should not be instantiated directly by user code, but rather `by op_control()`.
OperationControl <- R6Class("OperationControl",
cloneable = FALSE,
public = list(
.internal_rcpp_object = NULL,
initialize = function(op_control) {
if (class(op_control) != "Rcpp_INTERNAL_OperationControl") {
stop("'op_control' should be an internal Deephaven OperationControl. If you're seeing this,\n you are trying to call the constructor of an OperationControl directly, which is not advised.\n Please use the op_control function instead.")
}
self$.internal_rcpp_object <- op_control
}
)
)

#' @description
#' Creates an OperationControl that controls behavior on special values in `uby_em*` operations. The arguments `on_null`
#' and `on_nan` can take the following values:
#' 'poison': Allow bad data to poison the result. This is only valid for use with NaN.
#' 'reset': Reset the state for the bucket to None when invalid data is encountered.
#' 'skip': Skip and do not process the invalid data without changing state.
#' 'throw': Throw an exception and abort processing when bad data is encountered.
#' @param on_null Defines how an UpdateByOp handles null values it encounters. 'skip' is the default.
#' @param on_nan Defines how an UpdateByOp handles NaN values it encounters. 'skip' is the default.
#' @param big_value_context Defines how an UpdateByOp handles exceptionally large values it encounters.
#' The default value is 'decimal128'. The following values are available:
#' 'decimal128': IEEE 754R Decimal128 format. 34 digits and rounding is half-even.
#' 'decimal32': IEEE 754R Decimal32 format. 7 digits and rounding is half-even.
#' 'decimal64': IEEE 754R Decimal64 format. 16 digits and rounding is half-even.
#' 'unlimited': Unlimited precision arithmetic. Rounding is half-up.
#' @return OperationControl to be used in `uby_em*`.
#' @export
op_control <- function(on_null="skip", on_nan="skip", big_value_context="decimal128") {

if (!(on_null %in% c("poison", "reset", "skip", "throw"))) {
stop(paste0("'on_null' must be one of 'poison', 'reset', 'skip', or 'throw'. Got '", on_null, "'."))
}
if (!(on_nan %in% c("poison", "reset", "skip", "throw"))) {
stop(paste0("'on_nan' must be one of 'poison', 'reset', 'skip', or 'throw'. Got '", on_nan, "'."))
}
if (!(big_value_context %in% c("decimal32", "decimal64", "decimal128", "unlimited"))) {
stop(paste0("'big_value_context' must be one of 'decimal32', 'decimal64', 'decimal128', or 'unlimited'. Got '", big_value_context, "'."))
}
return(OperationControl$new(INTERNAL_op_control_generator(on_null, on_nan, big_value_context)))
}
Loading
Loading