-
Notifications
You must be signed in to change notification settings - Fork 11
/
predict_keras.Rmd
175 lines (146 loc) · 5.49 KB
/
predict_keras.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
---
title: "bupaR Docs | Customization with Keras"
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
eval = FALSE,
cache = FALSE
)
```
```{r echo = F, eval = T, out.width="25%", fig.align = "right"}
knitr::include_graphics("images/icons/predict.PNG")
```
***
# Customization with Keras
```{r setup, message = F, eval = T, warning = F}
library(processpredictR)
library(bupaR)
library(keras)
library(purrr)
```
We can also opt for setting up and training our model manually, instead of using the provided `processpredictR` methods. Note that after defining a model with `keras::keras_model()` the model no longer is of class `ppred_model` (as opposed to `processpredictR::create_model()`).
## Defining a new custom model
The layers on top of a custom model (previously defined in [adapting your model](https://bupaverse.github.io/docs/predict_adapt.html)) can be added as follows:
```{r}
new_outputs <- custom_model$model$output %>% # custom_model$model to access a model and $output to access the outputs of that model
keras::layer_dropout(rate = 0.1) %>%
keras::layer_dense(units = custom_model$num_outputs, activation = 'softmax')
custom_model <- keras::keras_model(inputs = custom_model$model$input,
outputs = new_outputs,
name = "new_custom_model")
custom_model
```
```
#> Model: "new_custom_model"
#> ________________________________________________________________________________
#> Layer (type) Output Shape Param #
#> ================================================================================
#> input_2 (InputLayer) [(None, 9)] 0
#> token_and_position_embedding_1 (To (None, 9, 36) 828
#> kenAndPositionEmbedding)
#> transformer_block_1 (TransformerBl (None, 9, 36) 26056
#> ock)
#> global_average_pooling1d_1 (Global (None, 36) 0
#> AveragePooling1D)
#> dropout_6 (Dropout) (None, 36) 0
#> dense_6 (Dense) (None, 64) 2368
#> dropout_8 (Dropout) (None, 64) 0
#> dense_8 (Dense) (None, 11) 715
#> ================================================================================
#> Total params: 29,967
#> Trainable params: 29,967
#> Non-trainable params: 0
#> ________________________________________________________________________________
```
```{r}
# class of the model
custom_model %>% class
```
```
#> [1] "keras.engine.functional.Functional"
#> [2] "keras.engine.training.Model"
#> [3] "keras.engine.base_layer.Layer"
#> [4] "tensorflow.python.module.module.Module"
#> [5] "tensorflow.python.trackable.autotrackable.AutoTrackable"
#> [6] "tensorflow.python.trackable.base.Trackable"
#> [7] "keras.utils.version_utils.LayerVersionSelector"
#> [8] "keras.utils.version_utils.ModelVersionSelector"
#> [9] "python.builtin.object"
```
## Preprocessing
### `processpredictR::tokenize()`
Before training the model we first must prepare the data. `processpredictR` provides the `tokenize()` function, but other alternatives (see [working with preprocessing layers](https://tensorflow.rstudio.com/guides/keras/preprocessing_layers)) can also be used.
```{r}
# the trace of activities must be tokenized
tokens_train <- df$train_df %>% tokenize()
map(tokens_train, head) # the output of tokens is a list
```
```
#> $token_x
#> $token_x[[1]]
#> [1] 2
#>
#> $token_x[[2]]
#> [1] 2 3
#>
#> $token_x[[3]]
#> [1] 2
#>
#> $token_x[[4]]
#> [1] 2 4
#>
#> $token_x[[5]]
#> [1] 2 4 5
#>
#> $token_x[[6]]
#> [1] 2 4 5 6
#>
#>
#> $numeric_features
#> NULL
#>
#> $categorical_features
#> NULL
#>
#> $token_y
#> [1] 0 1 2 3 4 5
```
### Pad sequences
Padding of sequences is required to make every input sequence of an equal length.
```{r}
# make sequences of equal length
x <- tokens_train$token_x %>% pad_sequences(maxlen = max_case_length(df$train_df), value = 0)
y <- tokens_train$token_y
```
## compile, fit, predict & evaluate
> Note that `keras::keras_model()` no longer returns a list, so we simply refer to the model as to an object.
Do not forget to compile the model
```{r}
# compile
compile(object=custom_model, optimizer = "adam",
loss = loss_sparse_categorical_crossentropy(),
metrics = metric_sparse_categorical_crossentropy())
```
We are now ready to train our custom model (the code below is not being evaluated).
```{r, eval=F}
# train
fit(object = custom_model, x, y, epochs = 10, batch_size = 10)
# see also ?keras::fit.keras.engine.training.Model
# predict
tokens_test <- df$test_df %>% tokenize()
x <- tokens_test$token_x %>% pad_sequences(maxlen = max_case_length(df$train_df), value = 0)
predict(custom_model, x)
# evaluate
tokens_test <- df$test_df %>% tokenize()
x <- tokens_test$token_x
# normalize by dividing y_test over the standard deviation of y_train
y <- tokens_test$token_y / sd(tokens_train$token_y)
evaluate(custom_model, x, y)
```
***
Read more:
```{r footer, results = "asis", echo = F, eval = T, collapse = F}
source("htmlbuttons.R")
create_buttons(df, "predict_keras.html")
```