[Implemented](Observed) data concept

[PavelBal]

Proposal for a structure for storing any kind of xy numerical data. This structure (class) can be used as a wrapper for DataRepository with extended functionality for convenient use in R and serve as a basis for data-related workflows such as figure creation with the TLF library and parameter identification.

I think it should always be used as an intermediate structure and it's functionality will be defined by the workflows it will be used it.

Name of the class

How should we name the class?

• DataRepository (not too obvious from my perspective...)
• XYData
• XYDataSet
• DataSet (will be used in following for now)

Class fields

• Name : string
• xValues : numeric array
• yValues : numeric array
• yError : numeric array
• LLOQ : numeric value
• xDimension
• yDimension
• MW : molecular weight (if applicable), used for unit conversion
  • Alternative: MW is also part of MetaData
• MetaData : name-value pairs

For this class, I propose not to follow the osps concept that all values are stored in base units. Instead, I propose to add the properties

• xUnit
• yUnit
• yErrorUnit

that define in which unit the xValues, yValues, and yError are given. I see multiple advantages for the user:

• If values are imported from an excel file, the same numerical values in DataSet are stored as they were in the excel file.
• Manual creation of a data set with a certain unit without having to manually re-scale.

Should multiple y-value columns per DataSet be allowed? I see potential issues how to manage e.g. error values etc...

Methods/functions for:

• Load from pkml
  • Already implemented
• Create from scratch
  • dataSet <- dataSet(xValues = c(1, 2, 3), yValues = c(2, 3, 4), yError = c(0, 0, 0.1))
  • If created from scratch, both x and y dimensions are set to Dimensionless. The used can change the dimension:
  • dataSet$xDimension <- "Time"
  • dataSet$yDimension <- "Concentration (molar)"
  • When changing the dimension, the units are set to default units of the dimension. In the above example, x values are 1, 2, 3 minutes, y values are 2, 3, 4 µmol/l. The user can change the unit:
  • dataSet$xUnit <- "h"
  • dataSet$yUnit <- "pmol/l"
  • x values are now 1, 2, 3 hours, y values are 2, 3, 4 pmol/l.
• Load from excel
  • Using the importer. It should be possible to use a configuration created in PKSim/MoBi or the create new configuration (and save it so it can be used in PKSim/MoBi).
• Create from simulation results
  • Maybe it should not be accessible by the user directly but used in the workflows.
• Export to data.frame (as the most common data structure for R users)

[PavelBal]

@msevestre @Yuri05 I would like to discuss it with you as soon as possible so we can start implementation.

[msevestre]

I like the idea of providing a user-friendly wrapper around DataRepository for our typical scenarios (dealing with observed data, with our without error)

• How should we name the class?

I vote for DataSet

Properties: Most of those properties should be direct warpper over existing method of DataRepository. A DataSet should NOT hold any data otherwise the serialization to and from pkml will miss it

I am going to call the wrapped DataRepository .wrappedData

name : => .wrappedData$name
xValues : =>
get{toUnit(.wrappedData$baseGrid$values, self$xUnit) (pseudocode)}
set{.wrappedData$baseGrid$values <-toBaseUnit(value, self$xUnit) (pseudocode)}

yValues : same as xValues, only with one of the non base grid columns (the one that has no auxiliary column)
yError : same as xValues, only with the auxiliary column set
LLOQ : comes from yValues directly (property of the column)
xDimension : delegate to the dimension of the xValues
yDimension delegate to the dimension of the yValues
yErrorDiension: THIS WAS missing. delegate to the dimension of the yError

MW : It is defined in the column as property. It should be set and read in g/mol (and not kg/mol)

MetaData : name-value pairs. delegate to the datarepo

[PavelBal]

Additional properties:

• errorType: Geometric or Arithmetic
• We don't have to explicitly store xErrorDimension - it must be the same as yError if errorType is arithmetic, or it is dimensionless if the error is geometric. Agree?
• However, we could allow yError to be in different unit than yValues (in case of arithmetic error).

What happens is the user changes the unit? Let's say a DataSet has self$xValues = c(1, 2, 3), self$xDimension = Time, self$xUnit = "Minutes". Now if

self$xUnit <- "hour(s)"
self$xValues
> 1, 2, 3

(in this case, in internally calls set{.wrappedData$baseGrid$values <-toBaseUnit(value, self$xUnit) (pseudocode)})

or

self$xUnit <- "hour(s)"
self$xValues
> 60, 120, 180

?

[msevestre]

I would say self$xValues does not change.
e.g. always 1, 2, 3

[msevestre]

@PavelBal I can take a stab at it if you're ok with it? It's going to allow me to see what's missing in core

[PavelBal]

@msevestre Sure, if you have time for this!

@hannaei Michael will take over for this class but you can still investigate how you would implement this and then "study" his implementation. Good opportunity to learn how to implement R-classes and functions that are based on .NET classes.

[msevestre]

I'll make a draft tomorrow and tag everyone for a discussion;-)

…

On Thu., Jul. 1, 2021, 11:38 a.m. Pavel Balazki, ***@***.***> wrote: @msevestre <https://github.com/msevestre> Sure, if you have time for this! @hannaei <https://github.com/hannaei> Michael will take over for this class but you can still investigate how you would implement this and then "study" his implementation. Good opportunity to learn how to implement R-classes and functions that are based on .NET classes. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#538 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAH6GVKF3BBM5FTIHTHRRTTTVSDXTANCNFSM47FUGATA> .

[msevestre]

What should be the behavior when setting a unit that does not fit with the dimension? Should we switch the dimension automatically to one matching the unit?
e.,g

dataSet <- DataSet$new()

HEre we will have some default set (xValues in Time, yValues in Concentration mass let say)
now I do
dataSet$yUnit <- "mol" (e.g. amount)

What do you expect to happen?

[PavelBal]

I would like to see an error here. To change the dimension, the user has first to set the dimension.

dataSet$xDimension <- "Amount"
(When changing the dimension, the unit is set to the base unit).
Now the user can do
dataSet$xUnit <- "mol"

[msevestre]

I am unsure about this. We want to make it easy to use. .forcing people to set dimension first seems to be a bit counter intuitive

[msevestre]

@PavelBal @hannaei
I don't see any updae to the PR I made a few weeks ago. Did you work on this at all? If yes, would you mind updating the code so that I can see the latest changes? You can also create a PR to the branch and then I can review and merge

[PavelBal]

We didn't have much time to work on this. But I am writing tests now and will add a PR.