conda install fails, biasCorrection isimip3 fails

[cs2239]

After clean install using miniconda as per github:

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "isimip3")
Error in match.arg(method, choices = c("delta", "scaling", "eqm", "pqm",  : 
  'arg' should be one of “delta”, “scaling”, “eqm”, “pqm”, “gpqm”, “mva”, “loci”, “ptr”, “variance”

R is also out of date. It is R version 3.6.1 (2019-07-05).

Also

WARNING: Your current version of transformeR (v1.7.2) is not up-to-date
WARNING: Your current version of downscaleR (v3.1.0) is not up-to-date

The other BC methods work but I want to use isimip3 and qdm, which are missing.

So I try clean new install using

    > library(devtools)
    > install_github(c("SantanderMetGroup/loadeR.java",
                 "SantanderMetGroup/climate4R.UDG",
                 "SantanderMetGroup/loadeR",
                 "SantanderMetGroup/transformeR",
                 "SantanderMetGroup/visualizeR",
                 "SantanderMetGroup/downscaleR"))

As isimip3 needs dask I install using conda

> out <- biasCorrection(y = y, x = x, newdata = newdata, method = "isimip3")
Time difference of 1 days
Time difference of 1 days
Time difference of 1 days
[2021-05-03 16:45:33] Argument precipitation is set as FALSE, please ensure that this matches
 your data.
[2021-05-03 16:45:34] Number of windows considered: 1...
[2021-05-03 16:45:34] Bias-correcting 1 members separately...
Error in py_run_file_impl(file, local, convert) : 
  ModuleNotFoundError: No module named 'dask'
Detailed traceback:
  File "<string>", line 122, in <module>
  File "/home/miniconda3/lib/R/library/reticulate/python/rpytools/loader.py", line 4
4, in _import_hook
    level=level
In addition: Warning messages:
1: In fillGridDates(y) : Undefined time zone
2: In fillGridDates(x) : Undefined time zone
3: In fillGridDates(newdata) : Undefined time zone

I am stuck on why isimip3 fails. dask module is installed.

[priyapatel1]

I'm also unable to install the package. I used both devtools and conda install and wasn't able to install either. I got the following error with R (I was working through the Jupyter Notebook " 'Statistical downscaling with climate4R: Contribution to the VALUE Intercomparison experiment'"

The operation couldn’t be completed. Unable to locate a Java Runtime.
Please visit http://www.java.com for information on installing Java.

Error: package or namespace load failed for ‘rJava’:
 .onLoad failed in loadNamespace() for 'rJava', details:
  call: system("/usr/libexec/java_home", intern = TRUE)
  error: (converted from warning) running command '/usr/libexec/java_home' had status 1
Error: package ‘rJava’ could not be loaded
Execution halted
ERROR: lazy loading failed for package ‘loadeR.java’
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/loadeR.java’
Error: Failed to install 'loadeR.java' from GitHub:
  (converted from warning) installation of package ‘/var/folders/ym/bpdlwttd35n1gjbfn9ws8yc40000gn/T//RtmpFiSY7e/fileb3e450a1c369/loadeR.java_1.1.1.tar.gz’ had non-zero exit status

With conda I got this:

PackagesNotFoundError: The following packages are not available from current channels:

  - climate4r

Current channels:

  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://conda.anaconda.org/r/osx-64
  - https://conda.anaconda.org/r/noarch
  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://conda.anaconda.org/santandermetgroup/osx-64
  - https://conda.anaconda.org/santandermetgroup/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

I'm very new to this so please excuse me if my error is obvious. Can someone help me figure out how to download this package?

[zequihg50]

Hi, would you try to install with the following command into a clean environment?

conda install -c santandermetgroup -c conda-forge -c r -c defaults climate4r=1.5.0 dask

This should install newer versions of the packages.

Please, note that the conda package is only available for linux at the moment.

[priyapatel1]

Oh, thank you for the clarification. How do I install on a Mac OSx operation system?

[zequihg50]

You might try the following:

conda create -n climate4R -c conda-forge -c r r-base=3.6 r-rjava r-devtools r-sampling r-easyverification r-deepnet r-gtools r-verification r-mapplots r-sticky r-kohonen r-presenceabsence r-padr r-ncdf4 r-base r-rcurl r-latticeextra r-earth r-ranger r-signal r-e1071 r-caret r-akima r-geosphere r-randomforest r-splancs r-climdex.pcic r-gdalutils r-glmnet r-keras r-spei r-dismo r-udunits2 r-rmarkdown r-tree r-evd r-vioplot 'r-spatstat.linnet<2' 'r-spatstat<2'

And from an R terminal:

install.packages("https://github.com/SantanderMetGroup/climate4R.UDG/tarball/v0.2.0",repos=NULL);library(climate4R.UDG);packageVersion("climate4R.UDG")
install.packages("https://github.com/SantanderMetGroup/loadeR.java/tarball/v1.1.1",repos=NULL);library(loadeR.java);packageVersion("loadeR.java")
install.packages("https://github.com/SantanderMetGroup/loadeR/tarball/v1.7.0",repos=NULL);library(loadeR);packageVersion("loadeR")
install.packages("https://github.com/SantanderMetGroup/loadeR.ECOMS/tarball/v1.4.6",repos=NULL);library(loadeR.ECOMS);packageVersion("loadeR.ECOMS")
install.packages("https://github.com/SantanderMetGroup/transformeR/tarball/v2.0.1",repos=NULL);library(transformeR);packageVersion("transformeR")
install.packages("https://github.com/SantanderMetGroup/downscaleR.keras/tarball/v0.0.2",repos=NULL);library(downscaleR.keras);packageVersion("downscaleR.keras")
install.packages("https://github.com/SantanderMetGroup/downscaleR/tarball/v3.3.1",repos=NULL);library(downscaleR);packageVersion("downscaleR")
install.packages("https://github.com/SantanderMetGroup/visualizeR/tarball/v1.6.0",repos=NULL);library(visualizeR);packageVersion("visualizeR")
install.packages("https://github.com/SantanderMetGroup/convertR/tarball/v0.2.0",repos=NULL);library(convertR);packageVersion("convertR")
install.packages("https://github.com/SantanderMetGroup/geoprocessoR/tarball/v0.2.0",repos=NULL);library(geoprocessoR);packageVersion("geoprocessoR")
install.packages("https://github.com/SantanderMetGroup/climate4R.climdex/tarball/v0.2.1",repos=NULL);library(climate4R.climdex);packageVersion("climate4R.climdex")
install.packages("https://github.com/SantanderMetGroup/climate4R.indices/tarball/v0.1.0",repos=NULL);library(climate4R.indices);packageVersion("climate4R.indices")
install.packages("https://github.com/SantanderMetGroup/loadeR.2nc/tarball/v0.1.1",repos=NULL);library(loadeR.2nc);packageVersion("loadeR.2nc")
install.packages("https://github.com/SantanderMetGroup/fireDanger/tarball/v1.1.0",repos=NULL);library(fireDanger);packageVersion("fireDanger")
install.packages("https://github.com/SantanderMetGroup/VALUE/tarball/v2.2.1",repos=NULL);library(VALUE);packageVersion("VALUE")
install.packages("https://github.com/SantanderMetGroup/climate4R.value/tarball/v0.0.2",repos=NULL);library(climate4R.value);packageVersion("climate4R.value")
install.packages("https://github.com/SantanderMetGroup/mopa/tarball/v1.0.0",repos=NULL);library(mopa);packageVersion("mopa")
install.packages("https://github.com/SantanderMetGroup/drought4R/tarball/v0.2.0",repos=NULL);library(drought4R);packageVersion("drought4R")
install.packages("https://github.com/SantanderMetGroup/climate4R.datasets/tarball/v0.0.1",repos=NULL);library(climate4R.datasets);packageVersion("climate4R.datasets") 

[cs2239]

Thanks for this. I created a clean python env and executed the command: conda install -c santandermetgroup -c conda-forge -c r -c defaults climate4r=1.5.0 dask Install was fine, then I tried a toy example... require(climate4R.datasets) data("EOBS_Iberia_tas") data("CORDEX_Iberia_tas") y <- EOBS_Iberia_tas x <- CORDEX_Iberia_tas data("CORDEX_Iberia_tas.rcp85") newdata <- CORDEX_Iberia_tas.rcp85 call to biasCorrection still fails... out <- biasCorrection(y = y, x = x, newdata = newdata, method = "isimip3")Time difference of 1 daysTime difference of 1 daysTime difference of 1 days[2021-05-24 09:02:18] Argument precipitation is set as FALSE, please ensure that this matches your data.[2021-05-24 09:02:19] Number of windows considered: 1...[2021-05-24 09:02:19] Bias-correcting 1 members separately...Error in py_run_file_impl(file, local, convert) : ModuleNotFoundError: No module named 'dask'Detailed traceback: File "<string>", line 122, in <module> File "/home/dev/miniconda3/envs/climate4r/lib/R/library/reticulate/python/rpytools/loader.py", line 44, in _import_hook level=level>

…

On Thu, May 20, 2021 at 7:47 PM Ezequiel Cimadevilla Alvarez < ***@***.***> wrote: Hi, would you try to install with the following command into a clean environment? conda install -c santandermetgroup -c conda-forge -c r -c defaults climate4r=1.5.0 dask This should install newer versions of the packages. Please, note that the conda package is only available for linux at the moment. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN4QSA2ECFUJKD4KDZ4RMMTTOVDK5ANCNFSM44BEQGPQ> .

[cs2239]

#toy example based on biasCorrection help
library(downscaleR)
require(climate4R.datasets)
data("EOBS_Iberia_tas")
data("CORDEX_Iberia_tas")
y <- EOBS_Iberia_tas
x <- CORDEX_Iberia_tas
data("CORDEX_Iberia_tas.rcp85")
newdata <- CORDEX_Iberia_tas.rcp85

#test reasonable BC methods for tas
out <- biasCorrection(y = y, x = x,newdata = newdata, method = "delta")
#no warnings, no errors

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "eqm")
#no warnings, no errors

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "pqm")
#no warnings, no errors

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "qpqm")
There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
  1: In min(p, na.rm = TRUE) : no non-missing arguments to min; returning Inf
#warnings trigger on ocean pixels?

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "mva")
#no warnings, no errors

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "dqm")
Error in lm.fit(cbind(1, seq_along(s)), s) : NA/NaN/Inf in 'y'
> sum(is.nan(y$Data))
[1] 398905
> yy <- y
> yy$Data[is.nan(y$Data)] <- 0
> sum(is.nan(yy$Data))
[1] 0 #no NA/NaN/Inf in 'y' now!
out <- biasCorrection(y = yy, x = x, newdata = newdata, method = "dqm")
Error in lm.fit(cbind(1, seq_along(s)), s) : NA/NaN/Inf in 'y'
#even after replacing no_data values with 0, still fails
#x and newdata are clean, yy is clean after replacing NaN with 0
#ocean pixels again?

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "qqm")
Warning messages:
  1: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
#not sure what is triggering this, likely ocean tiles again as a tie on land with tas seems odd

out <- biasCorrection(y = y, x = x, newdata = newdata, method = "isimip3")
Error in
py_run_file_impl(file, local, convert) : ModuleNotFoundError: No module
named 'dask'Detailed traceback: File "<string>", line 122, in <module> File
"/home/dev/miniconda3/envs/climate4r/lib/R/library/reticulate/python/rpytools/loader.py",
line 44, in _import_hook level=level>