Sub-daily Run

[Nooshdokht-Bayatafshary]

Dear developers,

I am currently in the process of integrating 6-hour meteorological data, encompassing air temperature, dew point, wind, and solar radiation, to compute ET0 using the LISVAP model. While progressing, I've encountered certain aspects of the model manual that warrant clarification, particularly regarding the setting for DtSec.

I'm seeking guidance on whether it is advisable to set DtSec as 3600*6 to align with the 6-hour time step for the LISVAP module. Alternatively, would it be more suitable to set DtSec as 86400, considering daily meteorological inputs, and then configuring ReportSteps for 6-hour outputs?

According to my interpretation of the LISVAP manual, some simplifying assumptions related to the radiation balance are deemed invalid at time steps smaller than days. The manual suggests using LISVAP exclusively for daily time intervals (with DtSec always set to 86400). However, I am concerned about the impact on accuracy due to the wide range of radiation and temperature within 24 hours. The uniform distribution assumed for the 6-hour output in LISVAP might not adequately capture variations, potentially affecting the precision of ET calculations. I would appreciate any insights or recommendations about this.

Best regards,
Nooshdokht

[StefaniaGrimaldi]

Dear @Nooshdokht-Bayatafshary,

thank you for your enquiry.

We started working on the technical implementation of LISVAP to make sure that reading and writing modules make correct use of the sub-daily time stamps.

You are correct, some further discussion is required to assess the suitability of the approach for sub-daily applications.

Please be aware the the current LISVAP master code generates 6-hourly outputs, but each 6-hourly step within the same day has the same value. This solution was implemented as a trade-off between data availability and LISFLOOD model requirements for the implementation of the operational European Flood Awareness System of the Copernicus Emergency Management Service.

However, if our understanding is correct, you are planning on using 6-hourly (or hourly) weather input data. Could you please share the exact list and source of weather data that you are planning to use (e.g. could you please clarify "solar radiation")? As explained in the manual, LISVAP implements several solutions to comply with different conditions of data availability. Furthermore, for instance, we found this relevant application of FAO’s Penman-Monteith equation for hourly steps: https://www.nature.com/articles/s41597-021-01003-9. Knowing the exact list of weather variables will allow more in-depth investigations.

Kind regards,
on behalf of the developers team,
Stefania

[Nooshdokht-Bayatafshary]

Dear Stefania,

Thank you for your response and insights into the ongoing technical implementation of LISVAP.

Regarding your inquiry about the weather input data, for average air temperature, dew point, and wind speed, I use the synoptic stations' 6-hour data in my area, which undergo spatial interpolation and are converted into a map. Additionally, I employ the FAO 56 recommended equation (equation 28) for calculating sub-daily downward surface solar radiation. The plan is to execute the LISVP model with the EFAS option and generate 6-hourly outputs.

Best regards,
Nooshdokht

[StefaniaGrimaldi]

Dear @Nooshdokht-Bayatafshary,
thank you for providing more details.
We would like to let you know that we have not forgotten about your request!
We are having a few busy weeks: we will contact you for further discussions by the end of the month.
Kind regards,
Stefania

[StefaniaGrimaldi]

Dear @Nooshdokht-Bayatafshary

We are glad to announce LISVAP v1.3.0!

@ggomes76 implemented all the required technical changes to the input and output routines: LISVAP now correctly reads sub-daily inputs and produces sub-daily outputs.

Please note that the equations have not been changed.
As stated in the documentation, and as you correctly reported, LISVAP was originally developed for applications with daily time steps.

It is great to know that you are using FAO 56 recommended equation 28 to compute the Extraterrestrial radiation for hourly or shorter periods (Ra) https://www.fao.org/3/X0490E/x0490e07.htm#calculation%20procedures
This is indeed a required improvement for sub-daily computations. Therefore, we would like to ask you if you are willing to collaborate and include your routine into LISVAP. If needed, @ggomes76 will provide technical support for GitHub.

Regarding your plan "The plan is to execute the LISVAP model with the EFAS option and generate 6-hourly outputs."
The computation of the net absorbed radiation within EFAS protocol makes use of measured downward surface solar radiation.

However, due to the lack of sub-daily measurements, EFAS 6hourly output is generated using daily inputs and the 6-hourly output steps have all the same values (option: "output6hourly"). This strategy seems different from your requirements. Do you have downward surface solar radiation data? As an alternative solution, FAO 56 suggests equation 35, which however requires the actual duration of sunshine. Equation 35 is not yet implemented in LISVAP.

Please note that the current LISVAP implementation the computation of the Soil heat flux for sub-daily intervals should be updated using eq. 45 and 46 https://www.fao.org/3/X0490E/x0490e07.htm#calculation%20procedures

The development of LISVAP for sub-daily implementations is currently not envisaged in our operational plan. Nevertheless, we are keen to collaborate with you and we highly encourage your contribution to the development. Therefore, we would like to learn your point of view on the analysis above, as well as your proposals for improvements.

Kind regards,
on behalf of the developers team,
Goncalo and Stefania

[Nooshdokht-Bayatafshary]

Dear Goncalo and Stefania,

Thank you for reaching out and for the detailed explanation regarding the latest version of LISVAP (v1.3.0). I'm thrilled to hear about the improvements made by @ggomes76 in integrating sub-daily input and output routines into LISVAP. It's certainly a significant step forward for users like me who require finer time resolutions for our computations.

I am indeed using the FAO 56 recommended equation 28 for computing Extraterrestrial radiation (Ra) for hourly or shorter periods. Integrating this routine into LISVAP would be a valuable addition, and I'm more than willing to collaborate on this front. I have already developed the code for sub-daily Ra computations in Python, and I believe with some adjustments and your team's guidance, we can smoothly integrate it into LISVAP.

However, I must confess that I'm relatively new to working within GitHub and I'm not yet fully proficient with the LISVAP codebase. Thus, I anticipate needing some assistance and guidance from your team as we navigate through the integration process.

I'm also grateful for your suggestion regarding equation 35 as an alternative solution for computing downward surface solar radiation in the absence of accurate synoptic station data. This recommendation comes as a significant help, as acquiring precise data for my region has been a challenge.

Although I understand that the development of LISVAP for sub-daily implementations is not currently part of your operational plan, I am enthusiastic about collaborating with you to enhance LISVAP's capabilities in this regard. I appreciate your openness to collaboration and your encouragement of contributions from users like myself.

I look forward to further discussions on how we can move forward with these proposed improvements and contribute to the advancement of LISVAP together.

Kind regards,
Nooshdokht

[Nooshdokht-Bayatafshary]

Dear developers,

I have been utilizing the LISVAP model for a year now, processing data in 6-hour increments. However, I've encountered an issue regarding the output maps generation. Regardless of how we configure the ReportSteps parameter for the output maps, they are consistently being written on an hourly basis.

Upon inspecting the source code, it appears that the 'internal.time.frequency' parameter may not be correctly propagated between the relevant functions. It seems logical that such a parameter should be defined in the settings file. However, I couldn't find the relevant time-related constant in the Ifuser section of the settings file. This absence might be contributing to the issue with the output maps generation. Nevertheless, even without explicit definition, making the following adjustments in the lisflood-lisvap/src/lisvap/utils/writers.py file (lines 315-329) should resolve the issue, allowing the outputs to be generated according to the configured ReportSteps.

if flag_time:
    # In case output6hourly==True, replicate four daily maps to get the 6 hourly output (EFCC-2316)
    # The timestep needs to increase by 4
    if output6hourly:
        time_frequency = 6
        for i in range(4):
            map_idx = output_index * 4 + i
            nf1.variables[time_variable][map_idx] = (timestep * 4 - 4 + i) * time_frequency
            nf1.variables[prefix][map_idx, :, :] = mapnp
    else:  # Generate daily output
        if timestep_stride < 86400:
            time_frequency = int(timestep_stride / 3600)
        nf1.variables[time_variable][output_index] = (timestep - 1) * time_frequency  # timestep - time_frequency
        nf1.variables[prefix][output_index, :, :] = mapnp
else:
    nf1.variables[prefix][:, :] = mapnp
nf1.close()

I would greatly appreciate any feedback or suggestions regarding this matter.

Best regards,
Nooshdokht

[gnrgomes]

Dear @Nooshdokht-Bayatafshary,
The time frequency was not defined in the settings file because we don't need it to be defined there since it is defined already in the meteo input files and is read from the netCDF metadata during the first access to the meteo files. It should be filled with the value 6 if you have 6hourly data with timeunits in hours as it should be.
The way it was developed allows to have any type of timeunits and frequency. You just need to setup DtSec=21600 in the settings file to process 6hourly maps and then you need your input files to be netCDF compliant (in our case CF 1.6), which means timunits and time frequency need to be setup in the netCDF files.

In netCDF CF 1.6, the different time units patterns available include (date is only an example):

1. "seconds since 1970-01-01 00:00:00"
2. "seconds since 1970-01-01 00:00:00 UTC"
3. "days since 1970-01-01"
4. "hours since 1970-01-01 00:00:00"
5. "minutes since 1970-01-01 00:00:00"
6. "milliseconds since 1970-01-01 00:00:00"
7. "microseconds since 1970-01-01 00:00:00"
8. "nanoseconds since 1970-01-01 00:00:00"

These patterns are used to specify the time units for the time coordinate variable in a netCDF dataset.

Could you please share with us an example of input file that you intend to use? I might try to help.

[Nooshdokht-Bayatafshary]

Dear @gnrgomes,

Thank you very much for your detailed explanation. I appreciate your assistance in clarifying the necessity of defining timeunits and frequency in the settings file and within the netCDF metadata. Your insights have been incredibly helpful.

I apologize for the oversight on my part regarding the setup of units and frequency. I've attached a sample input file created using the xarray package in Python (ta.zip). I'm currently in the process of adding the necessary attributes as you mentioned, including setting up the time units and frequency according to the CF 1.6 convention.

Once again, thank you for your guidance, and I'll make sure to implement these adjustments in my input files.

Best regards,
Nooshdokht

[gnrgomes]

Dear @Nooshdokht-Bayatafshary ,

If you just want to test there is no need to recreate the files, you could try just to add the property feature to the time variable of each of your files.
Backup your files before using any of the methods bellow that I write here as examples. If you use the CDO method you don't need the backup because it will create a new file.

By using NCO:
ncatted -O -a frequency,time,c,c,"6" your_file.nc

By using CDO:
cdo setattribute,time@frequency="6" your_file.nc modified_file.nc

Or just directly in python:

import netCDF4 as nc
nf = nc.Dataset('your_file.nc', 'a')
nf.variables['time'].setncattr('frequency', '6')
nf.close()