# SticsRFiles

    library(SticsRFiles)

## Introduction

The goal of `SticsRFiles` is to perform manipulations of all types of files related to the input and outputs of the STICS model.

This article presents the design of the package and its basic features, i.e. the main functions to deal with the XML files. For a complete introduction to SticsRPacks (managing files, running simulations, plotting, optimization…), see the tutorial from the SticsRPacks package.

## Concepts

### Input files

#### Text files

The executable of the STICS model reads text files with standard names and format to import the inputs describing a single unit of simulation (USM), e.g. one crop for one year.

Here’s a typical list of the input files for the STICS executable (without the optional files):

    workspace
    │
    ├── 📜climat.txt               # The meteorological data for the USM
    ├── 📜ficini.txt               # The initialization
    ├── 📜ficplt1.txt              # Parameters for the plant to be simulated
    ├── 📜fictec1.txt              # The management applied to the soil and crop
    ├── 📜new_travail.usm          # The general configuration parameters for the USM
    ├── 📜param.sol                # Soil parameters
    ├── 📜station.txt              # The site parameters (*e.g.* altitude, latitude)
    ├── 📜tempopar.sti             # More general parameters not in the other files
    ├── 📜tempoparv6.sti           # Parameters for the custom versions of STICS
    └── 📜var.mod                  # Variables to write in the outputs

Because these files only describe one USM at a time and can be tedious to explore and parameterize, we usually don’t interact with them directly, but through another program: JavaSTICS.

#### XML files

JavaSTICS is a graphical user interface used to easily create input text files for the STICS executable according to the user’s choices, and for managing STICS simulations. JavaSTICS saves the parameter values and options choices in XML files.

The XML files store more information than the text files: not only do they store the parameter values but also their description, maximum and minimum boundary values, all existing formalisms, the different choices allowed, and more importantly they allow for the management of several USMs in the same folder (called workspace), and can help make successive USMs.

It is important to note that only JavaSTICS interact with the XML files, not the STICS executable. The STICS input text files are then automatically created by JavaSTICS when running a simulation, just before calling the STICS executable.

### SticsRFiles

`SticsRFiles` is an R package that uses JavaSTICS from the command line to manage the XML and the text files. We can generate XML or text files, get and set parameter values, import simulations outputs, and manage observation files.

Advanced features also include:

- The generation of XML input files from templates included in the package, and
- The production of XML input files using a mailing process from XML templates and Excel sheets containing multiple simulations contexts

## Short example

Here’s a simple example usage of `SticsRFiles` using an example workspace.

### Example data

All the example data used in this article are available from the `data` repository in the `SticsRPacks` organization.

`SticsRFiles` provides a function to download it from the command line. Please execute the following command in R:

    library(SticsRFiles)
    example_data <- SticsRFiles::download_data(
      example_dirs = "study_case_1",
      stics_version = "V10.0"
    )

The example data is downloaded by default in a temporary folder.

For the sake of readability, we’ll declare the workspace path and the path to the plant file here. But remember the functions can be applied to any XML files or workspaces.

    workspace <- file.path(example_data, "XmlFiles")
    plant_file <- file.path(workspace, "plant", "maisopti_plt.xml")

### Example usage

#### Find a STICS variable

`get_var_info()` helps to get any STICS variable name by doing a fuzzy search. For example to get all variables with `lai` in their names, you would do:

    SticsRFiles::get_var_info("lai")
    #>              name
    #> 5       albedolai
    #> 242        exolai
    #> 338        innlai
    #> 353        lai(n)
    #> 354 lai_mx_av_cut
    #> 355        laimax
    #> 356     laisen(n)
    #> 742         splai
    #> 805       ulai(n)
    #>                                                           definition
    #> 5                   albedo of the crop including soil and vegetation
    #> 242              reduction factor on leaf growth due to water excess
    #> 338 reduction factor on leaf growth due to NNI (nitrogen deficiency)
    #> 353                                          leaf area index (table)
    #> 354            LAI before cut (for cut crops , for others = lai(n) )
    #> 355                                          maximum leaf area index
    #> 356                      leaf area index of senescent leaves (table)
    #> 742                source to sink ratio of assimilates in the leaves
    #> 805                                relative development unit for LAI
    #>            unit type
    #> 5            SD real
    #> 242         0-1 real
    #> 338 innmin to 1 real
    #> 353      m2.m-2 real
    #> 354          SD real
    #> 355      m2.m-2 real
    #> 356      m2.m-2 real
    #> 742          SD real
    #> 805         0-3 real

Sometimes it is also useful to search in the variable definition instead of its name. To do so, you can use the `keyword` argument like so:

    SticsRFiles::get_var_info(keyword = "lai")
    #>                   name
    #> 5            albedolai
    #> 171 diftemp1intercoupe
    #> 172 diftemp2intercoupe
    #> 242             exolai
    #> 334     inn1intercoupe
    #> 336     inn2intercoupe
    #> 338             innlai
    #> 353             lai(n)
    #> 354      lai_mx_av_cut
    #> 355             laimax
    #> 356          laisen(n)
    #> 360               leai
    #> 742              splai
    #> 745     str1intercoupe
    #> 746     str2intercoupe
    #> 747     stu1intercoupe
    #> 748     stu2intercoupe
    #> 801           tustress
    #> 805            ulai(n)
    #>                                                                                                         definition
    #> 5                                                                 albedo of the crop including soil and vegetation
    #> 171        mean difference between crop and air temperatures during the vegetative phase (emergence - maximum LAI)
    #> 172       mean difference between crop and air temperatures during the reproductive phase (maximum LAI - maturity)
    #> 242                                                            reduction factor on leaf growth due to water excess
    #> 334                               average NNI during the cut (cut crop vegetative phase: emergence to maximum LAI)
    #> 336                             average NNI during the cut (cut crop reproductive phase: maximum LAI  to maturity)
    #> 338                                               reduction factor on leaf growth due to NNI (nitrogen deficiency)
    #> 353                                                                                        leaf area index (table)
    #> 354                                                          LAI before cut (for cut crops , for others = lai(n) )
    #> 355                                                                                        maximum leaf area index
    #> 356                                                                    leaf area index of senescent leaves (table)
    #> 360                                                                                 Leaf+ear area index = lai +eai
    #> 742                                                              source to sink ratio of assimilates in the leaves
    #> 745      average stomatal water stress index during the vegetative phase (emergence - maximum LAI) of forage crops
    #> 746    average stomatal water stress index during the reproductive phase (maximum LAI  - maturity) of forage crops
    #> 747   average turgescence water stress index during the vegetative phase (emergence - maximum LAI) of forage crops
    #> 748 average turgescence water stress index during the reproductive phase (maximum LAI  - maturity) of forage crops
    #> 801                      reduction factor on leaf growth due to the effective water stress  (= min(turfac,innlai))
    #> 805                                                                              relative development unit for LAI
    #>            unit type
    #> 5            SD real
    #> 171     degreeC real
    #> 172     degreeC real
    #> 242         0-1 real
    #> 334         0-2 real
    #> 336         0-2 real
    #> 338 innmin to 1 real
    #> 353      m2.m-2 real
    #> 354          SD real
    #> 355      m2.m-2 real
    #> 356      m2.m-2 real
    #> 360      m2.m-2 real
    #> 742          SD real
    #> 745         0-1 real
    #> 746         0-1 real
    #> 747         0-1 real
    #> 748         0-1 real
    #> 801         0-1 real
    #> 805         0-3 real

#### Get informations on parameters

`get_param_info()` can be used giving a part of parameters names, here is a subset of the parameters names containing `lai`:

    head(get_param_info(param = "lai"))
    #>             name    file   min max
    #> 225    cielclair STATION     0   1
    #> 305 codeclaircie  PARTEC     1   2
    #> 339    codelaitr  PARPLT     1   2
    #> 408    codlainet  PARPLT     1   2
    #> 556      dlaimax  PARPLT 5e-06 0.5
    #> 557  dlaimaxbrut  PARPLT 5e-06 0.5
    #>                                                                                       definition
    #> 225             fraction of sunny hours allowing the inversion of thermal gradient with altitude
    #> 305                     option to simulate fruit removal: 1 = no, 2 = yes (for smallest fruits)
    #> 339          option to calculate the intercepted radiation according to: 1 = LAI, 2 = soil cover
    #> 408 option to calculate the LAI: 1 = net LAI, 2 = difference between gross LAI and senescent LAI
    #> 556                                                    maximum rate of net daily increase of LAI
    #> 557                                                  maximum rate of gross daily increase of LAI

it can also be used by giving a keyword that will be searched in the parameters names and definitions, here is a subset of the parameters names containing `plant`:

    head(get_param_info(keyword = "plant"))
    #>                      name   file  min max
    #> 4                   adens PARPLT   -2   0
    #> 6                    adil PARPLT    1   7
    #> 7                 adilmax PARPLT    3  10
    #> 57     albedomulchresidus  PARAM 0.05 0.8
    #> 58  albedomulchresidus(1)  PARAM 0.05 0.8
    #> 59 albedomulchresidus(10)  PARAM 0.05 0.8
    #>                                                                 definition
    #> 4                                         Interplant competition parameter
    #> 6       parameter of the critical dilution curve [Nplante]=adil MS^(-bdil)
    #> 7  parameter of the maximum dilution curve [Nplante]=adilmax MS^(-bdilmax)
    #> 57                                                   albedo of plant mulch
    #> 58                                                   albedo of plant mulch
    #> 59                                                   albedo of plant mulch

#### Get parameter values

`get_param_xml()` is used to get the values of a parameter in an XML file. For example if we want to get `dlaimax`, we would do:

    dlaimax <- get_param_xml(plant_file, "dlaimax")
    dlaimax
    #> $maisopti_plt.xml
    #> $maisopti_plt.xml$dlaimax
    #> [1] 0.00321 0.00321 0.00321 0.00321 0.00321

But this function is way more powerful than just that. You can also get the values for all parameters in a given formalism (`formalisme` in French, yes some variables are still written in French in STICS). To do so, use the `select` argument like so:

    values <- get_param_xml(plant_file,
      select = "formalisme",
      select_value = "radiation interception"
    )
    unlist(values) # For pretty-printing
    #> maisopti_plt.xml.codetransrad        maisopti_plt.xml.forme
    #>                             1                             1
    #>     maisopti_plt.xml.rapforme        maisopti_plt.xml.adfol
    #>                             4                             1
    #>      maisopti_plt.xml.dfolbas     maisopti_plt.xml.dfolhaut
    #>                             5                             5

#### Set parameter values

We can also change the value of a parameter by programming using `set_param_xml()`. It is used similarly to `get_param_xml()`. For example if we want to increase `dlaimax` by 30%:

    new_values <- unlist(dlaimax) * 1.3
    set_param_xml(file = plant_file, param = "dlaimax", values = new_values, overwrite = TRUE)

Don’t forget to use the `overwrite` argument and set it to `TRUE`. It is `FALSE` by default to avoid any mishandling.

New values written in the file can be checked:

    dlaimax <- get_param_xml(plant_file, "dlaimax")
    dlaimax
    #> $maisopti_plt.xml
    #> $maisopti_plt.xml$dlaimax
    #> [1] 0.004173 0.004173 0.004173 0.004173 0.004173

#### Generate observations files

We can generate observation files from a `data.frame` using `gen_obs()`.

Lets create some dummy `data.frame` first:

    obs_df <- data.frame(
      usm_name = "Test", ian = 2021, mo = 3:10, jo = 1,
      `masec(n)` = 0.1 * 3:10
    )

Then we can write the data to a file using `gen_obs()`:

    gen_obs(df = obs_df, out_dir = "/path/to/dest/dir")

#### Read observations files

We can read the observation files in a workspace using `get_obs()`. Note that all observation files should be named after the USM they are linked to. See the help page for more details, e.g. about intercrops.

    obs <- get_obs(workspace)
    #> bo96iN+.obs
    #> bou00t1.obs
    #> bou00t3.obs
    #> bou99t1.obs
    #> bou99t3.obs
    #> lu96iN+.obs
    #> lu96iN6.obs
    #> lu97iN+.obs

#### Read simulation files

Likewise, we can read the observation files in a workspace using `get_sim()`:

    sim <- get_sim(workspace)
    #> Warning in get_file_(workspace = x, usm_name = usm_name, usms_filepath =
    #> usms_path, : Not any sim file detected in
    #> workspace/tmp/RtmpjkDYAq/data-master/
    #> study_case_1/V10.0/XmlFiles

But as there aren’t any simulations yet in the workspace, the function will return an error. To make a simulation, head to SticsOnR. Then to plot both observations and simulations, you can use CroPlotR.