VegXshiny

Veg-X is a standard for the exchange of vegetation data. VegXshiny is an interactive web application for converting datasets into Veg-X and back.

The increasing digitization of research data has motivated data standardization efforts across scientific disciplines. For vegetation data, Veg-X (Wiser et al., 2011) is such a standard. It provides both flexibility and precision in representing vegetation data of different origins and formats. It's scope is the exchange of data-sets both between vegetation scientists and between vegetation scientists and database operators.

VegXshiny was developed as a GUI-based application for creating Veg-X documents and is build around the R package VegX (De Cáceres, 2018). It also allows the conversion from Veg-X documents to other formats.

The figure below shows a typical workflow. Start with the 'Upload' tab of the main menu. The second step is the actual import, where dialogs guide the user through the process of mapping data to the appropriate Veg-X elements. Once these steps have been completed, the VegX code can be reviewed and downloaded.

The 'Upload', 'Import', 'Review', 'Download' and 'Action Log' tabs have their own help sections, which describe their functionality in detail.

-------------

Package development

Christian König, Sebastian Schmidtlein

Acknowledgments

The development of VegXshiny is endorsed by the International Association for Vegetation Science (IAVS) and received funding by the German Research Foundation (DFG, project number 460840087.

Development of the R package VegX : Miquel De Cáceres

Veg-X standard development: Brad Boyle, Miquel De Cáceres, Martin Kleikamp, Christian König, Robert K. Peet, Sebastian Schmidtlein, Nick Spencer, Susan K. Wiser

References

Wiser, S.K., Spencer, N., de Cáceres, M., Kleikamp, M., Boyle, B., Peet R.K. (2011): Veg-X - an exchange standard for plot-based vegetation data. Journal of Vegetation Science 22: 598-609, doi:10.1111/j.1654-1103.2010.01245.x .

De Cáceres (2018): VegX: Vegetation data in Veg-X. R-package https://iavs-org.github.io/VegX

How Veg-X is structured

The heart of the Veg-X standard is its XML schema. An XML schema is composed of nested tags (elements) with defined attributes that describe the structure of an xml document. Veg-X uses this approach to specify what a valid Veg-X document looks like. The current version of the Veg-X schema defines 27 main elements that capture information about plots, taxa, observations and other aspects of a vegetation data set. Each main element may have a varying number of internal nodes that further demarcate thematic sections. Finally, leaf elements of a Veg-X document contain the data. A comprehensive overview of the Veg-X format can be found here .

Below is an interactive tree viewer of the Veg-X schema. Expand an element by clicking the plus icon (+). Additional information about the schema definition of the element (e.g. data type, number of occurrences, etc.) is displayed when hovering the mouse over it. The search box at the top of the tree allows to search for specific node names throughout the schema.


Frequently asked questions

The scope of Veg-X is the exchange of data-sets both between vegetation scientists and between vegetation scientists and database operators. VegXshiny can be used to convert smaller data sets to Veg-X and for the conversion of Veg-X documents into table formats.
Each main VegXshiny section (Upload, Import, Review, Download, Action Log) has a help section where the functionality is described in detail.
Veg-X is extremely flexible in accepting all kinds of vegetation data, but VegXshiny does not cover everything. Possible is, for example, the import of records of species in plots (e.g. species cover in layers) together with ancillary information about plots and observations. Currently unsupported is plotless sampling. The plan is to add other Veg-X elements little by little. If you need a feature urgently, you can post a request on Github .
Turboveg files, and of course Veg-X itself, should not need further preparation to be imported. Tabular data must often be rearranged but this can be done in the 'File editor' in the 'Upload' section. Tables need to be in 'long' table format and records need to be uniquely assigned by plot IDs; time-dependent plot observations (like species cover) also need a date.
Check the log. It can sometimes provide useful details as to why the data could not be imported. Typical import problems are treated in the dedicated help section
The online version of VegXshiny currently supports uploads up to a size of 50 MB. If you are handling larger files you can install VegXshiny from github and run the application locally:
install.packages("devtools")
devtools::install_github("iavs-org/VegXshiny")
VegXshiny::run_app(max_upload_size_MB = 99) # set limit and run
Use the following code to install VegXshiny from github and run the application:
install.packages("devtools")
devtools::install_github("iavs-org/VegXshiny")
VegXshiny::run_app(max_upload_size_MB = 99) # set limit and run

Converting a vegetation table into the Veg-X exchange format

The video explains how to transform a vegetation table with species covers into the Veg-X exchange format (01:07 Preparation of 'long tables', 08:46 Import into Veg-X).

If required, further videos can be provided here. Please let us know what you need.

Contact

For questions and feedback, please open an issue on Github

Privacy

All uploaded data is deleted at the end of a session. No data is collected and no cookies used.

Impressum

Anbieter:

Karlsruher Institut für Technologie (KIT)

Rechtlicher Sitz:
Kaiserstr. 12
76131 Karlsruhe
Deutschland
Tel. +49 721 608-0
Email: info@kit.edu
Umsatzsteueridentifikationsnummer: DE266749428

Rechtsform: Körperschaft des öffentlichen Rechts
Vertretungsberechtigt: Prof. Dr. Oliver Kraft

Projektkoordination:
Prof. Dr. Sebastian Schmidtlein
Institut für Geographie und Geoökologie
Kaiserstr. 12
76131 Karlsruhe
Deutschland
Email: schmidtlein@kit.edu

Links:

Die Web-Seiten des Karlsruher Instituts für Technologie enthalten Verweise (Links) zu Informationsangeboten auf Servern, die nicht der Kontrolle und Verantwortlichkeit des Karlsruher Instituts für Technologie unterliegen. Das Karlsruher Institut für Technologie übernimmt keine Verantwortung und keine Garantie für diese Informationen und billigt oder unterstützt diese auch nicht inhaltlich.

Upload files

Help with uploading

This is the single entry point for user-supplied data. All files that contain information intended for import need to be uploaded here. To upload files, click the upload button and browse your local file system. You can select and upload multiple files at once by pressing [CTRL]. Currently, the following file types are supported:

Tabular data:

  • .csv (comma-separated)
  • .txt, .tsv (tab-separated)
  • .xls, .xslx

Turboveg 2:

  • xml

Veg-X:

  • xml

If an upload of tabular data fails, check the following possible reasons or fixes:

  • Column names are expected in the first line and their number must match the number of columns.
  • If you have a text file and commas as column separators, use '.csv' as file extension. In case of tabs use '.tsv' or '.txt'.
  • Space is currently not recognized as a delimiter.

Completed uploads are listed under 'Uploaded Files' with a corresponding icon for the file type.

Clicking on one of the file icons under 'Uploaded files' will activate the 'File Editor'. Depending on the file type, this may either be a text editor (for xml data) or a spreadsheet editor (for tabular data). Click on the 'Edit' tab above the data view to access functions for editing and reshaping the uploaded files.

Help with editing

Regarding tabular data:

Check out the video tutorial on the entry page!

For import into Veg-X, the data must be in 'long' format, where each metric has only one column, even if it has been measured on different objects. For example, species cover values all go into one column and there is a second column indicating the species to which the values belong. Records are identified by plot ID or, in the case of time-dependent observations like of species cover, by plot ID and date.
Feeling lost? We are preparing video tutorials for common use cases and will link to them here as they become available.

Basic table editing can be reached using the context menu (right click on a column or row name when the table is in editing mode). The options are self-explanatory.

There is currently a bug in the underlying packages that prevents the table from being saved after removing scattered columns. You can work around this by first saving the object under a new name and then removing separated individual columns or blocks of columns one by one. Save the object after each removal.

More tools are available via the buttons at the top:

Save edits
Save: Save edits overwriting the current object
Save as: Save edits as a new object
Reshape table
Pivot: Transform table from 'wide' to 'long' format. Opens an input screen with detailed instructions
Transpose: Turn rows into columns
Crop: Removal of table sections
Merge columns: Merge two columns into one. This can be needed when data points are identified by more than one column. The merged column is appended as the last column.
Split column: Reverse the merging of columns
Row to colnames: Create column names from a row
Column to rownames: Create rownames from a column
Rownames to column: Transform rownames into a new column
Colnames to row: Transform column names into a new row
Edit values
Edit colnames: Beautify column names
Format date: For import, dates must be listed in a column. If they are not, transpose the table (and back-transpose if necessary). Veg-X requires dates to be in the international ISO 8601 format (YYYY-MM-DD, e.g. 2024-05-30). You need to specify what type of date you have to start the conversion. For example, if the dates are numbers generated during an Excel import from a Windows machine, enter the string 'excel-win' (without quotation marks) in the 'Original format' field. The string for MAC Excel is 'excel-mac'. Other definitions follow R conventions. For example, if your values look like 17.12.2023, the full conversion specification is %d.%m.%Y. For 02/21/18 it is %m/%d/%y. Common symbols are %d (day of month as number), %m (month as number), %b (month abbreviated like Jan), %B (month full name like January), %y (year 2 digit), %Y (year 4 digit). The separators are used literally (like ' ' for space). See the R documentation for more information. If a valid original format has been entered, a conversion example will appear at the bottom.
Discard edits: Exit the edit mode without saving

Describe your project and its contributors (datasets are assigned in the next steps)


Import static plot properties (time-dependent observations are assigned in the next step)


Import your observation data


Categories

Check your entries


Project
Plots
Observations

Help with reading tabular data

Check out the video tutorial on the entry page!

While the import from Turboveg is a one step procedure, tabular data comes in many flavours so that VegXshiny needs to be told which columns of a data table goes in which Veg-X container. The import dialogue guides through this procedure.

Importing requires the data to be organized so that each variable has its own column and each plot or observation has its own row. There should be only one column for each type of variable. For example, if you measured species cover, you need one column for species covers, another column for the respective species, and columns for plot IDs, dates and maybe vegetation layers. A date is only needed for time-dependent observations. The 'File editor' in the Upload section provided tools for transforming the data accordingly.

The import of tabular data is structured into five steps:

  1. Project information
  2. Provide a general description of the project and its contributors. No files need to be assigned in this step.

  3. Plot information
  4. Describe static plot properties (like surface area of the plot. This is distinguished from features that belong to a plotObservation, such as the date of a recording). The expected data set contains one plot per row. Each row is identified by a unique plot ID (mandatory).

    Each mapped plot property needs to be linked to a measurement method. Prefabricated descriptions of common measurement methods are available in the corresponding mapping dialog. For example, if plot area was measured in square metres, selecting "Plot area/m2" as measurement method will define an appropriate method and link it to the measurement values during import. If none of the predefined methods is applicable, a custom method can be defined by selecting the "...add custom method" option.

  5. Observations
  6. Add time-dependent observations, such as species cover, vegetation layers, or soil properties. All observations in Veg-X refer to a plotObservation, i.e. a sampling event at a specific plot at a specific point in time. Observation data of any type are identified at least by a unique combination of plot ID and date .

    As in the case of plot properties, measurements during observations require defined measurement methods. A dropdown menu provides a list of predefined methods. If none of these predefined methods is applicable, a custom method can be created by selecting the "...add custom method" option.

    Note that currently VegXshiny does not support imports into the Veg-X container for measurements applying to an entire plant community, such as successional stage (called communityObservations). The same applies to measurements related to individual organisms, such as DBH (called individualOrganismObservations).

  7. Check inputs
  8. Check your mapped inputs before starting the import. Color-coded boxes indicate the status:

    • Green : Inputs complete, ready for import.
    • Red : Inputs incomplete, review before import.
    • Grey : Inputs incomplete, ignored during import.

    Only when all input categories are marked either green or grey, you will be able to proceed with the import.

  9. Import
  10. Pressing Import and confirming the dialog will create a new Veg-X document from the specified mappings. Your current Veg-X document will be overwritten by this.

    Note that the import may take a while when working with large input files. A progress bar in the lower left corner indicates the current status. Once the import is finished, you can review the imported Veg-X file in the Review Section or export it under Download.

Pick a source file uploaded in the upload section

Document summary


Help with importing Turboveg

You can import data from Turboveg 2 standard XML files .

Choose an uploaded Turboveg 2 XML file and press the read button. Once the extraction is finished, a summary is displayed.

Read Turboveg file, review the summary and run the import. The Import button will open a dialog where you can select, which 'undefined' Turboveg header data you want to import (i.e. data lacking information that is required by pre-defined Veg-X elements). The selected fields will be attached as user-defined to their corresponding plot node in the Veg-X document. Pressing Proceed will then start the conversion to Veg-X.

Pick a source file uploaded in the upload section

Document summary


Help with reading Veg-X

Read Veg-X, review the summary and run the import.

Inspect, edit and validate your current Veg-X document


                      
                    

Help with reviewing Veg-X

Typically, no action is required here and you can continue with the download.

You can edit the data after pressing the 'Edit' button but caution is advised, as manually editing a Veg-X document can quickly invalidate it. Note that edits cannot be undone once saved.

Pressing the 'Validate' button tests whether the current Veg-X document conforms to the Veg-X XML Schema and whether there are any potential problems with references to nodes. Validation errors and other problems are listed in the action log.

Help with downloading

If a valid Veg-X document is available, an overview of the structure (with the number of nodes of different types) is shown on this page. The Veg-X document can then be downloaded as:

  • xml : Veg-X xml file
  • zip : Set of 'long tables' - a format that can be easily processed by generic database systems. While these tables are provided as a zip archive of csv files, the same tables can also be downloaded as elements of an R list object:
  • rds : R list object. Use readRDS() to read it into R.
  • csv : A comma-separated 'wide table' or 'vegetation table') for import into spreadsheet programs.

Note that Turboveg 3 is able to import Veg-X XML documents.

Action log

Help with the action log

The Action Log records user actions and application messages during a session. This includes file uploads and edits, import messages, Veg-X document edits and validations and file exports. If any problem occurrs during the session, the Action Log may provide relevant information.