View session timetable
Digital Poster/Demo
License: Creative Commons Attribution 4.0 International (CC-BY-4.0)

  • 18:15 – 19:00
Community members
View details

Enhanced metadata handling and automation of soil moisture time series integration into the operational data hub “International Soil Moisture Network (ISMN)”

Digital Poster/Demo
In session Postersession No. 2 , Sept. 3, 2025, 18:15 – 19:00
Exact timing: 18:15 – 19:00

Böhmer, Fay1 , Korres, W.2 , Olarinoye, T.3 , Kramer, K.2 , Dietrich, S.3 , Zink, M.3
  1. International Soil Moisture Network (ISMN) @ Federal Institute of Hydrology (BfG)
  2. Federal Institute of Hydrology (BfG)
  3. International Center for Water Resources and Global Change (ICWRGC)

Soil moisture is a critical environmental variable, that has a big impact on hydrological extremes, plant water availability and climate processes. Hence, accurate and extensive data on in situ soil moisture is crucial to assess these environmental impacts. The International Soil Moisture Network (ISMN, https://ismn.earth ) collects such data from several monitoring networks providing a harmonized, quality controlled and free accessible archive.

Collecting and integrating soil moisture data from various data providers presents significant challenges, as they use diverse measurement technologies and data formats. Next to daily ingesting data into the database, keeping the metadata updated and clean remains challenging. ISMN’s existing metadata mapping between input data and database was potentially leading to data corruption due to non-robust and weak data processing.

To ensure reliable data ingestion and mitigate corruption risks, we significantly enhanced the data integration and harmonization process. This involved improving data download robustness, developing an automated procedure for metadata mapping, and implementing automated metadata checks and retrievals. After successfully testing the automated detection and incorporation of metadata changes (e.g. sensor types, exchange dates) the maintenance efforts and human interventions were reduced significantly. Future work will focus on expanding this functionality to other providers and fully incorporating it into our operations.

The correct incorporation of accurate metadata is crucial and is anticipated to enhance the usability of soil moisture data and enabling consistent comparisons across different datasets, and supporting robust scientific analyses. Still, information of changes to observational networks, such as relocation of stations or replacement of measurement equipment, is difficult to get hold off.