fdsnxml2inv: inconsistent timespan behavior when agregating location

[petrrr]

This report is derived from an observation we did while resoving an webservices problem.

Our station inventory is managed as native FDSN StationXML and converted into SC Inventory XML as part of the configuration process. In one specific case the coordinates for different epochs differ, while the location code remains unchaged. We understand that while FDSN does not define rules on the exact usage of the location code, SC Inventory data model requires a location to have the same coordinates. This situation is usually handled by fdsnxml2inv by providing a warning and use only one set of coordinates for all comprised epochs. In our case these informations are not used, so this modification of metadata seems acceptable.

Processing /tmp/20231124_161019-build_seiscomp3_db_from_stationxml/stationxml/IV_.stationxml
 - parsing StationXML
 - converting into SeisComP-XML
W  IV.CPOZ location '' starting 2011-07-19 10:55:00 has conflicting coordinates: using the last read
   lat 40.8211 != 40.8211
   lon 14.1191 != 14.1187
   elevation 52 != 2

However, the problem we encountered is that the timespans in this situation are then handled in an incoherent way. The station correctly reports an open timespan. On the other hand, the location reports an endtime corrisponding to the first epoch and thus exluding the later two epochs. The two epochs are still retained in the resulting Inventory XML. Below the output from scinv ls:

station CPOZ   Darsena Pozzuoli - Stazione Osservatorio Vesuviano
      epoch 2011-07-19 10:55:00
      location __
        epoch 2011-07-19 10:55:00 - 2022-03-31 10:00:00
        channel HHE
          epoch 2011-07-19 10:55:00 - 2022-03-31 10:00:00
        channel HHE
          epoch 2022-04-11 11:00:00 - 2023-03-01 00:18:00
        channel HHE
          epoch 2023-03-01 00:18:00
        channel HHN
          epoch 2011-07-19 10:55:00 - 2022-03-31 10:00:00
        channel HHN
          epoch 2022-04-11 11:00:00 - 2023-03-01 00:18:00
        channel HHN
          epoch 2023-03-01 00:18:00
        channel HHZ
          epoch 2011-07-19 10:55:00 - 2022-03-31 10:00:00
        channel HHZ
          epoch 2022-04-11 11:00:00 - 2023-03-01 00:18:00
        channel HHZ
          epoch 2023-03-01 00:18:00
        channel HNE
          epoch 2023-03-01 00:18:00
        channel HNN
          epoch 2023-03-01 00:18:00
        channel HNZ
          epoch 2023-03-01 00:18:00

This behaviour seems unexpected and may be considered a bug. We understand that SC Inventory cannot represent the original StationXML information unchanged. However, if the change of representation is acceptable, location timespan should be set accordingly.

I attach the problematic files.

CPOZ_failure.tgz

[gempa-jabe]

We will check it.

[gempa-jabe]

The sensor location merges all channels with the same geo location (lat, lon, elev) in one epoch spanning all of the channel epochs. But here we have an issue. As the geo location is part of the sensor location element and not the stream and the index of the sensor location is code and start, the following set cannot be represented:

HHE 2011-07-19T10:55:00 ~ 2022-03-31T10:00:00 40.8211 14.1187 2
HHN 2011-07-19T10:55:00 ~ 2022-03-31T10:00:00 40.8211 14.1191 52
HHZ 2011-07-19T10:55:00 ~ 2022-03-31T10:00:00 40.8211 14.1191 52

HHE 2022-04-11T11:00:00 ~ 2023-03-01T00:18:00 40.8211 14.1191 52
HHN 2022-04-11T11:00:00 ~ 2023-03-01T00:18:00 40.8211 14.1191 52
HHZ 2022-04-11T11:00:00 ~ 2023-03-01T00:18:00 40.8211 14.1191 52

HHE 2023-03-01T00:18:00 ~ ................... 40.8211 14.1191 52
HHN 2023-03-01T00:18:00 ~ ................... 40.8211 14.1191 52
HHZ 2023-03-01T00:18:00 ~ ................... 40.8211 14.1191 52

HNE 2023-03-01T00:18:00 ~ ................... 40.8211 14.1191 52
HNN 2023-03-01T00:18:00 ~ ................... 40.8211 14.1191 52
HNZ 2023-03-01T00:18:00 ~ ................... 40.8211 14.1191 52

The index is ("", 2011-07-19T10:55:00) and only one sensor location can exist with that index. But one item (HHE) has a different (may erroneous) geo location and we cannot have two sensor locations with the same index. Thus a conflict.

[gempa-jabe]

Can we close the issue?

[jsaul]

Judging from the updated inventory for station CPOZ the answer is probably "yes". ;)

Nevertheless there may be better ways to handle this kind of inventory errors. We have also seen this several times, even in GSN inventories (e.g. inconsistent sensor depths), and it can be a pain to debug. The fdsnxml2inv strategy seems to be to try to repair this, but as we can see the result is not always satisfactory. Fixing the issue upstream is the right way and in order to make this more likely to actually happen, fdsnxml2inv should not attempt to repair this but either

• complain and fail loudly or
• skip the affected station

[gempa-jabe]

Feedback from the original author of the issue would be nice if the issue is solved or not.

[petrrr]

Hi @gempa-jabe and @jsaul !

I confirm that we now able to handle the station after we have corrected the metadata.
We also understand that the SC Inventory has a limited possibility to represent what would in principle be possible in FDSN StationXML (even if it might not be sensible).

However, the scope of this issue was a bit different. That is, a somewhat inconsistent behaviour to handle this case during conversion. I'll try to summerize our point:

• We believe it is okay, to accept that some StationXML cannot be correctly represented in SC Inventory and there converting exactly.
• However, there should be a clear decision (1) okay or (2) not okay for conversion; Differences in coordinates are collabsed in one set of coordinates or not;
• Then however, if a strategy is choose it should remain valid and used in all contexts;

The problem I see here is that there seems not to be one single and consistent strategy, see the details of the scinv ls output above.

• For the station the time span is epoch 2011-07-19 10:55:00 .. (open). This somewhat implies, that the last channel epochs, e.g. channel HNZ ; epoch 2023-03-01 00:18:002023-03-01 00:18:00 are included.
• On the other hand, the location epoch is epoch 2011-07-19 10:55:00 - 2022-03-31 10:00:00. This would mean at "location level" these last epochs are considered excluded from the specific location;
• But then again, the "excluded" channel epochs are explicitly listed as part part of the location;

This inconsistent translation leads to unexpected behaviour, but makes it extremly difficult to understand what exactly is going on and to debug such situation. At first it seems the epochs have been included. There is only one tiny detail, which makes the difference and access to data does not work anymore.

This issue is about avoiding this inconsistency and support clearer reporting to users.

We think we understand now the fundamental cause of why fdsnxml2inv behaves in this strange way. But we suggest that the situation is handled differently, so it supports correct feedback to the operator.

[jsaul]

@gempa-jabe
It should be enough to simply ignore the sensor coordinates when grouping channels by sensor location code and epoch start. Sensor coordinates are afterwards only checked for consistency. In case of a mismatch, an error is generated and the epoch (or station) is ignored.

@petrrr
If you mean by possibilities the ability to e.g. place the sensor N component 300 km away from the E component, I agree that not being able to do so in SC is a limitation indeed.

[petrrr]

@petrrr
If you mean by possibilities the ability to e.g. place the sensor N component 300 km away from the E component, I agree that not being able to do so in SC is a limitation indeed.

This is not really what I intended to say. However, FDSN StationXML in principle would allow to allow to

(1) provide slightly different coordinates for different streams under the same location code e.g. sensor a vs. sensor b (short period vs. broad band vs. accelerometer); different cooridinates could be also different depth. Not necessary good practice, but possibile in StationXML, and not directly translatable to SC Inventory;

(2) provide slightly different coordinates for different epochs; this case probably is more common when a sensor needed to be moved slightly. In FDSN StationXML, it would be possible to keep the location code and just specify the new coordinates; Such a StationXML file is not directly translatable to SC Inventory.

As mentioned already, I do not argue for such a practice or would encourage it. But we may see such metadata. In StationXML this can be represented but not translated into SC Inventory. Anyway, I am not too worried about this fact. We will deal with this.

This issue was only about keeping the behaviour consistent and transparent, and to avoid hidden issues with the inventory, which then are difficult to debug.