fdsnws: extended inventory network description causing no-show

[filefolder]

Noticing that FDSNWS won't return any station data on a particular network if the inv[0].description field is:

The Vulcan array consists of a short term (~13 day) nodal array targeting the Vulcan Iron-Oxide Copper-Gold (IOCG) prospect. The Vulcan nodal array is part of the South Australian Governments Department of Energy and Mining (DEM) Accelerated Discovery Initiative (ADI) project 'Integrated Exploration Under Deep Cover: Geophysics, Geochronology, Geochemistry' research collaboration between the University of Adelaide and Fortescue Metals Group (FMG) Resources Pty Ltd. 100 SmartSolo IGU-16HR 3C short period AuScope nodes were deployed in February 2022. All data recorded at 250 Hz, with a preamplifier gain of 24 dB. The aim of the project is to image the local seismic velocity structures in the top 4 km.

however this works:

The Vulcan array consists of a short term (~13 day) nodal array targeting the 'Vulcan Iron-Oxide Copper-Gold' (IOCG) prospect

I assume there is a character limit? Or something else I am missing? Seems to import and behave normally otherwise. This one is a bit longer than average so don't recall running into this problem before.

v 6.4.4

[gempa-jabe]

You probably mean inv.network[0].description. The default database schema character limit is 255 characters. I don't know how this could break fdsnws but we will look into it.

[filefolder]

Thanks.. it's a network object so whichever makes sense.

I should also mention that this is set while the file is a fdsnXML file, and then imported to SC. An abbreviated version is attached if it helps.

2X.test.fdsnxml.txt

[gempa-stephan]

I converted the FDSNXML file to SCXML and run fdsnws using the inventory file

fdsnxml2inv -f 2X.test.fdsnxml.txt > 2X.test.xml
fdsnws --inventory-db 2X.test.fdsnxml.xml

I was able to retrieve inventory information as text, SCXML and FDSNStationXML through the URL http://localhost:8080/fdsnws/station/1/query?nodata=404.

Next I created a test database.

$ mysql -u root -p
MariaDB [(none)]> CREATE DATABASE seiscomp_inv_test CHARACTER SET utf8mb4;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON seiscomp_inv_test.* TO sysop@localhost;
exit
$ mysql -u sysop -p seiscomp_inv_test
source ~/seiscomp/share/db/mysql.sql
exit

and then tried to store the inventory into it which resulted in an error:

scdb -i 2X.test.xml -d localhost/seiscomp_inv_test --debug
...
09:35:29 [debug] Found database version v0.13.0
Parsing file '2X.test.xml'...
Time needed to parse XML: 00:00:00.000362
Document object type: Inventory
Total number of objects: 24
Writing Inventory into database
..........09:35:29 [error] execute("insert into Network(_oid,_parent_oid,archive,code,description,end,end_ms,institutions,netClass,region,remark_used,restricted,shared,start,start_ms,type) values (19,4,'','2X','The Vulcan array consists of a short term (~13 day) nodal array targeting the Vulcan Iron-Oxide Copper-Gold (IOCG) prospect. The Vulcan nodal array is part of the South Australian Governments Department of Energy and Mining (DEM) Accelerated Discovery Initiative (ADI) project \'Integrated Exploration Under Deep Cover: Geophysics, Geochronology, Geochemistry\' research collaboration between the University of Adelaide and Fortescue Metals Group (FMG) Resources Pty Ltd. 100 SmartSolo IGU-16HR 3C short period AuScope nodes were deployed in February 2022. All data recorded at 250 Hz, with a preamplifier gain of 24 dB. The aim of the project is to image the local seismic velocity structures in the top 4 km.','2023-11-30 00:00:00',0,'','','','0','0','1','2022-02-09 20:43:55',0,'')") = 1406 (Data too long for column 'description' at row 1)
09:35:29 [error] writing object with type 'Network' failed
.
While writing 11 objects 1 errors occured
Time needed to write 11 objects: 00:00:00.003285

As @gempa-jabe indicated the description is to long and can't be stored into the database. As a result the entire network and it's stations are not written to the DB, see

mysql -u sysop -p seiscomp_inv_test
MariaDB [seiscomp]> SELECT COUNT(*) FROM Network where code="2X";
+----------+
| COUNT(*) |
+----------+
|        0 |
+----------+

[gempa-jabe]

@gempa-stephan, according to your findings I conclude that scinv does not raise an error in that case and therefore the error is hidden to the user. We need to verify that as well and improve scinv in that regard.

[gempa-stephan]

I created a test queue in scmaster.cfg and tested scinv:

scinv sync 2X.test.xml --no-keys --debug -H localhost/inv_test

Indeed no error is raised. The 2X network is silently dropped. Next I shortened the inventory description and successfully synchronized the network:

scinv sync 2X.test.short.xml --no-keys --debug -H localhost/inv_test

The output of the first and second run only differs in some debug messages.