Mimic USGS web services?

[wilsaj]

So I've recently been falling in love with the USGS's new REST interface (see: http://waterservices.usgs.gov/rest/IV-Service.html) and I think it would be worth implementing it in WOFpy.

We've had some discussions about implementing a WaterOneFlow REST interface (see issue #2) and I still think that is worth doing. But this a separate thing.

The USGS service interface is not WaterOneFlow and doesn't try to be but I think that implementing it in WOFpy would just require some new DAO methods and/or making existing methods a little bit more general. USGS also has an analogous SOAP interface that we could try to mimic as well, but I'm not sure if that would be worth the effort.

The advantages of the USGS services are:

1. A lot fewer queries for most tasks. For example: In WOF, if you want to query M variables across N sites you have to make approximately N * M queries (give or take, depending on if you want to make an additional GetSiteInfo request to check what variables are available at a given site). In the USGS service you can get away with asking approximately N questions because you can get all data across variables for a single site or even a set of sites if you want to be greedy, instead of having to make a GetValues call for each site+variable pair.
2. You can query for all data that has been modified since some date across sites/variables. This is a huge plus for scraping data - particularly when QA/QC status is subject to change.
3. The default return of any query (with no datetimes) is to give just 1 value, the most current value.
4. Querying data by county and state codes makes GIS-y workarounds largely unnecessary.
5. These things all combine naturally. So in a single query you can ask things like "what is the latest water temperature value for all sites in Texas"?
6. Some other niceties like accepting time durations (e.g. give me all data for the past three days).

The only real downside I can think of is that it is not really a standard, so we would need to keep up with any changes they make. I don't imagine that's going to be much of an issue given the nature of the service.

Anyway, something to think/talk about.

[dharhas]

+1 from me.

The USGS rest interface seems to match the way we ingest and use data much better that the WOF v1.0 does. I think it makes sense. One question is whether it is possible/worth the effort to talk to folks at the USGS and other places and come up with some sort of REST standard.

• dharhas