Skip to content

intermittent fails/successes when requesting a subset of data using shared dimensions #308

@Mikejmnez

Description

@Mikejmnez

The Context

I am attempting to download a subset of a very simple test dataset, with a tree repr as follows:

.SimpleGroup.nc4.h5
├──SimpleGroup
│  ├──Y
│  ├──X
│  ├──Temperature
│  └──Salinity
├──time
├──Z
├──Pressure
└──time_bnds

In this scenario, I want a subset of Salinity, which has dimensions time, Y and X, each a 1D array. NOTE: time has a single value and lies at root, whereas the rest of the dimensions and Salinity lie within /SimpleGroup.

To request the subset I construct the url using constraint dimensions. For example (needs escaping, but this is more readable):

http://test.opendap.org/opendap/dap4/SimpleGroup.nc4.h5.dap?dap4.ce=/time=[0:1:0];/SimpleGroup/Y=[0:1:9];/SimpleGroup/X=[10:1:19];/SimpleGroup/Salinity

the problem

I get intermittent 400 or 200 responses when requesting a dap response using the above url. I tried with pydap ( via requests.Session), firefox/Chrome web api, and curl. When it works (200 response) I get the right thing, but when I get a 400 error ,I get the following response:

<?xml version="1.0" encoding="UTF-8"?>
<dap4:Error xmlns:dap4="http://xml.opendap.org/ns/DAP/4.0#" httpcode="400">
 <dap4:Message>libdap error transmitting DAP4Data: Invalid constraint parameters: At least one of the start, stride or stop
 specified do not match the array variable.</dap4:Message>
  <dap4:Context />
  <dap4:OtherInformation />
</dap4:Error>

Nathan shared the following BES command is:

<?xml version="1.0" encoding="UTF-8"?>
<bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="ajp-nio-0.0.0.0-8009-exec-3_32" reqUUID="7611633c-c442-4d16-8ddf-1c60f0d985ff" clientId="/-0" clientCmdCount="16">
  <bes:setContext name="bes_timeout">300</bes:setContext>
  <bes:setContext name="dap_explicit_containers">no</bes:setContext>
  <bes:setContext name="errors">xml</bes:setContext>
  <bes:setContext name="olfsLog">206.192.248.66|&amp;|curl#8#7#1|&amp;|-|&amp;|-|&amp;|1757632470915|&amp;|ajp-nio-0.0.0.0-8009-exec-3_32-7611633c-c442-4d16-8ddf-1c60f0d985ff|&amp;|HTTP-POST|&amp;|/opendap/hyrax/dap4/SimpleGroup.nc4.h5.dap|&amp;|-</bes:setContext>
  <bes:setContext name="xml:base">http://test.opendap.org/opendap/dap4/SimpleGroup.nc4.h5</bes:setContext>
  <bes:setContext name="max_response_size">0</bes:setContext>
  <bes:setContext name="max_variable_size">0</bes:setContext>
  <bes:setContext name="uid">not_logged_in</bes:setContext>
  <bes:setContext name="dap4_checksums">false</bes:setContext>
  <bes:setContainer name="catalogContainer" space="catalog">/dap4/SimpleGroup.nc4.h5</bes:setContainer>
  <bes:define name="d1" space="default">
    <bes:container name="catalogContainer">
      <bes:dap4constraint>/time=[0:1:0];/SimpleGroup/Y=[0:1:9];/SimpleGroup/X=[10:1:19];/SimpleGroup/Salinity</bes:dap4constraint>
    </bes:container>
  </bes:define>
  <bes:get type="dap" definition="d1">
    <bes:contentStartId>22a85fea:1993b0f148a:-7ffd@opendap.org</bes:contentStartId>
    <bes:mimeBoundary>Part_32326138356665613a31393933623066313438613a2d37666665</bes:mimeBoundary>
  </bes:get>
</bes:request>

This may be an edge case. When I consider an example in which all dimensions exist in the same Group in which the data array I am attempting to subset lives, it works every time. But in this scenario where time is at root and the rest of the data is within SimpleGroup, I get this intermittent behavior...

Metadata

Metadata

Assignees

No one assigned

    Labels

    dap4Work related to DAP4 support

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions