Breaking equivalent items

[nicolasaunai]

One of the conditions for "items" to be considered "independent" is that they different VariableFillPattern types.

https://github.com/LLNL/SAMRAI/blob/develop/source/SAMRAI/xfer/RefineClasses.cpp#L392

 equivalent &= (!data1.d_var_fill_pattern ==
                  !data2.d_var_fill_pattern);
   if (equivalent && data1.d_var_fill_pattern) {
      auto& d1 = *(data1.d_var_fill_pattern);
      auto& d2 = *(data2.d_var_fill_pattern);
      equivalent &= (typeid(d1) == typeid(d2));
   }

Multiple source/dest/scratch sets can be registered to a single RefineAlgorithm and be non equivalent in that regard.
Refine items will be stored in the RefineClasses as non equivalent.

However, when creating the internal patch level during schedule execution, the VariableFillPattern that may have been provided by the user is replaced by a BoxGeometryVariableFillPattern to fill the entire temporary patch data.

here : https://github.com/LLNL/SAMRAI/blob/develop/source/SAMRAI/xfer/RefineSchedule.cpp#L996

  for (int nd = 0; nd < num_refine_items; ++nd) {
         RefineClasses::Data item = d_refine_classes->getRefineItem(nd);
         item.d_var_fill_pattern = bg_fill_pattern;
         coarse_schedule_refine_classes->insertEquivalenceClassItem(item);
      }

While I understand that in that internal context the entire PatchData needs to be filled, replacing the user VariableFillPattern has the side effect to now treat all registered variables as equivalent.

Contrary to SAMRAI that encodes the geometry with the type of PatchData, we have a single PatchData override for all geometries that are rather handled in a third party object. As a result, we have a single BoxGeometry, and a single kind of Overlap, Variable etc. all delegating geometrical details to that other object.

The internal replacement of the user VariableFillPattern by the BoxGeometryVariableFillPattern therefore results in the overlap being calculated by the geometry of the first registered item, which in our case is wrong for all others...

I think we somehow abuse the VariableFillPattern trick but that is the only way we found to register several items to the algorithm and not having only the geometry of the first used.

[nicolasaunai]

This is related to #170 (comment) and #106

[nselliott]

This method you describe goes outside of expected or recommended usage of BoxGeometry within RefineSchedule. There is an assumption that each PatchDataFactory will return a BoxGeometry object that is implemented to compute overlaps that match the mesh centering of the specific PatchData.

One possible way to work around this would be to add something to the RefineClasses::Data struct to give you a way to force an item to always be non-equivalent to all other items.

[nicolasaunai]

We could probably do that sure, but we do not want to maintain our own fork, would you consider PRs of this kind?

[nselliott]

Yes, we would consider that.

A different idea, which I don't know if it would work in your code, would be to make your third party object create an overlap that's a superset of all of your possible overlaps when it is given BoxGeometryVariableFillPattern. If your PatchData copy and packStream/unpackStream implementations could determine what to do with such an overlap for each different instance of your PatchData, then your specialized VariableFillPattern will still be applied on your fine level destination data when the refinement operator is called.

[nicolasaunai]

ok thanks. In the mean time we have been developing another patchdata subclass encapsulating N different geometries, with N internal overlaps etc. and it seems to do the job.

Yet, I think it could be nice to have a virtual function defined in the PatchDatafactory, say, bool PatchDataFactory::isNonEquivalentItem() const or something, that returns false on default and be added to the checks in RefineClasses::itemsAreEquivalent() ,

equivalent &= ( ! pdf1->isNonEquivalentItem()  and  ! pdf2->isNonEquivalentItem() );

and can be overriden to true by user PatchDataFactory to force non-equivalence.