CrossCorrelation Client issue

[martinschorb]

Dear @trautmane ,

I am trying to make the CrossCorrelation Client available and run into an error:

$ java -cp render/render-ws-java-client/target/render-ws-java-client-4.0.1-SNAPSHOT-standalone.jar -Xms5G -Xmx5G -Djava.awt.headless=true -XX:+UseSerialGC org.janelia.render.client.CrossCorrelationPointMatchClient --baseDataUrl xxx:8080/render-ws/v1 --owner TestCollectionOwner --ccFullScaleStepSize 200 --ccSubpixelAccuracy True --ccCheckPeaks 50 --ccMinResultThreshold 0.6 --renderScale 0.5 --matchModelType TRANSLATION --collection TestCollection --ccFullScaleSampleSize 200 --pairJson .../tile_pairs_ND_20_z_0_to_1286_dist_1.json
09:37:19.064 [main] INFO  [org.janelia.render.client.ClientRunner] run: entry
09:37:19.432 [main] INFO  [org.janelia.render.client.CrossCorrelationPointMatchClient] runClient: entry, parameters={
  "matchClient" : {
    "baseDataUrl" : "render.embl.de:8080/render-ws/v1",
    "owner" : "TestCollectionOwner",
    "collection" : "TestCollection"
  },
  "featureRender" : {
    "renderScale" : 0.5,
    "renderWithFilter" : true,
    "renderWithoutMask" : true
  },
  "featureRenderClip" : { },
  "maxFeatureSourceCacheGb" : 2,
  "correlation" : {
    "fullScaleSampleSize" : 200,
    "fullScaleStepSize" : 200,
    "minResultThreshold" : 0.6,
    "checkPeaks" : 50,
    "subpixelAccuracy" : true
  },
  "matchDerivation" : {
    "matchRod" : 0.92,
    "matchModelType" : "TRANSLATION",
    "matchIterations" : 1000,
    "matchMinInlierRatio" : 0.0,
    "matchMinNumInliers" : 4,
    "matchMaxTrust" : 3.0,
    "matchFilter" : "SINGLE_SET",
    "matchFullScaleCoverageRadius" : 300.0
  },
  "pairJson" : [
    ".../tile_pairs_ND_20_z_0_to_1286_dist_1.json"
  ]
}
09:37:21.149 [main] INFO  [org.janelia.render.client.CrossCorrelationPointMatchClient] generateMatchesForPairFile: pairJsonFileName is .../tile_pairs_ND_20_z_0_to_1286_dist_1.json
09:37:21.150 [main] INFO  [org.janelia.alignment.match.RenderableCanvasIdPairs] load: entry, path=.../tile_pairs_ND_20_z_0_to_1286_dist_1.json
09:37:21.228 [main] INFO  [org.janelia.alignment.match.RenderableCanvasIdPairs] load: exit, loaded 1286 pairs
09:37:21.265 [main] INFO  [org.janelia.alignment.match.CanvasRenderParametersUrlTemplate] getTemplateForRun: returning render.embl.de:?scale=0.5&filter=true&excludeMask=true&normalizeForMatching=true
09:37:21.375 [main] ERROR [org.janelia.render.client.ClientRunner] run: caught exception
java.lang.NullPointerException: null
	at org.janelia.alignment.match.MatchFilter.<init>(MatchFilter.java:60)
	at org.janelia.alignment.match.CanvasCorrelationMatcher.<init>(CanvasCorrelationMatcher.java:47)
	at org.janelia.render.client.CrossCorrelationPointMatchClient.generateMatchesForPairs(CrossCorrelationPointMatchClient.java:169)
	at org.janelia.render.client.CrossCorrelationPointMatchClient.generateMatchesForPairFile(CrossCorrelationPointMatchClient.java:130)
	at org.janelia.render.client.CrossCorrelationPointMatchClient$1.runClient(CrossCorrelationPointMatchClient.java:95)
	at org.janelia.render.client.ClientRunner.run(ClientRunner.java:38)
	at org.janelia.render.client.CrossCorrelationPointMatchClient.main(CrossCorrelationPointMatchClient.java:101)
09:37:21.376 [main] INFO  [org.janelia.render.client.ClientRunner] run: exit, processing failed after 0 hours, 0 minutes, 2 seconds

This line does not look correct to me:

09:37:21.265 [main] INFO  [org.janelia.alignment.match.CanvasRenderParametersUrlTemplate] getTemplateForRun: returning render.embl.de:?scale=0.5&filter=true&excludeMask=true&normalizeForMatching=true

It is not listing the full base URL.
Could that be the problem?

[trautmane]

Hi @martinschorb - yes, I think you need to add the protocol (http or https) to your URL. Change --baseDataUrl xxx:8080/render-ws/v1 to --baseDataUrl http://xxx:8080/render-ws/v1.

See https://github.com/saalfeldlab/render/blob/ibeam_msem/render-ws-java-client/src/main/java/org/janelia/render/client/parameter/MatchWebServiceParameters.java#L21 .

[trautmane]

It looks like you also need to add a --matchMaxEpsilonFullScale value, maybe something like --matchMaxEpsilonFullScale 5.0. That's what is causing the NPE. The client should check for this and warn you up-front. So, keep this issue open and I'll add the check sometime soon (and close the issue then).

[martinschorb]

works now, thanks!

[martinschorb]

Hi,

Do you have an idea why I run into occasional

Exception in thread "Thread-75" java.lang.ArrayIndexOutOfBoundsException: Index 39440 out of bounds for length 39440
	at mpicbg.imglib.container.basictypecontainer.array.ByteArray.getValue(ByteArray.java:68)
	at mpicbg.imglib.type.numeric.integer.GenericByteType.getValue(GenericByteType.java:83)
	at mpicbg.imglib.type.numeric.integer.UnsignedByteType.get(UnsignedByteType.java:107)
	at mpicbg.imglib.type.numeric.integer.UnsignedByteType.getIntegerLong(UnsignedByteType.java:113)
	at mpicbg.imglib.type.numeric.integer.IntegerTypeImpl.getRealFloat(IntegerTypeImpl.java:63)
	at mpicbg.stitching.PairWiseStitchingImgLib$1.run(PairWiseStitchingImgLib.java:522)
	at java.base/java.lang.Thread.run(Thread.java:833)

These images are some almost identical test patterns. So CC values that I get are rather high. Could this be a reason?

[martinschorb]

Here they are, in case you like to check:

test_images.zip

[martinschorb]

These are the parameters:

{
  "meshCellSize": 64,
  "minMeshCellSize": 0,
  "x": 0,
  "y": 0,
  "width": 463,
  "height": 340,
  "scale": 1,
  "areaOffset": false,
  "convertToGray": false,
  "quality": 0.85,
  "numberOfThreads": 1,
  "skipInterpolation": false,
  "binaryMask": false,
  "excludeMask": false,
  "doFilter": false,
  "addWarpFieldDebugOverlay": false,
  "fillWithNoise": false,
  "tileSpecs": [
    {
      "tileId": "4.0",
      "layout": {
        "sectionId": "2",
        "temca": "TIFslice",
        "camera": "TIFslice",
        "pixelsize": 20
      },
      "z": 2,
      "minX": 0,
      "minY": 0,
      "maxX": 463,
      "maxY": 340,
      "width": 463,
      "height": 340,
      "minIntensity": 0,
      "maxIntensity": 255,
      "mipmapLevels": {
        "0": {
          "imageUrl": "file://.../tif_testdata/4.0.tif"
        }
      },
      "transforms": {
        "type": "list",
        "specList": [
          {
            "type": "leaf",
            "className": "mpicbg.trakem2.transform.AffineModel2D",
            "dataString": "1.0000000000 0.0000000000 0.0000000000 1.0000000000 0.0000000000 0.0000000000"
          }
        ]
      },
      "meshCellSize": 64
    }
  ],
  "minBoundsMeshCellSize": 64
}

[trautmane]

Hi @martinschorb - I was able to recreate the ArrayIndexOutOfBoundsException using your example. I'm not certain, but I think part of the problem is that you are using "correlation" : { "fullScaleSampleSize" : 200, "fullScaleStepSize" : 200, ... for tiny 463x340 tiles. Reducing the fullScaleSampleSize and fullScaleStepSize from 200 to 10 made the exception go away for me. This is @StephanPreibisch 's code, so he might be able offer a better explanation or even a fix or improved error check.

I want to mention that we only briefly used this code experimentally to see if it helped connect same layer tile pairs with small overlap areas containing mostly resin. It wasn't really intended for matching cross layer tiles or to be used more broadly. In fact, we ended up using a different approach for the original resin tile overlap cases. Is there some reason you are using this instead of SIFT for deriving match points?

[martinschorb]

We are having issues with some problematic data that has some charging effects and a jump between tiles.
Using the SIFT client with a rigid preset still leads to enormous scaling being applied.
My idea was to add CC point matches with highly downsampled tiles (scale 0.1) to provide the solver some additional cues to where they should end up.

For this data we have some X-Ray tomography as low-res "ground truth" for the alignment and we clearly see SIFT going wild. I tried different parameters, extending the zrange of the cross-layer tilepairs to 5 and various transformation types.

What would be your approach for fixing such dramatic distortions? Do you manually go into the tilespecs and change transformations? Or is my idea of adding more low-scale pointmatches and hoping for the solve to find proper ones the way to go?

Here's the X-Ray:

Here's what I got from Render with SIFT:

[trautmane]

Wow - that SIFT alignment did not go so well. Can you share with me a few tiles (e.g. source/unaligned tile specs and images) from a few z layers around the first big jump along with the SIFT match derivation parameters you used? I'd like to experiment with derivation and alignment to see if I run into the same trouble you did. If I do, I could also use the example to get ideas for alternative approaches from folks smarter than me at Janelia.

[martinschorb]

Hi @trautmane

I have uploaded the requested tiles and parameters here:
https://oc.embl.de/index.php/s/sUHRzdJw2bvAL4L

There was some debris/charging going on during the acquisition and it was even missing one tile image entirely.
However the tilepairs were included and it also gave some point matches of the affected tiles:

Also the point matches where there is the visible jump in the tile images (mostly 310-311) look good:

Let me know if you need more data.
Thanks for giving it a go.