Integration test failures (Scientific linux 2.6.32-279.19.1.el6.x86_64)

[w-martin]

Java version 1.7.0_09-icedtea OpenJDK Runtime Environment (rhel-2.3.4.1.el6_3-x86_64)

testKernel01ToKernel26 1.3 sec Passed
testKernel01ToKernel33 0.66 sec Failed
testKernel26ToKernel01 0.24 sec Passed
testKernel26ToKernel33 0.56 sec Failed
testKernel33ToKernel01 0.21 sec Passed
testKernel33ToKernel26 0.43 sec Passed

Test set: TestSuite

Tests run: 6, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 3.726 sec <<< FAILURE!
testKernel01ToKernel33(diffr.integration.DiffPatchIntegrationTest) Time elapsed: 0.667 sec <<< FAILURE!
java.lang.AssertionError:
Expected: is
got:
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:21)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
at diffr.integration.DiffPatchIntegrationTest.testDiffrPatchr(DiffPatchIntegrationTest.java:78)
at diffr.integration.DiffPatchIntegrationTest.testKernel01ToKernel33(DiffPatchIntegrationTest.java:35)

testKernel26ToKernel33(diffr.integration.DiffPatchIntegrationTest) Time elapsed: 0.568 sec <<< FAILURE!
java.lang.AssertionError:
Expected: is
got:
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:21)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
at diffr.integration.DiffPatchIntegrationTest.testDiffrPatchr(DiffPatchIntegrationTest.java:78)
at diffr.integration.DiffPatchIntegrationTest.testKernel26ToKernel33(DiffPatchIntegrationTest.java:45)

[w-martin]

No failures on OSX 10.8.2 (Darwin 12.2.0)
Java 1.7.0_06-b24

[w-martin]

$ diff kernel33.txt diffr.new 
28369c28369
<  *  Copyright  ?  2009 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>
---
>  *  Copyright  �  2009 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>

Encoding issue?

[w-martin]

Failures also occur in Fedora 17 (3.6.2-4.fc17.x86_64) with java 1.7.0_09-icedtea

That's supposed to be the copyright symbol. Works fine under OSX because the default charset is MacRoman and so is the file encoding (probably because we created the file in osx). We need some way of detecting the encoding before reading the file. This also raises the question of how to handle a diff of two files with different encodings? Otherwise we can only reliably support input files with no funky characters.

[jkozlowski]

I don't think it's possible to reliably guess the encoding of a file... http://illegalargumentexception.blogspot.co.uk/2009/05/java-rough-guide-to-character-encoding.html

That's the problem with text files, I think it's best to save those tests as UTF-8 or 16 and that should potentially work...

Jakub

On Feb 4, 2013, at 8:26 PM, William Martin notifications@github.com wrote:

Failures also occur in Fedora 17 (3.6.2-4.fc17.x86_64) with java 1.7.0_09-icedtea

That's supposed to be the copyright symbol. Works fine under OSX because the default charset is MacRoman and so is the file encoding (probably because we created the file in osx). We need some way of detecting the encoding before reading the file. This also raises the question of how to handle a diff of two files with different encodings? Otherwise we can only reliably support input files with no funky characters.

—
Reply to this email directly or view it on GitHub.

[jkozlowski]

Another itneresting one: http://www.joelonsoftware.com/articles/Unicode.html

On Feb 4, 2013, at 8:49 PM, Jakub Kozlowski mail@jakub-kozlowski.com wrote:

I don't think it's possible to reliably guess the encoding of a file... http://illegalargumentexception.blogspot.co.uk/2009/05/java-rough-guide-to-character-encoding.html

That's the problem with text files, I think it's best to save those tests as UTF-8 or 16 and that should potentially work...

Jakub

On Feb 4, 2013, at 8:26 PM, William Martin notifications@github.com wrote:

Failures also occur in Fedora 17 (3.6.2-4.fc17.x86_64) with java 1.7.0_09-icedtea

That's supposed to be the copyright symbol. Works fine under OSX because the default charset is MacRoman and so is the file encoding (probably because we created the file in osx). We need some way of detecting the encoding before reading the file. This also raises the question of how to handle a diff of two files with different encodings? Otherwise we can only reliably support input files with no funky characters.

—
Reply to this email directly or view it on GitHub.

[w-martin]

What if we use byte arrays instead of strings - if we can still do line by line, which I think is possible because the new line and carriage return characters are the same for all 8-bit encodings?
This would be encoding-independent and work for 8-bit encodings, as gnu diff does (or certainly gnu diff works for our test files, which is why I'm somewhat against changing the test files).

Wouldn't that require excluding EOLs since they are different depending on the OS?

[jkozlowski]

I have to think about this...

From what I remember UTF is a variable length encoding and indeed the first byte is the same as ASCII for backwards compatibility.

Therefore if we read in bytes this should work, in a sense that we will ignore character encodings and just compare bytes -> comparisons should still work.

And yes EOLs will have to be handled separately. This shouldn't require much changes to the SuffixTree implementation, the only thing I use there, really, is iterating through the thing line by line. So as long as it'll be given a list of lists of bytes it should work.

I would probably opt in writing a small wrapper around byte arrays to lazily cache hashCode so be able to use it for comparisons... Or maybe there is something like that: http://docs.oracle.com/javase/6/docs/api/java/nio/ByteBuffer.html, I don't know I didn't work much with byte buffers so far...

That is my understanding.

On Feb 6, 2013, at 4:33 PM, Ben Couste notifications@github.com wrote:

Wouldn't that require excluding EOLs since they are different depending on the OS?

—
Reply to this email directly or view it on GitHub.

[w-martin]

Works fine on FreeBSD where the default encoding is ISO8859-1.
During the build there were these warnings:

[WARNING] Using platform encoding (ISO8859-1 actually) to copy filtered resources, i.e. build is platform dependent!

[jkozlowski]

I think these are standard Maven warnings if you forget to set the build encoding:

http://maven.apache.org/general.html#encoding-warning

On Feb 17, 2013, at 11:21 AM, William Martin notifications@github.com wrote:

Works fine on FreeBSD where the default encoding is ISO8859-1.
During the build there were these warnings:

[WARNING] Using platform encoding (ISO8859-1 actually) to copy filtered resources, i.e. build is platform dependent!
—
Reply to this email directly or view it on GitHub.