Use newer slices.Sort in pruneDuplicates

[Juneezee]

The new slices.Sort function introduced in Go 1.21 ¹ is faster than the older sort.Slice approach. The documentation for sort.Slice also recommends using slices.Sort in most cases ².

Benchmark:

func BenchmarkSort(b *testing.B) {
	array := []uint64{0, 42, 10, 8}

	b.Run("sort.Slice", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			sort.Slice(array, func(i, j int) bool {
				return array[i] < array[j]
			})
		}
	})

	b.Run("slices.Sort", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			slices.Sort(array)
		}
	})
}

Result:

goos: linux
goarch: amd64
pkg: github.com/FastFilter/xorfilter
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkSort/sort.Slice-16         	 4290086	       284.0 ns/op	      56 B/op	       2 allocs/op
BenchmarkSort/slices.Sort-16        	62395050	        19.53 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/FastFilter/xorfilter	3.739s

Footnotes

1. https://pkg.go.dev/slices#Sort ↩

2. https://pkg.go.dev/sort#Slice ↩

[lemire]

Wow

[Juneezee]

@lemire Happy New Year! Does this PR look good to you, or is there anything else you would like me to address?

[lemire]

It will be part of the next release.