Skip to content

Conversation

@gldeng
Copy link

@gldeng gldeng commented Jan 10, 2025

Fix to the following issue:

Issue: Empty baseName causes Merger, Relayer and Firehose services to fail.
Root Cause: When walking GCP bucket directory, it returns the directory as the first object

panic: wrong filename format: ""

goroutine 101 [running]:
github.com/streamingfast/bstream.MustNewOneBlockFile(...)
        /home/runner/go/pkg/mod/github.com/streamingfast/bstream@v0.0.2-0.20240916154503-c9c5c8bbeca0/oneblockfile.go:107
github.com/streamingfast/firehose-core/merger.(*ForkAwareDStoreIO).DeleteForkedBlocksAsync.func1({0xc00114b5f0?, 0xc00114b5f0?})
        /home/runner/go/pkg/mod/github.com/streamingfast/firehose-core@v1.6.6/merger/merger_io.go:317 +0x13c
github.com/streamingfast/dstore.(*GSStore).WalkFrom(0xc000702ba0, {0x3144838, 0x4aef260}, {0x0, 0x0}, {0x0, 0x0}, 0xc0023fefa8)
        /home/runner/go/pkg/mod/github.com/streamingfast/dstore@v0.1.1-0.20241011152904-9acd6205dc14/gsstore.go:291 +0x727
github.com/streamingfast/firehose-core/merger.(*ForkAwareDStoreIO).DeleteForkedBlocksAsync(0xc00093f500, 0x1, 0x0)
        /home/runner/go/pkg/mod/github.com/streamingfast/firehose-core@v1.6.6/merger/merger_io.go:313 +0xdb
github.com/streamingfast/firehose-core/merger.(*Merger).startForkedBlocksPruner.func1()
        /home/runner/go/pkg/mod/github.com/streamingfast/firehose-core@v1.6.6/merger/merger.go:104 +0xa2
created by github.com/streamingfast/firehose-core/merger.(*Merger).startForkedBlocksPruner in goroutine 97
        /home/runner/go/pkg/mod/github.com/streamingfast/firehose-core@v1.6.6/merger/merger.go:97 +0x225

…e as the first item when listing the directory
@maoueh
Copy link
Contributor

maoueh commented Jan 10, 2025

Is this happening because you are using a GCP bucket directly without any "directory"?

@gldeng gldeng changed the title Fix: Walk shouldn't return empty baseName (i.e. object representing the directory itself0 Fix: Walk shouldn't return empty baseName (i.e. object representing the directory itself) Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants