Feature request - enable FLAC 1.5's new chained bitstream feature

[jprjr]

Feature request

Hi there - this is a request (and accompanying PR #2216) to use the new chained ogg decoding in FLAC 1.5+.

I see there's a few issues on the subject, some with misleading/wrong information, outdated test streams, the issue is related to some specific product, etc - so, I'm hoping to have a fresh discussion in this issue. If you'd prefer I open or re-use an issue, please let me know and I'll move conversation there.

The previous related issues: #1771 and #2122

To quickly recap: the core issue is with Icecast radio streams, using FLAC in Ogg containers with chained bitstreams. In FLAC before 1.5 - whenever the FLAC decoder would hit the end of the Ogg bitstream, it would continue to consume bytes of data but not produce any further audio output, signal there's new metadata, and so on. It would just read data until the actual input stream ends.

The FLAC project refers to each of these Ogg bitstreams as "links", so as you decode an Icecast stream, you start decoding the first "link," then when the Ogg bitstream closes and a new one opens, you finish the "link" and start decoding a new "link".

FLAC 1.5 adds the ability to continue decoding through multiple links, at a minimum it requires 2 changes to applications:

1. Applications need to call FLAC__stream_decoder_set_decode_chained_stream() before initializing the decoder.
2. Applications need to handle the new FLAC__STREAM_DECODER_READ_STATUS_END_OF_LINK in the read callback by calling FLAC__stream_decoder_finish_link()

On FLAC 1.5 - if you don't make these changes, then when the link ends, the decoding ends as well (so at least it doesn't just hang forever like in 1.4). Issue #2122 indicated something was fixed by using the new library, I'm unsure if they considered "fixed" to be "the decoding no longer hangs forever" or if "fixed" means "decoding continues on link changes."

#2216 at the moment, represents the bare minimum needed and will work with well-formed streams with chained Ogg. One example stream with chained FLAC-in-Ogg is Radio Calico at https://www.radio-calico.com/ - with a stream URL at http://radio3.radio-calico.com:8080/calico

However there's a few potential situations I'm not sure how to handle, or may need some refactoring elsewhere:

Poorly-formed streams

I'm going to define a "well-formed" stream as one that doesn't change sample rate, channel count or bit depth between links. I'm sure no Icecast stream is going to intentionally create streams that change any of these parameters, and maybe the "fix" is to just inform the stream author that they shouldn't change these parameters.

One option for handling poorly-formed streams may be changing FlacCommon's OnStreamInfo and OnFirstFrame methods such that - if the initialized property is already true, compare the incoming sample rate, bit depth, channels to the configured ones and (somehow) create an error if there's a mismatch.

Since libFLAC's FLAC__StreamDecoderMetadataCallback doesn't return any value that MPD could use to put the decoder into an error state - I'm guessing the error state would need to be saved somewhere in MPD and surfaced in the decoder loop?

Or again - this is likely such a rare occurrence, maybe it's not worth spending a lot of time trying to detect.

Chained Ogg Files

This is also probably going to be pretty rare. If I createda chained ogg flac file (just concatenate some ogg files together) and load that into MPD, it loads information like the length, metadata, etc from the first link in the file.

Playback works - once the end of the first link is reached, MPD will continue playing. Information such a the duration will be incorrect since that doesn't update for the next link. Also interestingly, the metadata does not update, but I'm not sure if that's an issue file metadata being handled differently from stream metadata.

Again - I don't think there's a lot of people out there creating files like this so, this probably isn't worth spending a lot of time on solving.

[jprjr]

A few other things to think about:

In my write-up on how to handle "poorly-formed streams" - I just wrote everything under the assumption that a change in bit-depth etc is an error state, but - maybe the right answer there is for MPD to allow re-performing initialization mid-stream. Do the other Ogg-based decoders (Vorbis, Opus) allow this? I'm unsure.

Additionally, chained Ogg files could be considered a container type. There's a new FLAC API function - FLAC__stream_decoder_find_total_samples - that will process all the links in the file, and afterwards you can use FLAC__stream_decoder_get_link_lengths to get both the total number of links, as well as the lengths of each. Or alternatively, when scanning the file use FLAC__stream_decoder_skip_single_link to iterate links (get metadata etc) and check FLAC__stream_decoder_get_state to detect the end.

That would be a pretty big change though and introduce questions like, would single-bitstream Ogg FLAC files still be treated as just a single file or as a container with a single entry? What about chained Ogg Opus and Ogg Vorbis files? Is that a feature worth having? It seems like it could be a nice alternative to files + cue sheets.

[homayunmhmadi]

Thanks very good