Multithreaded NT/NQ to Jelly conversions #240
Description
When converting NT/NQ to Jelly, the main bottleneck is parsing the input in NT/NQ. This can't really be sped up, it's as fast as it will get.
We could however, run the conversion in parallel. If we split the input into chunks of, say, 100k triples, we could send these chunks to parallel converters. These would have to be either completely independent with checkpointing... or we could pipeline their outputs through a transcoder to recompress the output.
When splitting we can seek by an arbitrary offset in the file (like a few MB) and then look for the next \n. This will require extensive buffering, so memory usage will be significantly higher.