From 894efa63d1ffd6bc57e74b739816f35185c0a879 Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Tue, 1 Sep 2020 13:03:15 +0100 Subject: [PATCH 1/7] Add note on f3probe assumptions to readme address issue #144 --- README.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.rst b/README.rst index e9b5e1b..16b2a22 100644 --- a/README.rst +++ b/README.rst @@ -48,6 +48,11 @@ user:: .. warning:: This will destroy any previously stored data on your disk! +f3probe assumes that higher bits of the drive block address are simply +not connected in hardware. If you come across a fake drive that behaves +differently, please open an issue reporting the details. + + Correcting capacity to actual size with f3fix --------------------------------------------- From 767e4a8f0d5a1e928eea16a38a683866b18c3a88 Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Tue, 1 Sep 2020 13:05:10 +0100 Subject: [PATCH 2/7] remove duplicated empty line --- README.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/README.rst b/README.rst index 16b2a22..739ed86 100644 --- a/README.rst +++ b/README.rst @@ -52,7 +52,6 @@ f3probe assumes that higher bits of the drive block address are simply not connected in hardware. If you come across a fake drive that behaves differently, please open an issue reporting the details. - Correcting capacity to actual size with f3fix --------------------------------------------- From 67337d7998e9501970901d8d91bc06fe7e69e1a6 Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Tue, 1 Sep 2020 13:22:31 +0100 Subject: [PATCH 3/7] Correct overflow protection `sizeof()` returns the number of bytes, not bits, and since this is as an unsigned type and the value typically less than 10 the assertion currently does not fail but also fails to detect problems when `MAX_N_BLOCK_ORDER` is too big as the comparison is with the the very big number resulting from an unsigned overflow, typically 2^32-2. --- libprobe.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libprobe.c b/libprobe.c index 082f095..a3553d1 100644 --- a/libprobe.c +++ b/libprobe.c @@ -690,7 +690,7 @@ uint64_t probe_device_max_blocks(struct device *dev) /* Make sure that there is no overflow in the formula below. * The number 10 is arbitrary here, that is, it's not tight. */ - assert(MAX_N_BLOCK_ORDER < sizeof(int) - 10); + assert(MAX_N_BLOCK_ORDER < 8*sizeof(int) - 10); return /* find_cache_size() */ From 702846c195636fd641def35cbbc9c897e9f34cdb Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Tue, 1 Sep 2020 13:03:15 +0100 Subject: [PATCH 4/7] Add note on f3probe assumptions to readme address issue #144 --- README.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.rst b/README.rst index e9b5e1b..16b2a22 100644 --- a/README.rst +++ b/README.rst @@ -48,6 +48,11 @@ user:: .. warning:: This will destroy any previously stored data on your disk! +f3probe assumes that higher bits of the drive block address are simply +not connected in hardware. If you come across a fake drive that behaves +differently, please open an issue reporting the details. + + Correcting capacity to actual size with f3fix --------------------------------------------- From 74a1a836575659e9ccc34f3c3fe6ea27ddd270dd Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Tue, 1 Sep 2020 13:05:10 +0100 Subject: [PATCH 5/7] remove duplicated empty line --- README.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/README.rst b/README.rst index 16b2a22..739ed86 100644 --- a/README.rst +++ b/README.rst @@ -52,7 +52,6 @@ f3probe assumes that higher bits of the drive block address are simply not connected in hardware. If you come across a fake drive that behaves differently, please open an issue reporting the details. - Correcting capacity to actual size with f3fix --------------------------------------------- From 67a7e0825b141834866c3d48ff5a0dc848e55c56 Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Wed, 2 Sep 2020 16:34:58 +0100 Subject: [PATCH 6/7] section techincal details --- README.rst | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/README.rst b/README.rst index 739ed86..da7a5ba 100644 --- a/README.rst +++ b/README.rst @@ -48,10 +48,6 @@ user:: .. warning:: This will destroy any previously stored data on your disk! -f3probe assumes that higher bits of the drive block address are simply -not connected in hardware. If you come across a fake drive that behaves -differently, please open an issue reporting the details. - Correcting capacity to actual size with f3fix --------------------------------------------- @@ -356,3 +352,32 @@ personalize F3 to your specific needs:: Please notice that all scripts and use examples above assume that f3write, f3read, and the scripts are in the same folder. + +Technical Details +----------------- + +f3write writes as many 1 GiB big files as necessary to fill the flash disk. +Each file is filled with blocks (typically 512 bytes) containing (a) the +overall byte offset and (b) pseudo-random data derived from the byte offset +with a 64-bit linear congruential generator. The former guarantees that +no two blocks written are the same. The latter ties the contents of each +block to the byte offset in a way that is not easily predictable by the +flash disk. f3read checks this content of each block for correctness. + +f3probe assumes that the flash disk ignores higher bits of any block +address used in block operations. In hardware, this simply means that the +corresponding address lines are not connected. f3probe determines the +highested number of address bits that can be reliably used with the flash +disk. Starting with a lower bound of bits required to address all blocks +in the first 1 MiB correctly (fake disks will have at least this much real +storage to avoid immediate error messages when accessing the flash disk in +MS Windows) and an upper bound that is sufficient to address all blocks +reported by the flash disk, f3probe narrows down the the number of address +bits by repeatedly testing a candidate number of bits and updating lower +or upper bound depending on the outcome of the candidate test. Fake flash +is indicated as soon as the upper bound is lowered. However, f3probe +continues narrowing down the number of address bits to report the useable +size of the flash disk that can, for example, be used with f3fix. + +If you come across a fake drive that does not simply discard high bits of +block addresses please open an issue reporting the details. From 84e273def1f0fd58466a2400ea8f6e36e4845d6f Mon Sep 17 00:00:00 2001 From: Joachim Wagner Date: Wed, 2 Sep 2020 17:20:02 +0100 Subject: [PATCH 7/7] document f3read output categories --- README.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.rst b/README.rst index da7a5ba..85d346a 100644 --- a/README.rst +++ b/README.rst @@ -363,6 +363,12 @@ with a 64-bit linear congruential generator. The former guarantees that no two blocks written are the same. The latter ties the contents of each block to the byte offset in a way that is not easily predictable by the flash disk. f3read checks this content of each block for correctness. +Blocks with a wrong byte offset are reported as overwritten if the +pseudo-random data is correct or only contains 1 or 2 errors. Blocks with +more than 2 errors in the pseudo-random data are reported as corrupted, +irrespective of the correctness of the byte offset. Blocks with the +correct byte offset but containing 1 or 2 errors in the pseudo-random data +are reported as changed. f3probe assumes that the flash disk ignores higher bits of any block address used in block operations. In hardware, this simply means that the