Following a few interesting conversations recently, I got interested in the idea of 'bit flip' – an occasion where a single binary bit changes state from a 0 to a 1 or from a 1 to a 0 inside a file.
I wrote a very inefficient script that sequentially flipped every bit in jpeg file, saved the new bitstream as a jpeg, attempted to render it in the [im] python library, and if successful, to calculate an RMSe error value for the new file.
I've not really had much time to take this further at the moment, but its an academic notion I'd be interested in exploring some more.
I'm not sure if a bit flip is a theoretical or 'real' threat on modern storage devices – in the millions of digital objects that have passed through my hands in the past 10 years, I've never knowingly handled a random bit flip errored file. I'd be interested in any thoughts / experiences / observations on the topic.
Please see the attached file for some pretty pictures.
Feel free to get in touch if you want any more data – images, RMSe data or scripts.
February 15, 2013 @ 1:26 pm CET
Jay,
Volker has done some nice work on this:
Heydegger, V.: Just One Bit in a Million: On the Effects of Data Corruption in Files. In: Agosti, M. et al. (Eds.): Research and Advanced Technology for Digital Libraries, ECDL 2009, LNCS 5714 (2009). (=Proceedings of the 13th European Conference ECDL, published by Springer ) .
Also, the British Library has seen bit rot in its Digital Library System, detected when recalculating hashsums. They are real.
Best wishes,
Angela
February 15, 2013 @ 10:50 am CET
I'll share this one as well, as (to me as a DP geek…) it's a rather interesting one. Judging from the way the corruption has been cropped, this seems to have occured during the post digitisation, processing stage (de-skew, cropping etc).