One of the things that had me scratching my head when I read Stasiland was the news that the agency tasked with re-assembling millions of documents shredded by the East German secret police had hired a bunch of people to sit there and try to do this by hand. Because piecing together shredded documents is pretty hard, the result was a projected finish date for this project of somewhere around the year 2619.
I wondered when it might dawn on the German government that perhaps the task of sorting thousands of tiny bits of paper according to complex criteria could be done more efficiently by one of those bright, shiny objects sitting around on office workers’ desks. You know, those magic, blinking boxes that somehow perform complex calculations automatically!
Dawn has finally broken, according to this fascinating piece in Wired. The geniuses of the Fraunhofer institute, who brought us the unfathomable blessing of the .mp3, are on the case — with big sacks of government money:
The data for the 400-bag pilot project is stored on 22 terabytes worth of hard drives, but the system is designed to scale. If work on all 16,000 bags is approved, there may be hundreds of scanners and processors running in parallel by 2010. (Right now they’re analyzing actual documents, but still mostly vetting and refining the system.) Then, once assembly is complete, archivists and historians will probably spend a decade sorting and organizing. "People who took the time to rip things up that small had a reason," Nickolay says. "This isn’t about revenge but about understanding our history." And not just Germany’s — Nickolay has been approached by foreign officials from Poland and Chile with an interest in reconstructing the files damaged or destroyed by their own repressive regimes.
This kind of understanding isn’t cheap. The German parliament has given Fraunhofer almost $9 million to scan the first 400 bags. If the system works, expanding up the operation to finish the job will cost an estimated $30 million. Most of the initial cost is research and development, so the full reconstruction would mainly involve more scanners and personnel to feed the paper in.
Wouldn’t it be cool if this ambitious project spins off a bunch of interesting new applications and inventions? It would be like the space program! But with a uniquely German twist: the mammoth technological challenge involves looking backwards, to thoroughly document all the horrible things Germans did to each other and to other people.*
* For the record, let it be said Germans have largely stopped doing horrible things to each other and to other people. Good for them!