1. 5

“Institutional Memory and Reverse Smuggling” is the memoir of an anonymous petroleum engineer who was came out of retirement to help his old company relearn how one of its chemical plants worked. The company had forgotten how the plant worked because, in his words

It was designed by a group that no longer exists, in a company that has since merged, in an office that has been closed, using non-digital methods that are no longer employed.

As bad as this problem is in other areas of engineering, it’s especially acute online, where linkrot and the death of free services mean that content disappears from the internet on a mass scale every day. Ironically, this essay itself was almost a victim to that. It was originally posted to http://wrttn.in which is a (now defunct) site that allowed people to post notes and short essays. If I hadn’t copied this essay and e-mailed to myself (like its subject did with his private blueprints) it would have been lost too.

EDIT: Thanks to Obormot for giving me a proper place to host this essay for public consumption.

EDIT 2: I think this is a high value problem for rationalists to solve, given that many problems would be much easier to reason about if you knew about all the prior approaches to the problem that had been tried and discarded. If information disappears or becomes inaccessible, that becomes impossible, and you no longer find yourself able to stand on the shoulders of giants. However, I don’t see very many other people talking about the problem of information preservation, and those that are don’t really seem to be doing a good job of it.

  1.  

  2. 5

    Emphatically seconding all of this. Some comments, from the standpoint of my own areas of expertise:

    I think this is a high value problem for rationalists to solve, given that many problems would be much easier to reason about if you knew about all the prior approaches to the problem that had been tried and discarded.

    Indeed; but perhaps even more frustratingly, even more tragically, is the fact that many problems would be easier to solve if you knew about all the prior approaches to the problem that had been tried and found successful, and then forgotten about!!

    This is a pervasive problem in UX design—and very few people working in the field realize or acknowledge it. The operating systems, applications, and other various software systems of the past are dense with brilliant solutions to many UX problems, amazing designs that are perfect for many of today’s systems (and are, typically, easier to implement today than they were back then, thanks to the march of technology)—but no one knows about them, and instead, we get amateurish, clumsy designs in our current systems, that are inferior to the solutions of old. We could do better, much better, without even having to invent new solutions from scratch—if only the best of the past was known by all designers of today!

    This, among other reasons, is why I’ve said many times that my #1 piece of advice to today’s UX designers would be “stop innovating”. The fumbling attempts, of today’s designers, to solve problems that have already been solved before, are almost invariably much worse in almost every way than the best solutions of past generations—and why shouldn’t they be? We’re comparing today’s average to yesterday’s best!

    We see, again and again, UX paradigms that were perfected once, be clumsily reinvented on new platforms—in the browser, on the mobile device… why reinvent? Why not take what has already been invented and perfected, and adapt it? Why not learn from the successes of the past?


    Re: Hypothesis’s comments on indexing: indeed, and, again, emphatically seconded. On this topic too, from my own perspective:

    The annals of academic HCI research contains vast fallow fields of amazing ideas. Working designers could benefit tremendously from seeking inspiration there. And, yes, it’s all quite unimpeachably archived; there are proceedings of all the conferences, and there are journals, etc., just like in any proper academic field…

    … but how the heck do you find anything in that modern-day Library of Babel? It’s not like there’s some centralized database, it’s not like you can easily search all the proceedings of every article and every poster and every conference paper and every website of every HCI researcher… but why the heck not? It’s quite doable in principle; tagging already exists (sample paper), full-text indexing/search is a solved problem from a technical perspective, so all that remains is actually building such a system…

    But of course there are problems of copyright and open access and so forth. In practice, it ain’t happening, not legally. And that’s a shame, because a great deal of value is being left on the table.

    1. 2

      Are you familiar with the Internet Archives work on emulation of old computer artifacts? They have a version of MAME that runs right in the browser and lets you experience (within a certain approximation) very old computer media that would otherwise be inaccessible to anyone who isn’t a dedicated hobbyist.

      For example, this is now accessible to people:

      https://archive.org/details/BeyondCyberpunkMacintosh

    2. 3

      I think this is a high value problem for rationalists to solve, given that many problems would be much easier to reason about if you knew about all the prior approaches to the problem that had been tried and discarded. If information disappears or becomes inaccessible, that becomes impossible, and you no longer find yourself able to stand on the shoulders of giants. However, I don’t see very many other people talking about the problem of information preservation, and those that are don’t really seem to be doing a good job of it.

      Actually, this is one of my major interests.

      Why do you think I insisted that all the stories on Whistling Lobster be especially well indexed?

      Why do you think I was so enamored when it turned out that the Lobsters software sends all submitted stories to archive.is?

      Why do you think the feature I decided was most important for me to try and work on was search, of all things?

      I’ve participated quite a bit in stuff like Archive Team precisely because I think that we need to be preserving knowledge now before we realize we want it later.

      As for how to get organizations to do this, I don’t know. I’m currently pursuing an internship in which I try to document a lot of institutional knowledge in a white paper. I think that ‘person who preserves things’ is one of those roles that is probably unnamed in the business context.

      1. 3

        Why do you think I was so enamored when it turned out that the Lobsters software sends all submitted stories to archive.is?

        It doesn’t seem to automatically send the links though, it just provides the search bookmark. If you check on the ‘cached’ link for this story for example, there are no results on archive.is.

        EDIT: Is it even possible to automatically archive stuff there given how it’s constantly making human visitors solve captchas?

        1. 2

          Confirmed that the caching isn’t automatic, but I just saved the link for this story, and didn’t have to solve and captchas. Perhaps it is automatable, after all?

          1. 1

            This should now be fixed.

          2. 1

            Oh dear, that’s really unfortunate. I’ll have to work on that then. Thanks for pointing it out.

        Recent Comments