Nothing seems built to last anymore. Autos, home appliances, electronics, generally any mass-produced product eventually breaks down and succumbs to the clutches of planned obsolescence. Humans too, will one day be made redundant and vanish amidst a world increasingly governed by AI. Established occupations in many industries and fields have already slowly started to disappear as companies relentlessly pursue automation and cost-cutting human capital replacement initiatives. No one is immune. Not even Lewis Hamilton, who apparently has exceeded his shelf life. So it should come as no surprise that generally anything published online disappears sooner or later.
The transience of Web content is a long-established problem, says Digital Preservationist David Rosenthal who describes the Web as an evanescent medium where URLs are subject to two kinds of change:
- Content drift, when a URL resolves to different content than it did previously.
- Link rot, when a URL no longer resolves.
Link rot and content drift are perhaps nothing more than a minor inconvenience if you happen to be looking for something specific published online in the distant past. Good luck with that. The quality of Google search results have been steadily declining for years now. Coupled with the rising specter of AI-generated content and ever-evolving algorithms that prioritize monetization over relevance, the Web has become a hostile place for journalists, writers and researchers.
“The ephemerality of the web isn’t just a problem for journalists. Any area of work that is reliant on the written record – a growing share of which is constituted by web content – is vulnerable to linkrot and content drift.”
By some estimates, the indexed Web contains ~4.25 billion pages. Most people probably couldn’t care less about a missing page or broken link here and there. The disappearance of information online is insidious. It can occur rapidly over news and social media platforms where a culture of immediacy pushes content into algorithmically-curated clickbait that exists in a constant state of flux. Or it can be much slower, occurring less conspicuously years after a piece has been published, through various means including:
- the technical failure of Web server software or hardware
- domain and/or hosting plan expiration
- content migration or deletion by site administrators and creators during updates and redesigns
In 2013 Felix Salmon wrote a great piece on the spread of link rot for Reuters, ironically no longer at its original location: http://blogs.reuters.com/felix-salmon/2013/06/28/the-spread-of-link-rot/, moved to a completely different permalink: https://www.reuters.com/article/world/the-spread-of-link-rot-idUS2435401678/
“.. the institution of the permalink is dying away as we move away from the open web; if you’re not even on the web (if, for instance, your content comes in the form of a show on Netflix), then the very concept makes no sense. What’s more, we’ve moved into a world of streams, where flow is more important than stock, and where the half-life of any given piece of content has never been shorter; that’s not a world which particularly values preserving that content for perpetuity.
And of course it has never been easier to simply delete vast amounts of content at a stroke.”
Yes indeed. The Web is full of severed hyperlinks and dead ends perpetrated by the simple act of deletion.
Not long after Elon Musk took control of Twitter, beginning its slow descent into the polarizing cesspool of crap it’s now become, I decided to delete my account effectively erasing all my Tweets going back ~10 years. So if anyone out there retweeted one of my posts or shared any links back-and-forth with followers, I’m afraid it’s all completely gone for good. Sorry, I too am guilty of contributing to this problem.
Who cares, I can hear you say. I love my algorithmic echo chambers, thank you Elon.
Social media has become primarily a conduit for advertising and disposable infotainment, 90% of which will be created by machines in 15 years. So it’s worth asking, what’s really worth saving? For many people the Web seems to work just fine as-is with its machine-learning bots creating new and titillating material 24/7.
These may be trivial matters, sure, but consider the impact of linkrot on legal research and case citations. It’s been reported 49% of all links cited in U.S. Supreme Court decisions were broken as of 2013. I imagine that number has grown much higher over the past 10 years but I wasn’t able to find any recent or verifiable studies on this particular subject.
And, unfortunately, that’s exactly where we’re at —a ‘World Wide Web’ full of fractured nodes of communication in decline. Ephemerality beats permanence online. Happy surfing.
If you’re interested in reading further on this subject, a 2024 report published by the Pew Research Centre, When Online Content Disappears (HTML, PDF), is chock-full of fascinating statistics on the prevalence of link rot and various other forms of digital decay.