# INTERNET HISTORIAN A patient Wayback Machine archiver • MIT, on PyPI • ships with a Claude Code skill ## Why you should care The web forgets faster than people think. Pew Research Center measured it in 2024: 38% of webpages that existed in 2013 were no longer accessible ten years later (https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/), and a quarter of all pages that existed at any point between 2013 and 2023 are already gone. Even the durable-looking stuff rots: 23% of news pages contain at least one broken link. The Internet Archive's Wayback Machine can preserve almost any public page, but only if someone asks it to, at the right time, and keeps asking when the Archive is too busy to answer. That "keep patiently asking" part is tedious for a human and perfect for software. Internet Historian is that software: it does the asking, forever, in the background. ## The story It was built for one purpose: saving ちいかわ (Chiikawa) pages before they vanish. Official sites, the anime, the shops, the fan wikis. Fandom pages are exactly the kind of web the statistics above eat first: personally maintained, commercially fragile, gone the day someone stops paying for hosting. It shipped in early July 2026 and promptly outgrew its fandom. The same patience works for your press mentions, your old blog, the recipe site you cite in your dissertation. I now run a self-preservation collection with my own press links in it, on a 90-day refresh, because the archive should not find out about link rot from me at the worst possible moment. ## How it works You feed it pages three ways: paste URLs, import a browser bookmarks export, or use discover, which takes a subject ("Chiikawa") and finds its official site, Wikipedia article, and real external links with zero API keys. The demo recording in the README is a real run: discover found and queued 63 genuine Chiikawa pages. Everything lands in a local SQLite queue that a background job drains every ten minutes. Pages move from queued to submitted to archived, and a page is only marked dead after three failed confirmations spaced a day apart, because a throttled Archive is not a dead URL. If the Archive is busy, that is weather, not failure. The queue waits and tries again. For as long as it takes. ## Polite by design (the SRE part) The Internet Archive is a nonprofit running shared infrastructure, so the archiver runs on an explicit politeness budget, the same way you would budget error rates for a production service: - It always leaves two of your Save Page Now capture slots free, so a greedy queue never locks you out of archiving something by hand. - It caps itself at 5 attempts per page per day, half of what the service allows. - When throttled, it backs off exponentially, up to a full day. - It asks the server to skip pages archived within the last 30 days instead of burning a capture on a duplicate. None of these numbers are guesses; they are written into the config and verified against it in the README's diagrams. The tool optimizes for never losing a URL, not for speed. That is a deliberate trade, and it is the whole personality of the thing. ## The hard line Every tool I ship with an AI skill gets a line the AI is not allowed to cross. Internet Historian ships with a Claude Code skill, so you can say "archive my Chiikawa stuff" in plain language and walk away. But it only ever archives what you asked for: no auto-crawling, no wandering off to scrape whole sites, no deleting anything, ever. The queue is append-only history, which is fitting for an archiver. ## Where it is now v0.3.0, MIT licensed, installable with pipx install internet-historian. The test suite runs green on a 9-job CI matrix (Ubuntu, macOS, and Windows across Python 3.10 to 3.12), so "works on other platforms" is tested rather than assumed: macOS gets the fully automatic background setup, and Linux and Windows get working cron and Task Scheduler recipes out of the box. The README carries five verified explainer diagrams (what it does, setup flow, why not by hand, under the hood, polite by design) if you want the deeper mechanics. View on GitHub (https://github.com/ssskay/internet-historian) ## Sources - Pew Research Center, "When Online Content Disappears" (https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/) (May 17, 2024): the link rot and digital decay numbers above. - The internet-historian README (https://github.com/ssskay/internet-historian): politeness budget, retry policy, discover demo, CI matrix. - Internet Archive: Save Pages in the Wayback Machine (https://help.archive.org/help/save-pages-in-the-wayback-machine/): the service this politely leans on. *Canonical page: https://sarakay.me/case-studies/internet-historian.html*