Checking old Hacker News links for dead domains
EDITOR BRIEF
A tool used Hacker News and Algolia APIs to test older story URLs without scraping the site itself. It found different dead-link rates in older high-point stories, while today's front page showed no failures in the sample. Many broken links also had archived copies near the original submission date. The author also found that 401, 403, and 429 responses should not be counted as dead links because they often mean bot blocking.
INSIGHTS
This is a good example of careful data checking: status codes can be misleading, so you need to define what “broken” means before measuring. If you build crawlers or link checkers, next try handling redirects, archive lookups, and bot-block responses separately.
Learn more with these courses
CodeFriends courses that build on this story. Practice in the browser with nothing to install.
- Automating Tasks with Python and AIStreamline Your Workflow with Python Automation.Beginner25 Hours
- Python Programming 101Learn Python in just 20 hours! Kickstart your programming journey with this beginner-friendly course.Beginner20 Hours
- Introduction to Web Development (Light)Master HTML, CSS, and JavaScript in just 10 hours.Beginner10 Hours
COMMENTS
Loading comments…