User-agent: ia_archiver Disallow: / To exclude the Internet Archive's crawler (and remove documents from the Wayback Machine) while allowing all other robots to crawl your site, your robots.txt file should say: http://www.archive.org/about/exclude.php