Change is afoot…
Within a few moments of President Obama being sworn in, a new whitehouse.gov was revealed. The new site looks great, and promises greater transparency about the actions of our President and government.
Along those lines, check out its robots.txt file, in entirety:
User-agent: *
Disallow: /includes/
Compare this to the robot.txt from Bush’s era, and I think you’ve got a lovely little microcosm; one that points to a hopeful, and open future.
12 Comments
Show the robots.txt from Bush’s first day and you might have something. Otherwise; “baaaaaaaaaaaaaaaaa”
Give it four years. You’ll likely see /economy, /iraq and /oh-my-god-we\’re-turning-into-France in there.
The old robots.txt was simply blocking text-only versions of pages. The full versions of the pages were available, not blocked and were fully indexed by google. Run a search and you’ll see.
> /oh-my-god-we\’re-turning-into-France
…that’s a feature, not a bug.
You have too much time on your hands, man.
Disallow: /privacy.html
The Barrakessiah may not disallow webpages, but he’ll disallow your right to own so-called *assault* weapons.
That is treason.
Bif Powell +1. Also,
@Jacob Johnson: “The Barrakessiah may not disallow webpages, but he’ll disallow your right to own so-called *assault* weapons.” “That is treason.”
No. Criticizing the President’s policies is basically not even frowned upon anymore, let alone considered treasonous.
I am *deeply* unnerved by the presence of a folder called /photos/text/. Ascii art? maybe?!
Evidently, the “text” directories contained versions of various documents for printing, and the webmasters wanted those excluded to avoid them being double-counted by search engines (although Google is already pretty good at filtering such duplicates). So this isn’t really a sign of greater transparency, probably. On the other hand, it is probably a sign of superior web design.
@Jacob Johnson …. those “so-called” assault weapons. We feel for you. We really do. (What would Freud say about men with such fears. Not hard to guess.)
It seems the BBC has picked up on this and got it completely wrong: http://www.jamiedigi.com/2009/01/bbc-gets-it-completely-wrong-about-whitehousegov-robotstxt/
14 Trackbacks/Pingbacks
[...] Codeulate. I hope you like that post! The Next Web Blog covers start-up news from all over the world (not [...]
[...] siracusa: http://codeulate.com/?p=24 [3:54pm] tachikoma: Title: Obamas New Robots.txt [3:55pm] jonathan: can you explain the bush one to [...]
[...] New White House website (with improved robot exclusion list). [...]
[...] January 20, 2009 in Uncategorized Change is in the air. Including the robots.txt at whitehouse.gov: [...]
[...] the old site, the new white house site is now open to search engines. Codeulate analyzes the robots.txt file. The current robots.txt consists of two lines, allowing search engines [...]
[...] Obama’s New Robots.txt. addthis_url = ‘http%3A%2F%2Fj.snyder.name%2F2009%2F01%2F20%2Fobamas-new-robotstxt’; addthis_title = ‘Obama%26%238217%3Bs+New+Robots.txt’; addthis_pub = ”; [...]
[...] Obama’s New Robots.txt : Codeulate. (tags: politics robots.txt obama bush) [...]
[...] last a a question about Obama’s New Robots.txt and the new White House website which has a blog (and an RSS [...]
[...] Yes even Robots have become a topic of partisan pot shots! [...]
[...] While doing research on robot.txt files, which sit on a web site and tell visiting search engine spiders what pages and directories to skip when they index your pages, I found this article at codeulate.com regarding the new whitehouse.gov’s robot.txt file. [...]
[...] Obama’s New Robots.txt codeulate.com/?p=24 [...]
[...] Le robots.txt du nouveau site de la Maison Blanche (1 ligne) versus l’ancien (nnn lignes) [...]
[...] Obama’s New Robots.txt Change is afoot… [...]
[...] Obama’s New Robots.txt : Codeulate. [...]
Post a Comment