blog

Wednesday, September 5, 2007

 

Whitehouse robots.txt

A robots.txt file on a web site instructs search engines to ignore certain pages. Basically, this is a list of web pages on whitehouse.gov that they don’t want search engines to keep a record of.




Leave a Reply

guidelines