#robots.txt file to point web spiders in the correct direction User-agent: * Disallow:/about Disallow:/FILES Disallow:/images Disallow:/REPORT Disallow:/scripts Disallow:/UPDATE