# /robots.txt file for www.rbg.vic.gov.au # google seems to be able to handle the fungi catalogue, so we give them # their own entry User-agent: googlebot Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /avh/tmp/ # Other bots get lost in the catalogue, so we disallow that bit User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /avh/tmp/ Disallow: /fungi/cat/ Disallow: /dbpages/ Disallow: /plant_science/fungi/cat/ Disallow: /plant_science/online_plant_information/viclist/ Disallow: /biodiversity/fungi/cat/ Disallow: /demosite/ Disallow: /updatedesign/ Disallow: /old/