# # robots.txt for http://www.familylawwebguide.com.au/ # # Please note: There are a lot of pages on this site, and there are # some misbehaved spiders out there that go _way_ too fast. If you're # irresponsible, your access to the site may be blocked. # User-agent: AhrefsBot Disallow: / User-agent: Yandex Disallow: / User-agent: Baiduspider Disallow: / User-agent: magpie-crawler Disallow: / User-agent: * Disallow: /guide/pg/contact_us Disallow: /backend.php?mode=ocf_ Disallow: /*find_post Disallow: /*first_unread Disallow: /*new_topic Disallow: /*new_post Sitemap: http://flwg.com.au/data_custom/sitemaps/index.xml Sitemap: http://flwg.com.au/data_custom/sitemaps/news_sitemap.xml