-
- 437 Posts
Hi guys, sorry to hijack but I am a little lost with the robots.txt file too. I have added one to my root with the following:
# Robots.txt
User-agent: *
Disallow: /connectors/
Disallow: /core/
Disallow: /manager/
Disallow: /logs/
Disallow: /setup/
Sitemap: http://www.pawsforwildlife.co.uk/sitemap.xml
I am not really sure what else I need if anything? I have removed the assets folder as it contains my images.
Regards,
DS
www.PawsForWildlife.co.uk
www.Borntobrick.co.uk
-
- 437 Posts
Thankfully once my sitemap was corrected and validated, my robots.txt also passed
www.PawsForWildlife.co.uk
www.Borntobrick.co.uk