How to set up the robots.txt file in Joomla

The robots.txt file and SEO

As mentioned, the robots.txt file is in your site root folder. It contains info on which folders should be indexed and not. I can also include information about your XML sitemap.

There are just two tips I would recommend regarding SEO and the robots.txt file:

1. Remove exclusion of images

For reasons I don't understand, the default robots.txt file in Joomla is set up to exclude your images folder. That means your images will not be indexed by Google and included in their Image Search. And that's something you would want, as it adds another level to your sites search engine visibility.

To change this, open your robots.txt file and remove the line that says:

Disallow: /images/

By removing this line, Google will start indexing your images on the next crawl of your site.

2. Add a reference to your sitemap.xml file

I've talked about the Sitemap XML file previously, in my post on How to get your Joomla site indexed in Google. If you have a sitemap.xml file (and you should have!), it will be good to include the following line in your robots.txt file:

sitemap:http://www.domain.com/sitemap.xml

Naturally, this line needs to be adjusted to fit your domain and sitemap file. In my case, I use the Xmap component to create the Sitemap XML file automatically.

So, the line looks like this for Joomlablogger.net:

sitemap:http://www.joomlablogger.net/component/option,com_xmap/lang,en/sitemap,1/view,xml/