The robots.txt file is a small file located in the root folder of your Joomla site. The file contains instructions to the search engines on what to index and what to leave out.
Some people confuse the robots.txt file with the .htaccess file. The difference is significant. The first one only gives instructions to search engines - and most search engines respect it. The robots.txt file is not used for anything related to security. Nevertheless, it can have an impact on Search Engine Optimization.
The .htaccess file, on the other hand, is used to reconfigure the settings of your Apache server, redirect URLs and other server related tasks. More about that in a later post.
SEO - or Search Engine Optimization - is a complex area of web development. I've been working with SEO for a number of years, and with Joomla and SEO for almost 4 years now.
In this post, I talk about ten important factors you should consider when optimizing your site.
For me, SEO is about what you do every day. SEO is a long term strategy - not a quick fix.
If you implement these factors, you will see increased search engine traffic to your site. All without breaking your neck performing them ;)