How to create a domain / subdomain specific robots.txt file

This is how to create a robots.txt file that only appears under selected domain name / subdomain names. A common case where this may be needed is if you are setting up a testing domain name whilst a site is being built and you want to avoid the scenario where you forget to alter the file and accidentally deny the website from being seen by search engines.

Firstly create a robots.txt file as normal but give it a slightly different name. In my case mine is called "robots-staging.txt":

Then you need to add a htaccess rule (I.e. you need to edit your .htaccess file) that for the domain in question re-writes the file name to appear as robots.txt so that it can be picked up by search engine spiders. In this case "testdomain.scorchsoft.com" would be the one that we would like the subdomain to appear under:

And that is it, the robots.txt will now appear for the domain "testdomain.scorchsoft.com" but will not show up for any other domain pointed to the hosting in question.