When redirecting the root domain of my site to the www subdomain every URL works except /robots.txt

58 Views Asked by At

This issue only popped up a month ago when Moz reported it is no longer able to crawl our site, because the campaign is configured for our root domain, but all of our traffic gets forwarded to the 'www' subdomain.

So 'example.com/some-page.html' gets forwarded correctly to 'www.example.com/some-page.html' (as well as all other pages on our site).

'www.example.com/robots.txt' also works correctly.

'example.com/robots.txt' gets a connection timeout.

I've tested all of this thoroughly with https://httpstatus.io/

Our DNS registrar is GoDaddy, our site is hosted on Heroku, and our platform is Ruby on Rails.

0

There are 0 best solutions below