disallow some image folders

1.7k Views Asked by At

I am making my robots.txt file. But I am a little bit insecure about how to make disallow Googlebot-Image. I want to allow the Google bot to crawl my site, except for the disallow I have made below. This is what I made:

User-agent: Googlebot
Disallow:

Disallow: /courses/
Disallow: /portfolio/portfolio-template.php/

User-agent: Googlebot-Image
Disallow: 
/images/graphics/erhvervserfaring/
/images/graphics/uddannelse/

sitemap: http://www.example.com/sitemap.xml

Should the User-agent: Googlebot and User-agent: Googlebot-Image be written together, so it is this instead?:

User-agent: Googlebot-Image User-agent: Googlebot Disallow:

Disallow: /courses/
          /portfolio/portfolio-template.php/
          /images/graphics/erhvervserfaring/
          /images/graphics/uddannelse/
1

There are 1 best solutions below

3
On BEST ANSWER

No, you should write them separately along with the Disallow information. Also, you should copy the Disallow information too.

User-agent: Googlebot-Image
Disallow: /courses/
Disallow: /portfolio/portfolio-template.php/
Disallow: /images/graphics/erhvervserfaring/
Disallow: /images/graphics/uddannelse/

User-agent: Googlebot
Disallow: /courses/
Disallow: /portfolio/portfolio-template.php/
Disallow: /images/graphics/erhvervserfaring/
Disallow: /images/graphics/uddannelse/

As a reference, you can see what Facebook and what apple did in their robots.txt.