New to the community? Start here

Smush CDN robots.txt

We’re using your Smush CDN for our images. Our site audit on sem rush is showing that your robots.txt is blocking our image links.

Why is that and is there a way to fix this?

Fri, 19 Jul 2019 12:31:17
More Smush Pro discussions
Smush Pro
  • Adam
    • Support Gorilla

    Hello mark

    I hope you’re well today and thank you for your question!

    We’ve asked our Smush developers for consultation on this and I’m awaiting for their response. Please keep an eye on this ticket and we’ll update you here as soon as we get to know more.

    Kind regards,

    Adam

  • Adam
    • Support Gorilla

    Hello mark

    Yes and I’m sorry for delayed response.

    I got confirmation that we’re not blocking “anything” on our end that way on purpose, no robots.txt lockout. Also, images stored in CDN are available even directly via URL (as they should be).

    However, I did test it too and it seems that some crawlers are indeed denied access – though they falsely recognize that as a standard robots.txt block while it seems to be related to some additional security tools of the underlying infrastructure. This part is still being checked by Smush developers and I’m waiting for their feedback on this as they need to actually investigate it. I’ll update you here as soon as possible, once I got some explanation myself.

    Kind regards,

    Adam