New to the community? Start here

No link to sitemap.xml in robots.txt

Hi,

I use sitemaps in SmartCrawl. But there is no link to sitemap.xml in robots.txt

https://www.zijemehomemade.cz/robots.txt

Is it necessary to set it in some advanced settings or so? I didn’t find nothing like that (and think SmartCrawl should do it automatically…:wink:

Thanks,

Jiri

Thu, 23 Aug 2018 10:09:13
More SmartCrawl Pro discussions
SmartCrawl Pro
  • Adam
    • Support Gorilla

    Hi Jiri,

    I hope you’re well today and thank you for your question!

    That’s a known issue and currently SmartCrawl actually… does not add it. Yet :slight_smile: We already got it on a feature list so I’m pretty sure it will be added to the plugin in future.

    As of now, adding proper rule to the robots.txt manually is the way to go. However, if you are not yet using a “physical” robots.txt file, I’d recommend adding it to the native WP virtual robots.txt instead of creating a file. That can be done with a small code snippet:

    <?php
    

    
# Append directives to the virtual robots.txt
    
add_filter( 'robots_txt', 'robots_mod', 10, 2 );
    
function robots_mod( $output, $public ) {
    
$output .= "nSitemap: YOUR_SITE_SITEMAP_URL_HERE";
    
return $output;
    
}

    To apply it to your site:

    – make sure that you put a sitemap URL into the code

    – create an empty file with a .php extension (e.g. “sitemap-in-robots.php”:wink:

    – put the code into that file

    – upload the file into the “/wp-content/mu-plugins” folder on the site (if there’s no “mu-plugins” inside “wp-content” folder, just create empty one first).

    Best regards,

    Adam