New to the community? Start here

[Smart Crawl Plugin] Smart Crawl plugin, I am seeing some recommendations but I don't know ho

This is the recommendation of the plugin.

Recommendation

Good news! We located the robots.txt file and search engine crawlers have access to do their job. However, we haven't been able to find any sitemap information in it.

Wed, 30 Jan 2019 10:10:04
More SmartCrawl Pro discussions
SmartCrawl Pro
  • Dimitris Kalliris
    • Support Team Lead

    Hello there William Holvoet,

    hope you’re doing well and thanks for reaching us! :slight_smile:

    SmartCrawl doesn’t create any robots.txt file in the current version (2.2.4.1), so you will have to manually create such a file, if you’re not using a custom robots.txt file already, and add the following lines in there:

    Sitemap: https://www.example.com/sitemap.xml
    
User-agent:*
    
Disallow:

    Replacing https://www.example.com/sitemap.xml with the actual link of your sitemap file of course. And create the robots file in the root folder of your WP installation.

    This is an improvement that’s already in the pipeline for a future SmartCrawl update, so you won’t have to make such a thing again.

    Warm regards,

    Dimitris