New to the community? Start here

SmartCrawl Sitemap URL Crawler Not Working Properly.

I am trying to add the 48 URLs are missing from the sitemap. And when I hit add to sitemap it does nothing.

Also showing an error “Too many server errors” https://monosnap.com/file/jLl1yjjfy9yYbm2r9WAUDm4Arx5fRk

Please help.

Sun, 20 May 2018 10:28:25
More SmartCrawl Pro discussions
SmartCrawl Pro
  • JMorris
    • I do geek stuff

    Hello Christopher

    I hope you are well today.

    The problem is that there’s a large number of folders and files on your site that have incorrect file system permissions. You have many folders with 750 (rwx-rx–:wink: permissions and files with 640 (rw-r–:wink: permissions. These will not work given the nature of the folders. The last column in the permissions needs to be set to read (4) for files and read/execute (rx) folders. Otherwise, the web server will not be able to serve those files to the public. This is why SmartCrawl’s crawler keeps returning server errors.

    I attempted to connect to your account using SSH, but your host has disabled this feature. Unfortunately, due to the sheer volume of incorrect permissions, it’s not really possible to correct them all via FTP or cPanel File Manager.

    Would you please contact your host and have them run the following commands from inside your public_html/ directory, via ssh.

    Fix directories:

    find ./ -type d -exec chmod -fv 755 {} ;

    Fix directories:

    find ./ -type f -exec chmod -fv 644 {} ;

    Once your host has performed this step, SmartCrawl should be able to properly crawl your entire site… And you’ll find BuddyPress should start working correctly too! :wink:

    Please let us know if this resolves your problem.

    Best regards,

    James Morris