New to the community? Start here

MultiDB for performance gain for less than 100 site's network

I am not expecting to go beyond 100 sites in the network but 3 of these sites currently runs on multiple servers (on WISA stack) with following configurations physical loadbalancer -> 2 Application Servers -> database server.

We are transitioning every site over wordpress to consolidate everything and have a common user-base.

In the documentation it is listed multiple times that multi-db woudl not help much for less than 5k blogs/sites but does not say a word about the situations where you would be running multiple huge traffic sites from it assuming to be able to put every high traffic site on a separate physical database server using multiDB plugin would help.

Please let me know if I should be using this plugin or start looking for some alternative like hyperDB.

Thanks!

Wed, 4 Dec 2013 11:05:03
  • Alexander
    • DEV MAN’s Mascot

    HI @chodhry,

    The goal of Multi-DB is to avoid needing to query multiple blogs at the exact same time. This doesn’t become a performance issue until you get into the thousands of blogs – you will eventually run into the issue that the databases server slows down because WordPress requests need to take turns working with the same database.

    When talking about high traffic on a smaller amount of sites, I’d recommend looking into a caching system first. This way you could eliminate the database being called on each page request. The database cold just be called when the cacheing expires or content is updated.

    W3TC for example can do in memory cacheing – so a copy of the most highly requested data would simply be stored in RAM so the database doesn’t need to be called every page request.

    Honestly I wouldn’t jump at using multiple databases just yet. It’s possible (and simpler) to manage a few high traffic sites from a single db.

    Best regards,

  • chodhry
    • Site Builder, Child of Zeus

    Alex thanks for your reply.

    Just wanted to clarify that by high traffic I mean 30+ Million UVs for over 100 Million PVs on monthly basis. Currently there are over 1000 content items being added every day with 15K approximate visitors at any given time. By Dec 2014 we are expecting to have grown to 50+ Million UVs for over 300 Million page views with 2000+ daily post submissions and over 5000 comments and 25K+ concurrent visitors (excluding bots).

    Data posted from google analytics/chartbeat.

    At the moment 3 of these sites despite having strong caching system runs on a cluster of 3 servers each. So there are 9 high end servers for just 3 sites and some times when there is a breaking news we end up having errors due to servers getting choked.

    Apart from that, the data in all three sites could be kept separate which seems more secure.

    So do you still think that a single database will be able to handle it all?

  • Alexander
    • DEV MAN’s Mascot

    Hi @chodhry,

    You would certainly get some performance increase out of Multi-DB, but I’m still not sure it’s the right tool for the job.

    Multi-DB is meant for the automatic dispersal of multiple sites across multiple databases. However by using the VIP databases, you could simply have it move each site to it’s own database.

    This would introduce a performance increase – but if you’re just creating a new database for each site it wouldn’t be noticeable.

    You’d really need to setup an entirely new database server (more VPS instances) to host each of the VIP databases. Otherwise you’d still have the same bottleneck as it would be one server handling everything.

    Now, I’m not a system administrator, and I know very little about Windows based deployments. If you’re working with those kinds of numbers, you might want to consider some consulting for your architecture.

  • Alexander
    • DEV MAN’s Mascot

    @chodhry,

    If you’re positive that the current bottleneck is database access, yes this will result in greater performance.

    You might want 4 database servers. 1 for each of the main sites, and 1 for the rest of the network.

    Multi-DB will need to create at least 16 DBs to work with, so on the fourth server, you can create the rest of those. Then the other 3 can be VIP databases.

    Best regards,