New to the community? Start here

[The Hub] Throttling requests to a single hosting server

0

I have had an issue every time I try to use the hub to update a plugin across all my sites, etc. Because all my sites are hosted on 1 of 2 servers, it ends up sending my load rate into the clouds and locking things up with some failures as a result as well. It would be great if the hub could detect that the connections were to a single server and stretch out the requests.

Wed, 29 Dec 2021 16:28:48
More The Hub discussions
The Hub
  • Pawel Pela
    • Ex Staff

    Hello Nathan Lyle !

    I trust you’re having a great week!

    We do have some data that would allow this (IP of the sites), though it wouldn’t always be possible due to proxy servers such as CloudFlare being in the middle and effectively cloaking the original IP address.

    Not really sure if that would be possible anyway, but I’ll share a task with the Hub team to let them know of your request and hopefully they will be able to figure something out in the future.

    Best regards,
    Pawel

  • Tony G
    • Mr. LetsFixTheWorld

    Today seems to be my day for commenting on older posts…

    Pawel said:

    > Not really sure if that would be possible anyway,

    Not possible? I’m stunned whenever I see those words. We can do anything with some thought, dedication, skill, time, and funding. Of course this is possible. The internet couldn’t function as it does today if everything were as synchronous as things we see here. It’s actually tough to count the number of ways that we can operate asynchronously right now – there are too many options and it changes rapidly. It’s difficult now to even find “highly synchronous” code, given AJAX, async, promises, callbacks, webhooks, etc …

    I made a similar request a couple years ago. Every day I get about eight Down/Up notices from DEV on the same sites. It seems at some point all of them get hit at about the same time from the Uptime server, my server blocks the flood, so DEV responds by sending me a flood of emails to tell me my sites are down.

    The solution to these challenges is dirt simple: Accept a request to do an update or uptime check. Do not do the network processing immediately, queue it.

    1) Store all requests rather than processing them synchronously or in parallel.
    2) A secondary process (multiple crons or services that look for such requests) checks the queue of requests and pulls them out one at a time for processing.
    3) These processes check a table for recent accesses to any given IP, and if a site has been accessed within the last 30 seconds just move on to the next request.
    4) Add more logic in here that is intended to throttle too many connections to a domain, to a subdomain, to an email address, to an email domain, etc.

    Again, put everything into a queue and separate the process of accepting/generating requests from the process of handling the requests. This dove-tails ideally with another note I wrote here today about separating notification content from notification media (like up/down notices and HTML emails).

    If you then get complaints about sites not updating fast enough, phase 2 of this enhancement would be to add Hub controls at the site level so that admins can determine when to throttle connections. All this does is make that 30 second number a variable rather than a constant. 30 seconds is just an example number, and for some purposes it’s unreasonable. For some processes a more reasonable number could be 3 seconds, just don’t do operations in parallel. Since we know this will be an issue, a UI to control values for various notifications and connections should be implemented in phase 1, but I don’t want to seem too greedy…

    Even if the processing is throttled as described, it’s possible that on queuing to another server (CDN, etc) that requests will get further delayed, or even cached and processed in-batch – deducing the efficacy of the throttling. Deal with that issue when it’s evident. Don’t avoid async processing because some other process down the road might mess it up for you.

    OK, that’s enough outta me. TY

    • Pawel Pela
      • Ex Staff

      Hello Tony G !

      Agreed on most of the points, it looks like you have a good knowledge of software topics! This is actually a good concept for a new system, but let me explain what I meant earlier in relation to an existing system.

      Just to clarify, by “not possible” I didn’t mean it’s not possible to implement due to lack of technology – I was rather commenting on the scale and complexity of this kind of a system. There’s a massive difference between setting up such system for a handful of sites, but once you have multiple millions of those, the complexity increases massively. As always, it’s WordPress, so a lot of additional variables need to be taken into account as well.

      Systems that are in use and on which millions of people rely daily need to be changed and updated extremely carefully, so it’s not just a matter of pulling it apart and replacing with something else.

      Kind regards,
      Pawel

      • Tony G
        • Mr. LetsFixTheWorld

        Hey bud! In all exchanges here, just remember, that we’re almost always in sync – even when we’re almost not. LOL

        What gets me uptight about this is that everyone knew when the Hub was being designed that there would be multiple requests generated simultaneously for many different functions. Who (rhetorically) are the amateurs who designed and coded processes that would essentially DDoS client systems and email boxes? Yeah, this is never going to be re-written, “it’s impossible”, but this situation was easily avoidable from the first box drawn on a whiteboard.

        And no, the process I described above has nothing to do with WordPress. A form is submitted on the Hub. The data is parsed and passed from WordPress to other handlers that I hope aren’t even on the same system. It’s pure data, no WordPress involved. The handler that receives these requests is the process that needs to be refactored to dequeue and re-enqueue transactions that will be processed (send an email, update a plugin, check uptime…). If that code IS in WordPress, then I need to reconsider how smart I truly believe the folks there are.

        It’s possible that we were talking about different components. So maybe with this clarification someone may go “hey, he’s right, this wouldn’t be such a big deal after all” … or maybe someone will go “he’s actually right that we botched that part of the implementation, it might be best not to continue that discussion”. I’d be happy either way. :)

        And yes, I’ve been doing this for over 40 years. Our clients pay a lot of money for my assistance to help design and develop systems like this – and we’ve never had to re-engineer anything significant.

        Thanks as always for the exchanges.