New to the community? Start here

[Defender Pro] Aggregate data from other security auditing plugins

0

Looking at the WP Security Audit Log plugin I’m envious of its audit logging of changes in yet other plugins.

We see requests in these forums from time to time for WPMU DEV plugins to support features which are present in other plugins. I really don’t like that approach. I believe it’s better for Dev to dove-tail with other plugins rather than trying to duplicate functionality.

For this request, rather than asking for a duplication of the same logging found in that other security plugin, or others, I’m asking for an enhancement to Defender which retrieves audit data from other plugins, and integrates that data into the Defender Audit Log. With this approach, Dev doesn’t need to integrate with Yoast SEO, WooCommerce, or other plugins to capture their events, we just need one code module to retrieve data from one plugin. Of course a site that wants that logging data needs to install that separate security plugin. But once this task is complete we will continue to get all of the benefits of enhancements to that plugin and Dev doesn’t need to do anything!

Following on with that concept – I believe such an integration should be implemented as an add-on, like Forminator supports a number of add-ons for other plugins. So we can have the first one for WP Security Audit Log, another for IP Geo Block logs, another for Ban Hammer, etc.

The result is that Defender will provide a single dashboard for a wide variety of audit logging, far beyond anything that Dev has the desire or ability to build-in. This is something that I believe is unique in the industry, and that should draw more people to use this plugin.

Thanks.

Thu, 29 Aug 2019 16:37:21
More Defender Pro discussions
Defender Pro
    • Tony G
      • Mr. LetsFixTheWorld

      I try really hard to reduce staff workload when I see anything that looks like it will increase the load. This is entirely self-serving. I do this because I want more out of my vendor WPMU DEV, more code, more features, more documentation. I want DEV to process more tickets in the Huge volume of tickets that have already been approved. One way I can get what I want is to make it easier for a lot of other people to get what they want. This request can save processing of a lot of other requests for specific kinds of audit logging in Defender. It may also help to get new users for Defender, new subscribers for WPMU DEV, and thus more revenue to continue paying staff to do what I want. So while this seems like a lot of work, I see it more like a small investment toward reducing other expenses.

      To reduce the load even further, if Defender gets an enhancement with an API to support injection of audit data, then someone can create a plugin that interfaces other plugin audit data. Hey, I have no idea – this might already be in there. Maybe a simple look at the code will reveal a function that can be invoked. This could actually be used by Dev to support its own add-ons, but if the concept of add-ons for Defender isn’t yet “a thing” in the eyes of management or developers, then I’ll be happy with simple hooks, functions, documentation, or sample code published here.

  • Tony G
    • Mr. LetsFixTheWorld

    Over the last few days I was securing one of my sites, and for the first time I implemented Fail2Ban. That software monitors log files, and when defined regex conditions are met it triggers actions, specifically to ban IP addresses through IPTables. …At least that’s how I understand it so far, and it seems to work like that. :slight_smile:

    What I’ve unknowingly described for this ticket is the essence of Fail2Ban, which has filters for a lot of software products. Now I’m thinking that filters can be created for the logs of other WP plugins, and the action taken when a rule is matched would be to update the Defender log, rather than banning an IP.

    I don’t know if anyone is using Fail2Ban like that but I suspect someone is. For all I know that could be a core/advertised feature – to do things other than actual banning via IPTables.

    If nothing else, one thought I had was that it would be helpful to have a Fail2Ban filter for the Defender logs, so that when it detects an intrusion we can then ban the offending IP at the server level. This will help with other WP sites on the same server and other applications. I could even take the IPTable rule and copy it to my other systems : If some bad actor tries to hack one of my systems/sites, I don’t want them to have access to ANY of my systems in my cloud network. I could do this one myself very easily with no changes required in Defender or WP.

    This isn’t a big deal guys. Think big, and sometimes small solutions can help to get you there.

    HTH

  • Mark
    • The Crimson Coder

    This I suspect is where The Hub and Defender should come in. Defender already stores its logs in The Hub, much like Defender itself has a series of settings as to when to Ban an IP. We should be able to have an amalgamated set of settings across all sites, and possibly opt in/out to a global (all WPMUDEV users function) list where we can define temporary/permanent block rules and have those as published lists for Defender (at a minimum) to block, ideally I would be able to add this list directly to my iptables firewall (via CSF).

    This would help when you have “slower” attacks using compromised servers where a server attacks a single entry point once across multiple sites, the global Audit log parsing could potentially detect this and then ban it across all sites (ie 1 remote client tries to login to multiple websites and fails – to an individual site that looks like a username/password error – across a multitude of sites thats a hack attempt).

    Regards

    Mark

  • Tony G
    • Mr. LetsFixTheWorld

    My understanding of Fail2Ban was mostly correct, just incomplete.

    Fail2Ban uses Filter config files to decide if a log indicates a reason to take an action. It then uses Action config files to execute commands based on what was found in the filter.

    So for this application we’re talking about individual Filter files, one for defender logs, and one for each other WP plugin log of interest. Then a single Action file which bans an aggressive IPs in Defender. I would take that further with an Action that aggregates the IPs into a network-accessible file, and a cron process that periodically rebuilds local ban tables using the latest aggregate list of IPs. The local network ban table can be augmented with IPs from DNSBLs that aggregate data exactly as I’ve described. Gosh, maybe I should have just asked for an enhancement to Defender to make use of specified block list services.

    This is all fairly trivial. I wish I had time to create and sell the related software/plugins.

    About how to ban at the server tier rather than the WP application tier : I started with the idea that IPTables was the only option, but looking at the Actions files I found that there are a lot of actions that can be executed. IPTables itself is being deprecated in favor of the “new and improved” NFTables. I’ve setup Fail2Ban for IPTables but will transition/upgrade to NFTables as time permits.

    There’s a Lot that we can do with the technology and infrastructures that are available. It seems all of us are subject to factors including time, money, risk, and reward.
    – If someone perceives more risk, they’ll be more inclined to spend more time on a problem.
    – As the perception of risk increases, the time vs money factor gets re-evaluated and we spend money rather than time.
    – I’ll spend time to do this for myself because the risk is high enough, but not high enough to spend a lot of money.
    – Neither I nor WPMU DEV prioritizes creating solutions like this for others, because (speaking for myself but it seems obvious that this applies to Dev as well) the potential reward isn’t great enough that it justifies our time. Dev would need to do a lot of work, and give away the features as a membership benefit. So their time/reward ratio is really unbalanced. I (or someone more qualified) would need to get some guarantee of reward first before spending any time. Doing this as FOSS simply won’t work.

    So as I’m musing about all of this stuff, I think : Wouldn’t it be cool to link forum threads here with Forminator polls, where people can vote, not only on whether they want something from Dev, but whether they want it bad enough that they’ll pay someone else. Consider : When we ask Dev for a feature, we may or may not get it – ever. That’s not a slam. That’s reality. We never know when they will process a request, when it will cross that threshold where the risk/pain of not having it outweighs the time/money to provide a new feature. What we do not see here or elsewhere is a metric that helps us to prioritize third-party development : “If Dev doesn’t intend to do this within the next year, I’d pay someone else to do it.” Dev might see the results of such a poll and say “wow, they really do want it, we should either do it for free or for-fee”.

    Alas, until then all we can do is talk about stuff here and hope.

    </musing-mode>

    • Kasia Swiderska
      • Support nomad

      Hello Tony G ,

      What we do not see here or elsewhere is a metric that helps us to prioritize third-party development : “If Dev doesn’t intend to do this within the next year, I’d pay someone else to do it.” Dev might see the results of such a poll and say “wow, they really do want it, we should either do it for free or for-fee”.

      I’m confused a little bit – you mean those polls should be visible for 3rd party developers or our developers?
      Like, our developers and basically the whole plugin teams keep track of the requests and they can see which features are highly requested.

      kind regards,
      Kasia

      • Tony G
        • Mr. LetsFixTheWorld

        Kasia wrote:

        I’m confused a little bit – you mean those polls should be visible for 3rd party developers or our developers? Like, our developers and basically the whole plugin teams keep track of the requests and they can see which features are highly requested.

        :laughing: I know you’re thinking “we already do that”. This is different.

        Recognizing the best of intent, we know Dev can’t process requests unless there is some fiscal justification for the effort. You need to get “something” out of it, either keeping clients happy or getting new clients. That’s basic business and we all “get it”.

        What I’m talking about is an added bit of weight to requests. “Not only do I think it’s a good idea but I want to pay for that feature.” This is separates the casual request from others. It gives Dev more to think about. You guys might decide you can’t process a request because there are only 20 people who support it. But right now you don’t have a metric that tells you that 5 of those 20 are willing to fund the entire development project. And you don’t have a metric that tells you that if you had that feature right now that another 50 might be willing to pay a subscription fee for it. Very few people click that ThumbsUp button to support a new idea, but people are much more willing to pay for functionality that is advertised as already existing. If you want confirmation of that, just look at the millions of installations of premium plugins for WordPress – people are willing to buy what’s there, but they aren’t willing to support new development.

        With more transparency we can get a better view of whether a feature is worthy of development. For example: Dev decides that feature X isn’t worth their effort. Rather than leaving forum threads hanging for years with no response, you tell us : “No, we don’t plan to do this. Sorry. It’s not worth out time.” …. Brutal but we all know it’s true. With that final statement out, it puts more visibility on the feature for WordPress admins as well as third-party developers. Some people will decide “OK, if Dev won’t do it, I can live without it”. Others will decide “Wait, no I really do want this, even though I only had one ThumbsUp vote.” Their voice should be heard … and it should be heard by developers anywhere who might accept the challenge and the potential rewards.

        Dev doesn’t need to process every request. Just tell us what you’re not processing and let’s facilitate a mechanism that gives feature requests a fair hearing in the court of public judgement. Give supply and demand an opportunity to meet when you decide you don’t want to create supply for an unacceptable level of demand. The feature isn’t worth it to Dev, but it might be quite lucrative for someone else to create that same feature. And this is why there is a huge market of free and for-fee plugins outside of core WP development. In the WordPress Trac, we know what they’re not going to do in core. From there it’s up to others to decide what they want to do. With WPMU DEV, we have no idea if or when you’ll process anything. And again, that’s no criticism, that’s really just normal business.

        I hope that helps to clarify this concept which probably should be in a completely different thread. Unfortunately, the discussion probably won’t be seen by many people at all. THAT is a part of the problem that I’m citing. Astute readers here will notice that this issue of feature creation is exactly the same as the topic of this thread : We can’t solve a problem unless we know about it. I can’t block hackers on server X if I don’t know that they’ve already attacked server Y. It doesn’t make sense that I should let my websites get violated by the same people who I know have already attacked my other servers and sites. I don’t have any good way to let our colleagues here know that my un-announced development server is being abused daily by a specific block of IP addresses. Everyone has to find that out for themselves, possibly by being attacked themselves. And right now someone else here is attempting to block a hacker who will just move over to my system. That makes no sense that we casually let that happen.

        There’s no good reason for persisting isolated data silos in today’s integrated world – especially with the specific tools that we here are all managing right this very minute.