New to the community? Start here

Hosting Space Stats seem off

Hi,

I was having a few issues with this site (since resolved), but one thing I noticed was the space allocation seems a bit off. The Total used is 6G out of 10G (which is possibly about right given it backed up a large zip file). However the graph underneath and the stats seem to be confusing the break down and the totals. So Production Files shows 5.5GB (both in the text and the graph) – a dh shows about 2.2 GB, with the backup overhead being 3.3G (I obviously can't check that ), this is shown separately in the text and the graph, now 2.2 + 3.3 is the 5.5G mentioned as production files (and the rest of the breakdown would then add up to 6G). The graph shows them separately as well, so what should be a 60/40 split on the graph is actually closer to 90/10.

The Staging is actually deactivated but has some minor space (about 121 MB) allocated.

Lastly the DB storage is set at 874M, now I understand this is probably calculated from the filesystem, but if I go to phpMyAdmin the DB is only listed as 50M, this overhead seems a bit much.

Overall the total seems about right, but the breakdown seems to be a bit "confused" so it may be worth taking a look.

Regards

Mark

Tue, 14 May 2019 11:54:38
  • Aaron
    • Ex Staff

    Hi there, thanks for the feedback. I understand it’s hard to wrap your head around. The amount are accurate, but hard to understand because it’s using a ZFS filesystem. That means different parts are incremental of other parts as they are based on clones or snapshots, and a lot of other strange things that’s hard to understand without some really in depth technical documentation. And yes the DB space is the filesystems that mysql actually uses. It can have a lot of overhead per mysql instance as InnoDB preallocates space for its log files, data, indexes, etc. So even with no DB the overhead runs about 650MB.

    All I can really tell you is that those graphs are meant to simply answer the question that many user’s kept asking, which is “what is using up my storage space?”. And this should help a little bit.

  • Mark
    • The Crimson Coder

    Hi Aaron

    Well in that case somebody needs to go back and re look at the graphs and totals!

    My total usage is clearly stated at s 6gb out of 10gb. The graph should show ( from what I can tell ) coloured break down up to 60% and then light gray ( for free space ) for the other 40%.

    Additionally the breakdown below the graph adds up to around 8.5gb not 6 so which is correct?

    If it’s all correct and parts are displayed twice then the graph doesn’t make sense and the totals need that explaining.

    Regards

    Mark

  • Mark
    • The Crimson Coder

    What’s even funnier is that the graph just re-emphasises how incorrect they are. I was just messing around with it on the phone, and noticed that there is a pop up on the colours which gives percentages of use.

    The production files is listed as 80.1% on a bar that includes free space ( this 80.1% would be correct if the bar was just the used space as per the total ). If I total up all the non free space percentages it comes to just under 140%. The bar itself is mixed up as it includes free space and yet the percentages ( based on the available numbers ) are based on total used ( hence the colours don’t finish where they should ).

    It still looks to me as if you are counting backups twice. It may be a perception here but I would think at least some documentation explaining why it doesn’t add up would be good!

    Mark

  • Mark
    • The Crimson Coder

    Last point for the night, I only really noticed this on one of my hosts, because it has gotten a bit bigger. The other 2 are fairly small ( only 2g each) with backups in under 100m. You really don’t see this ( or I haven’t ) until you get a larger file system AND backups, in this case the backups are listed as 2G.

  • Aaron
    • Ex Staff

    They won’t always add up. The total will always be perfectly accurate as that’s is total zpool usage.

    The individual values and graphs are just to give our best approximation of how big each type is relative to the other in how much space it uses. Due to the nature of ZFS filesystem you can’t perfectly separate storage used by snapshots and clones from their parent filesystem. That backup overhead should rotate out on its own soon though, it’s rare to see that much except right after deleting a lot of data.

    This may help if you can wrap your head around it: https://blogs.oracle.com/solaris/understanding-the-space-used-by-zfs-v2