Permit restores when free disk space is available !
Today i've facing a BIG problem :
- I have a DO 2GB Server with 40GB disk
- My apps (total) take about 20GB of disk (50%)
- One of my apps take about 5MB
- I can't restore that app (or any other app) as i have "no free available disk space" ??
After contacting the support they told me that "Yes, if all your applications size is 20 GB, then you need more then 20GB of free disk space as system count total size of all the hosted applications, not the single app."
So, with about 20GB of free disk space, we just CAN'T (by ourselves, without open a ticket and wait 6hrs as per SLA time) to restore a ridiculously small 5MB app ??
Cloudways, it's time to review you engineering... ! Sorry guys, this is just to bad...
Clouwd
shared this idea
This is now done.
Cloudways Team
-
AdminCloudways
(Admin, Cloudways)
commented
You will appreciate that not everyone at CW has knowledge about all inner workings of stuff :)
Problem with the query parameter is that we don't just store plain files in S3, we store compressed incremental backups, so totalling the S3 bucket for a given application doesn't give a precise picture of data usage when restored locally (compression rate, point in time you restore from ... will affect final restored size).
Don't worry though, we will work out something :)
Cloudways Team
-
Clouwd
commented
Ok your explication is much better than the one received on the support ticket ! I can understand now. But i'm happy to see that someone (Pere?) takes care about his customers ;)
Just for the records, the AWS CLI supports the --query parameter which takes a JMESPath expressions.
This means you can sum the size values given by list-objects using sum(Contents[].Size) and count like length(Contents[]).
This can be be run using the official AWS CLI :
aws s3api list-objects --bucket BUCKETNAME --output json --query "[sum(Contents[].Size), length(Contents[])]"
-
AdminCloudways
(Admin, Cloudways)
commented
You are right about this. We will be taking a closer look.
Main issue here is that we store backups on S3 and there is no way to calculate how much space a given backup on S3 takes. We could assume that the backup size is roughly the same size of the current live application on the server, but we have found corner cases where a customer deleted large amounts of data in the live application (so live application size was small) and when trying to restore backup (much bigger), disk was filled and consequently serious problems arose. So we took most probably a too conservative approach.
In any case, going to review the restore process to improve it and allow for maximum balance between usability and security.
Cloudways Team