What are some useful tips for disk space management?
It is important to make sure that the server does not fully run out of disk space. If it does, the services running in the server will quit working, and some of the database tables could even become corrupted. Here are some things to consider, to help ensure that your server has enough disk space to work correctly.
Backups
Although backups do themselves take disk space, it is still very important to have them. To protect against data loss, it is always strongly recommended not only to keep multiple sets of backups, but also to keep copies of at least some of them in an offsite location. Although we do create our own backups of the VPSs, there are multiple reasons it is important not only for you to create your own backups as well, but also to keep at least one of the copies offsite.
For cPanel/WHM servers, you can read more about how to configure automatic backups and some of our recommendations for them here. For manually triggering backups at the time you want them via cPanel, you can read more here. For DirectAdmin servers, you can configure backups from any of the three access levels and it is possible to store the generated backup both locally and remotely. For servers still using Plesk, (Plesk is no longer available for new servers) you can learn more about the backup system here.
On the Dedicated Servers, the backups are by default configured to be stored on the secondary disk, so even in the event that the partition for the backups becomes completely full, it will not interfere with the functionality of the daemons (though it would cause those backup runs to fail, and those backup files to become corrupted). But on a VPS, there is only a single partition, so it is very important to make sure the backups being generated and stored there will not cause the disk to become completely full.
Overall, when choosing a backup configuration, it is important to consider all of the following factors:
- How much disk space you have available in the server
- How much disk space each account takes up
- Whether/how many external locations you have available to you in order to store copies of backups outside of the server
- How long ago you want to be able to revert accounts to
For more specific recommendations, (for cPanel/WHM servers) please read more here.
Account Quotas
One way to help prevent the server from running out of disk space is by using account quotas to limit how much disk space each account is allowed to use. In cPanel, the quota for a specific account can be adjusted in WHM at
Home >> Account Information >> List Accountsby clicking the plus (+) button next to the desired account, clicking the “Change Quota” button that then appears under the account information, and using the interface there.
It is strongly recommended to have a quota on each account, and for the sum of the account quotas to be enough less than the total size of the server to leave room for the system files.
It is important to note that with default settings, database sizes are not calculated. This setting is chosen as default for performance reasons, since there is considerable overhead with calculating database sizes. For this reason, the database sizes are by default displayed as zero, and do not count against the accounts’ disk space quotas. They do, however, still take up space on the disk. For this reason, with default settings, it is possible for the accounts to use more disk space than they have been allocated. To prevent this from happening, you could enable database size calculations by setting “Include databases in disk usage calculations” to “On” in WHM at
Home >> Server Configuration >> Tweak Settings -> SQLBut remember that this is likely to slow down the server, so you will need to decide for your server how likely it is for database disk space usage to become a problem vs. database disk space calculation to cause a problem with server load, or which issue you would rather risk happening.
It is also important to note that (in a cPanel server) files owned by “nobody” will not count against the cPanel user’s disk space quota. This is yet another reason to use mod_ruid2 (or, if this is not possible, suphp) so that the site files will be correctly owned by the cPanel user.
Monitoring and Troubleshooting
It is strongly recommended to keep an eye on how much disk space the server is using, to help ensure it does not completely fill up its disk. On a cPanel server, it is strongly recommended to enable all of the following settings in WHM at
Home >> Server Configuration >> Tweak Settingsin the section “Notifications”, to help keep you and your resellers/site admins informed when the disk space usage needs attention:
- System disk space usage warnings
- Account system disk usage “warn” percentage
- Account system disk usage “critical” percentage
- Disk quota usage warnings
- Account disk quota “warn” percentage
- Notify admin or reseller when disk quota reaches “warn” state
- Account disk quota “critical” percentage
- Notify admin or reseller when disk quota reaches “critical” state
- Account disk quota “full” percentage
- Notify admin or reseller when disk quota reaches “full” state
- Email mailbox usage warnings
- Mailbox disk quota “warn” percentage
- Mailbox disk quota “critical” percentage
- Mailbox disk quota “full” percentage
If your server has mailman enabled, it is also strongly recommended to enable “Include mailman in disk usage calculations” in the “Mail” section of “Tweak Settings”.
In DirectAdmin or Plesk servers, it is recommended to have similar or equivalent configurations enabled where possible.
If more disk space is being used than expected, it is recommended to check what is using the disk space. Where to check first will depend on how you find out about it.
If the server has used up enough of its disk space overall that it is causing services (daemons) to be interrupted, you might not be able to use the cPanel (or DirectAdmin or Plesk, as applicable) interface to investigate. Sometimes even if the panel is not working, you will be able to connect using SSH. If SSH is not working either, you can open a Support Ticket for assistance 24x7x365.
Troubleshooting via SSH
If you are logged into ssh as root, you can check the server’s overall disk space usage with the df utility. For example, you can check the disk space usage in the root / partition as follows:
root@host [~]# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/simfs 50G 34G 17G 68% /In this example, the / partition has a total of 50 GB of disk space, of which it is using about 34 GB, leaving roughly 17 GB remaining. This comes out to about 68% of the disk space being full.
If you are using a VPS, the / partition is the only one you need to check, but on a Dedicated Server there are multiple partitions. You may also want to check how full the /backup partition is. For that, you can use this command:
# df -h /backupThe output will appear similar as in the above example, but with data for /backup instead of /.
If you want to check how many backups are present and how big they are, you can check with this command:
root@host [~/support]# find /backup -type d -name accounts -print0 | xargs -0 du -sch | sort -h
4.0K /backup/2017-07-02/accounts
4.0K /backup/2017-07-03/accounts
4.0K /backup/2017-07-04/accounts
4.0K /backup/2017-07-05/accounts
4.0K /backup/2017-07-06/accounts
4.0K /backup/2017-07-07/accounts
4.0K /backup/monthly/2017-07-01/accounts
4.0K /backup/weekly/2017-06-25/accounts
4.0K /backup/weekly/2017-07-02/accounts
5.0G /backup/monthly/2017-06-01/accounts
5.0G /backup/monthly/2017-06-15/accounts
10G totalIn this case, a Remote Destination was configured, and later the server was configured not to retain the backups locally. The backups that were already present before that were not removed, but all subsequent ones were deleted from the server after successful transfer to the remote destination.
Although this only looks for the account backups, these are usually larger than the system backups.
If you decide you want to remove one or more of the backups to save space, you can use the `cd` command to go to the desired folder, and then use the `rm -v` command to remove the specific files you would like to remove.
After checking the total disk space usage of the server, it is useful to find how much is being used by each account. You can get a general overview using repquota:
# repquota -asHere is example output showing the first few lines of the output:
root@host [~]# repquota -as | head
*** Report for user quotas on device /dev/simfs
Block grace time: 00:00; Inode grace time: 00:00
Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
root -- 29533M 0 0 246k 0 0
bin -- 44980 0 0 70 0 0
daemon -- 28 0 0 6 0 0
nobody -- 144M 0 0 36320 0 0
mailnull -- 2732 0 0 197 0 0 The first column shows the username. The amount of disk space used by the account is shown under “used” in the section “Block limits”. So in this example, we can see that user “root” is using 29533 MB of space (about 29 GB). If you do the command as just repquota -as rather than repquota -as | head as here, then you will see the full repquota output rather than just the first ten lines, and included in the list should be all the cPanel users.
So if you are not able to log into cPanel/WHM you can still check which users are using roughly how much disk space. However, remember that if “Include databases in disk usage calculations” is not enabled, these numbers will not include the disk space used by the databases.
If there is a particular account using more disk space than expected, it is recommended to determine where the disk space is being used. You can find the size of folders (including their contents) with the du utility as follows:
root@host [~]# cd /path/to/folder/
root@host [/path/to/folder]# du -hd 1 .
932K ./.cphorde
84K ./etc
8.0K ./.sqmaildata
4.0K ./.trash
12K ./.htpasswds
20K ./.ssh
4.0K ./perl5
8.0K ./.subaccounts
63M ./public_html
636M ./mail
8.5M ./.git
2.2M ./.spamassassin
62M ./tmp
24K ./ssl
580K ./.cpanel
80K ./cache
8.0K ./public_ftp
8.0K ./.cpaddons
630M ./logs
4.0K ./.sqmailattach
100K ./.razor
1.4G .You can sort these by size and display only the largest ones as follows:
root@host [/path/to/folder]# du -hd 1 . | sort -h | tail
100K ./.razor
580K ./.cpanel
932K ./.cphorde
2.2M ./.spamassassin
8.5M ./.git
62M ./tmp
63M ./public_html
630M ./logs
636M ./mail
1.4G .Here, the last entry 1.4G . is showing us that folder ., which represents the folder we are in at the time, is 1.4 GB in size. This is the sum of all the contents of the folder we are in, which includes loose files (which are not displayed in the list) and subfolders (which are displayed in the full list, but only the nine largest are shown in this second shorter list). If you want to show loose files as well, you can do so with the -a option on du, as follows:
root@host [/path/to/folder]# du -had 1 .
932K ./.cphorde
4.0K ./.contactemail
84K ./etc
8.0K ./.sqmaildata
4.0K ./.trash
12K ./.htpasswds
20K ./.ssh
4.0K ./perl5
4.0K ./.bash_logout
8.0K ./.subaccounts
4.0K ./.bashrc
4.0K ./.ftpquota
4.0K ./.bash_profile
63M ./public_html
636M ./mail
4.0K ./.lastlogin
0 ./access-logs
8.5M ./.git
2.2M ./.spamassassin
0 ./www
62M ./tmp
24K ./ssl
0 ./README
580K ./.cpanel
4.0K ./.dns
4.0K ./.gitconfig
80K ./cache
8.0K ./.bash_history
8.0K ./public_ftp
4.0K ./.gitignore
8.0K ./.cpaddons
630M ./logs
4.0K ./.sqmailattach
4.0K ./cpbackup-exclude.conf
100K ./.razor
4.0K ./.my.cnf
1.4G .As before, you can sort the output and show only the largest items, as follows:
root@host [/path/to/folder]# du -had 1 . | sort -h | tail
100K ./.razor
580K ./.cpanel
932K ./.cphorde
2.2M ./.spamassassin
8.5M ./.git
62M ./tmp
63M ./public_html
630M ./logs
636M ./mail
1.4G .In this case, none of the loose files are as big as the ninth-biggest subdirectory, so none of the loose files are showing up in the shorter list. But if, for example, you had created a backup via cPanel and forgotten to remove it, it could be useful to see the file sizes alongside the folder sizes, to determine where the disk space usage is coming from.
For example, if I create a large file in this folder named temp:
root@host [/path/to/folder]# ls -hl temp
-rw-r--r-- 1 root root 1.0G Jul 31 10:38 tempit will not show up in du without using -a. But we can see the larger total size of the directory, which can tell us that we should check for large files:
root@host [/path/to/folder]# du -hd 1 . | sort -h | tail
100K ./.razor
580K ./.cpanel
932K ./.cphorde
2.2M ./.spamassassin
8.5M ./.git
63M ./tmp
65M ./public_html
634M ./logs
637M ./mail
2.4G .Using -a then shows us the files as well, which gives us more detail of what is taking up space:
root@host [/path/to/folder]# du -had 1 . | sort -h | tail
580K ./.cpanel
932K ./.cphorde
2.2M ./.spamassassin
8.5M ./.git
63M ./tmp
65M ./public_html
634M ./logs
637M ./mail
1.1G ./temp
2.4G .Troubleshooting via cPanel/WHM
If you are able to still log into WHM and cPanel, you can use the “List Accounts” interface in WHM at
Home >> Account Information >> List Accountsto see how much space each account is currently taking. You can even sort the accounts by disk space usage by clicking the title of the “Disk Used” column. Note that if databases are not being included in quota calculations, the accounts are likely taking more space than will be displayed here.
If there is an account in particular you would like to investigate, you can click on the “cP” icon of the row in the “cPanel” column, to open the cPanel interface for the account. From there, you can go to the “Disk Usage” page in the “Files” section, to see a breakdown of approximately how that account is using its disk space. How to proceed from there depends heavily on what category is using most of the disk space.
Temporary Files
Usually temporary files will not be the main cause of the server running out of disk space. However, it is very important not to remove the /tmp/ directory itself, or any socket files or subdirectories that might be present. Before removing the files, check how much space they are taking up:
# find /tmp -type f -print0 | xargs -0 du -sch | sort -h | tailIf the files in /tmp are large enough to make a difference, you may want to consider removing them. To do so, in a cPanel server, this command is likely to be safe:
# find /tmp -type f -exec rm -v {} +Trash Folders – cPanel File Manager
cPanel File Manager has a trash folder for each account, so that if something is deleted by accident from the File Manager, it can be brought back relatively easily. But, this trash folder is not automatically emptied. If any of your users make extensive use of the File Manager in cPanel, especially if they do not often use the Empty Trash button within File Manager, it is possible that these trash folders can get quite large.
via cPanel
These trash folders can be emptied within cPanel File Manager by clicking the “Empty Trash” button.
via SSH
These trash folders are named .trash and are direct subdirectories of each user’s home directory. If logged into SSH as root, you can check which accounts are taking the most space with their File Manager trash folders as follows:
root@host [~]# du -sch /home*/*/.trash | sort -h | tail
4.0K /home/user1/.trash
4.0K /home/user2/.trash
4.0K /home/user3/.trash
12K totalIn this case, each of the trash folders is empty, but if you do need to remove items from the trash folders, you can do so with this command:
# find /home*/*/.trash -depth -mindepth 1 \( \( ! -type d -exec rm -v {} + \) -o \( -exec rmdir -v {} + \) \)Trash Folders – IMAP
If your users use IMAP to connect to their mail folders, then when they delete messages via their mail client they will usually at first be moved to a trash folder. In cPanel servers, it is possible to configure the IMAP trash folders to “Autoexpunge”, which means that messages which have been in these trash folders a certain number of days will be automatically removed. Within the filesystem, these folders are named .Trash, capitalized, rather than lowercase like the File Manager trash folders.
Autoexpunge
In a cPanel server, The “Autoexpunge” option can be found in WHM at Home >> Service Configuration >> Mailserver Configuration. Also on this page is the option “Trash Expire Time” which controls how many days old a message is kept in the trash folder before it is automatically removed.
via cPanel
In order to find these .Trash folders within the File Manager, it is necessary first to make sure “Show Hidden Files (dotfiles)” is enabled in the Settings. Once this has been done, you should be able to find the folders in their locations. There are two places to look:
- directly in the “mail” folder, that is,
mail/.Trashand - in each email account’s base folder, for example
mail/domain.tld/someemail/.Trash
If you started File Manager from the site’s docroot, you will need to go up one level in order to see the mail folder.
via Webmail
If you have not yet enabled the new mdbox mailbox format, you can see the contents of the email accounts from the webmail interface of the overarching email account of the cPanel account itself. You may need to “subscribe” to the relevant folders in the settings within the preferred webmail application. But, if you have already converted from maildir format to the new mdbox format, the symlinks necessary for this method of access will not work.
via SSH
Since the Trash folders of IMAP are in two specific locations, they can be found from commandline, similarly to the trash folders of the File Manager. You can check their sizes as follows:
root@host [~]# du -sch /home*/*/mail/{*/*/,}.Trash | sort -h | tail
16K /home/user1/mail/.Trash
16K /home/user2/mail/.Trash
16K /home/user3/mail/.Trash
16K /home/user4/mail/.Trash
16K /home/user5/mail/.Trash
16K /home/user6/mail/.Trash
24K /home/user7/mail/.Trash
32K /home/user7/mail/domain.tld/address1/.Trash
436K /home/user7/mail/domain.tld/address2/.Trash
604K totalIn this example, the largest IMAP Trash folder is the one for address2@domain.tld, under cpanel account user7.
You can also combine this technique with the one shown above, to find the sizes of all the file manager and imap trash folders together:
root@host [~]# du -sch /home*/*/{.trash,mail/{*/*/,}.Trash} | sort -h | tail
16K /home/user1/mail/.Trash
16K /home/user2/mail/.Trash
16K /home/user3/mail/.Trash
16K /home/user4/mail/.Trash
16K /home/user5/mail/.Trash
16K /home/user6/mail/.Trash
24K /home/user7/mail/.Trash
32K /home/user7/mail/domain.tld/address1/.Trash
436K /home/user7/mail/domain.tld/address2/.Trash
616K totalIn this case, the largest file manager trash folder was smaller than the smallest imap trash folder.
Similarly, if you want to empty all of the trash folders at once, you can do so as follows:
# find /home*/*/{.trash,mail/{*/*/,}.Trash} -depth -mindepth 1 \( \( ! -type d -exec rm -v {} + \) -o \( -exec rmdir -v {} + \) \)And remember, if you need any help, you can always open a Support Ticket for assistance.