NCDU-fu

Sometimes it feels to have some meager sysadmin competencies, such as knowing how to quickly identity where large files are in a particular hosting account. This issue comes up from time-to-time when someone discovers all their storage space has been eaten up, but they are not sure where and why.  Often this is a symptom of a larger problem, such as an error_log run out of control which suggests bad process for a particular application, etc. That was the case on a ticket this morning, and luckily I knew the NCDU command. What is NCDU, you ask?

Ncdu is a disk usage analyzer with an ncurses interface. It is designed to find space hogs on a remote server where you don’t have an entire graphical setup available, but it is a useful tool even on regular desktop systems. Ncdu aims to be fast, simple and easy to use, and should be able to run in any minimal POSIX-like environment with ncurses installed.

In other words, a script for a remote server that finds big files. You can install it on your server, and then run it by navigating in command line to the offending account, which on our cPanel servers would live at /home/offendingaccount and running the command NCDU. After that, it will list all the directories and their sizes, followed by files.  You can then locate the directory with the largest file usage, and then change to that directory and run NCDU again until you find the offending file. In the example this morning, it was a 6 GB error_log in a directory running WordPress, easy fix to clean out space, and a good heads up things in that account need to be checked for a bad plugins, theme, etc.

The life of a Reclaimer is always intense

Where Did All My Disk Space Go?

Sometimes you may find that you just plain need more disk space. Perhaps you're regularly uploading large files to build an online archive for a project or you work with video files that need to be stored locally. As Reclaim Hosting has grown over the years we've consistently increased our storage options and luckily upgrading is an easy process! That being said we always encourage folks to diagnose the source of disk space issues to be sure there's not any other issues at play. Automated backups you weren't aware of in a folder on your account because of a plugin, or a log file that has grown rapidly could be the source and simply deleting a few files could free up large amounts of space in your account. But how to find that large needle in the haystack?

cPanel has a rudimentary tool in the control panel called Disk Usage which is found under the Files section. Disk Usage will calculate the size of every folder in your account and display it in a nice graph. However the biggest caveat that makes this a poor solution is that the Disk Usage tool will not let you navigate to subfolders. More often than not you'll find that the public_html folder which houses all your web content is the largest, but that could still leave you searching blindly for more information on where within that folder the culprit is.

Disk Usage in cPanel

Disk Usage Detail View

Because of the limitations of this tool we add a program to our servers called Ncdu. Ncdu runs from the command line so you'll need to be logged in via terminal with your SSH account, however you'll find the interface pretty friendly. Once logged on simply navigate to the folder you want information about and type the ncdu command to get a text-based graphical representation of the largest folders.

ncdu

Because the folders and files are ordered by size I recommend starting at the root of your account and running the tool. You can use the arrow keys to move up or down and enter to navigate into a folder and get a readout of the size of its contents. In our experience it makes quick work of analyzing disk space issues and finding any opportunities to clear out the cruft. When you're finished simply hit the q key to quit.

We've added ncdu to many servers but if for some reason it's not available in your account on Reclaim Hosting put in a support ticket and we'd be happy to make sure it gets added.