10 Unix Commands Every Web Developer Should Know

10 Unix Commands Every Web Developer Should Know

Unix servers run the internet, and, as a web developer, you should know how to use this powerful operating system. In the beginning, Unix can be a little intimidating, especially for someone coming from a strictly graphics-oriented OS. There’s no magical ‘undo’ button that can fix a mistake you made and there are not very many “Are you sure you want to remove ‘test.txt’?” messages unless you specify the tag to do so (‘rm -i test.txt’ would ask for confirmation before deleting). I’ve compiled a list of useful commands and examples of situations to use them in. I’m going to assume you know the very basics (ls, cp, mv, mkdir, rm, cd, etc.) so I’ll skip over these.

Searching in Unix

Our first two commands deal with searching in Unix involving both file names and the contents in a file. The following commands take string literals or regular expressions as the search string, another tool every web developer must know.

1. find

Locating a file by name inside the current directory and all subdirectories


find . -type f -name 'nameofthefile.txt'

You can also use the * metacharacter if you want to find every file ending in .txt


find . -type f -name '*.txt'

Find can also search for directory names using the ‘d’ flag


find /usr -type d -name 'bin'

2. grep

Grep is useful when you need to search for content within a file. The following command will search through every file in the current directory and all files in subdirectories and return the line in the file that matches the regular expression. I escape the period to tell grep I want to find a literal period and to not use the dot regular expression metacharacter.


grep -r 'immense\.net' .

Grep and find can work together to search the contents of the files that are found using the find command. The following two commands are different ways to do the same thing: search through all php files looking for the string ‘propel’.


find . -type f -name '*.php' -exec grep 'propel' '{}' \;
find . -type f -name '*.php' | xargs grep 'propel'

If you don’t understand how pipes work you should learn how to use them.

3. sed

Sed is a very powerful “stream-oriented” editor. In conjunction with regular expressions and the find command, you can perform some complex search and replace functions across a set of files. Consider a file named “testing.txt” which contains the text “This is only a test”. The following command would alter the word “test” to “disaster”.


sed -i '' s/test/disaster/g testing.txt

The -i flag means “edit in place”. You may optionally pass a parameter to this flag which will make a backup copy of the file(s) you are modifying and append the extension you pass it. Consider our original file and original content; the following command will replace the word “test” for “disaster” and will make a copy of the original file called “testing.txt.orig” in its untouched state.


sed -i '.orig' s/test/disaster/g testing.txt

Sed is an extremely powerful editor, and it can modify many files simultaneously when combined with the ‘find’ command from above. The following command will search through the current directory and all subdirectories and replace the string “http://” with the string “https://” in each html file.


find . -type f -name '*.html' -exec sed -i 's/http\:\/\//https\:\/\//g' {} \;

Notice that certain characters must be escaped in the search parameters of sed. The curly braces “{}” pass each filename returned from ‘find’ as an argument into ‘sed’ and the “\;” is required by exec as a termination character. This is only one small sample of what sed can do and I highly recommend reading the sed man page to learn more options.

File Manipulation and Examination

Every entity in the Unix operating system is a file. It may be called a directory, socket, or stream, but they all have inode numbers and they are all files. As such, file manipulation and examination is an extremely important topic. Below are just a few of the many very important file handling programs.

4. tail

Tail is very simple. It shows you the tail end of a file; just the last few lines. This is invaluable when it comes to error logs. You can use the ‘f’ flag to “follow” the end of the file so every time a program writes to that log you see it instantly. Use “CTRL+C” to end execution of tail.


tail -f /var/log/php.log

5. more

‘More’ is a command that shows you the contents of a file. You can also use ‘cat’ to view a file but ‘more’ will paginate it and give you some options. To begin reading a file:


more filename.txt

This will launch the ‘more’ program and you will see “–MORE–(#%)” at the bottom of your screen if your file has more lines than can fit on your terminal screen. Hitting the ‘Return’ key will move the screen down one line, ‘Spacebar’ will move a page forward, ‘b’ will move a page backward, ‘=’ will display your current line number, ‘v’ will launch the vi text editor (covered next) at the current line, ‘q’ will quit and ‘/’ will allow you to enter a regular expression to search for. Remember that a regex can be as simple as a plain string.

6. vi

Vi is a great text editor. People will argue that emacs is a better option or that pico is easier, but I find that vi is a great solution for doing quick text modifications. Emacs is the big mature Unix editor and pico is a little too simple, so I use vi most of the time because it offers a lot of solid features without a huge learning curve. After reading this article you should read more about vi.

Start vi with a new or existing filename:


vi filename.php

You’re now in the editor. Within the editor are two modes: command and insert. To enter command mode at any point hit the ‘ESC’ key. Most commands are prefixed with a colon ‘:’ so to quit out of the program enter command mode and type ‘:q’ (without quotes) followed by the ‘Return’ key. Fire up vi again and, this time, go into insert mode by pressing the ‘a’ key. You can start typing whatever you need just like in any other text editor. To save and quit hit the ‘ESC’ key to enter command mode and type ‘:wq’. The ‘w’ means write the file and ‘q’ means quit vi. If you wanted to quit and not save your changes you could type ‘:q!’ in command mode. There are hundreds and hundreds of good keyboard shortcuts to use while in command and insert mode and you can read about them in the article I linked to above.

Backups and Transfers

As a web developer, backing up files and transferring data is critical. Sometimes you just need to move a ton of files from one server to your local machine but sometimes you need to transfer them to another server. This section covers archiving files with tar, backing up a MySQL database with mysqldump and transferring files to another machine with scp.

7. tar

Tar is a program that archives files into a single package that you can move around easily. After using the tar command ,you are left with a single tar file which is referred to as a ‘tarball’. A tarball is compressed ,but you can compress it further by adding the ‘z’ flag which gzips it into a smaller file. This is how I typically use tar:


tar cvzf mytarball.tgz .

The ‘c’ flag means “create a tarball”, the ‘v’ flag stands for “verbose mode” or “show every file that is compressed as it happens”, the ‘z’ flag means “gzip the tarball further (hence the .tgz extension instead of .tar)” and the ‘f’ flag means “use the tarfile argument as the name of the tarfile”. The argument ‘mytarball.tgz’ is the name of the file. You can name it whatever you like; the extension does not matter – I just like using .tgz. The last argument ‘.’ means include all files in the current directory and all subdirectories. After running this command you now have a compressed ‘mytarball.tgz’ file. You can move this file around wherever you need it. If I needed to download an entire site via FTP I would run this command and then just download the single tarball rather than downloading many files individually.

The opposite of archiving is unarchiving (o rly?). To unarchive a gzipped tarball use the following:


tar xvzf mytarball.tgz

It’s that simple. The flags are the same except you use ‘x’ instead of ‘c’ which means ‘extract’. This will uncompress your tarball in the current directory you are in, maintaining the original directory hierarchy.

8. scp

Scp is useful in conjunction with tar. It allows you to securely copy a file from one server to another. To do this you must know the address of your destination server and you must have a username and password for a user with write privileges in the location you want to put the file. For this example, say I want to move that tarball I created earlier to a server ‘someserver.com’ and I have write privileges with the user ‘myuser’ and password ‘mypassword’:


scp mytarball.tgz myuser@someserver.com:/home/myuser

You would then be prompted for the password of ‘myuser’. This would copy the file to the directory /home/myuser on the host server. If you were on the host server already and wanted to copy a file from a remote server you would do:


scp anotheruser@someotherserver.com:/home/anotheruser/filetocopy.txt .

You would then be prompted for that users password. This would copy the file ‘filetocopy.txt’ from a remote server to your current director. This is essentially ‘getting’ a file whereas the first example was ‘putting’ a file.

9. mysqldump

This command is biased towards those developers who use MySQL. I’m positive there are identical commands for other RDBMS. Say you needed to move an entire site with it’s database from one server to another. You’ve tarred up all the files, and you’ve moved them to the new server via scp, but how do you get the database? It’s simple:


mysqldump -uroot -p databasename > dumpfile.sql

You will then be prompted for a password and, depending how large the database is, it may take a few seconds to return you to the command prompt. Substitute the ‘root’ parameter for whatever username has access to the database you are trying to dump. The ‘>’ redirects standard output to standard input and places the dumped data in the file ‘dumpfile.sql’. You can then scp that file to your new server and use the ‘mysql’ command to import the dumpfile.sql contents into whatever new database you want:


mysql -uroot -p newdatabase < dumpfile.sql

The database ‘newdatabase’ must already exist. Notice the reverse ‘< ‘ redirection operator which will send the contents of the dumpfile.sql file into the database specified.

Process Control

10. top

Think of top like ‘Task Manager’ in Windows or ‘Activity Monitor’ in Mac OS X. Simply run:


top

And you are shown your running processes. It is a realtime view unlike ‘ps’ which also shows your running processes. To quit top hit the ‘q’ key. The first column displayed in top is the PID. Every process has a unique ID and you use this ID if you need to kill a process. Do not just kill processes for fun you should only do this if you know for sure there won’t be any adverse effects:


kill -9 1234

Where the -9 flag is the SIGKILL signal that no process can ignore. It will force quit the process with PID ‘1234’. There are many other signals you can use on a process that you can see here.

1 reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *