SUMMARY
General Linux Commands
ISSUE
General Commands
ls
lists files, directories, etc. within a directory.
- -l long listing showing permissions, size, etc. (required for all the rest of these flags)
- -h human-readable values for size
- -S sort by size, largest to smallest
- -t sort by time modified, newest to oldest
- -r sort by reverse of other flag (i.e. ls -ltr gives files sorted from oldest to newest)
- -A show all files, including hidden files (will not show . and .. as -a does)
- -R recursive listing, shows all files in all directories and subdirectories of the current path
- -i shows inode numbers (inodes are structures that store information about files, directories, etc.)
cd
changes directory
- cd .. changes to parent directory
- cd <relative path like backups> goes to relative path within directory
- cd <absolute path like /backups> goes to absolute path
- cd without any flags goes to the logged-in user's home directory (also represented by ~)
- cd - goes to previous directory (directory before the last change of directory)
mkdir
create directory
mkdir /tmp/1 would create a directory called 1 in /tmp
- -p creates parent directories as needed
mkdir /tmp/1/2/3/4 would create all of the needed parent directories if they did not exist already
- -v verbose
rmdir
remove directory
rmdir /tmp/1 would remove that directory (if it is empty).
- -v verbose
mv
moves and/or renames files/directories
- -f forces the move--no prompt before overwriting
- -v verbose; show what is being done
cp
copies files, directories, etc.
- -p preserve mode, ownership, timestamps when copying
- -R or -r copies directories recursively
- -v verbose
pwd
print working directory (current directory)
rm
removes files, can remove directories as well
- -f force removal--do not prompt
- -r recursive removal; allows removal of files and directories. BE CAREFUL WITH THIS!! KNOW WHERE YOU ARE!!
- -v verbose
man
shows manual pages for a given program or topic
- running man tar will show the manual pages for the archiving program tar
cat
concatenates multiple files, prints files to screen
- -A shows all non-printing characters like tabs, end-of-line characters, etc.
- -n numbers all lines
head
show the first part of a file; by default the first 10 lines are shown
- -n <#> will give the first # lines of a file
- -<#> will also give the first # lines of a file
- i.e., head -n 15 a.txt and head -15 a.txt give the first 15 lines of a.txt
tail
show the last part of a file; by default the last 10 lines are shown
- -n <#> will give the last # lines of a file
- -<#> will also give the last # lines of a file
- -f will follow a log file as new data is added to it, outputting the data to screen
- i.e., tail -n 15 a.txt and tail -15 a.txt give the last 15 lines of a.txt
less
less is a pager program that will let you move back and forth within a file
- i.e., less a.txt will open up a.txt and allow you to page up and down within it.
- Note: typing gg goes to the beginning of the file, G goes to the end, q quits
date
shows the current date and time
- to convert from a Unix epoch timestamp to a human-readable format, type date -d @<UNIX epoch timestamp>
uptime
shows how long the system has been up, and the system load average over the last 1 minute, 5 minutes, and 15 minutes
top
gives you a running listing of the current processes running, sorted by CPU usage
ps
similar to top, except that the listing is static.
- ps aux is the way I most often run ps.
- ps -leaf is another common way to run ps.
- ps aux | grep <processname> will show you information for the given processname.
clear
clears the screen
df
shows disk free space, disk used space, etc.
- -hT will give you human-readable values and show the filesystem type.
- -BG will give you block sizes in gigabytes.
du
will show disk usage for files, directories, etc.
- -s gives high-level summary (1 directory deep)
- -h gives human-readable information.
- -BG gives block-size in gigabytes
- -c gives a total at the bottom.
- du -shc * is very helpful.
free
shows free memory
- -m shows memory in megabytes
echo
prints to screen
- -e allows for escape characters (like \n for newline)
- to print the contents of a variable called BASH, do
echo $BASH
- to append words to a file called a.txt
echo end of file is here >> a.txt
bc
calculator program best used with echo. by default only gives integers. to give numbers after decimal use scale
- divide 86763 by 843 with 3 decimal points in the result
echo scale=3; 86763/843 | bc 102.921
- divide 8000 by 3600 and just leave whole number
echo 8000/3600 | bc 2
- to convert hex FF3 to decimal
echo ibase=16; FF3 | bc 4083
- to convert 1027 to hex
echo obase=16; 1027 | bc 403
Bash Basics
Bash is the default command line shell for Linux. As you will be logged in via Putty to many backup appliances, you will become very familiar with bash.
- To redirect the output of a command to a file, use the > sign.
- b.sh > a.txt will create a.txt and put the standard output of the script b.sh into the file a.txt
- >> is similar to > but instead appends to a given file.
- b.sh >> a.txt will add the output of b.sh to a.txt
- | allows you to run additional commands on the output stream of the previous command. You can use as many pipes as desired.
- ps aux | grep ssh will run a process listing and then look for the lines containing ssh with the grep utility.
- using backticks ` will do command-line substitution--that is, it will run the command within the backticks
- i.e., one helpful command to run will give you a listing of files recursively in a given directory, and sort by time.
ls -lhtr `find . -type f`
Using Bash History
- the history command will show command history
- !! runs previous command
- !-5 runs the fifth to last command
- !ls runs previous command starting with ls--BE CAREFUL WITH THIS
- ^abc^def substitute def for abc in the previous line
- !123 runs history item 123
- Ctrl-R allows you to search your history from most recent command to oldest
- hit Ctrl-R, type vim, and if you do not see the right vim command, type Ctrl-R again and it will show you the command for vim prior to that
- Ctrl-K kill (erase) rest of line
- Ctrl-U erase part of line before cursor
- Ctrl-X, Ctrl-U to undo your previous command line edit
- Ctrl-A to go to beginning of line
- Ctrl-E to go to end of line
- Alt-F to go forward a word
- Alt-B to go back a word
- Alt-shift-8 (Alt-asterisk) gives all possible tab completion matches
- Ctrl-Alt-E expands ~ (to the path for the home directory) and asterisks (shell globbing) and command line substitution
vim basics
The program vim is a fantastic command-line editor. You can run vim <filename> to open the editor. The editor is opened in command mode, and you can change to insert mode through various methods--most often by hitting the letter i. To return to command mode, hit the ESC key. I strongly recommend using the built-in program vimtutor as this will allow you to learn more about vim; just type vimtutor. While in command mode, you can delete, run special commands to look for lines containing certain information, etc. All of the following is for when you are in command mode.
- :w saves the file
- :q! quits, discarding changes
- :q quits
- :wq saves the file and quits
- :e! reopens the current file, discarding changes
- i goes to input mode at current character
- A goes to input mode at the end of the current line
- x deletes the current character
- dd deletes the current line
- u undoes the last change; this can be run multiple times
- Ctrl-R is for redo
- 0 goes to the beginning of the line
- $ goes to the end of the line
- gg goes to the beginning of the document
- G goes to the end of the document
- /searchpattern searches forward in the document for the searchpattern
- ?searchpattern searches backward in the document for the searchpattern
- :n goes to the next file if multiple files have been opened.
- :N or :prev goes to the previous file in multiple files
- :ar shows the list of files that are being edited (the vim command-line arguments)
- :set list shows nonprinting characters
- :set ic makes searches case-insensitive.
- :set hls highlights searches (default behavior most of the time)
- :set noh stops highlighting the given search pattern
- :set nu shows line numbers
- :set ff=dos converts EOL format to dos
- :set ff=unix converts EOL format to unix
- :g/regexp/d deletes all lines containing regexp
- :v/regexp/d deletes all lines except those containing regexp
- :%s/regexp1/regexp2/g substitutes regexp2 for regexp1 every occurrence in the document.
- :history shows vim history
You can also search through history by hitting / or ? or : and using the up and down arrows.
screen
screen is another great Linux utility. This will allow you to work through multiple windows on a given system. Ctrl-A is the primary meta key that is used for screen.
Ctrl-A, c to create new screen Ctrl-A, :caption always--this gives a caption at the bottom of the screen Ctrl-A, Ctrl-A to flip between windows Ctrl-A, <window #> to go to that window Ctrl-A, n goes to next window Ctrl-A, p goes to previous window Ctrl-A, ' to switch to a given window Ctrl-A, to show all windows Ctrl-A, d to detach screen Ctrl-A, K to kill current window Ctrl-A, :number <#> to change order of screen
exit will exit a screen and show screen is terminating when you have exited the last screen
screen -ls lists screens. if there are multiple, specify the PID for the one you would like to connect to screen -x will allow you to join a current screen session screen -d -r will allow you to detach and reattach a screen
Parsing through Text Files
grep
search files for patterns (regular expressions)
- -i ignore case
- -v find all lines except those containing this pattern
- -l list files where matches are found, not individual lines from the files
- -A<#> for context list # of lines after match was found
- -B<#> for context list # of lines before match was found
- -n show line number of file where match was found
- -r recursive--find files in initial directory and all subdirectories
grep <pattern> <filename>
egrep allows for extended regular expressions and easy searching for multiple patterns
ps aux | egrep 'tasker|devmonitor' root 1035 0.0 0.0 115544 9680 ? SNs Sep29 2:17 /usr/bp/bin/tasker root 18802 0.0 0.0 108800 5216 ? SNs Sep29 13:34 /usr/bp/bin/devmonitor root 30530 0.0 0.0 61188 832 pts/4 S+ 20:09 0:00 egrep tasker|devmonitor
sort
sorts lines, usually alphabetically by default using the first field in each line
- -u show unique lines only
- -n numeric sort
- -r reverse order
- -k<#> sort by this field number
cat a.txt | sort -nr -k2 would sort the lines in a.txt numerically in reverse using field 2
sed
stream editor, can be used to replace strings and manipulate files and output streams
- -i inline, perform this on a file rather than throwing the output to the screen (standard out)
- to substitute the number 7 for 4 throughout an entire file you would do the following (g is for global rather than the first occurrence per line)
sed -i a.txt 's/4/7/g'
Network Commands
nmap <IP address or host name> will show ports open/closed at a given IP or host name
Processes and Files
kill
kills a process by process number.
- i.e., kill 5900 kills process with PID of 5900.
- the default kill signal is -15 (SIGTERM, which allows process to cleanup), but if all else fails when trying to kill a process, kill -9 (SIGKILL) is a rude way to kill a process.
killall
killall is similar to kill but allows you to use a process name, and it will kill multiple processes with that name
file
will tell you what kind of data is contained in a file (tar archive, etc.)
- just run file <filename>
chmod
change permissions on a file
- +x will make a file executable.
chown
change ownership of a file/directory, can point to user and/or group
- chown root /backups/a.txt would change ownership of a.txt to root.
- chown -R root:root /home would change ownership of all files in /home and subdirectories to the root user and root group.
ln
creates hard links for files or soft (symbolic) links for directories or files
- typical format ia for symbolic links is
ln -s <target path> <path for link to be created>
- example--you could remove logs.dir and create a symbolic link in its place to a new directory that you had created as /backups/logs.dir
ln -s /backups/logs.dir /usr/bp/logs.dir
mount
mount drives or network shares, etc.
- typical format is
mount <target> <mount point>
- i.e.
mount /dev/mapper/vgbackup-lvbackup /backups
If a drive is already listed in /etc/fstab, you can often just use the mount point to mount it.
- i.e.
mount /backups
umount
unmount drives, network shares, etc.
umount /backups
Note: If you get an error about files being in use, you may need to cd out of that directory. Also you can use lsof to see what files are open.
lsof
list open files
- common usage:
lsof | grep <mount point>
-
- i.e.
lsof | grep /backups
find
utility for finding files
- typical usage is
find <what directory to start from> -type <d for directory, f for file, etc.>
- i.e. to find files with a name containing 123 or 456, do the following:
find /backups -type f | egrep '123|456'
tar
used to create or extract from archives
- to create a tarball (tar of gzipped files) of /usr/bp/logs.dir at /tmp/logs.tgz
tar -czvf /tmp/logs.tgz /usr/bp/logs.dir
- to extract from a log file (normally extracts as relative paths to the current directory, and will overwrite. BE CAREFUL!)
tar -xzvf /tmp/logs.tgz
- view files within an archive
tar -tvf /tmp/logs.tgz
rpm
redhat package manager for installing, upgrading packages, etc. Be very careful with rpm and check with a senior engineer before doing anything!
- show all packages on system
rpm -qa
- show all unitrends packages in order
rpm -qa unitrends* | sort or rpm -qa | grep unitrends
- update packages
rpm -Uvh *.rpm
yum
easier way to update packages, especially when dependencies are needed etc.
- yum list updates shows updates available
- yum update updates all packages with updates available
- yum update unitrends* updates all unitrends packages
- yum localupdate updates packages from a local directory. Sometimes this is a workaround if packages have to be transferred via an unusual route (if WAN access is not available, etc.)
File Transfer
wget
download files using ftp
wget ftp://ftp.unitrends.com/bp/latest_build/airbag.tar
- show current IP address
wget -qO - icanhazip.com
- get multiple files using dash (reads from standard input)
wget - ftp://ftp.unitrends.com/bp/latest_build/Windows/PC61_Universal.exe ftp://ftp.unitrends.com/bp/latest_build/Windows/md5sums
scp
secure copy protocol, use to securely transfer files using ssh
- -P port number
- -C compress
- to transfer abc.txt from a customer DPU to your home directory on the tunnel server. use your username rather than jsnipes
scp -P 222 -C abc.txt jsnipes@tunnel.unitrends.com:~/
- to grab output.txt from a remote system and transfer it to current directory
scp -C root@uvault1a:/tmp/output.txt .
NOTES
Commands only