Unix terminal survival kit
Do you have to manage a server?
Do you need to perform some actions in bulk?
Do you want impress your peers with a “hacking” skills?
Whatever the reason, it can be much faster to complete some tasks using a Terminal than with graphical applications and menus. Another benefit is allowing access to many more commands and scripts.
Important information
UNIX representation
In UNIX everything is represented by a process or file. A process is an executing program. Files are collections of data organized by a directory structure.
Files can be identified by absolute or relative paths. For example:
|
|
case-sensitive
Everything written in the terminal is case-sensitive. When the command is ls
, neither Ls
, lS
nor LS
will work.
Files and directories are also case-sensitive, eg foobar.txt
and FoObAr.txt
are two different files, even if they are in the same directory.
Beware of blank spaces
If you want to create/access/delete a file or directory that has a space in its filename, you can either put the whole filename in quotation mark "
or escape the space using the backslash \
:
|
|
Copy/paste
To copy or paste on the terminal, ctrl+c
and ctrl+v
won’t work.
Instead, we must use ctrl+shift+c
and ctrl+shift+v
.
⚠️ ctrl+c
is used to terminate the program
Suspending processes
ctrl+z
will suspend the current process.
Using fg %1
will resume the job in foreground whereas bg %1
will resume in background.
To list all suspended jobs, just call jobs
.
man the hell up
Use man
whenever you aren’t sure about a command or its options…
|
|
Shortcuts
tab
Always use the tab
button to autocomplete your command. It’s really useful to prevent any typos.
ctrl+r
“Reverse-i-search” is a shortcut to display a list of commands you have already used. It’s based on history
.
!!
Re-execute last command:
|
|
!$
Execute last command’s value:
|
|
ctrl+q
“Parks” the current command you’re currently typing. Useful if you forgot to perform another command before executing another.
It may only be available on zsh.
ctr+x+e
Edit a command you’re currently typing. Useful if it’s a long command.
Navigation
Credit to Clément Chastagnol.
Of course, bear in mind these keyboard shortcuts depends on your unix distribution, your shell, your configuration, …
Super user VS sudo
There are two ways to run administrative applications:
- run as “super user” (root) with the
su
command - take advantage of
sudo
(Substitute User DO)
sudo
allows an user to run a program as another user (most often as the root user).
|
|
Basic commands
File and folder navigation
|
|
File and directory handling
|
|
File content
|
|
Processes handling
|
|
Folders definition
|
|
File and folder permissions
Permissions are managed in three distinct scopes or classes:
- user
- group
- others
Classes
From wikipedia:
Files and directories are owned by a user. The owner determines the file’s user class. Distinct permissions apply to the owner.
Files and directories are assigned a group, which define the file’s group class. Distinct permissions apply to members of the file’s group. The owner may be a member of the file’s group.
Users who are not the owner, nor a member of the group, comprise a file’s others class. Distinct permissions apply to others.
The effective permissions are determined based on the first class the user falls within in the order of user, group then others. For example, the user who is the owner of the file will have the permissions given to the user class regardless of the permissions assigned to the group class or others class.
Permissions
From wikipedia:
Unix-like systems implement three specific permissions that apply to each class:
- The read permission grants the ability to read a file. When set for a directory, this permission grants the ability to read the names of files in the directory, but not to find out any further information about them such as contents, file type, size, ownership, permissions.
- The write permission grants the ability to modify a file. When set for a directory, this permission grants the ability to modify entries in the directory. This includes creating files, deleting files, and renaming files.
- The execute permission grants the ability to execute a file. This permission must be set for executable programs, in order to allow the operating system to run them. When set for a directory, the execute permission is interpreted as the search permission: it grants the ability to access file contents and meta-information if its name is known, but not list files inside the directory, unless read is set also.
|
|
From wikipedia:
Another method for representing Unix permissions is an octal notation. This notation consists of at least 3 digits.
Each digit represent a different component of the permission: owner, group and others.
Each digit is the sum of its component bit in the binary numeral system:
read
= 4write
= 2execute
= 1which means:
- 7 = read + write + execute
- 6 = read + write
- 5 = read + execute
- 3 = write + execute
|
|
⚠️ Please, just don’t do a chmod -R 777 *
… for obvious security issues…
Stdin/Stout
There are three main file descriptors:
stdin
is the input from the keyboardstdout
is outputstderr
is the error output
Type | Symbol |
---|---|
stdin | 0< |
stdout | 1> |
stderr | 2> |
stdin
Allows you take standard input from a file:
|
|
stdout
|
|
stderr
|
|
There is a special file on the Linux system called /dev/null
which can be considered as the “Bin”,
but once information has gone to this file, it’s gone forever.
|
|
Mixing everything
|
|
Appending to file
|
|
Pipes
You can connect two commands together so that the ouput from one program becomes the input of the next program
by using the |
:
|
|
Executing a command within a command
|
|
Scripting
VIM
When editing a file in a linux server, you do not have the choice of your favorite GUI notepad or IDE.
You will have to stick with a terminal editor, like vi, vim, nano,…
I have my preference on vim so I will share just a few tips on how to use it without being lost.
First of all, vim has 3 modes:
insert
mode: you write text as if in normal text editor- use the
i
button to switch to this mode - use the
a
button to switch to this mode and move the cursor to the next character - use the
o
button to switch to this mode and add a newline
- use the
normal
mode: provides efficient ways to navigate and manipulate texts- use the
Esc
button to switch to this mode
- use the
visual
mode: select text using movement keys before deciding what to do with it- use the
v
button to switch to this mode - use the
shift+v
buttons to select lines instead of characters
- use the
VIM movement
In normal
mode, you can navigate by using the arrow buttons, but vim has been thought for productivity.
So instead of using the arrow buttons, and thus moving your right hand, vim has a set a buttons for which
you won’t need to move your palms and be able to reach every functionnalities vim has to offer.
So use the following to navigate in your file:
h
: move leftj
: move downk
: move upl
: move right
You can also move by words:
w
: moves to the start of the next worde
: moves to the end of the wordb
: moves to the beginning of the word
If you want to navigate quickly in your file:
ctrl+d
: page downctrl+u
: page upgg
: go to the first lineshift+g
: go to the last line<number>g
: go to line5g
: go to line 510g
: go to line 10
0
: move cursor to the beginning of the line$
: move cursor to the end of the line*
: find next occurrence of the word under the cursor#
: find previous occurrence of the word under the cursor%
: go to matching parenthesis/brackets
VIM editing
You don’t have to be in insert
mode to edit text. You can also edit portion of your file in normal
mode.
x
: delete the character under the cursorX
: delete the character to the left of the cursorr
: replace only one character under the cursorcw
: remove character until the end the word and switch toinsert
modec$
: remove character until the end of the line and switch toinsert
modedw
: delete the first word on the right side of the cursor and copies the contentdd
: delete the line and copies the contentyw
: copy wordyy
: copy linep
: paste the copied content.
: repeat the previous command
Search and replace
Still in normal
mode:
/
: search from top to bottom?
: search from bottom to top
Use n
and N
to search next and previous occurrence, respectively.
To replace, you can use the same syntax as sed
command:
:s/wordtoreplace/replacedword/
: only replace the first instance of “wordtoreplace” of the line where the cursor is:%s/wordtoreplace/replacedword/
: apply to the entire file:%s/wordtoreplace/replacedword/g
: apply to all instances
Other VIM commands
In normal
mode:
:w
: save file:q
: quit file edition:q!
: force quit:wq
: save and quit file edition:x
: save and quit the file edition:set invnumber
: display/hide line numbers:set invpaste
: format/do not format copied content (useful when copy pasting formatted code)
In insert
mode:
- press the
ctrl+n
for autocompletion
Undo
You can undo the last action by pressing the u
button in normal
mode.
If you want to undo the undo, you can press the ctrl+r
buttons.
Going further with VIM
I did not cover everything, only what’s needed to be able to perform basic edition on files using VIM.
There are lot of tutorials out there:
- VIM offial website
- VIM wikipedia
- openvim
- VIM adventures
:help tutor
: launch VIM own tutorial- …
Writing a SH script
Linux doesn’t care about extension. So if your script file name is foobar.txt
, you can still execute it.
However, it’s considered as best practice to have the extension .sh
for script files.
To run a script file, you cannot just type script.sh
, you will need to precede the script by the PATH to the script,
ie /path/to/script.sh
, or if it’s in the current directory ./script.sh
.
⚠️ Don’t forget to add the execute
permission to your file: chmod +x /path/to/script.sh
Shebang or hashpling #!
We need to tell the script what shell it’s going to run under as the user that will execute the script may not use the shell needed to execute the script.
For example, if we want to force bash, we need to add the following at the first line of the script:
|
|
If we want to use another shell, like Korn shell:
|
|
Exit
The standard way to exit a script file is by returning the number 0
:
|
|
If the script exists with anything other than 0 (a number between 1 and 255), that means there was an error.
Functions
To declare a function, all you need is to declare like this:
|
|
Variables
You can define variables simply like this:
|
|
If you want to add a parameter in your function:
|
|
$#
: represents the number of parameters$0
: represents the script filename$1
: represents the first parameter$2
: represents the second parameter$3
: represents the third parameter- …
Here a sample script that will display a help message if there are no parameter provided:
|
|
Loops
|
|
Unit test
There are tools like Bash Automated Testing System (or bats) that can help you test your scripts.
Customizing your terminal
You can customize your terminal by editing your .bashrc
, .zshrc
, … to:
- add aliases
- add environment variables
- add plugins
- …
There are lot of resources out there:
- oh-my-zsh is a good start to customize your terminal
- dotfiles on github list a set of setups you can inspire from for your computer setup
- my dotfiles
Your terminal can also feel like $HOME