Typical commands and tasks helpfull using Linux/Unix systems.

To test a website's performances:

ab -t 30 -c 5 %WEBSITE%

apachectl configtest

<VirtualHost %IP%:80>
ServerName %YOUR.HOSTNAME.COM%
Redirect permanent / http://%DESTINATIONDOMAIN%/%DESTINATIONPAGE%
</VirtualHost>

%IP% can be * for any IP addresses.

<VirtualHost %IP%:80>
ServerName %YOUR.HOSTNAME.COM%
ProxyRequests Off
DocumentRoot /var/www
<Proxy *>
   Order deny,allow
   Allow from all
</Proxy>

ProxyPass / http://%DESTINATIONIP%/
ProxyPassReverse / http://%DESTINATIONIP%/
</VirtualHost>

%IP% can be * for any IP addresses.

To delete all .lck files in sub-folders:

for i in `find . -name '*.lck'`; do rm -f $i; done

rm !(%PATTER%)

for i in {1..254}; do echo $i; done

echo " "
echo -n " Do that? y/n: "
read ok
echo " "
if [ $ok = "y" ]
then
  echo "OK!"
fi
if [ $ok = "n" ]
then
  echo "NOK!"
fi

$((%VARIABLE1%+%VARIABLE2%))

Convert hexadecimal number 2F to decimal 47:

echo $((0x2F))

or

echo $((16#2F))

Convert sexatrigesimal (base 36) to decimal:

echo $((36#XY))

${variable1}${bariable2} 

Wait some time before go ahead:

wait %SECONDS%

cat /root/file.txt|
while read g
do
         
        echo "$g is the content of the line...."

done

Append to the crontab line:

2>&1 > /dev/null

Or disable globally inserting this line at the beginning of crontab:

MAILTO=""

date -d "-2 days" +'%Y-%m-%d' 

fdisk /dev/sdX
mkfs -t FSTYPE /dev/sdXY 

ls -l /dev/disk/by-uuid/
vol_id /dev/sdXY 

apt-get install scsitools

or

wget http://rescan-scsi-bus.sh/

and then

rescan-scsi-bus.sh

hdparm -B 254 /dev/sdX

du %FOLDERROOT% | sort -rn

du -sh * | sort -h

du -scb

iostat -d -x 1 10

All files search:

find %PATH% -name '%FILNAME%*'

locate %FILE%

Locate a binary, source, and manual page files for a command:

whereis %PATTERN%

wich %PATTERN%

Get the public IP from ssh session:

curl ipconfig.me

Exclude lines containing pattern:

grep -v %PATTERN%

gzip -d FILE.gz

gzip file.tar 

gzip -c FILE > FILE.gz

Before installing VMWareTools from .tar.gz file, install the following packages:

sudo apt-get install build-essential
sudo apt-get install linux-headers-`uname -r`

For debian OS, do the following command:

export CC=/usr/bin/gcc-4.1

Install on ubuntu:

apt-get install lvm2 dmsetup mdadm reiserfsprogs xfsprogs

Format disks and create a new partition with file-system “8e - Linux LVM”:

fdisk /dev/sdX

Initialize disks for LVM:

pvcreate /dev/sdXY [/dev/sdWZ]

Create a new volume group:

vgcreate %VGNAME% /dev/sdXY [/dev/sdWZ]

Create a new logical volume that use all the free space:

lvcreate -n %LVNAME% -l 100%FREE %VGNAME%

The disk is now see as device under /dev/%VGNAME%/%LVNAME%

Create:

lvcreate -n %LVNAME% -l 100%FREE %VGNAME%

Extend:

lvextend -L%SIZE% /dev/%VGNAME%/%LVNAME%

lvextend -l +100%FREE /dev/%VGNAME%/%LVNAME%

Shrink [Warning]:

lvreduce -L%SIZE% /dev/%VGNAME%/%LVNAME%

Send an email from command line or script:

echo "This is the first line of the body\n and this the second" | mail -s "The subject" recipient1[,recipient2,...] -- -f senderaddress

In some versions -e switch need to be specified if \X characters are used.

On linux systems, mail command is in mail-utils package (debian and ubuntu)

arping -I %IF% -c %#OFRETRIES% %IPADDRESS%

Do traceroute and ping:

mtr www.google.com

Simple check:

nc -z %HOST% %PORT%

Scan a class C subnet and detect hosts:

for i in {1..254}; do ping -c 1 -W 1 %SUBNET%.$i | grep 'from' | cut -d' ' -f 4 | tr -d ':'; done

Get a bar-graph of network usage per interface (right or left key to change interface)

nload

To retrieve the public IP address:

curl ip.appspot.com

To retrieve the public IP address, the location and the ISP name:

curl -s "http://www.geody.com/geoip.php?ip=$(curl -s icanhazip.com)" | sed '/^IP:/!d;s/<[^>][^>]*>//g'

openssl req -verify -text -in cert.req

openssl x509 -text -in cert.req

openssl rand -base64 12

openssl pkcs12 -export -out %OUTPUTFILENAME%.pfx -inkey %PRIVATEKEY.KEY% -in %PUBLICKEY.CRT% -certfile %INTERMEDIATECERT.CRT%

To export the private key:

openssl pkcs12 -in %PFXFILE%.pfx  -nocerts -out %PRIVATEKEY%.pem

To remove password for private key:

openssl rsa -in %PRIVATEKEY%.pem -out %PRIVATEKEY-NOPWD%.pem

To extract the certificate (public key), run the OpenSSL command:

openssl pkcs12 -in %PFXFILE%.pfx -clcerts -nokeys -out %PUBLICCERT%.pem

openssl rsa -in www.key -out new.key

installer xfsdump, xfsprogs
ajouter   ramdisk_size=2621440 dans les options de chargement du kernel (GRUB)
mkfs -t xfs -q /dev/ram1 2621440
mount -t ramfs /dev/ram1 /path -o defaults,rw

sed -i -e 's/archive.ubuntu.com\|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list

dpkg --get-selections

Check running process:

pgrep %PROCNAME%

rename %EXPRESSION% %REPLACETEXT% %FILES%

rename -v .doc .txt *.doc

rsync -ave ssh --exclude="System Volume Information" --exclude="RECYCLER" . root@10.0.0.5:/var/path/ 

sed -e 's#LINE##g' sourcefile  > destinationfile 

sed '/^$/d' sourcefile > destinationfile

sed -e 's/^.*\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\).*/\1/g' 

sed -e 'y/\t/ /;s/ *$//;s/^.* //' file

Print the Nth line of FILE:

sed -n 'N'p FILE

Print every Nth line starting from line M:

sed -n ‘M~N’p FILE

Print odd lines:

sed -n ‘2~1’p FILE

Print Mth line to Nth line:

sed -n ‘M,N’p FILE

Print the last line:

sed -n ‘$’p FILE

Print line corresponding the PATTERN (can be regex expression):

sed -n /PATTERN/p FILE

Print lines beetween PATTERN and Nth line:

sed -n ‘/PATTERN/,Np’ FILE

Print lines beetween Nth line and PATTERN:

sed -n ‘N,/PATTERN/p’ FILE

Print the lines which matches the pattern and next N lines following the matched line:

sed -n ‘/PATTERN/,+Np’ FILE

Delete the Nth line of FILE:

sed -n 'N'd FILE

Delete Mth line to Nth line:

sed -n ‘M,N’d FILE

Replace the first occurrence of ABC with DEF:

sed 's/ABC/DEF/' FILE

Replace all occurrences of ABC with DEF:

sed 's/ABC/DEF/g' FILE

Replace the 3th occurrence of ABC with DEF:

sed 's/ABC/DEF/3' FILE

Replace all occurrences of ABC with DEF and write output to stdout and OUTFILE:

sed -n 's/ABC/DEF/gpw OUTFILE' FILE

Replace ABC with DEF in lines containing GHI pattern:

sed '/GHI/s/ABC/DEF/g' FILE

Replace the last 4 characters of all lines with ABC:

sed 's/....$/ABC/'

Delete characters after # symbol and delete empty lines:

sed -e 's/#.*//;/^$/d' FILE

Convert DOS newlines (CR/LF) to Unix format:

sed 's/.$//' FILE

To modify/replace direct in a file:

sed -i -e 's/new/archive/g' FILE

Uncomment or add aliases in ~/.bashrc file

Execute the line N of the history record:

!%N%

Execute the command N lines back:

!-%N%

Execute the last command:

!!

Get the Nth argument of the last command.

!!:%N%

Get the last argument of the last command.

!!:$

or

!$

Reexecute the most recent COMMAND command, string from the start:

!%COMMAND%

Reexecute the most recent COMMAND command, string appearing anywhere in the command:

!?%COMMAND%

Keybinding | Description
 Control-a | Position cursor at the start of the line
 Control-e | Position the cursor after the last character of the line
 Control-k | Delete all text after (and including) cursor
 Control-u | Delete all text before (not including) cursor
 Control-w | Delete immediate WORD before cursor
 Control-d | When text exist => delete char under cursor (think delete). When no text => exit shell
 Control-r | Reverse index search, Control-r to search again
 Control-l | Clear display
 Control-c | Cancel current command, return empty prompt

... 2>&1 > /dev/null

snmpwalk -v 1 -c %COMMUNITY% %AGENTIP%

Sort on first numerical field:

sort -n

Sort on first numerical field from higest to smallest:

sort -rn

Connect on client server and do the following:

ssh-keygen
scp .ssh/id_dsa.pub %USER%@%REMOTEHOST%:./id_dsa.pub
ssh %USER%@%REMOTEHOST%
mkdir .ssh
chmod 700 .ssh
cd .ssh
touch authorized_keys
chmod 600 authorized_keys
cat ../id_dsa.pub >> authorized_keys
rm ../id_dsa.pub
quit

ssh -N -f %USER%@%PROXYHOST% -L[%BINDLOCALADDRESS%:]%LOCALLISTENINGPORT%:%REMOTEHOST%:%REMOTEHOSTPORT%

sudo adduser existing_username_to_add_to_admin_group admin

if [ "$1" = "ABC" ]
then
echo abort
exit
fi

dmidecode 

dmidecode -s system-serial-number 

for i in `ps aux | awk '/%PATTERN%/ {print $2;}'`; do kill $i; done

List all samba shares:

smbclient -L %SAMBASERVER%

lsof | grep deleted

locale-gen fr_CH.UTF-8

Verify that the locale is present:

locale -a

watch vmstat -sSM

Update

/etc/hostname
/etc/hosts

and execute

/etc/init.d/hostname.sh

lsb_release -a
uname -a 

select-editor

apt-get install console-data
dpkg-reconfigure console-setup

Comment the following line in ~/.bashrc file:

alias ls='ls --color...

or replace with:

alias ls=ls

tar -cvf file.tar filestotar 

tar -czvf file.tar.gz filestotar 

Create a local compressed tarball from remote host directory:

ssh %USER%@%HOST% “tar -cf - %/PATH/TO/TAR%” | gzip %FILENAME%.tar.gz

untar multiple files: for i in *.tar.gz; do tar -xvzf $i; done

Filter on one port:

tcpdump port %PORT%

Filter on more ports:

tcpdump port %PORT% and port %PORT2%

Filter to exclude port:

tcpdump port not %PORT%

Filter on host:

tcpdump host %HOST%

Filter on interface:

tcpdump -i %INTERFACE%

tcpdump -x -s 1500

To write results to file:

tcpdump -w %FILENAME%

To read the results from file:

tcpdump -r %FILENAME%

Read the standard input comparing adjacent lines and write a copy of each unique input line to the standard output:

... | uniq

To delete all repeated lines, need to sort before:

... | sort | uniq

usermod -a -G %GROUP% %USER%

Add

syntax off

in file ~/.vimrc

wc -w FILE

wc -l FILE

wc -m FILE

wc FILE

Recursively retrieve a website:

wget -r %URL%

The command /usr/sbin/sendmail -q forces the queue to be sent. Use the command mailq to tell what's stacked up in the queue

By default, sed prints every line. If it makes a substitution, the new text is printed instead of the old one. If you use an optional argument to sed, “sed -n,” it will not, by default, print any new lines. I'll cover this and other options later. When the “-n” option is used, the “p” flag will cause the modified line to be printed. Here is one way to duplicate the function of grep with sed:

 <code> sed -n 's/pattern/&/p' file </code>

at (atrm atq)
wall

[ parameter FILE ] OR test parameter FILE

Where parameter can be any one of the following:

• -e: Returns true value if file exists
• -f: Return true value if file exists and regular file
• -r: Return true value if file exists and is readable
• -w: Return true value if file exists and is writable
• -x: Return true value if file exists and is executable
• -d: Return true value if exists and is a directory

Examples Find out if file /etc/passwd file exists or not

Type the following commands: $ [ -f /etc/passwd ] && echo “File exists” || echo “File does not exists” $ [ -f /tmp/fileonetwo ] && echo “File exists” || echo “File does not exists” Find out if directory /var/logs exists or not

Type the following commands: $ [ -d /var/logs ] && echo “Directory exists” || echo “Directory does not exists” $ [ -d /dumper/fack ] && echo “Directory exists” || echo “Directory does not exists” You can use conditional expressions in a shell script:

#!/bin/bash FILE=$1

if [ -f $FILE ]; then

 echo "File $FILE exists"

else

 echo "File $FILE does not exists"

fi

-eq

  is equal to

  if [ "$a" -eq "$b" ]

-ne

  is not equal to

  if [ "$a" -ne "$b" ]

-gt

  is greater than

  if [ "$a" -gt "$b" ]

-ge

  is greater than or equal to

  if [ "$a" -ge "$b" ]

-lt

  is less than

  if [ "$a" -lt "$b" ]

-le

  is less than or equal to

  if [ "$a" -le "$b" ]

<

  is less than (within double parentheses)

  (("$a" < "$b"))

⇐

  is less than or equal to (within double parentheses)

  (("$a" <= "$b"))

>

  is greater than (within double parentheses)

  (("$a" > "$b"))

>=

  is greater than or equal to (within double parentheses)

  (("$a" >= "$b"))

count links: ls -ld

while true; do date; ssh <YOUR HOST HERE> "echo" && echo "HOST UP" && break; sleep 60; done

seq 1 255 | parallel -j+0 'nc -w 1 -z -v 192.168.1.{} 80'

 echo "- - -" > /sys/class/scsi_host/host%ID%/scan

echo 1 > /sys/block/sda/device/rescan

#echo “1” > /sys/class/scsi_device/%DEVICE%/device/rescan
use fdisk to resize the partition by deleting the existing partion and creating a new, larger partition at the same starting cylinder 'pvresize -v' (the output of pvresize is not immediately clear; use the -v flag and compare vgdisplay's output of usable disk space for confirmation) If you're uncomfortable resizing the disk partition or the space is not contiguous, simply create a new partition, 'pvcreate' and then vgextend. The space is now available inside lvm, consume as normal.

find . -type f -newermt "2010-01-01" ! -newermt "2010-06-01"

split screen to multiple command lines: tmux

RDP through SSH

ssh -f -L3389:<RDP_HOST>:3389 <SSH_PROXY> "sleep 10" && rdesktop -T'<WINDOW_TITLE>' -uAdministrator -g800x600 -a8 -rsound:off -rclipboard:PRIMARYCLIPBOARD -5 localhost

Get server version:

curl -Is http://%HOST% | grep -E '^Server'

update-rc.d autorun.sh defaults

shuf -n4 /usr/share/dict/words | tr -d '\n'

Sort by column:

ps aux | sort -nk 6

Bckp file:

cp httpd.conf{,.bk}

save command output to image by unixmonkey21861

ifconfig | convert label:@- ip.png

Find out how much data is waiting to be written to disk

grep ^Dirty /proc/meminfo

Display line number: cat -n file.txt nl file.txt > file_numbered.txt

Create iso from cd (package wodim)

readom dev=/dev/scd0 f=/path/to/image.iso

Similarly, if you want to burn your newly creating ISO, stay away from 'dd', and use:

wodim -v -eject /path/to/image.iso

lower() { echo ${@,,}; }

upper() { echo ${@^^}; }

Echo a command, then execute it from All commands by hfs $ v () { echo “$@”; “$@”; }

You can use this in shell scripts to show which commands are actually run. Just prepend every “critical line” with “v˽”.

$TMP=/tmp

echo “Let me create a directory for you”

v mkdir $TMP/new

In scripts this can be more useful than “set -x”, because that can be very verbose with variable assignments etc.

Another nice use is if you prepend every “critical” command with “v”, then you can test your script by commenting out the actual execution.

Expand shortened URLs by atoponce (http://pthree.org) $ expandurl() { curl -sIL $1 | grep ^Location; } expandurl() { wget -S $1 2>&1 | grep ^Location; }

Clear filesystem memory cache by mariusbutuc (http://mariusbutuc.com) $ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

Run a command when a file is changed by anhpht $ while inotifywait -e modify /tmp/myfile; do firefox; done

Empty Bind9 cache by ironmarc $ rndc flush

/etc/init.d/nscd restart

openssl req -new -x509 -nodes -out server.crt -keyout server.key

Create a file server, listening in port 7000 by anhpht $ while true; do nc -l 7000 | tar -xvf -; done

At client side: tar c myfile | nc localhost 7000 ##Send file myfile to server tar c mydir | nc localhost 7000 ## Send directory mydir to server

Create a local compressed tarball from remote host directory ssh user@host “tar -cf - /path/to/dir” | gzip dir.tar.gz

find all symlinks to a file

find / -lname path/to/foo.txt

Exclude lines with patter:

grep -v %PATTERN%

rescan-scsi-bus.sh: wget http://rescan-scsi-bus.sh/

iSCSI: iscsiadm -m iface -I iface0 –op=new iscsiadm -m iface -I iface1 –op=new iscsiadm -m iface -I iface0 –op=update -n iface.hwaddress -v\ 00:16:3E:XX:XX:XX iscsiadm -m iface -I iface1 –op=update -n iface.hwaddress -v\ 00:16:3E:XX:XX:XX

The above example sets up two interfaces. You get the hardware address for the NICs you want to use by checking the output of the “ifconfig” command. The interfaces must each be configured with an IP address that can reach the iSCSI target. Update: If you want to script the process, you can source the ifcfg-ethX files for the interfaces you want to use. Then you can refer to the HWADDR variable, e.g. . /etc/sysconfig/network-scripts/ifcfg-eth1 iscsiadm -m iface -I iface1 –op=update -n iface.hwaddress -v $HWADDR

Now you're ready to do your iSCSI discovery and login. Ping the iSCSI group IP address from each NIC to make sure it's reachable, and then:

iscsiadm -m discovery -t st -p 10.X.X.X
iscsiadm -m node --loginall=all
iscsiadm -m session

Mount on reboot:

iscsiadm -m discovery -t st -p %TARGETIP%
iscsiadm -m node --targetname "%TARGETLUNIQN%" --portal %TARGETIP% --login

mount -t cifs //server-name/share-name /mnt/cifs -o username=shareuser,password=sharepassword,domain=nixcraft

Change system language: /etc/default/locale:

LANG="en_US.UTF-8"
LANGUAGE="en_US:en"

If need, edit ~/.pam_environment:

LANG=en_US
Language=en_US

When I say broken “sudo dpkg-reconfigure locales” does not yield an interface that allows you to select and deselect locales. It simply generates the locales mentioned in “/var/lib/locales/supported.d/local” file. Therefore if you want to generate a bunch of locales you will need to add them to this file and re-run “dpkg-reconfigure locales”. NOTE: One locale per line. For a list of valid locales you can search through “/usr/share/i18n/SUPPORTED”.

echo 3 | tee /proc/sys/vm/drop_caches
free -m

To make sure you always have X amount of Ram available (in kilobytes), run: sysctl -w vm.min_free_kbytes=X To empty the page cache and the dirty write buffers with all committed transactions, run: sync && sysctl -w vm.drop_caches=3

get year-month-day from date

DATE=`date +%Y-%m-%d`

get year-month-day hour:minute:second from date

DATE=`date +%Y-%m-%d:%H:%M:%S`

let “arithmetic expression”