Typical commands and tasks helpfull using Linux/Unix systems.
To test a website's performances:
ab -t 30 -c 5 %WEBSITE%
apachectl configtest
<VirtualHost %IP%:80> ServerName %YOUR.HOSTNAME.COM% Redirect permanent / http://%DESTINATIONDOMAIN%/%DESTINATIONPAGE% </VirtualHost>
%IP% can be * for any IP addresses.
<VirtualHost %IP%:80> ServerName %YOUR.HOSTNAME.COM% ProxyRequests Off DocumentRoot /var/www <Proxy *> Order deny,allow Allow from all </Proxy> ProxyPass / http://%DESTINATIONIP%/ ProxyPassReverse / http://%DESTINATIONIP%/ </VirtualHost>
%IP% can be * for any IP addresses.
To delete all .lck files in sub-folders:
for i in `find . -name '*.lck'`; do rm -f $i; done
rm !(%PATTER%)
for i in {1..254}; do echo $i; done
echo " " echo -n " Do that? y/n: " read ok echo " " if [ $ok = "y" ] then echo "OK!" fi if [ $ok = "n" ] then echo "NOK!" fi
$((%VARIABLE1%+%VARIABLE2%))
Convert hexadecimal number 2F to decimal 47:
echo $((0x2F))
or
echo $((16#2F))
Convert sexatrigesimal (base 36) to decimal:
echo $((36#XY))
${variable1}${bariable2}
Wait some time before go ahead:
wait %SECONDS%
cat /root/file.txt|
while read g
do
echo "$g is the content of the line...."
done
Append to the crontab line:
2>&1 > /dev/null
Or disable globally inserting this line at the beginning of crontab:
MAILTO=""
date -d "-2 days" +'%Y-%m-%d'
fdisk /dev/sdX mkfs -t FSTYPE /dev/sdXY
ls -l /dev/disk/by-uuid/ vol_id /dev/sdXY
apt-get install scsitools
or
wget http://rescan-scsi-bus.sh/
and then
rescan-scsi-bus.sh
hdparm -B 254 /dev/sdX
du %FOLDERROOT% | sort -rn
du -sh * | sort -h
du -scb
iostat -d -x 1 10
All files search:
find %PATH% -name '%FILNAME%*'
locate %FILE%
Locate a binary, source, and manual page files for a command:
whereis %PATTERN%
wich %PATTERN%
Get the public IP from ssh session:
curl ipconfig.me
Exclude lines containing pattern:
grep -v %PATTERN%
gzip -d FILE.gz
gzip file.tar
gzip -c FILE > FILE.gz
Before installing VMWareTools from .tar.gz file, install the following packages:
sudo apt-get install build-essential sudo apt-get install linux-headers-`uname -r`
For debian OS, do the following command:
export CC=/usr/bin/gcc-4.1
Install on ubuntu:
apt-get install lvm2 dmsetup mdadm reiserfsprogs xfsprogs
Format disks and create a new partition with file-system “8e - Linux LVM”:
fdisk /dev/sdX
Initialize disks for LVM:
pvcreate /dev/sdXY [/dev/sdWZ]
Create a new volume group:
vgcreate %VGNAME% /dev/sdXY [/dev/sdWZ]
Create a new logical volume that use all the free space:
lvcreate -n %LVNAME% -l 100%FREE %VGNAME%
The disk is now see as device under /dev/%VGNAME%/%LVNAME%
Create:
lvcreate -n %LVNAME% -l 100%FREE %VGNAME%
Extend:
lvextend -L%SIZE% /dev/%VGNAME%/%LVNAME%
lvextend -l +100%FREE /dev/%VGNAME%/%LVNAME%
Shrink [Warning]:
lvreduce -L%SIZE% /dev/%VGNAME%/%LVNAME%
Send an email from command line or script:
echo "This is the first line of the body\n and this the second" | mail -s "The subject" recipient1[,recipient2,...] -- -f senderaddress
In some versions -e switch need to be specified if \X characters are used.
On linux systems, mail command is in mail-utils package (debian and ubuntu)
arping -I %IF% -c %#OFRETRIES% %IPADDRESS%
Do traceroute and ping:
mtr www.google.com
Simple check:
nc -z %HOST% %PORT%
Scan a class C subnet and detect hosts:
for i in {1..254}; do ping -c 1 -W 1 %SUBNET%.$i | grep 'from' | cut -d' ' -f 4 | tr -d ':'; done
Get a bar-graph of network usage per interface (right or left key to change interface)
nload
To retrieve the public IP address:
curl ip.appspot.com
To retrieve the public IP address, the location and the ISP name:
curl -s "http://www.geody.com/geoip.php?ip=$(curl -s icanhazip.com)" | sed '/^IP:/!d;s/<[^>][^>]*>//g'
openssl req -verify -text -in cert.req
openssl x509 -text -in cert.req
openssl rand -base64 12
openssl pkcs12 -export -out %OUTPUTFILENAME%.pfx -inkey %PRIVATEKEY.KEY% -in %PUBLICKEY.CRT% -certfile %INTERMEDIATECERT.CRT%
To export the private key:
openssl pkcs12 -in %PFXFILE%.pfx -nocerts -out %PRIVATEKEY%.pem
To remove password for private key:
openssl rsa -in %PRIVATEKEY%.pem -out %PRIVATEKEY-NOPWD%.pem
To extract the certificate (public key), run the OpenSSL command:
openssl pkcs12 -in %PFXFILE%.pfx -clcerts -nokeys -out %PUBLICCERT%.pem
openssl rsa -in www.key -out new.key
installer xfsdump, xfsprogs ajouter ramdisk_size=2621440 dans les options de chargement du kernel (GRUB) mkfs -t xfs -q /dev/ram1 2621440 mount -t ramfs /dev/ram1 /path -o defaults,rw
sed -i -e 's/archive.ubuntu.com\|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list
dpkg --get-selections
Check running process:
pgrep %PROCNAME%
rename %EXPRESSION% %REPLACETEXT% %FILES%
rename -v .doc .txt *.doc
rsync -ave ssh --exclude="System Volume Information" --exclude="RECYCLER" . root@10.0.0.5:/var/path/
sed -e 's#LINE##g' sourcefile > destinationfile
sed '/^$/d' sourcefile > destinationfile
sed -e 's/^.*\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\).*/\1/g'
sed -e 'y/\t/ /;s/ *$//;s/^.* //' file
Print the Nth line of FILE:
sed -n 'N'p FILE
Print every Nth line starting from line M:
sed -n ‘M~N’p FILE
Print odd lines:
sed -n ‘2~1’p FILE
Print Mth line to Nth line:
sed -n ‘M,N’p FILE
Print the last line:
sed -n ‘$’p FILE
Print line corresponding the PATTERN (can be regex expression):
sed -n /PATTERN/p FILE
Print lines beetween PATTERN and Nth line:
sed -n ‘/PATTERN/,Np’ FILE
Print lines beetween Nth line and PATTERN:
sed -n ‘N,/PATTERN/p’ FILE
Print the lines which matches the pattern and next N lines following the matched line:
sed -n ‘/PATTERN/,+Np’ FILE
Delete the Nth line of FILE:
sed -n 'N'd FILE
Delete Mth line to Nth line:
sed -n ‘M,N’d FILE
Replace the first occurrence of ABC with DEF:
sed 's/ABC/DEF/' FILE
Replace all occurrences of ABC with DEF:
sed 's/ABC/DEF/g' FILE
Replace the 3th occurrence of ABC with DEF:
sed 's/ABC/DEF/3' FILE
Replace all occurrences of ABC with DEF and write output to stdout and OUTFILE:
sed -n 's/ABC/DEF/gpw OUTFILE' FILE
Replace ABC with DEF in lines containing GHI pattern:
sed '/GHI/s/ABC/DEF/g' FILE
Replace the last 4 characters of all lines with ABC:
sed 's/....$/ABC/'
Delete characters after # symbol and delete empty lines:
sed -e 's/#.*//;/^$/d' FILE
Convert DOS newlines (CR/LF) to Unix format:
sed 's/.$//' FILE
To modify/replace direct in a file:
sed -i -e 's/new/archive/g' FILE
Uncomment or add aliases in ~/.bashrc file
Execute the line N of the history record:
!%N%
Execute the command N lines back:
!-%N%
Execute the last command:
!!
Get the Nth argument of the last command.
!!:%N%
Get the last argument of the last command.
!!:$
or
!$
Reexecute the most recent COMMAND command, string from the start:
!%COMMAND%
Reexecute the most recent COMMAND command, string appearing anywhere in the command:
!?%COMMAND%
Keybinding | Description Control-a | Position cursor at the start of the line Control-e | Position the cursor after the last character of the line Control-k | Delete all text after (and including) cursor Control-u | Delete all text before (not including) cursor Control-w | Delete immediate WORD before cursor Control-d | When text exist => delete char under cursor (think delete). When no text => exit shell Control-r | Reverse index search, Control-r to search again Control-l | Clear display Control-c | Cancel current command, return empty prompt
... 2>&1 > /dev/null
snmpwalk -v 1 -c %COMMUNITY% %AGENTIP%
Sort on first numerical field:
sort -n
Sort on first numerical field from higest to smallest:
sort -rn
Connect on client server and do the following:
ssh-keygen scp .ssh/id_dsa.pub %USER%@%REMOTEHOST%:./id_dsa.pub ssh %USER%@%REMOTEHOST% mkdir .ssh chmod 700 .ssh cd .ssh touch authorized_keys chmod 600 authorized_keys cat ../id_dsa.pub >> authorized_keys rm ../id_dsa.pub quit
ssh -N -f %USER%@%PROXYHOST% -L[%BINDLOCALADDRESS%:]%LOCALLISTENINGPORT%:%REMOTEHOST%:%REMOTEHOSTPORT%
sudo adduser existing_username_to_add_to_admin_group admin
if [ "$1" = "ABC" ] then echo abort exit fi
dmidecode
dmidecode -s system-serial-number
for i in `ps aux | awk '/%PATTERN%/ {print $2;}'`; do kill $i; done
List all samba shares:
smbclient -L %SAMBASERVER%
lsof | grep deleted
locale-gen fr_CH.UTF-8
Verify that the locale is present:
locale -a
watch vmstat -sSM
Update
/etc/hostname /etc/hosts
and execute
/etc/init.d/hostname.sh
lsb_release -a uname -a
select-editor
apt-get install console-data dpkg-reconfigure console-setup
Comment the following line in ~/.bashrc file:
alias ls='ls --color...
or replace with:
alias ls=ls
tar -cvf file.tar filestotar
tar -czvf file.tar.gz filestotar
Create a local compressed tarball from remote host directory:
ssh %USER%@%HOST% “tar -cf - %/PATH/TO/TAR%” | gzip %FILENAME%.tar.gz
untar multiple files: for i in *.tar.gz; do tar -xvzf $i; done
Filter on one port:
tcpdump port %PORT%
Filter on more ports:
tcpdump port %PORT% and port %PORT2%
Filter to exclude port:
tcpdump port not %PORT%
Filter on host:
tcpdump host %HOST%
Filter on interface:
tcpdump -i %INTERFACE%
tcpdump -x -s 1500
To write results to file:
tcpdump -w %FILENAME%
To read the results from file:
tcpdump -r %FILENAME%
Read the standard input comparing adjacent lines and write a copy of each unique input line to the standard output:
... | uniq
To delete all repeated lines, need to sort before:
... | sort | uniq
usermod -a -G %GROUP% %USER%
Add
syntax off
in file ~/.vimrc
wc -w FILE
wc -l FILE
wc -m FILE
wc FILE
Recursively retrieve a website:
wget -r %URL%
The command /usr/sbin/sendmail -q forces the queue to be sent. Use the command mailq to tell what's stacked up in the queue
By default, sed prints every line. If it makes a substitution, the new text is printed instead of the old one. If you use an optional argument to sed, “sed -n,” it will not, by default, print any new lines. I'll cover this and other options later. When the “-n” option is used, the “p” flag will cause the modified line to be printed. Here is one way to duplicate the function of grep with sed:
<code> sed -n 's/pattern/&/p' file </code>
at (atrm atq)
wall
[ parameter FILE ] OR test parameter FILE
Where parameter can be any one of the following:
Examples Find out if file /etc/passwd file exists or not
Type the following commands: $ [ -f /etc/passwd ] && echo “File exists” || echo “File does not exists” $ [ -f /tmp/fileonetwo ] && echo “File exists” || echo “File does not exists” Find out if directory /var/logs exists or not
Type the following commands: $ [ -d /var/logs ] && echo “Directory exists” || echo “Directory does not exists” $ [ -d /dumper/fack ] && echo “Directory exists” || echo “Directory does not exists” You can use conditional expressions in a shell script:
#!/bin/bash FILE=$1
if [ -f $FILE ]; then
echo "File $FILE exists"
else
echo "File $FILE does not exists"
fi
-eq
is equal to
if [ "$a" -eq "$b" ]
-ne
is not equal to
if [ "$a" -ne "$b" ]
-gt
is greater than
if [ "$a" -gt "$b" ]
-ge
is greater than or equal to
if [ "$a" -ge "$b" ]
-lt
is less than
if [ "$a" -lt "$b" ]
-le
is less than or equal to
if [ "$a" -le "$b" ]
<
is less than (within double parentheses)
(("$a" < "$b"))
⇐
is less than or equal to (within double parentheses)
(("$a" <= "$b"))
>
is greater than (within double parentheses)
(("$a" > "$b"))
>=
is greater than or equal to (within double parentheses)
(("$a" >= "$b"))
count links: ls -ld
while true; do date; ssh <YOUR HOST HERE> "echo" && echo "HOST UP" && break; sleep 60; done
seq 1 255 | parallel -j+0 'nc -w 1 -z -v 192.168.1.{} 80'
echo "- - -" > /sys/class/scsi_host/host%ID%/scan
echo 1 > /sys/block/sda/device/rescan
#echo “1” > /sys/class/scsi_device/%DEVICE%/device/rescan
use fdisk to resize the partition by deleting the existing partion and creating a new, larger partition at the same starting cylinder
'pvresize -v' (the output of pvresize is not immediately clear; use the -v flag and compare vgdisplay's output of usable disk space for confirmation)
If you're uncomfortable resizing the disk partition or the space is not contiguous, simply create a new partition, 'pvcreate' and then vgextend.
The space is now available inside lvm, consume as normal.
find . -type f -newermt "2010-01-01" ! -newermt "2010-06-01"
split screen to multiple command lines: tmux
RDP through SSH
ssh -f -L3389:<RDP_HOST>:3389 <SSH_PROXY> "sleep 10" && rdesktop -T'<WINDOW_TITLE>' -uAdministrator -g800x600 -a8 -rsound:off -rclipboard:PRIMARYCLIPBOARD -5 localhost
Get server version:
curl -Is http://%HOST% | grep -E '^Server'
update-rc.d autorun.sh defaults
shuf -n4 /usr/share/dict/words | tr -d '\n'
Sort by column:
ps aux | sort -nk 6
Bckp file:
cp httpd.conf{,.bk}
save command output to image by unixmonkey21861
ifconfig | convert label:@- ip.png
Find out how much data is waiting to be written to disk
grep ^Dirty /proc/meminfo
Display line number: cat -n file.txt nl file.txt > file_numbered.txt
Create iso from cd (package wodim)
readom dev=/dev/scd0 f=/path/to/image.iso
Similarly, if you want to burn your newly creating ISO, stay away from 'dd', and use:
wodim -v -eject /path/to/image.iso
lower() { echo ${@,,}; }
upper() { echo ${@^^}; }
Echo a command, then execute it from All commands by hfs $ v () { echo “$@”; “$@”; }
You can use this in shell scripts to show which commands are actually run. Just prepend every “critical line” with “v˽”.
$TMP=/tmp
echo “Let me create a directory for you”
v mkdir $TMP/new
In scripts this can be more useful than “set -x”, because that can be very verbose with variable assignments etc.
Another nice use is if you prepend every “critical” command with “v”, then you can test your script by commenting out the actual execution.
Expand shortened URLs by atoponce (http://pthree.org) $ expandurl() { curl -sIL $1 | grep ^Location; } expandurl() { wget -S $1 2>&1 | grep ^Location; }
Clear filesystem memory cache by mariusbutuc (http://mariusbutuc.com) $ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
Run a command when a file is changed by anhpht $ while inotifywait -e modify /tmp/myfile; do firefox; done
Empty Bind9 cache by ironmarc $ rndc flush
/etc/init.d/nscd restart
openssl req -new -x509 -nodes -out server.crt -keyout server.key
Create a file server, listening in port 7000 by anhpht $ while true; do nc -l 7000 | tar -xvf -; done
At client side: tar c myfile | nc localhost 7000 ##Send file myfile to server tar c mydir | nc localhost 7000 ## Send directory mydir to server
Create a local compressed tarball from remote host directory ssh user@host “tar -cf - /path/to/dir” | gzip dir.tar.gz
find all symlinks to a file
find / -lname path/to/foo.txt
Exclude lines with patter:
grep -v %PATTERN%
rescan-scsi-bus.sh: wget http://rescan-scsi-bus.sh/
iSCSI: iscsiadm -m iface -I iface0 –op=new iscsiadm -m iface -I iface1 –op=new iscsiadm -m iface -I iface0 –op=update -n iface.hwaddress -v\ 00:16:3E:XX:XX:XX iscsiadm -m iface -I iface1 –op=update -n iface.hwaddress -v\ 00:16:3E:XX:XX:XX
The above example sets up two interfaces. You get the hardware address for the NICs you want to use by checking the output of the “ifconfig” command. The interfaces must each be configured with an IP address that can reach the iSCSI target. Update: If you want to script the process, you can source the ifcfg-ethX files for the interfaces you want to use. Then you can refer to the HWADDR variable, e.g. . /etc/sysconfig/network-scripts/ifcfg-eth1 iscsiadm -m iface -I iface1 –op=update -n iface.hwaddress -v $HWADDR
Now you're ready to do your iSCSI discovery and login. Ping the iSCSI group IP address from each NIC to make sure it's reachable, and then:
iscsiadm -m discovery -t st -p 10.X.X.X iscsiadm -m node --loginall=all iscsiadm -m session
Mount on reboot:
iscsiadm -m discovery -t st -p %TARGETIP% iscsiadm -m node --targetname "%TARGETLUNIQN%" --portal %TARGETIP% --login
mount -t cifs //server-name/share-name /mnt/cifs -o username=shareuser,password=sharepassword,domain=nixcraft
Change system language: /etc/default/locale:
LANG="en_US.UTF-8" LANGUAGE="en_US:en"
If need, edit ~/.pam_environment:
LANG=en_US Language=en_US
When I say broken “sudo dpkg-reconfigure locales” does not yield an interface that allows you to select and deselect locales. It simply generates the locales mentioned in “/var/lib/locales/supported.d/local” file. Therefore if you want to generate a bunch of locales you will need to add them to this file and re-run “dpkg-reconfigure locales”. NOTE: One locale per line. For a list of valid locales you can search through “/usr/share/i18n/SUPPORTED”.
echo 3 | tee /proc/sys/vm/drop_caches free -m
To make sure you always have X amount of Ram available (in kilobytes), run: sysctl -w vm.min_free_kbytes=X To empty the page cache and the dirty write buffers with all committed transactions, run: sync && sysctl -w vm.drop_caches=3
get year-month-day from date
DATE=`date +%Y-%m-%d`
get year-month-day hour:minute:second from date
DATE=`date +%Y-%m-%d:%H:%M:%S`
let “arithmetic expression”