Bash for Everyone — Part 2

Part-2 — Learn Core Linux Commands

  • Working with commands – type, which, help, man, info, whatis, alias
  • Exploring the file system commands – ls, pwd, file, more, less
  • Manipulating files and directories commands – cp, mv, mkdir, rm
  • Redirection commands – cat, sort, uniq, grep, wc, head, tail
  • Permissions commands – id, cdmod, su, sudo, passwd
  • Processes commands – ps, top, bg, fg, kill, killall, shutdown
  • Environment commands – printenv, set, vim
  • Networking commands – ping, traceout, dig, ip, netstat, wget, curl, ifconfig etc
  • Searching of files commands – locate, find
  • text processing commands – cut, sed, awk, parallel
  • more commands – clear, history

Working with commands 

type – Display’s command type

man type //Type Command manual page 
type commands

which – Display which program will be executed

man which //Which command manual p
which ls

help – Get help

help
help cd
mkdir --help

man – Display manual pages

info – Display commands info entry

man info 
info coreutils

whatis – very brief description of the command

man whatis 
whatis ls

alias – Create an alias for a command.

alias l.='ls -d .* --color=tty'
alias ll='ls -l --color=tty'
alias ls='ls --color=tty'
unalias which //removing alias

 

Exploring the file system commands 

ls – list directory contents

man ls

Useful ls commands

ls -lt --reversels
ls -li
ls > list.txt
ls -l
LC_ALL=C ls
ls -l "some_file"

lsof – list open files

pwd – Return working directory name

man pwd

file – Determine file types

man file
file filename

more – file perusal filter for crt viewing

man more

less – view file content

a comic about less

Here is some of the text in it

less is a pager

that means it lets you view (not edit) text files or piped in text

man uses your pager (usually less) to display man pages

many vim shortcuts
work in less

/: search
n/N: next/prev match
g/G beginning/end of file

 

Manipulating files and directories Commands 

cp – copy files and directories

man cp
cp file.html /usr/local/bin

mv – move and rename files and directories

man mv
mv file.html /usr/localbin // moving files
mv file.html file2.html //renaming files

mkdir – create directories

man mkdir 
mkdir somedirectory
mkdir dir1 dir2 dir

rm – remove files and directories

Caution: Be careful with rm

man rm
rm file.txt
rm -i //interective - if this option is not defined, rm will delete files silently.
rm -r //recursive recursively delete directories.
rm -f //force delete.
rm -v //Display informative messages.
rm -rf file1 dir1 //if nither file1 or dir1 exists rm will countinue silently.

 

Redirection Commands 

Redirection makes it possible to control where the output of command goes to, and where the input of command comes from.

stdin - standard input stream (eg- keyboard)
stdout - standard output stream (eg- monitor)
stderr - standard error output.
# Below cat-command will execute and redirect its error to (stderr) #to the bit bucket
cat file.txt 2>/dev/null
# below echo-command will execute and redirect its normal outout (stdout).
echo "there was an error" 1>&2

cat – concatenate files

man cat #Manual page
cat 1.txt 2.txt > new.txt
cat >new.txt 1.txt 2.txt
>new.txt cat 1.txt 2.txt

sort – Sort or merge records (lines) of text and binary files.

man sort
cat -n file.txt // file cat with no of lines.
cat company_ip | sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n ipaddr.list

Wow, that’s ugly. Here it is in the old format:

cat company_ip | sort -t. +0n -1 +1n -2 +2n -3 +3n -

uniq – report or omit repeated lines

man uniq

grep – print matching pattern

man grep
grep root /etc/passwd
grep -n root /etc/passwd
grep -v bash /etc/passwd | grep -v nologin
grep -c false /etc/passwd
grep -i ps ~/.bash* | grep -v history

wc – printv newline, word, and byte count for each file

man wc

head – output first part of the file

tail – output last part of the file

Permission Commands 

id – Display user identity

man id

chmod – change a file’s mode

man chmod
chmod u+x script.sh
chmod +x script.sh

su – Substitute user identity or run the shell as another user

man su

sudo – Excute a shell as another user

man sudo

passwd – Modify a user’s password

man passwd

 

Processes Commands 

ps – Report current process

ps x
ps aux
ps -ef
ps -ef | grep stuck_process
kill -9 5607

When a process starts up several instances, killall might be easier. It takes the same option as the kill command but applies on all instances of a given process.

top – Display task

bg – put a job in the background

fg – put a job in the foreground

kill – send a signal to a process

killall – kill processes by name

 

Environment commands 

printenv – print all or part of the environment

Env and printenv commands used to display the environment variable.

printenv or env
man printenv 
printenv | less 
printenv USER

set – set shell options

set | less
set -o // display all shell options

vim – ViIMproved. a programmer’s text editor.

man vim

Benefits of using vim

vim is always available & vim is lightweight and fast

vi filname-txt
Enter "i" to edit
:q to exit and save
:q! to force exit and save
o - The line below the current line.
O - The line above the current line.

Networking Commands —

Important networking files within the local machine.

  • /etc/hosts — Name to the Ip address
  • /etc/networks — Network name to the IP address
  • /etc/protocol — Protocol name to the Protocol number.
  • /etc/services — TCP/UDP names to the port number.

ping – Send an ICMP ECHO_REQUEST to network hosts

man ping

traceroute – print the route packets trace to a network host, Route taken by packets to a specfic Ip Address.

man traceroute

Dig – DNS lookup Utility

netstat – how network status, what connection is active between the local machine and another network machine.

man netstat 
netstat -ie
netstat -r

netcat – Netcat is a simple Unix utility which reads and writes data across network connections,

Iptable – adminstration tool for IPv4/IPv6 packet filtering and NAT.

Ip – IP is the transport layer protocol used by the internet protocol family.

wget – The non-interactive network downloader

man wget

curl – transfer a URL

man curl

Getting subdomains from curl using certspotter.com

curl -s https://certspotter.com/api/v0/certs\?domain\=deliveroo.co.uk | jq '.[].dns_names[]' | sed 's/\*\.//g' | tr -d "\"" | sort -u

Searching for files commands —

locate – locate the file by name

man locate 
locate bin/zip
locate zip | grep bin

find – search for filesman find

find ~
find ~ | wc -l
find ~ -type d | wc -l
find ~ -type f | wc -l
find ~ -type f -name "*.JPG" -size +1M | wc -l 840

text processing commands

cut — cut out a selected portion of each line of a file.

man cut

sed – Stream Editor is used to perform basic transformation on read text from a file or a pipe. sed is also sometimes known as bash editor.

http://www.pement.org/sed/sed1line.txt

awk – pattern-directed scanning and processing language

AWK: Effective AWK Programming: A User’s Guide for GNU Awk

the basic function of awk is to search files for lines or other text unit text containing one or more pattern. when a line matches one of the patterns, special action is performed on that line.

awk 'EXPRESSION { PROGRAM }' file(s)

The variables $1, $2, $3, …, $N hold the values of the first, second, third until the last field of an input line. The variable $0 (zero) holds the value of the entire line.

man awk
ls -l | awk '{ print $5 $9 }'
history | awk 'BEGIN {FS="[ \t]+|\\|"} {print $3}' | sort | uniq -c | sort -nr | head
Remove duplicate lines: awk '!a[$0]++'

Parallel

We can use the parallel command to resolve the multiple javascript URLs present in a text file.

we can use TomNomNom way back URL to get javascript files URLs.

waybackurls deliveroo.com | grep ".js" > deliveroo-js.txt
cat deliveroo-js.txt | parallel -j50 -q curl -w 'Status:%{http_code}\t Size:%{size_download}\t %{url_effective}\n' -o /dev/null -sk

More commands

clear – clear the terminal

man clear

History – Display the content of the history list

histroy | less
!88 - bash will expand “!88” into the contents of the 88th line in the history list
!! - Repeat the last command

Git — the stupid content tracker

Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both

high-level operations and full access to internals.

One-Liners

ASN — An autonomous system number (ASN) is a unique number assigned to an autonomous system (AS) by the Internet Assigned Numbers Authority (IANA).

ASN Example : - AS63086

https://iptoasn.com/

CIDR(Classless Inter-Domain Routing or supernetting ) — is a way to allow more flexible allocation of Internet Protocol (IP) addresses than was possible with the original system of IP address classes.

A CIDR network address looks like this under IPv4:

193.30.250.00/18

https://www.cidr-report.org/as2.0/autnums.html

Get CIDR from ASN numbers.

whois -h whois.radb.net -- '-i origin AS63086' | grep -Eo "([0-9.]+){4}/[0-9]+" | head

CIDR to IP addresses using nmap

nmap -sL 104.36.192.0/24 | grep “Nmap scan report” | awk ‘{print $NF}’

Finding Up hosts using NMAP.

nmap -sP 104.36.192.0/21 -oG uber-ips.txt

Grep fro UP hosts only.

cat uber-ips.txt | grep Up | cut -d” ” -f2

Saving UP hosts as uber-up-hosts.txt

Running masscan on uber-up-hosts.txt

masscan -iL uber-up-hosts.txt -p80,443,8080,8000,9000,8888,9999 --rate 10000 --open

Command line basic shortcuts

ctrl + a – move cursor to the begining of the line
ctrl + e – move cursor to the end of the line.
Alt+f – move one word forward
Alt+b Move cursor one work backword
ctrl+l- clear the clean (clear command alternative)

#Personal alias
#.bashrc file
alias adbf="adb forward tcp:31415 tcp:31415"
alias pt="source ./venv/bin/activate"
alias aa="work/src/github.com/aquatone"
alias convert="/opt/tools/convert.sh"
alias shot="python /Users/sahil/opt/recon/webscreenshot/webscreenshot.py"
alias mobsf="docker run -it -p 8000:8000 opensecurity/mobile-security-framework-mobsf:latest"
alias th="trufflehog"
alias goaltdns="goaltdns -w /work/src/github.com/subfinder/goaltdns/words.txt"
alias ds="/work/src/github.com/eur0pa/dirsearch-go/cmd/dirsearch/dirsearch -2 -f -waf -t 50  -w /recon/dir/personal.txt -u"
alias dss="work/src/github.com/eur0pa/dirsearch-go/cmd/dirsearch/dirsearch  -w /recon/dir/personal.txt -u"
alias ds1="/work/src/github.com/eur0pa/dirsearch-go/cmd/dirsearch/dirsearch -2 -f -waf -t 50  -w /recon/dir/personal.txt -L"
alias grep="grep --color=auto"
alias py="python"
alias py3="python3"
alias k="knockpy"
alias dir1="python3 /recon/dir/dir.py -w recon/dir/personal.txt -e * -L"
alias dirs="python3 /recon/dir/dir.py -e *"
alias dir="python3 recon/dir/dir.py -w opt/recon/dir/personal.txt -e * -u"
alias wc="wc -l"
alias sf="subfinder -nW -silent"
alias s="screen"
alias mkdir="mkdir -pv"
alias recon="cd opt/recon"
alias turbo="python /recon/turbo/turbo.py -d"
alias bucket="ruby /recon/bucket/bucket.rb"
alias l.='ls -d .* --color=tty'
alias ll='ls -l --color=tty'
alias ls='ls --color=tty'
alias vi='vim'
alias which='alias | /usr/bin/which --tty-only --read-alias --show- dot --show-tilde'

these are the only tip of the iceberg,

more one-liners?

References

Thanks to all of the following peoples for creating awesome content.

Bash Cookbook by Carl Albing, JP Vossen, and Cameron Newham
The Linux Command Line by William Shott
Penetration Testing with the Bash Shell by Keith Makan

Closing Gifs.

 

One thought on “Bash for Everyone — Part 2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s