Unix

Where it all started

Wednesday, Sep 9, 2020

Unix

User Management

Warning: Although this account disables root login via password entry, it does not disable root login via SSH keys. To disable SSH access to the root account, set PermitRootLogin no in /etc/ssh/sshd_config

Using coreutils on macOS

Note: If you're using these snippets make sure you've installed coreutils

date

The date command lets you get the date in your terminal. It contains the -I flag which allows you to get the date in ISO8601 format. -Is to print the ISO8601 date as well as the current time up to seconds.

Examples

disown

If you have a process running in the background of your terminal, you can release it. The process will continue, retaining the original PID, but it will no longer be part of your terminal session.

# disown the process with job id %2
disown %2

stat

You can use stat to get information about a file.

du

The du command stands for disk usage. It can tell you information about how much space a given file on your computer is taking up.

Viewing the size of a directory

Note: For file sizes, du rounds up to the nearest block. If there is a file that is less than 1TB in size, du will still report the size as 1T when the flag -BT is specified.

You can also set the environment variable DU_BLOCK_SIZE to have a default setting for the size of blocks reported by du

lsof

The lsof command stands for list open files. On the surface, this doesn't seem useful, you'd know what files are open on your computer. However, UNIX thinks of everything as a file. This includes sockets, ports, etc.

mkdir

The -m flag allows you to specify the octal permissions code for the files inside the directory to be created.

head

By default, head will return the first 10 lines of the specified file. You can use the -n flag to specify the number of lines that should be returned.

tail

By default, tail will return the last 10 lines of the specified file. You can use the -n flag to specify the number of lines that should be returned.

gzip

Although more people are familiar with the .zip extention written by microsoft, there also exists the GNU zip .gz extension, which has a slightly higher performance benchmark in comarison to .zip.

dig

The UNIX dig command, which stands for "Domain Information Groper" is a program used to query DNS servers for the records held by domain names.

dig @1.1.1.1 helpful.wiki A +short
# => 185.199.108.153

tar

tar is a tool used for archiving files onto tape drives, which is why the program stands for tape archive. Today it is still used to create a compressed copy of a directory. tar is used to create tarballs, a term used to denote an archive .tar file.

Useful flags

Short form Long form Function
-c --create Create a new .tar tarball
-r --append Append files to a existing tarball
-t --list List the contents of a tarball
-u --update Append a file to the archive, even if an existing version of that file is already in the archive
-x --extract Extract the files contained within an archive
N/A --delete Delete the file from the archive
-k --keep-old-files Don't replace existing files when extracting files from an archive
-f --file Specify the archive file to be created, or the one to be extracted from
-z --gzip Filter the archive command through gzip
-a --auto Use the archive file's suffix to automatically determine which compression program to use
-v --verbose Verbosely list files as they are being processed.
-w --interactive Ask for confirmation for every action
-h --dereference Follow any symlinks and archive the files they point to
-C --dir Change to a different directory before performing any operations
-O --to-stdout Extract files to standard output
-P --absolute Don't strip leading slashes from file names when creating an archive
-N --newer Only store files newer than DATE, which, if starting with a / or a . will be calculated in reference to the file specified as DATE.
N/A --utc Print file modification times in UTC
N/A --exclude-vcs Exclude version control systems like .git
N/A --excluse-vcs-ignores Excludes files from VCS ignore files like .gitignore
N/A --overwrite Overwrite existing files when extracting
N/A --remove-files Remove files from the disk after adding them to the archive
N/A --skip-old-files Don't replace existing files when extracting, silently skip over them
N/A --xform / --transform Use sed to replace the following expression when transforming the file names
N/A --strip n Strip n leading components from the file names on extraction
N/A --quoting Specify the style of quoting when listing directory contents, accepts shell as a valid argument

Creating an archive

When creating an archive, the -C or --dir option is very useful. It allows you to specify which directory to be in for the subsequent operations. In the case of archiving a file, by default tar will create the archive including every directory specified in the path. If you tried to type tar -cf archive.tar ~/example then it would actually create an archive of /Users/tommy/example where the root directory is /Users, containing a single directory /tommy, which itself contained a single directory /example. To avoid this, you must specify which directory tar should operate from. In this case, it should be the parent directory of example which is /Users/tommy.

Examples

Tip: tar can automatically determine which compression algorithm to use based on the specified style extension when using the -a flag

tar -f archive.tar -t
tar --file archive.tar --list
# -> example/spaced out.txt
tar --file archive.tar --list --quoting shell

tree

Getting Started

brew install tree

wc

Note: Herestrings implicitly append a trailing \n newline, which is why wc returns three characters.

rs and datamash

awk

FS

man

Typically, a manpage for a given command is located at /usr/share/man/man1/command.1

Tip: This is a vague title, but when you do sudo apt install yarnpkg you can't use the command yarn, instead you have to use yarnpkg which is unfamiliar. In order to restore it to its normal state, you can run the following command:

sudo update-alternatives /usr/bin/yarn yarn /usr/bin/yarnpkg 1

You can also do the same for python3

sudo apt install python3
sudo apt install python3-pip
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1
sudo update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1

${PATH}

POSIX

File Systems

File System Hierarchy

Volumes & Drives

The collection of non-volatile memory stored on a computer is kept in one or more Volumes. The collection of data is the Volume, and the device this volume exists on is what's known as the Drive. Back in the day, these drives were implemented as discs, hence the term Floppy Disk Drive and Hard Disk Drive. These days, we're moving on from disc drives, instead transitioning to Solid State Drives, which use transistors and circuits to store the volume (which contains the file system).

The commands for performing various operations are different on macOS and Linux. I'll try to include an example of how to do each in the relevant section. (I ran out of time while writing this, so this is to be continued for now...)

curl

PAC Files

A Proxy Auto-Config PAC file contains a JavaScript function that decides which proxy a given network connection (URL) should use. curl does not support PAC files, but I'm putting this note here for now.

cron

Example crontab using zsh globbing

SHELL=/usr/local/bin/zsh

# When the computer reboots,
# delete any files in ~/tmp that haven't been accessed in over a day
@reboot rm ~/tmp/**/*(.ad+1) 2> ~/tmp/cron.log

# When the computer reboots,
# delete any directories in ~/tmp that are empty
@reboot rm -rf ~/tmp/**/*(/^F) 2> ~/tmp/cron.log

# Month, Hour, Day, Month, Weekday
#   *     *     *     *       *

# @yearly
0 0 1 1 * echo 'Once a year' &> ~/log.txt

# @monthly
0 0 1 * * echo 'Once a month' &> ~/log.txt

# @weekly
0 0 * * 0 echo 'Once a week' &> ~/log.txt

# @daily
0 0 * * * echo 'Once a day' &> ~/log.txt

# @hourly
0 * * * * echo 'Once an hour' &> ~/log.txt

zip

less

Useful Flags

ln

Useful Flags

coreutils exclusive flags

sudo

Useful Flags

who

diff

chsh

uname

tr

visudo & vipw

These are two special commands to manage the super user and password files on your computer

/etc/localtime

The file that determines the timezone of the computer is located at /etc/localtime

To change what time it is, make a symbolic link from another timezone, and store it at this location.

motd

dns-sd

The dns-sd diagnostic tool is useful for network scanning. If you are trying to find users to connect to via ssh, this is a great way to find out what their multicast DNS (mDNS) name is, such as tommy.local

Filesystem Hierarchy Standard

Disk Utility Functions

Encoding / Decoding Text

Binary Files