2019独角兽企业重金招聘Python工程师标准>>>

Joshua Levy, Trust me. I'm a professional.

Votes by Kartik Ayyar, Robin Green, Prasad Chakka, Srinath Sridhar, and 2180 more.

Here is a selection of command-line tips that I've found useful when working on Linux. The emphasis is on somewhat less-known techniques that are generally important or useful to technical users. It's a bit long, and users certainly don't need to know all of them, but I've done my best to review that each item is worth reading in terms of projected time savings, if you use Linux heavily.

To get more information on a command mentioned, first try "man <command name>". In some cases, you must install a package for this to work -- try aptitude or yum. If that fails, Google it.

Basics

  • Learn basic Bash. Actually, read the whole bash man page; it's pretty easy to follow and not that long. Alternate shells can be nice, but bash is powerful and always available (learning mainly zsh or tcsh restricts you in many situations).
  • Learn vim. There's really no competition for random Linux editing (even if you use Emacs or Eclipse most of the time).
  • Know ssh, and the basics of passwordless authentication, via ssh-agent, ssh-add, etc.
  • Be familiar with bash job management: &, Ctrl-Z, Ctrl-C, jobs, fg, bg, kill, etc.
  • Basic file management: ls and ls -l (in particular, learn what every column in "ls -l" means), less, head, tail and tail -f, ln and ln -s (learn the differences and advantages of hard versus soft links), chown, chmod, du (for a quick summary of disk usage: du -sk *), df, mount.
  • Basic network management: ip or ifconfig, dig.
  • Know regular expressions well, and the various flags to grep/egrep. The -o, -A, and -B options are worth knowing.
  • Learn to use apt-get or yum (depending on distro) to find and install packages.

Everyday use

  • In bash, use Ctrl-R to search through command history.
  • In bash, use Ctrl-W to kill the last word, and Ctrl-U to kill the line. See man readline for default keybindings in bash. There are a lot. For example Alt-. cycles through prevous arguments, and Alt-* expands a glob.
  • To go back to the previous working directory: cd -
  • If you are halfway through typing a command but change your mind, hit Alt-# to add a # at the beginning and enter it as a comment (or use Ctrl-A, #, enter). You can then return to it later via command history.
  • Use xargs (or parallel). It's very powerful. Note you can control how many items execute per line (-L) as well as parallelism (-P). If you're not sure if it'll do the right thing, use xargs echo first. Also, -I{} is handy. Examples:

find . -name \*.py | xargs grep some_function

cat hosts | xargs -I{} ssh root@{} hostname

  • pstree -p is a helpful display of the process tree.
  • Use pgrep and pkill to find or signal processes by name (-f is helpful).
  • Know the various signals you can send processes. For example, to suspend a process, use kill -STOP [pid].  For the full list, see man 7 signal
  • Use nohup or disown if you want a background process to keep running forever.
  • Check what processes are listening via netstat -lntp. See also lsof.
  • In bash scripts, use set -x for debugging output. Use set -e to abort on errors. Consider using set -o pipefail as well, to be strict about errors (though this topic is a bit subtle). For more involved scripts, also use trap.
  • In bash scripts, subshells (written with parentheses) are convenient ways to group commands. A common example is to temporarily move to a different working directory, e.g.

# do something in current dir

(cd /some/other/dir; other-command)

# continue in original dir

  • In bash, note there are lots of kinds of variable expansion. Checking a variable exists: ${name:?error message}. For example, if a bash script requires a single argument, just write input_file=${1:?usage: $0 input_file}. Arithmetic expansion: i=$(( (i + 1) % 5 )). Sequences: {1..10}. Trimming of strings: ${var%suffix} and ${var#prefix}. For example if var=foo.pdf, then echo ${var%.pdf}.txt prints "foo.txt".
  • The output of a command can be treated like a file via <(some command). For example, compare local /etc/hosts with a remote one: diff /etc/hosts <(ssh somehost cat /etc/hosts)
  • Know about "here documents" in bash, as in cat <<EOF ....
  • In bash, redirect both standard output and standard error via: some-command >logfile 2>&1. Often, to ensure a command does not leave an open file handle to standard input, tying it to the terminal you are in, it is also good practice to add "</dev/null".
  • Use man ascii for a good ASCII table, with hex and decimal values.
  • On remote ssh sessions, use screen or dtach to save your session, in case it is interrupted.
  • In ssh, knowing how to port tunnel with -L or -D (and occasionally -R) is useful, e.g. to access web sites from a remote server.
  • It can be useful to make a few optimizations to your ssh configuration; for example, this .ssh/config contains settings to avoid dropped connections in certain network environments, not require confirmation connecting to new hosts, forward authentication, and use compression (which is helpful with scp over low-bandwidth connections):

TCPKeepAlive=yes

ServerAliveInterval=15

ServerAliveCountMax=6

StrictHostKeyChecking=no

Compression=yes

ForwardAgent=yes

Data processing

  • To convert HTML to text: lynx -dump -stdin
  • If you must handle XML, xmlstarlet is good.
  • For Amazon S3, s3cmd is convenient (albeit immature, with occasional misfeatures).
  • Know about sort and uniq (including uniq's -u and -d options).
  • Know about cut, paste, and join to manipulate text files. Many people use cut but forget about join.
  • It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via sort/uniq. Suppose a and b are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the -T option if /tmp is on a small root partition.)

cat a b | sort | uniq > c   # c is a union b

cat a b | sort | uniq -d > c   # c is a intersect b

cat a b b | sort | uniq -u > c   # c is set difference a - b

  • Know that locale affects a lot of command line tools, including sorting order and performance. Most Linux installations will set LANG or other locale variables to a local setting like US English. This can make sort or other commands runmany times slower. (Note that even if you use UTF-8 text, you can safely sort by ASCII order for many purposes.) To disable slow i18n routines and use traditional byte-based sort order, use export LC_ALL=C (in fact, consider putting this in your .bashrc).
  • Know basic awk and sed for simple data munging. For example, summing all numbers in the third column of a text file: awk '{ x += $3 } END { print x }'. This is probably 3X faster and 3X shorter than equivalent Python.
  • To replace all occurrences of a string in place, in files:

perl -pi.bak -e 's/old-string/new-string/g' my-files-*.txt

  • Use shuf to shuffle or select random lines from a file.
  • Know sort's options. Know how keys work (-t and -k). In particular, watch out that you need to write -k1,1 to sort by only the first field; -k1 means sort according to the whole line.
  • Stable sort (sort -s) can be useful. For example, to sort first by field 2, then secondarily by field 1, you can use sort -k1,1 | sort -s -k2,2
  • If you ever need to write a tab literal in a command line in bash (e.g. for the -t argument to sort), press Ctrl-V <tab> or write $'\t' (the latter is better as you can copy/paste it).
  • For binary files, use hd for simple hex dumps and bvi for binary editing.
  • Also for binary files, strings (plus grep, etc.) lets you find bits of text.
  • To convert text encodings, try iconv. Or uconv for more advanced use; it supports some advanced Unicode things. For example, this command lowercases and removes all accents (by expanding and dropping them):

uconv -f utf-8 -t utf-8 -x '::Any-Lower; ::Any-NFD; [:Nonspacing Mark:] >; ::Any-NFC; ' < input.txt > output.txt

  • To split files into pieces, see split (to split by size) and csplit (to split by a pattern).

System debugging

  • For web debugging, curl and curl -I are handy, and/or their wget equivalents.
  • To know disk/cpu/network status, use iostat, netstat, top (or the better htop), and (especially) dstat. Good for getting a quick idea of what's happening on a system.
  • To know memory status, run and understand the output of free and vmstat. In particular, be aware the "cached" value is memory held by the Linux kernel as file cache, so effectively counts toward the "free" value.
  • Java system debugging is a different kettle of fish, but a simple trick on Sun's and some other JVMs is that you can run kill -3 <pid> and a full stack trace and heap summary (including generational garbage collection details, which can be highly informative) will be dumped to stderr/logs.
  • Use mtr as a better traceroute, to identify network issues.
  • For looking at why a disk is full, ncdu saves time over the usual commands like "du -sk *".
  • To find which socket or process is using bandwidth, try iftop or nethogs.
  • The ab tool (comes with Apache) is helpful for quick-and-dirty checking of web server performance. For more complex load testing, try siege.
  • For more serious network debugging, wireshark or tshark.
  • Know strace and ltrace. These can be helpful if a program is failing, hanging, or crashing, and you don't know why, or if you want to get a general idea of performance. Note the profiling option (-c), and the ability to attach to a running process (-p).
  • Know about ldd to check shared libraries etc.
  • Know how to connect to a running process with gdb and get its stack traces.
  • Use /proc. It's amazingly helpful sometimes when debugging live problems. Examples: /proc/cpuinfo, /proc/xxx/cwd, /proc/xxx/exe, /proc/xxx/fd/, /proc/xxx/smaps.
  • When debugging why something went wrong in the past, sar can be very helpful. It shows historic statistics on CPU, memory, network, etc.
  • For deeper systems and performance analyses, look at stap (systemtap) and perf.
  • Use dmesg whenever something's acting really funny (it could be hardware or driver issues).

A couple other answers in the same vein:  What are the most useful "Swiss army knife" one-liners on Unix?  What are some lesser known but useful Unix commands?

Disclaimer: Just because you  can do something in bash, doesn't necessarily mean you should. ;)

转载于:https://my.oschina.net/zungyiu/blog/191012

What are some time-saving tips that every Linux us相关推荐

  1. 20 Linux Server Hardening Security Tips

    转自:http://www.cyberciti.biz/tips/category/gentoo-linux Securing your Linux server is important to pr ...

  2. html鼠标悬停弹出,纯css实现鼠标滑过弹出层效果

    复制代码代码如下: 无标题文档 *{margin:0; padding:0;} /* 所有元素外边距为0 内边距为0 */ /* body中元素字体大小为12px,字体样式为:"Times ...

  3. Android会加速地球的毁灭?

    并非危言耸听,以Android的普及能力,估计5年左右,中国将会有大量基于Android的手机系统,这个数子可能达到一个恐怖的数字.目前中国拥有 手机的数量已经突破 7亿台大关,5年时间估计会突破十亿 ...

  4. Redis持久化 - RDB和AOF

    一.持久化的作用 1. 什么是持久化 持久化(Persistence),即把数据(如内存中的对象)保存到可永久保存的存储设备中(如磁盘). 持久化Redis所有数据保持在内存中,对数据的更新将异步地保 ...

  5. dnslog 在 sql注入中的应用

    目录 1.DNSLOG的原理 2.Dnslog在常见数据库中SQL注入的实战 1.MySQL 2.msSQL 3.postgreSQL 4.Oracle 3.总结 1.DNSLOG的原理 DNS的解析 ...

  6. 应该知道的Linux技巧

    这篇文章来源于Quroa的一个问答<What are some time-saving tips that every Linux user should know?>-- Linux用户 ...

  7. 20145212 《信息安全系统设计基础》第2周学习总结

    20145212 <信息安全系统设计基础>第2周学习总结 教材学习内容总结 Vim基本操作 1.使用vim命令进入vim界面 vim后面加上你要打开的已存在的文件名或者不存在(则作为新建文 ...

  8. kali linux超级用户_如何优雅的在Linux上使用Powershell]

    点击上方"蓝字"关注我们吧! 译文声明 本文是翻译文章,文章原作者 TJ Null 文章来源:https://www.offensive-security.com 原文地址: ht ...

  9. 你应该知道的 Linux 命令行技巧

    2019独角兽企业重金招聘Python工程师标准>>> 这篇文章来源于Quroa的一个问答<What are some time-saving tips that every ...

最新文章

  1. 简单的CSS颜色查看工具
  2. linux sendto 对方关闭后性能,Gateway的请问sendToUid是否有性能问题?
  3. P3375 【模板】KMP字符串匹配 (KMP模板)
  4. CentOS 7下宿主机使用virsh console访问KVM的设置
  5. 关于Myeclipse2017 MemoryAnalyzer的安装
  6. WiFi行业将走向何方?
  7. Dapr云原生应用开发系列7:工作流集成
  8. jmeter mysql驱动jar包_用 Jmeter 做 Web 接口测试
  9. CountDownLatch原理简介和使用过程
  10. [ 逻辑锻炼] 用 JavaScript 做一个小游戏 ——2048 (详解版)
  11. yum使用总结(转)
  12. 百度竞价有没有好的推广方法?
  13. smbcontrol - 向smbd或nmbd进程发送消息
  14. CentOS 6.X yum源更新(阿里云默认最新更新为CentOS6.10)
  15. 苹果11微信表格服务器地址怎么填,苹果实用技巧:iPhone11手机微信打字怎么换行...
  16. 「TCG 规范解读」初识嵌入式和工业工作组
  17. Arduino ESP8266当中的yield 函数
  18. 做短视频的经历——成长与心酸
  19. Vue项目清理本地缓存并删除node_modules (清除不掉揍我)
  20. MATLAB台大郭彦甫老师课程笔记:第一课:基本操作与矩阵输入

热门文章

  1. PageOffice,word常用接口对象--Paragraph类
  2. ODOO权限管理,在两个方面设置权限
  3. 一个资深技术人的转型之路
  4. (八)流程控制 for标签和if标签
  5. 优云,新一代运维PaaS平台
  6. 也许每个农村出来的码农都有个田园梦
  7. 脚本-收集计算机信息
  8. 解决COOKIES存储中文乱码的问题
  9. 资源2010-01-29
  10. makefile 常用函数