Which Linux command or utility is simple, powerful, and surprisingly unknown to many people or used less often?
This could be a command or a piece of software or an application.
For example I’m surprised to find that many people are unaware of Caddy, a very simple web server that can make setting up a reverse proxy incredibly easy.
Another example is fzf. Many people overlook this, a fast command-line fuzzy finder. It’s versatile for searching files, directories, or even shell history with minimal effort.
Not powerful, but often useful,
column -t
aligns columns in all lines. EG$ echo {a,bb,ccc}{5,10,9999,888} | xargs -n3 a5 a10 a9999 a888 bb5 bb10 bb9999 bb888 ccc5 ccc10 ccc9999 ccc888 $ echo {a,bb,ccc}{5,10,9999,888} | xargs -n3 | column -t a5 a10 a9999 a888 bb5 bb10 bb9999 bb888 ccc5 ccc10 ccc9999 ccc888
batcat
It’s like cat but better. Great for when you just want to look at the contents of a file, without loading a whole text editor.
Oh also, tldr
My procedure for learning how to use a cli command goes tldr page -> --help if the tldr fails to help me -> THEN the full manpage
Gripes:
-
starship and all these shell frameworks are overbloated. Just write your own prompt command and be done with it.
-
restic, ongoing issue with the author to allow people to backup without a password. Seems like a no-brainer but he’s being difficult
Why would you want password less backups?
I understand if the reason is ‘just because’, but seriously, why? I just write down the password in a text file for restic --password and I am done.
write down the password where though, somewhere I can guarantee it will always be there 10 years from now? That’s a big ask of me
I keep mine in Bitwarden, I export that data every 3 months and store it in a Backblaze backup, I have it written on a piece of paper stored in a locked fire box in my house, and that paper scanned in my phone.
I can’t imagine not having at least one of those in 10 years and I can’t imagine all four failing in the same week.
Does that give you any helpful ideas that would work for you?
None that I can see persisting, as I move around a lot and my backups tend to get boxed up for periods of time before being unboxed. But, I appreciate the effort
That is true for lots of things.
Moreover I use one easy “default” password for all basic stuff, and its always the same known to my spouse and written down on paper.
At least my offsite backups are protected from prying eyes. Maybe uneeded for local backups, but doesn’t hurt to have.
10 years? Boy you are joung :)
I have encrypted files from w 20 years ago, and unencrypted files from 30 years ago.
And digitized stuff from analogic of 40 and 50 years ago.
-
Underrated
Both linked projects have over 60k+ stars on GitHub
Pick one
Inshellisense is teaching me a lot. :) It’s an autocompleter.
https://github.com/microsoft/inshellisense
Also, Atuin for history.
+1 to caddy. There are some services that set safe headers following the recommendations outlined by Mozilla but others don’t control headers as strictly. Caddy is the only web server that I found that supports loose default header values. These values will be selected unless the upstream application specifies their own values.
You can do something similar in nginx but it requires playing with maps and has a little more indirection than I’d like.
Just wish caddy was capable of starting as root and stepping down permissions like Nginx. I have certs being managed by other tools and have to make sure they are installed and chowned for caddy’s use when they are cycled.
I just started the process of switching from nginx Proxy Manager to Caddy yesterday, and even before setting up a single rule, I’m enjoying it more than NPM. Really wish I would have heard about it sooner!
I’m currently using NPM and don’t have any problems with it for at least my use case. Is there something I’m missing out on never having tried Caddy, or is it one of those no need to switch if there’s Nothing bugging you situations? That last bit is how I feel about Bazzite on the Steam Deck when people ask of they should switch.
I think a lot of people don’t realise that yt-dlp works for many sites, not just YouTube
I used it recently for watching a video from tiktok without having to use their god awful web UI and it was amazing
Also works on Twitch with the added benefit of NOT playing ads (you still get breaks, just with a placeholder screen instead of the commercial).
mpv has yt-dlp support built in, so it can just play the streams directly.
Wait how?
I just run
mpv $URL
This is the only way to watch twitch
With their huge, clunky UI and my 1080p screen, yeah it is.
mpv --ytdl URL. Read starting from --ytdl option in the mpv man page, you can even give specific yt-dlp options through --ytdl-raw-options.
Since everyone keeps mentioning yt-dlp I gotta ask: what’s wrong with the original youtube-dl? I keep using it, it works, it’s still being updated.
yt-dlp has sponsorblock features, youtube-dl does not.
There are minor feature differences there’s also a convenience factor: youtube-dl people for some reason stopped doing releases, so you can’t get a fresh version from pypi (only installing from github or their site). Yt-dlp is on pypi, including nightly builds.
The huge list of sites can be found here https://github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor
It also supports ripping playlists. Fantastic to archive a set locally…
A few that I use every day:
- Fish shell
- Starship.rs
- Broot (a brilliant filesystem navigator)
- Helix editor (My favorite editor / IDE, truly the successor to vim IMO)
- Topgrade (updates everything)
I heard about helix from you and I’ve used it for a year and a half or so now, it’s by far the best editor I’ve used so far and I can definitely vouch for it
Nice!
Just commenting to give more love to helix. It’s my favorite “small quick edits” editor.
I’ve actually been testing with fish recently coming from zsh, though I might wait until 4.0 fully releases before I make a more conclusive decision to move or not.
With that said, I remember looking through omf themes and stumbled onto Starship that branched off one of the themes and really liked the concept.
One thing that holds people back sometimes is that bash scripts that set environment variables don’t work by default. https://github.com/edc/bass is an easy solution
Helix is great thanks
Once Helix gets plugin support and someone makes a Clojure REPL plugin as good as Conjure I am never touching
vim
again!It does have clojure lsp support, but you’ll probably have to use a command line for most repls.
Yeah the clojure lsp support is top notch, but there being no support for “jacking in” to a repl is the big thing keeping me from using helix full time. There’s a way of doing it if you use kitty, but it’s pretty janky.
Could you explain them in more depth? I opened them and don’t know
Helix is a terminal based text editor. It’s much like vim / neovim, but unlike those editors it’s good to go right out of the box, no configuration or plugins needed to make it work well.
Topgrade is one I haven’t used, but it looks like its intended purpose is to let you upgrade your apps with one command, even if you use multiple different package managers (I.e. if you were on Ubuntu, you could use it to upgrade your apt packages, at the same time as your snap packages, as well as flatpak, nix, and homebrew if you’ve added those.)
Thank you for explaining. I would never have understood topgrade without your example :)
Fish is a replacement of bash that’s a bit more user friendly (has some cool auto completion features out of the box and more sane behaviour like handling of spaces when expanding variables). I personally started to use nutshell recently but unlike fish it’s very different from bash.
Starship is a “prompt” for various shells (that bit of text in terminal before you enter the command that shows current user and directory in bash). I haven’t used it but AFAIK it has many features like showing current time, integration with git, etc.
Yep, here’s my Starship prompt, for example:
So, I have it configured to show:
- the exit code of the last command (if it’s non-zero),
- the duration of the last command (if it’s longer than 2 seconds),
- the time (when the last command ended),
- the current directory,
- the current Git branch, and it also shows some Git status information, for example the
means I have something stashed,
- and finally the technology in use in a repository/directory, so in this case that repo uses Rust and the compiler version is 1.83.
Thanks for adding this. What does stashed mean
Oh, when you’re coding something in a Git repo and you realize that you need to make a different change before you continue coding (e.g. switch to a branch, pull newest changes, or just create a separate smaller commit for part of your change), then you can run
git stash push
to put away your current changes, then make your other change, and then rungit stash pop
to bring your ongoing changes back. I recommend readinggit stash --help
, if you want to use it.Sometimes, though, you might end up just taking it into a different direction altogether or simply forget that you had something stashed. That’s when that indicator comes in handy. Because while you can have multiple things stashed, I do find it’s best not to keep them around for too long. If you do want to keep them for longer, then you can always create a branch and commit it as WIP onto there, so that you can push it onto a remote repo.
This is sick!! Would you mind sharing your config?
Nope, I’m glad to share.
I personalized it from the “Gruvbox Rainbow” preset from here: https://starship.rs/presets/
So, you might prefer that, if you’re not, well, me.You will need to set up a NerdFont, like the Starship installation guide says.
Here’s my configuration:
Spoiler
"$schema" = 'https://starship.rs/config-schema.json' format = """ [$status](bg:color_red fg:color_fg0)\ [](fg:color_red bg:color_orange)\ [$cmd_duration](bg:color_orange fg:color_fg0)\ [](fg:color_orange bg:color_yellow)\ [$time](bg:color_yellow fg:color_fg0)\ [](fg:color_yellow)\ $line_break\ [$directory](bg:color_aqua fg:color_fg0)\ [](fg:color_aqua bg:color_blue)\ [$git_branch\ $git_status](bg:color_blue fg:color_fg0)\ [](fg:color_blue bg:color_bg3)\ [$c\ $rust\ $golang\ $nodejs\ $php\ $java\ $kotlin\ $haskell\ $python\ $docker_context](bg:color_bg3 fg:color_fg0)\ [](fg:color_bg3)\ $line_break\ $line_break""" palette = 'gruvbox_dark' [palettes.gruvbox_dark] color_fg0 = '#ffffff' color_bg1 = '#3c3836' color_bg3 = '#665c54' color_blue = '#458588' color_aqua = '#689d6a' color_green = '#98971a' color_orange = '#d65d0e' color_purple = '#b16286' color_red = '#cc241d' color_yellow = '#d79921' [status] disabled = false symbol = "" format = ' $symbol $status ' [username] format = ' $user ' [directory] format = " $path " truncation_length = 3 truncation_symbol = "…/" [directory.substitutions] "Documents" = " " "Downloads" = " " "Music" = " " "Pictures" = " " "Projects" = " " [git_branch] symbol = "" format = ' $symbol $branch ' [git_status] style = "bg:color_aqua" format = '$all_status$ahead_behind ' [nodejs] symbol = "" format = ' $symbol $version ' [c] symbol = " " format = ' $symbol $version ' [rust] symbol = "" format = ' $symbol $version ' [golang] symbol = "" format = ' $symbol $version ' [php] symbol = "" format = ' $symbol $version ' [java] symbol = " " format = ' $symbol $version ' [kotlin] symbol = "" format = ' $symbol $version ' [haskell] symbol = "" format = ' $symbol $version ' [python] symbol = "" format = ' $symbol $version ' [cmd_duration] format = ' $duration ' [time] disabled = false time_format = "%R" format = ' $time ' [line_break] disabled = false
Thanks!
I use fuck to fix typos
That’s fantastic, I can’t wait to go home and install it
If you’d map it to just
f
it’s even more handyYou gain efficiency, but lose fun
Game changer!
That sounds dangerous. I hope it never tries to fix anything with
rm
You have to confirm any command before it runs, so no more dangerous than baseline rm
this just reminds me of
please
which runs the previous command withsudo
sudo !!
Will also run the previous command with sudo, fwiw.
I use fuckit to fix exceptions
grep goes crazy if you know your regex
I can never get grep to work consistently on Mac and Linux. Now, ripgrep OTOH…
That’s because Macs generally use BSD-based command line tools instead of GNU ones. You have to do a lot of Homebrew jiggery-pokery to approximate a GNU environment. Know Your Tools: Linux (GNU) vs. Mac (BSD) Command Line Utilities
Alas, doesn’t fit my purpose since it requires action by the script user. I usually just use perl in those situations
Check out my chapter on GNU grep BRE/ERE for those wanting to learn this regex flavor: https://learnbyexample.github.io/learn_gnugrep_ripgrep/breere-regular-expressions.html (there’s also another chapter for PCRE)
jq - super powerful json parser. Useful by hand and in scripts
jq?
I use it occasionally but every time I need to do something a tiny bit more complex than “extract field from an object” I have to spend half an hour studying its manual, at which point it’s faster to just write a Python script doing exactly what I need it to do.
Check out https://www.nushell.sh/ I use it for exactly that, i.e. complex extract and convert files
I actually installed it recently out of curiosity, but I’m hesitant about learning its advanced features like that. At least jq is a standalone tool that’s more ubiquitous than nushell, so you can rely on it even in environments that you don’t fully control (e.g. CI like GitHub Actions). And if you use it in some public code/scripts then other people will be more familiar with it too.
yq can do both JSON and YAML :)
https://github.com/johnkerl/miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
Funny how this was one of the first tools I learnt once I “seriously” started my linux journey, lol
I’m a big fan of
screen
because it will let me run long-running processes without having to stay connected via SSH, and will log all the output.I do a lot of work on customers’ servers and having a full record of everything that happened is incredibly valuable for CYA purposes.
There is also zellij, which can do the same but also has modern functionality specific for development workspaces!
(Although
screen
ortmux
will still probably be more widely available on remote machines etc)I’d recommend
tmux
for that particular use. Screen has a lot of extras that are interesting but don’t really follow the GNU mentality of “do one thing and do it well.”Tmux / Screen is like the emacs/vim of the modern day Linux I think.
Screen is more than capable, but for those who have moved to Tmux, they will absolutely advocate for it.
When tmux was first released I was already so used to screen that I never really considered switching. What would some convincing arguments be for me to make the effort to switch now?
The thing that got me to switch was being able to maintain my pane layout between connections. The various window and pane management niceties (naming, swapping, listing and the like) got me to stay. Now you can keep your screen, but you’d have to pry tmux from my cold, dead, tty.
Tmux was purpose built for terminal multiplexing. You can assign session names for organizing and manipulating multiple instances. Send keys to and read output from detached sessions. It’s easy to script.
Tmux was purpose built for terminal multiplexing.
Was screen not purpose built for terminal multiplexing?
Sorry, it was, just not for exploring all of those instances at once. Should have called out the tiling function. Screen also built in a serial terminal emulator and started playing with a few other things.
This was a few years ago so maybe it has improved, but I found that screen would crash and lose my session history and layout too often. That was bad enough, but when it happened it had some bullshit error message about a dungeon roof falling in. I don’t mind some comedy in code or even the interface, but don’t make light of the user losing their stuff. I tried tmux and it is much more stable than screen was.
I know everyone likes tmux but screen is phenomenal. I have a .screenrc I deploy everywhere with a statusbar at the bottom, a set number of pre-defined tabs, and logging to a directory (which is cleaned up after 30 days) so I can go back and figure out what I did. Great tool.
Woah screen is seeing active development again? There was like a decade where it stagnated. So much so that different distros were packaging different custom feature patches (IIRC only Ubuntu had a vertical split patch by default?) Looking at it now, the new screen maintainers had to skip a version to not conflict with forks that had become popular.
When tmux stabilized I jumped ship immediately and never looked back.
tmux with control mode in iterm is god mode for me on all my machines. Absolutely love it.
nohup
is similarI’ve had nohup fail to keep things running after my session ended quite frequently. It’s like it just goes to the next step in the process then gives up.
It’s likely that you’re using a systemd based system and the admin hasn’t enabled
linger
for your user.The servers are very locked down, so I’m sure that’s part of our compliance requirements. I haven’t looked into fixing it because I just wrote a script to hit Enter every 10 minutes to keep it alive.
Ha! Faking key presses, truly an elegant weapon for a more civilized age. If it works, it works.
It’s not as useful, sadly. Nohup disconnects standard input, output, and error. With screen or tmux, you can reattach them later.
The pipe (
|
), which if you think about it is the basis for function composition.nano was and still is vital to me learning and using linux, I will not learn how to use vim so if the distro forces it to be default im not using it.
Why is editing text so convoluted for seemingly no reason… also hate that vim must be used for certain files.
It’s for people to memorize hundreds of arcane shortcuts and shit so they can feel like a smug hacker and gloat over the rest of us using other editors and getting just as much done as they are.
Also for graybeards that haven’t realized it’s not 1985 anymore.
For the average user you’re definitely right, but I will say for the sysadmin of headless systems, having a powerful cli editor is a godsend. While it may seem arcane and unnecessary, learning vim is easier than managing remote x or sshfs or copying files to and from a system.
I didn’t learn vim to be a contrarian; I learned it because it seemed (and still seems to be) the path of least resistance for many workflows.
Wait until you meet an emacs user! ;p
It’s for people that don’t want a big bulky IDE and are willing to put a little work in to get used to it. I do all my coding in the terminal with vim and tmux and I like the simplicity and that with two dotfiles I can migrate my whole development environment to whatever PC, server or RaspberryPi that I need.
I’ve used Vim for some pretty non-nerdy stuff. Like ripping my DVD collection, when I got to the TV section I had a lot of file names to modify in bulk, and Vim let me do that. Also guitar tablature, the ability to edit plaintext both horizontally and vertically is surprisingly handy. Just having a macro to be able to add a bar line saves a shocking amount of time.
It was mostly a joke. I was just trying to mess with people 😉
Which I’ll use as a lighthearted excuse to mention things like the block edit mode.
deleted by creator
Wow, who hurt you? Vim is fun, and just because you can make things work without it doesn’t mean it has no practical benefit. It’s nice to have an editor as powerful as an IDE that doesn’t require a graphical environment.
Hundreds of shortcuts is emacs, by the way. A major perk of modal editing and the vi editing language is that you can compose relatively few operations to accomplish many tasks rather than memorizing lots of more complex and specific shortcuts.
You can change your hate to love by using vim
one of my favorite linux youtubers is named vimjoyer so maybe one day I will try to learn it
I’ve liked helix a bit more. It takes less initial set up, and generally has the mentality of showing what you’re about to change before inputting a change command.
It’s totally worth it. But be aware that you might get some :w sprinkled over your documents you are forced to write in other editors or word processors which does not speak vim…
I use vim mode everywhere I can and vim in the console, it took a bit of effort to learn but it was fun and satisfying. Highly recommend, I’m a vim user now for 7 years.
You can change that by changing your editor global variable
Wow you triggered a lot of vim users !
Maybe give micro a shot :) It’s nano but more sane defaults and comes with customization in mind.
Yeah, to this day vim still isn’t intuitive for me, so I just use nano as it’s either often included or simple to install on most Distros.
Unless a script is hardcoded for vim I haven’t had to use it.
It’s weird but VIM is so powerful and I love it but i also agree it wouldn’t be the default just an option if you needed it. It’s like with notepad ++ on windows it’s wonderful but not everyone needs it from day one notepad will work just fine for basic typing.
vim must be used for certain files??
Cant remember exactly but it had something to do with a file relating to sudo and it only was allowed to be edited with a vim style editor.
There’s a separate command called
visudo
for this purpose.You CAN use any ol’ text editor but visudo has built-in validation specific to the sudoers file. This is helpful because sudoers syntax is unique and arcane, and errors are potentially quite harmful.
But visudo can use any editor if you set SUDO_EDITOR or EDITOR variables. If you don’t want to use vi(m) you should probably set EDITOR in your .bashrc and visudo and probably other programs will use your editor of choice.
visudo
?There may be certain times where it’s all that’s available, I think I remember having to edit fstab in some recovery state in vi
The
EDITOR
orVISUAL
environment variables are usually read by command line tools to launch your preferred editor. You could setVISUAL
tonano
before launching visudo and you would be editing the sudoers file in nano./etc/sudoers?
you can just edit that with nano or whatever, the visudo thing they tell you to use is goofy and I don’t like it
vim isn’t required for any files, you just followed online tutorials for how to edit those files instead of RTFM
terminal text editing is convoluted because it has to strike a balance between figuring out when a keypress is part of the text you’re typing, vs when it’s a command you’re using, and making sure that all the editor commands the designer wanted are accessible.
vim is great because it allows for thousands more editing commands and macros, and much more customization of the editor, up to allowing plugins that emulate other functionality. As it stands, my setup basically functions as a full, lightweight-ish, multi-language IDE that rivals Emacs or Visual Studio.
On top of all that, I don’t have to move my hands away from the homerow of keys to navigate or edit, which may not seem like much, but adds up to a lot of avoid typos and time saved from moving my hands to reach the arrows/delete/home/end/pgup/pgdn.
Some examples:
h
,j
,k
,l
move left, down, up, and right respectively, but they can be combined with a number to move that many rows or columns; e.g.6j
will move down 6 rowsdd
deletes a line, but using a number +d
+ a movement will delete that many characters/lines in the path of the cursor: e.g.34dl
will delete 34 characters to the right of the cursor,12dk
will delete 12 lines up.gg
will take you to the first line,G
will take you to the last, and number + either will take you to that line: e.g.3275gg
or3275G
will take you to line 3275and finally you can use
/text or regex pattern you want to search for
and Enter to search the document for the first occurence below your current location, and then usen
to search for the next occurence, orN
to search for the previousThat doesn’t even scratch the surface (that’s just the cheatsheet, which only scratches the surface), but if you can get a handle on only what I’ve said, and switching between input and command mode (
i
andEsc
respectively), the speedup to navigation alone will make it seem more sensible.And as always, don’t forget to
:wq
(write to file and quit)seems like you need to try micro. It’s like nano, but with more sensible standard keybinds imho, as well as syntax highlighting and global clipboard use.
igtfo<ESC>
:q!I used nano when I started but now I am using vim for one year already. I’d recommend taking a few days where you only use vim and I think you will see why people like it. With a few motions you can be much faster than you would be in Nano.
One of the big reasons I switched to nixos is that I mostly need to use the console only for updating my system by editing the configuration file using nano. I do very little besides that thankfully while the GUI side of linux gets better everyday.
I’m using Linux since 1998 and still like nano. I can use vi, but prefer nano when it’s available.
deleted by creator
Nano is hella confusing too. Since when is
^
= Ctrl?And why dont they tell you that Ctrl+S Ctrl+C Ctrl+X works?
^C has been notation for ctrl-c for decades.
I grew up with the ^ symbol meaning CTRL. Kids these days.
As for why: arbitrary choice, they just needed a printable character they could show on screen, for when people pressed it and the terminal echoed it back out to them.
Yes but I am not that old and never saw it anywhere. So while it makes as much sense as hjkl it is not beginner friendly.