This may make some people pull their hair out, but I’d love to hear some arguments. I’ve had the impression that people really don’t like bash, not from here, but just from people I’ve worked with.
There was a task at work where we wanted something that’ll run on a regular basis, and doesn’t do anything complex aside from reading from the database and sending the output to some web API. Pretty common these days.
I can’t think of a simpler scripting language to use than bash. Here are my reasons:
- Reading from the environment is easy, and so is falling back to some value; just do
${VAR:-fallback}; no need to write another if-statement to check for nullity. Wanna check if a variable’s set to something expected?if [[ <test goes here> ]]; then <handle>; fi - Reading from arguments is also straightforward; instead of a
import os; os.args[1]in Python, you just do$1. - Sending a file via HTTP as part of an
application/x-www-form-urlencodedrequest is super easy withcurl. In most programming languages, you’d have to manually open the file, read them into bytes, before putting it into your request for the http library that you need to import.curlalready does all that. - Need to read from a
curlresponse and it’s JSON? Reach forjq. - Instead of having to set up a connection object/instance to your database, give
sqlite,psql,duckdbor whichever cli db client a connection string with your query and be on your way. - Shipping is… fairly easy? Especially if docker is common in your infrastructure. Pull
Ubuntuordebianoralpine, install your dependencies through the package manager, and you’re good to go. If you stay within Linux and don’t have to deal with differences in bash and core utilities between different OSes (looking at you macOS), and assuming you tried to not to do anything too crazy and bring in necessary dependencies in the form of calling them, it should be fairly portable.
Sure, there can be security vulnerability concerns, but you’d still have to deal with the same problems with your Pythons your Rubies etc.
For most bash gotchas, shellcheck does a great job at warning you about them, and telling how to address those gotchas.
There are probably a bunch of other considerations but I can’t think of them off the top of my head, but I’ve addressed a bunch before.
So what’s the dealeo? What am I missing that may not actually be addressable?
I'm afraid your colleagues are completely right and you are wrong, but it sounds like you genuinely are curious so I'll try to answer.
I think the fundamental thing you're forgetting is robustness. Yes Bash is convenient for making something that works once, in the same way that duct tape is convenient for fixes that work for a bit. But for production use you want something reliable and robust that is going to work all the time.
I suspect you just haven't used Bash enough to hit some of the many many footguns. Or maybe when you did hit them you thought "oops I made a mistake", rather than "this is dumb; I wouldn't have had this issue in a proper programming language".
The main footguns are:
shellcheck. I have too. That's not a criticism. It's basically impossible to get quoting completely right in any vaguely complex Bash script.set -e, but then that breaks pipelines and conditionals, and you end up with really monstrous pipelines full ofpipefailnoise. It's also extremely easy to forgetset -e.No. If it's missing
$1will silently become an empty string.os.args[1]will throw an error. Much more robust.Absolutely not. Python is strongly typed, and even statically typed if you want. Light years ahead of Bash's mess. Quoting is pretty easy to get right in Python.
I actually started keeping a list of bugs at work that were caused directly by people using Bash. I'll dig it out tomorrow and give you some real world examples.
I don't disagree with your point, but how does
set -ebreak conditionals? I use it all the time without issuesPipefail I don't use as much so perhaps that's the issue?
It means that all commands that return a non-zero exit code will fail the script. The problem is that exit codes are a bit overloaded and sometimes non-zero values don't indicate failure, they indicate some kind of status. For example in
git diff --exit-codeorgrep.I think I was actually thinking of
pipefailthough. If you don't set it then errors in pipelines are ignored, which is obviously bad. If you do then you can't usegrepin pipelines.My sweet spot is
set -uebecause I like to be able to use things likeif grep -q ...; thenand I like things to stop if I misspelled a variable.It does hide failures in the middle of a pipeline, but it's a tradeoff. I guess one could turn it on and off when needed
Agreed.
Also gtfobins is a great resource in addition to shellcheck to try to make secure scripts.
For instance I felt upon a script like this recently:
Quotes are OK, shellcheck is happy, but, according to gtfobins, you can abuse tar, so running the script like this:
./test.sh /dev/null --checkpoint=1 --checkpoint-action=exec=/bin/shends up spawning an interactive shell...So you can add up binaries insanity on top of bash's mess.
I imagine adding
--so it becomestar -caf archive.tar.bz2 -- "$@"would fix that specific caseBut yeah, putting bash in a position where it has more rights than the user providing the input is a really bad idea
This runs into a part of the unix philosophy about doing one thing and doing it well: Extending programs to have more (absolutely useful) functionality winds up becoming a security risk. The shell is generally geared towards being a collection of shortcuts rather than a normal, predictable but tedious API.
For a script like that you'd generally want to validate that the input is actually what you expect if it needs to handle hostile users, though. It'll likely help the sleepy users too.
Meh, most in that list are just "if it has the SUID bit set, it can be used to break out of your security context".
I honestly don’t care about being right or wrong. Our trade focuses on what works and what doesn’t and what can make things work reliably as we maintain them, if we even need to maintain them. I’m not proposing for bash to replace our web servers. And I certainly am not proposing that we can abandon robustness. What I am suggesting that we think about here, is that when you do not really need that robustness, for something that may perhaps live in your production system outside of user paths, perhaps something that you, your team, and the stakeholders of the particular project understand that the solution is temporary in nature, why would Bash not be sufficient?
Wrong assumption. I’ve been writing Bash for 5-6 years now.
Maybe it’s the way I’ve been structuring my code, or the problems I’ve been solving with it, in the last few years after using
shellcheckandbash-language-serverthat I’ve not ran into issues where I get fucked over by quotes.But I can assure you that I know when to dip and just use a “proper programming language” while thinking that Bash wouldn’t cut it. You seem to have an image of me just being a “bash glorifier”, and I’m not sure if it’ll convince you (and I would encourage you to read my other replies if you aren’t), but I certainly don’t think bash should be used for everything.
You’ll probably hate this, but you can use
set -uto catch unassigned variables. You should also use fallbacks wherever sensible.Not a good argument imo. It eliminates a good class of problems sure. But you can’t eliminate their dependence on shared libraries that many commands also use, and that’s what my point was about.
And I’m sure you can find a whole dictionary’s worth of cases where people shoot themselves in the foot with bash. I don’t deny that’s the case. Bash is not a good language where the programmer is guarded from shooting themselves in the foot as much as possible. The guardrails are loose, and it’s the script writer’s job to guard themselves against it. Is that good for an enterprise scenario, where you may either blow something up, drop a database table, lead to the lost of lives or jobs, etc? Absolutely not. Just want to copy some files around and maybe send it to an internal chat for regular reporting? I don’t see why not.
Bash is not your hammer to hit every possible nail out there. That’s not what I’m proposing at all.
If you're proposing Bash, then yes you are.
I actually didn't know that, thanks for the hint! I am forced to use Bash occasionally due to misguided coworkers so this will help at least.
Not sure what you mean here?
Well if it's just for a temporary hack and it doesn't matter if it breaks then it's probably fine. Not really what is implied by "production" though.
Also even in that situation I wouldn't use it for two reasons:
I’m going to downvote your comment based on that first quote reply, because I think that’s an extreme take that’s unwarranted. You’ve essentially dissed people who use it for CI/CD and suggested that their pipeline is not robust because of their choice of using Bash at all.
And judging by your second comment, I can see that you have very strong opinions against bash for reasons that I don’t find convincing, other than what seems to me like irrational hatred from being rather uninformed. It’s fine being uninformed, but I suggest you tame your opinions and expectations with that.
About shared libraries, many popular languages, Python being a pretty good example, do rely on these to get performance that would be really hard to get from their own interpreters / compilers, or if re-implementing it in the language would be pretty pointless given the existence of a shared library, which would be much better scrutinized, is audited, and is battle-tested. libcrypto is one example. Pandas depends on NumPy, which depends on, I believe, libblas and liblapack, both written in C, and I think one if not both of these offer a cli to get answers as well. libssh is depended upon by many programming languages with an ssh library (though there are also people who choose to implement their own libssh in their language of choice). Any vulnerabilities found in these shared libraries would affect all libraries that depend on them, regardless of the programming language you use.
If production only implies systems in a user’s path and not anything else about production data, then sure, my example is not production. That said though, I wouldn’t use bash for anything that’s in a user’s path. Those need to stay around, possible change frequently, and not go down. Bash is not your language for that and that’s fine. You’re attacking a strawman that you’ve constructed here though.
If your temporary small script morphs into a monster and you’re still using bash, bash isn’t at fault. You and your team are. You’ve all failed to anticipate that change and misunderstood the “temporary” nature of your script, and allowed your “temporary thing” to become permanent. That’s a management issue, not a language choice. You’ve moved that goalpost and failed to change your strategy to hit that goal.
You could use Deno, but then my point stands. You have to write a function to handle the case where an env var isn’t provided, that’s boilerplate. You have to get a library for, say, accessing contents in Azure or AWS, set that up, figure out how that api works, etc, while you could already do that with the awscli and probably already did it to check if you could get what you want. What’s the syntax for
mkdir? What’s it formkdir -p? What about other options? If you already use the terminal frequently, some of these are your basic bread and butter and you know them probably by heart. Unless you start doing that with Deno, you won’t reach the level of familiarity you can get with the shell (whichever shell you use ofc).And many argue against bash with regards to error handling. You don’t always need something that proper language has. You don’t always need to handle every possible error state differently, assuming you have multiple. Did it fail? Can you tolerate that failure? Yup? Good. No? Can you do something else to get what you want or make it tolerable? Yes? Good. No? Maybe you don’t want to use bash then.
Yes, because that is precisely the case. It's not a personal attack, it's just a fact that Bash is not robust.
You're trying to argue that your cardboard bridge is perfectly robust and then getting offended that I don't think you should let people drive over it.
You mean "third party libraries" not "shared libraries". But anyway, so what? I don't see what that has to do with this conversation. Do your Bash scripts not use third party code? You can't do a lot with pure Bash.
Well that's why I don't use Bash. I'm not blaming it for existing, I'm just saying it's shit so I don't use it.
Handling errors correctly is slightly more code ("boilerplate") than letting everything break when something unexpected happens. I hope you aren't trying to use that as a reason not to handle errors properly. In any case the extra boilerplate is...
Deno.env.get("FOO"). Wow.What's the syntax for a dictionary in Bash? What about a list of lists of strings?