22
Help with sed commands (lemmy.dbzer0.com)
submitted 5 days ago* (last edited 5 days ago) by orsetto@lemmy.dbzer0.com to c/linux@lemmy.ml

Hi all! I have always only used sed with s///, becouse I've never been able to figure out how to properly make use of its full capabilities. Right now, I'm trying to filter the output of df -h --output=avail,source to only get the available space from /dev/dm-2 (let's ignore that I just realized df accepts a device as parameter, which clearly solves my problem).

This is the command I'm using, which works:

df -h --output=avail,source \
    | grep /dev/dm-2 \
    | sed -E 's/^[[:blank:]]*([0-9]+(G|M|K)).*$/\1/

However, it makes use of grep, and I'd like to get rid of it. So I've tried with a combiantion of t, T, //d and some other stuff, but onestly the output I get makes no sense to me, and I can't figure out what I should do instead.

In short, my question is: given the following output

$ df -h --output=avail,source 
Avail Filesystem
  87G /dev/dm-2
 1.6G tmpfs
  61K efivarfs
  10M dev
...

How do I only get 87G using only sed as a filter?

EDIT:

Nevermind, I've figured it out...

$ df -h --output=avail,source \
    | sed -E 's/^[[:blank:]]*([0-9]+(G|M|K))[[:blank:]]+(\/dev\/dm-2).*$/\1/; t; /.*/d'
85G
top 18 comments
sorted by: hot top controversial new old
[-] smeg@feddit.uk 10 points 5 days ago

Are you opposed to using awk? Printing out a specific column is basically the only thing I actually know how to do with it: df -h --output=avail,source | awk '/dm-2/ {print $1}'

[-] thingsiplay@beehaw.org 5 points 5 days ago

A perfect example when to use AWK.

[-] orsetto@lemmy.dbzer0.com 3 points 5 days ago

Are you opposed to using awk?

Not at all, I'm just not familiar with it so I find it confusing.

Although, looking at your command, i think I understand what it means

[-] smeg@feddit.uk 2 points 5 days ago* (last edited 5 days ago)

I had to check the syntax because I also don't use it enough to be sure, definitely a bit weird. Basically just grep for a regex and print the specified column.

[-] thingsiplay@beehaw.org 2 points 5 days ago

Just a thumb of rule to make sense of it: A column in AWK is by default any space separated part. You can change the column separator to any other character too with -F ":" in example would be a double colon. There is also a way to print all columns, but with certain exceptions. In example print all, but the third and fourth columns: ls -l | awk '{$3=""; $4=""; print $0}' . Admittely I forget this syntax often and have to look for it again.

[-] gnuhaut@lemmy.ml 8 points 5 days ago* (last edited 5 days ago)

You can do something like this to emulate grep:

sed -n '/myregex/p'

The -n suppresses auto-printing. That command should interpreted as: find line matching /myregex/ and then print (p) it.

You can then combine this with s (substitute):

sed '/myregex/ s/from/to/ p`

The complete command is then something like:

LANG=C df -h --output=avail,source | sed -n '/\/dev\/nvme0n1p2/ s/^\s*\([0-9.]\+[KMGT]\).*/\1/ p'

Note that the output can be something like 2.3G, but in my locale that would be 2,3G which is why I added LANG=C.

Easier IMHO is awk:

awk '/myregex/ {print $1}'

prints the first field.

[-] bizdelnick@lemmy.ml 2 points 5 days ago

You can then combine this with s (substitute):

sed '/myregex/ s/from/to/ p'

This is not combining commands. In your example p is a modifier to the s/// command.

[-] gnuhaut@lemmy.ml 2 points 4 days ago

Yeah you're right, thanks. I did figure this out on my own already, see my other comment:

Oh yeah that, so technically (and I was confused about this), the p in s/from/to/p is not the same as the p command, it’s a flag for the s command that tells it to print the output.

[-] orsetto@lemmy.dbzer0.com 2 points 5 days ago

The -n suppresses auto-printing

That's something I was missing! It makes the command look definitely less complicated.

Also, what is the difference between separating expressions with a space vs with a semicolon?

Easier IMHO is awk

Eh, that's another beast. For some reason tho I find sed more appealing.

[-] gnuhaut@lemmy.ml 2 points 5 days ago

What expressions?

I mean awk is more powerful, it has variables, function, can do arithmetic and format strings, and such proper scripting language features. And the GNU awk manual is rather well written.

[-] orsetto@lemmy.dbzer0.com 2 points 5 days ago

I meant to ask what is the difference between, i.e., sed '/myregex/ s/from/to/ p' and sed '/myregex/ s/from/to/ ; p', but while testing to explain what i meant I answered myself, and in the process I also understood what addresses are ehe

Right, awk is a proper programming language, right? that'll be for another day...

[-] gnuhaut@lemmy.ml 2 points 5 days ago

Oh yeah that, so technically (and I was confused about this), the p in s/from/to/p is not the same as the p command, it's a flag for the s command that tells it to print the output. You could do multiple commands like /re/ {s/a/b/;p} for the same result, by using a {} block.

If you do, say, /re/ s/a/b/; p those would be separate command, the first does the thing on lines matching /re/, while the p has no address range (e.g. regex) associated with it, so it gets executed for all the lines.

[-] orsetto@lemmy.dbzer0.com 2 points 5 days ago

I see. I guess what confused me was that i didn't understand what addresses were.

Thank you for your explanations :)

[-] gnuhaut@lemmy.ml 1 points 5 days ago

Yeah no problem.

What's maybe interesting is how sed came to be. Back in early days of Unix, they had ed as their editor (or, as some old manpage says, "Ed is the standard text editor.")

Sed has basically the same commands as ed. You typed 3d to delete the third line, or 10,20p to print lines 10 to 20. They only had teletypes back then, which are basically a keyboard and dot-matrix printer with one of those continuous papers for output, prior to when hardware terminals with CRT screens were a thing.

Anyway, someone thought it would be useful if you could put ed-style editing commands inside shell pipelines, and ed doesn't work for that. It is used for interactively editing files in place, and so gets commands from stdin. You can't pipe any files into it. So the "stream editor", sed, was born.

Also interesting: There were improved modified versions of ed going around like em and later ex. The original vi was a "visual mode" extension for ex, and you can still run ex/ed commands from vi by typing : first, e.g. you can delete line 3 by typing :3d inside vi.

[-] unlawfulbooger@lemmy.blahaj.zone 4 points 5 days ago* (last edited 5 days ago)

The easiest way is probably without sed, which you mentioned:

df -h --output=avail /dev/dm-2| tail -n1

But purely with sed it would be something like this:

df -h --output=avail,source | sed -n ‘/\/dev\/dm-2/s!/dev/dm-2!!p’

-n tells sed to not print lines by default

/[regex]/ selects the likes matching regex. We need to escape the slashes inside the regex.

s/// does search-and-replace, and has a special feature: it can use any character, not just a slash. So I used three exclamation points instead , so that I don’t need to escape the slashes. Here we replace the device with the empty string.

p prints the result

Check the sed man page for more details: https://linux.die.net/man/1/sed

[-] orsetto@lemmy.dbzer0.com 2 points 5 days ago* (last edited 5 days ago)

!/dev/dm-2!!p

This weired me out until I read the explanation. I'm so used to the slashes lol

In the end I've used the first command you wrote, because KISS, but I appreciate your explanation

Check the sed man page for more details

Yes that's been my only source so far, but to be honest it's really cryptic. it might just be that I'm used to syscalls man pages (also sed is kinda complex it can't be easy to write a single man page for it)

In the end I've used the first command you wrote, because KISS, but I appreciate your explanation

There’s no shame in combining multiple tools, that’s what pipelines are all about 😄.

Also there’s a different tool that I would use if I want to output a specific column: awk

df -h —output=avail,source | awk ‘/\/dev\/dm-2/ {print $1}’

For lines matching /dev/dm-2 print the first column. awk splits columns on whitespace by default.

But I would probably use grep+awk.

Sed is definitely a very powerful tool, which leads to complex documentation. But I really like the filtering options before using the search/replace.

You can select specific lines, with regex or by using a line number; or you can select multiple lines by using a comma to specify a range.

E.g. /mystring/,100s/input/output/g: in the lines starting from the first match of /mystring/ until line 100, replace input with output

[-] bizdelnick@lemmy.ml 2 points 5 days ago* (last edited 5 days ago)

Just disable printing out with the -n option and add the p modifier to the s/// command to print out lines where substitution has occured. sed -n 's/your-regexp/replacement/p'

this post was submitted on 15 Apr 2025
22 points (95.8% liked)

Linux

53346 readers
549 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS