77
DuckDB as the New jq (www.pgrs.net)
top 8 comments
sorted by: hot top controversial new old
[-] bugsmith@programming.dev 10 points 7 months ago

Love this. Always interesting to see novel ways of querying data in the terminal, and I agree that jq's syntax is difficult to remember.

I actually prefer nu(shell) for this though. On the lobste.rs thread for this blog, a user shared this:

| get license.key -i
| uniq --count
| rename license

This outputs the following:

╭───┬──────────────┬───────╮
│ # │    license   │ count │
├───┼──────────────┼───────┤
│ 0 │ bsd-3-clause │    23 │
│ 1 │ apache-2.0   │     5 │
│ 2 │              │     2 │
╰───┴──────────────┴───────╯

[-] Kissaki@programming.dev 1 points 7 months ago

Haha, I tried querying in Nushell before reading this and I was sure there was a better way. And indeed there is (especially since I was missing uniq). I'm still learning the available operators, but I enjoy the shell a lot, as well as its promised capabilities.

[-] bugsmith@programming.dev 2 points 7 months ago

I really like Nushell. I would not run it as a daily driver currently, as it mostly doesn't win me over from Fish, feature-wise, but I love having it available for anything CLI date pipeline work I need to do.

[-] LinearArray@programming.dev 6 points 7 months ago

Interesting

[-] Kissaki@programming.dev 5 points 7 months ago* (last edited 7 months ago)

While it doesn't provide an SQL interface, I've been using Nushell as my shell, which has native data operations.

I tried querying the same, and I'm still not fluent (this was my third or fourth bigger/practical data querying), but it works well and fast too when you know the syntax:

http get https://api.github.com/orgs/golang/repos | each {|x| get license} | get key | group-by --to-table | update items {|x| $x.items | length }

I've used Nushell for reading en-mass json files, generating command json files for stuff saved in excel files (you can natively open those too), and most recently to query log files for specific information and usage analysis.

/edit: This comment has the better nu solution.

[-] magic_lobster_party@kbin.run 4 points 7 months ago

I use jq almost every day, and I can admit its syntax is a bit cumbersome. Will give this a try.

[-] LPThinker@lemmy.world 3 points 7 months ago

I'll stick with nushell for terminal-first data interactions.

[-] bobburger@fedia.io 3 points 7 months ago

I love DuckDB. It's really nice to be able to put a bunch of parquet files in a directory and then do operations on them as if they were a sql database. You can even put an index on the files if you're feeling froggy.

this post was submitted on 22 Mar 2024
77 points (98.7% liked)

Programming

17314 readers
71 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 1 year ago
MODERATORS