66
The lost art of XML — mmagueta
(marcosmagueta.com)
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Follow the wormhole through a path of communities !webdev@programming.dev
CSV >>> JSON when dealing with large tabular data:
1 can be solved with JSONL, but 2 is unavoidable.
There ya go, problem solved without the unparseable ambiguity of CSV
Please stop using CSV.
Great, now read it row by row without keeping it all in memory.
Wdym? That's a parser implementation detail. Even if the parser you're using needs to load the whole file into memory, it's trivial to write your own parser that reads those entries one row at a time. You could even add random access if you get creative.
That's one of the benefits of JSON: it is dead simple to parse.
No:
Just user Zarr or so for array data. A table with more than 200 rows isn't ”human readable” anyway.
Yes..but compression
And with csv you just gotta pray that you're parser parses the same as their writer..and that their writer was correctly implemented..and they set the settings correctly
Compression adds another layer of complexity for parsing.
JSON can also have configuration mismatch problems. Main one that comes to mind is case (in)sensitivity for keys.
Nahh your nitpicking there, large csvs are gonna be compressed anyways
In practice I've never met a Json I cant parse, every second csv is unparseable