Stop using pickle already. Seriously, stop it! (mina86.com)

submitted 2 days ago by mina86@lemmy.wtf to c/python@programming.dev

5 comments fedilink hide all child comments

It is common knowledge that pickle is a serious security risk. And yet, vulnerabilities involving that serialisation format keep happening. In the article I shortly describe the issue and appeal to people to stop using pickle.

you are viewing a single comment's thread
view the rest of the comments

[-] NostraDavid@programming.dev 0 points 1 day ago* (last edited 1 day ago)

If you need to pickle your ML model, just use JobLib instead.

If you want to save a polars or pandas df, save files as parquet.

Both ways you can also use compression, so you'll save space as well. Use zstd if you need decent compression, or lz4 if you write and read speeds.

[-] mina86@lemmy.wtf 3 points 1 day ago

Joblib has the same drawback as pickle. From the documentation:

joblib.dump() and joblib.load() are based on the Python pickle serialization model, which means that arbitrary Python code can be executed when loading a serialized object with joblib.load().

joblib.load() should therefore never be used to load objects from an untrusted source or otherwise you will introduce a security vulnerability in your program.