55
Analyzing Data 180,000x Faster with Rust
(willcrichton.net)
Welcome to the Rust community! This is a place to discuss about the Rust programming language.
Credits
I can absolutely confirm. I work in a specialized industry where we have a team of PhDs in our R&D team that writes quick and dirty code in Fortran. That's what they know, and it's what they're most productive with.
Our production code is in Python. We took one of their solutions and made it way faster, and the main improvement was to restructure the code to not need everything in memory at once. Basically, they were processing data in 4D space, but only needed 3D worth of data at a time, and most of the time was being spent in memory allocation. So we drastically dropped memory usage and memory allocation by simply changing how they iterated, which also meant we could parallelize it, further increasing performance.
They're paid to come up with the algorithms, I'm paid to make it run fast in production. We looked into Rust, but for this project, Python got us well within a reasonable running time and Rust would've requested retaining a lot of our team, but it's still on the table if we need more performance.