With familiarity, can Rust’s intuitiveness match Python’s “from idea to deployment” speed?

Likely not at the start. Rust can take some time to learn to use it effectively it is not the fastest at throwing shit together quickly.

Or should I only learn Rust if I intend to create applications that need tight performance?

Also no. IMO rusts performance is only a nice by product of the language, yeah it encourages people to try it out, but they don't stay for the speed. They stay for the tooling and the feeling that once it compiles it will likely just work they way you intended. Rust forces you to think more about correctness and edge cases of your code - which does slow down initial idea to working prototype a bit. But IMO it quickly pays back dividends when you get something into production and it just works with no random crashing in the middle of the night.

It also makes refactoring a joy to do, where I hate refactoring in languages like python as you never know what you might have broken - likely something that you will find out only after you have deployed to production. Instead the compiler catches basically every thing that you missed before it will even let you run the code - so those edge cases are taken care of when you are developing, not after it fails in production.

I also find it is very nice with data processing/transformation as it lets you use functional coding styles which tend to lean towards clearer/easier to read series of data transforms.

If you want to learn it I would recomend starting out with the offical book, but you might also find zero to production or datawithrust interesting reads as well.

Context: I am an embedded software engineer. I write a lot of low level code that runs on microprocessors or in OS kernels, as well as networking applications and other things. I write a lot of C, I write some Rust, I write Elixir if I possibly can, I write a lot of Python (I hate C++ with a passion).

I don't think you want Rust. Python is unbeatable on "idea to deployment" speed. Python's downsides:

• Painful packaging/distribution if you want to get a load of people who don't have Python installed to run your thing (e.g the GUI program we currently maintain for talking to our hardware)
• Performance under some circumstances. There are some things that are not quick in Python. They're not always the things you expect because Python actually drops down to C modules for a lot of the number crunching that you might do. E.g. for ML you are basically using Python to plug a load of bits of fast C code together

Rust is good when you need at least one of:

• High speed
• Control over use of memory
• Low level systems programming (drivers etc.)
• Can't cope with a Garbage Collector
• Compiling to a microcontroller

If you're doing one of those and so have become expert in Rust, then it is actually excellent for a lot of other things. E.g. you might build your data processor in it, and then distribution is easy because it's just a single binary.

One option you might look at is Go. You get a lot of performance, you get good parallelism if you need it, it's designed to be easy to learn, and it also compiles programs to a single binary for easy distribution.

I’m usually a little suspicious of a new fancy language - because the language is only a part of the equation. Does it have good tooling and does it have awesome libraries?

I had a preconception that Rust is strong as a language (formally well structured, low shoot-yourself-in-the-foot potential, consistent, predictable) and that the tooling seemed strong (debuggers, editors, code completion, help, test frameworks), but I’ve always thought that it would lag with libraries. I mean compared to something like Python (« Batteries included ») or java, surely it is not yet compatible, right?.

So I chose a few of the less main-stream libraries that I use regularly… and Lo and behold! They exist for Rust, including Couchbase, SQLite, ECDH, DiffMatch. I can’t vouch for the completeness of those libs, but the fact that everything I looked for existed… that’s impressive.

I'd say it's powerful but not intitultive, at least in the sense of matching the intuitions that you already have. What learning it will do for you is make you develop new intuitions, that will make you a better programmer.

Rust would be an excellent fit for the type of work you describe. Assuming I understand the specifics correctly, the regex and serde crates would make the parsing + converting pretty effortless and fast.

The language itself also works really well for "data pipeline" type programs, thanks to its FP features/iterators and type system.

With familiarity, can Rust's intuitiveness match Python's "from idea to deployment" speed?

Yes and no. For experienced developers, the total time from "start" to "finished product" is probably going to be about the same in both. It's how that time is allocated that really distinguishes them.

Rust is going to make you put in more work up front compared to Python: negotiating with the compiler, getting your types in order, that sort of thing. The benefit is that what comes out the other end tends to be baked all the way through - "if it compiles, it works."

Python, being a dynamic scripting language, is going to make it easier to get a vertical slice or minimum viable product up and running. But when/if you scale up, you have to pay that time back fixing problems that Rust's static analysis could have caught at build time.

TL;DR - Python is good for "throwing something together" or writing load-bearing scripts that do one simple thing really well. Rust is a slower start that shines as complexity and scale increase.

you have to always ask your self: is the memory safe, performance etc worth the dev cost?

When you compare "idea to deployment" speed, a dynamic language will always win. However, much of this win is due to a dynamic language will let you deploy with a lot of bugs. So, you will then have to spend lot of time fixing production issues. Rust will force you to fix most of these issues before you can deploy, hence it feels slower in this aspect. I previously worked for 10 years with a huge perl code base, and I trade the deployment speed for stability in production any time.

Perl is still the best. 😁 😁

Hum... "Powerful" is not a word I associate with Rust... at all.

But it depends on what kind of "power" you want. Rust will give you lots of power about what kinds of things your programs do (and how performant they are), and not a lot about how to create your program with maximum productivity or minimum maintenance once your requirements change.

In particular, Rust won't give you a very good "time to first prototype" compared to Python. But it will beat Python on maintenance if your program gets large enough. The tools that make Rust more maintainable are available in other languages (Ocaml, Haskell), and if you are mostly writing parsers, those other languages will probably beat Rust on any metric.

Anyway, be prepared to learn a lot if you decided to go with any of those 3 language, and remember that "learning" always comes with feeling you are not good enough or that the language "just can't do this thing that is so easy on the one you know". Those feelings are rooted on a very small truth (you aren't good enough yet, and no, the new language does a different thing that is much better), just don't let it blind you from the larger picture.

Rust is a great language but it really has a lot of up-front costs. Whether you are learning it for the first time, or starting a new project, both. Python is always going to be faster "from idea to deployment."

I think like the other person said, start with the Rust book. It really is a perfectly good introduction. But what I think you'll find is if you want to be productive with Rust you will need to get the ground rules down or else you will be constantly tripping on the borrow checker and ownership rules. If you think you are getting somewhere with the book, try rewriting something you've done in Python. If that works out, great! If not, it's OK to accept that Rust might not be worth your time.

"intuitive" is extremely subjective, and based on your past experiences. I've coded in C++ for years, and some Python, too and was able to grasp many Rust concepts very quickly, while for others I struggled (and still am). I'd say that if you are looking for "intuitive", Rust ain't it. It's a system language, so it requires planning, it's definitely not the ideal language to slap a prototype quickly together, expecially as a beginner.

Most of the “automation” work that I do involves parsing data with regex, restructuring the data, converting the data into a modeled format and transforming something with that data.

These are the highlights of rust - regexes (the regex library that powers ripgrep), serialization/deserialization (serde, nom, pest, capnproto, etc), data manipulation (too many. perhaps apache arrow deserves a mention), etc. The language is designed for this sort of stuff.

Has anyone used Rust for network automation tools?

I'm in the same boat as you. I use python to manage an LXD cluster. I didn't choose Rust because Python has an API binding, but Rust didn't. It wouldn't have been too hard, considering that the API is mostly just JSON.

With familiarity, can Rust’s intuitiveness match Python’s “from idea to deployment” speed?

Unfortunately no! (in my personal experience). I have been using Rust for a decade (since before 1.0) and am more or less comfortable with it. I no longer fight with the compiler as most beginners do. Still, the compiler complains a lot and you have to think about why. This is not bad, IMO - all that thinking make you design better programs. There is always a sweet satisfaction of writing excellent code when you finish a Rust program - probably the reason why Rust developers love it so much. But you have to sacrifice some speed for that.

Intuitiveness is also subjective. My introduction to programming and computers was from the hardware side - from ALUs, DRAM, hardware pipelining etc. Rust's rules are very well defined, but is obscurely related to real problems you encounter in the hardware. In other words, Rust rules look confusing at first, but it makes sense once you see the error messages. You know what problem could have happened on the hardware if Rust didn't stop you. It's as if Rust's simple rules magically cover the problems you dread on the hardware.

All this means that Rust is easy to grasp for people who are already using C - OS, game engine, web engine and embedded system developers. For others who are mostly used to GC languages, it's an uphill battle with the compiler, until some day it clicks somehow. With Rust, it's very important to start right away with the (c lang) memory model and Rust's aliasing rules.

Or should I only learn Rust if I intend to create applications that need tight performance?

I wouldn't say no to learning a new language. But I always recommend a few languages because they introduce a radically different idea. Rust is one of them. My rant above probably gave you an idea why I would recommend Rust. The first is that Rust forces you to design programs properly - even though the rules are meant only to enforce memory safety. Second is that Rust gets you very close to the soul of the hardware. Python and even Go has some opaque layers in between you and the hardware. With Rust, you'll get every opportunity to get the best the hardware can offer. You get the same in C and C++ - but they also allow you to screw up.

I use Rust for even trivial tasks where a shell script would suffice. Besides the language features, Rust has excellent tooling and a rapidly growing library ecosystem. In many ways, it's easier to package Rust programs than Python ones. So Rust will work for your use case. But if development speed is more important to you than performance, then Go (similar to Python), Nim (much more similar to Python) and Zig are probably better choices. Repeating something here - Go is designed for the web. Numerous web backends are already on Go. And almost the entirety of Kubernetes is written in Go.

Just adding that you can also do Python - Rust interop fairly easily with something like https://github.com/PyO3/pyo3

So you can gradually adopt Rust for some of the more performance demanding parts and call them from Python. You will get a massive increase just from a simple rewrite. Though you would have the additional burden of 2 build toolchains. But as others have said, Rust is not just about the performance.

two words: yes