pretty trivial to do so

Yup. We have to "badge in" to our office, but the secretary will buzz you in if you ask nicely. Also, if you walk in with confidence as someone is entering/leaving, they'll hold the door for you. Or go in around the EOD when the cleaning staff are there and they'll let you in. All it takes is a very small amount of social engineering and you could steal a ton of stuff from my company.

But most people don't lose stuff like laptops at home or in their office, they lose them when traveling. Maybe you drop by a coffee shop on the way to work and someone filches your bag, or maybe you take a flight for work and someone swipes it while you're throwing something in the trash. They're not going to break into your home or your office, they'll snatch it while you're out in public and not paying particularly close attention.