If you rent, it's a possibility that you have a messed up landlord. But this is illegal to enter without notice that something needs to be fixed or you are informed they are going to do maintenance/spraying.
If any of your keyholders are control freaks, snoops or psychologically messed up people, you need to change the locks. Always change the locks on a new house or after you take back the keys from a keyholder, pet sitter, friend of sigificant other when breaking up.
I actually had an encounter with a psychopath with a violent history and his family of enablers. I had a habit of leaving my keys in my purse or sitting next to it when visiting. I did wonder if they would stoop so low as to copy them to get in so I changed the locks. It's better to be proactive that to pay for it! I always keep my keys on my person while visiting anyone now except relatives.
Never, ever leave the windows open while you are away. They should always be latched too. Put a sawed off closet pole or broom handle in the track on the sliding door too. Never ever hide a key outside either. A locksmith will only charge you $20-30 to pop the cylinder on the lock to get in.
You might also be starting into paranoia from a mental illness. You should talk to a professional if you change the locks and still feel this way.