Never has it been stated that he has been physically abused, not even Naruto himself when we learn more about his past in VotE. He was always ignored.
Then tell me, what other possible reason is there for Naruto to HATE the village? Shunning him/ignoring him doesn't work, since the Rookie Nine did it all throughout his childhood, and he never admits to hating them specifically. Insulting him doesn't work, as the various people I've listed all tried that, and no dice. If that's all he got from the village, then he wouldn't have hated the village. YET HE DID. What other reason besides physical abuse could exist to make him feel outright HATE, given that the stated alternatives have clearly shown themselves to not be enough?
Actually, he still would. If he gets people to notice him and be friends with him, maybe there's a chance those people would eventually try to help him when he's in a tight spot (the exact thing Iruka did when Mizuki tried to kill him). And maybe eventually, those people find a way to help him get through the abuse or stop it altogether. Or there could be more to this: His dream, as stated in Chapter 1, is to be the Hokage so people "acknowledge him". And as he himself states, the Hokage is the strongest person in the village. So try this interpretation on for size:
I want to be the Hokage because if I can somehow get to be the Hokage, everyone in this village will finally know that you can't mess with me, and so they'll stop hurting me.
Is it a darker interpretation of the Naruto dream we've all come to know and love? Perhaps. Is is consistent with the thoughts he had of his life before Iruka accepted him and before he finally found his so-called "precious people"? In my honest opinion, absolutely. On the same token, do I believe this interpretation still applies to him today? Certainly not. But that still doesn't mean I don't think it applied at one time...
What is stopping any ninja from killing him? The fact that even though Sandaime would not have watched over him 24/7, he still would have eventually noticed if Naruto had been killed. Someone would have opened their big mouths sometime, if only to off-handedly comment about their joy that "justice had been served" or something like that. And if all else failed, Sandaime would at least have begun to wonder where Naruto had disappeared off to if he had tried to visit /check up on him and found him missing. So to just kill off Naruto without any sort of viable alibi would have likely equated to suicide for a Konoha nin in the sense of risking Sarutobi's wrath. However, the stealing of the scroll finally provided them with the perfect opportunity; If they were to kill him, they could easily say that it was accidental, or that he gave them no other choice, or something like that. After all, Sandaime himself realized that there was a possibility that Naruto could have been dangerous with the scroll. Hence, we see the ninja all grouped-up together being gung-ho about going off to kill him on sight.
But abuse is different. It could be a punch or a kick (or two) when he's not looking, or beat-ups by kids who were silently encouraged and protected by their parents. Heck, it could even be as innocent-looking as an "accident" resulting in him being tackled, tripped up or something like that. It could be vandalization of property, or the ruining of his food when he goes off to eat. It could have been a rock or two thrown at him from a place he can't see, or a bucket of boiling hot water poured over him accidentally. I could go on and on. The key thing here is that it would have had to have been abusive enough to make Naruto utterly hate, yet something he could recover from naturally without visible signs for Sarutobi or anyone else to find (not a difficult task given that he had quick regeneration powers). And it would have had to have been a secret thing, given that something such as outrageous, continuous abuse in broad daylight would be pretty tough to cover up time after time. After all, the villagers aren't stupid; if they were clever enough to not do so much as talk about the Kyuubi with Sarutobi around, I doubt they'd risk themselves foolishly by abusing Naruto out in a street in broad daylight when they could easily do so in a back alley somewhere. The bottom line is that killing Naruto would have been too conspicuous, but secretly wrought physical abuse (if premeditated correctly), would not. Hence why I think Konoha didn't try to do the former until they had a perfect chance (Chapter 1), and why I think they did the latter chronically.
But then again, these are just my theories and my interpretations.

SkyStrider