I really haven't had an issue with them. I've been stung a few times, but usually I know what I did wrong. Once or twice my dogs made them mad and they ended up going for me.
I used to have this storage shed that was full of wasp nests. I don't mean just a few small nests, the wall was literally covered with wasps. I could go in there to fetch something and as long as I moved slowly they didn't seem to care. (Though I did try to avoid the day time and visit them when it was pretty cool and they were torpid.) Never even had a close call with those wasps.