Generally they avoid humans because we're weird, alien creatures that they're unsure about. Even the really big ones will ignore us or at worst give an exploratory bite then decide we taste wrong, most of the time.
The little guys are funny though, if you're surfcasting from a sand bar they'll leave you alone but quickly figure out that you're throwing all the paddlecrabs that try to steal the bait off your hook back into the water and hang about for some free lunch.