Psychologically it seems, smoothness can almost be seen as a "secondary sex characteristic" for of a lot of gay men(not for me, but I have noticed it in others). The ritual of actually shaving yourself could be interpreted by ones self as a sort of personal rite taken for the purpose of sex i.e., the ritual of shaving yourself, in order to satisfy those who like smooth men, is telling yourself you're doing something in order to have sex some time in the future.
Perhaps another level might be conditioning. If you know you're going to have sex and you like to be smooth or you know your partner likes no hair you'd probably have a quick shave if you had time. If you were going to some LGBT event with hopes of meeting something, you might including shaving in your bathroom agenda that day/weekend. every time sex (and the experience of sex with a freshly shaves pair of balls) happens it would condition stronger and stronger the link od shaving balls >having sex.
Then on the final level, yes of-course your handling yourself and I imagine, feeling all sort of sensations pre and post shaving. This feels strange on your face, so One can easily imagine when it's happening all over an organ associated with mostly pleasure, or in Stephen Fry's case pleasure only (for pissing is an enjoyable experience), then well, what would you expect would happen? your testicles are saying "pleasurable sensations? I'll get my hat".
All in all you got a pretty potent sub conscious, conditioned and physical cocktail of 'erection on the beach'.
I would never shave my balls. for me personally, I think it's weird.