It's my understanding that apart of that thinking was you didn't allow your wife to learn of these infidelities, you didn't bring home any STD's or other children and you did not disrespect your wife by flirting with other women in front of her or her friends. Your wife was your wife deserving of attention and respect.
While that's generally true, things were a bit more brazen in Hollywood, particularly under the studio system. As an example: One starlet - and I'm unfortunately blanking on the name - had a child out of wedlock, kept said child in an orphanage (in Europe, I think) for about a year until she decided to 'adopt' a child. And wasn't it such a great coincidence that this baby looked so much like her!
I'm not saying that Desi's behavior approached anything like that, but there was a definite "as long as my behavior doesn't get into the press, I can do whatever I want" feeling with the vast majority of stars at that time. The media wasn't there to knock them down, as it is today.