Bill Wyman was the first (AFAIK) to use fretless in r & r, in his case it was because the frets rattled so he pulled them. But he used it to good effect. Sure, you can get his sound on Paint It Black, Mother's Little Helper etc. with a fretted bass, but he might not have come up with those slides and runs otherwise.
To my simple mind, if it sounds good, it is good, whatever it was played on.