Fear of unfretted basses is unfounded - seriously. I was always hesitant, but when I finally made the jump I took to it like a duck takes to water (and surprised myself, I'm generally not talented learning new things, it's all hard work and patience with me, as my son once put it so disrespectfully, yet aptly: "
For your limited talent, you've turned into quite some bass player, dad."
). The secret is just doing it and not thinking too much about it. You'll be surprised how good your muscle memory will be.
That is not to say that fretted basses don't have their place. I alternate as whim strikes me. The band guys by now don't even notice anymore what I'm playing unless I do blatant "yeoooow" fretless slides.