Don't grind anything down.
A pair of pliers will do fine, get yourself a book (like Erlevine's guitar repair guide) and you'll get the info you need regarding how to pull
out frets from different wood/makes. Fender, as an example - put frets in sideways for a while, so if you pull them out sideways you won't
chip the wood. Wish I knew that when I was 16 and wanted to be Jaco. I unfretted maybe 4 basses I think. Filled slots with wood filler and epoxied the neck and all. For about 15 years, I played almost only fretless.
I actually unfretted a Gibson bass (1967 EB-0) in 2013 (see the LoZ fretless project posted here) just because I play short scale Gibsons now and want to keep it that way. Needed to see if they work as fretless instruments, but they don't. Too muddy and undefined, even with LoZ pickups. I filled with sawdust mixed with wood filler, easy to get out - and glad I did. Refretted it again shortly after.
The good thing, is that you get to plane/sand properly (in this case the fingerboard was oversprayed and ugly) and after the refret
it's good as new:-)
Still looking for a short scale gibson fretless project, though. I think a SB-300 or SB-350 will be a better candidate, having alder body and maple neck.
Anyway, it's easy and fun and all - if you just pay attention to the wood and be careful not chipping it apart.
The most difficult thing is to actually find an instument that works good as a fretless.
And prepare for learning how to refret, just sayin