Once you start using the low "B", it's hard to do without it. It gets integrated into your mental patterns.
That never happens with me. Whenever I spend some time with my 5ers and then revert to a 4er, my mind and my ears go: Now what should be wrong with that (high) D? But then my current band plays largely in keys such as A, G or E which do not cry out for a low B. I also find that if you have guitarists with standard tuning (no drop-D or anything like it), then a low D, C or B leaves a gap between your and the guitarists' frequencies.
The way I play, I need a high E on the G-string or even a double-ocatve 24-fret high G more often than "submerging" below an empty E-string low E. I heard way too much Glenn Hughes when I was young I guess. 5-string sounds in early 70ies hard rock that shaped my playing (Glenn Hughes, Roger Glover, Nick Simper, Jim Lea, Trevor Bolder, Alan Lancaster, Gary Thain, Martin Turner, Gene Simmons) were scarce.
Maybe I should get the 4er after all.
I shouldn't kid myself, I won't turn into a 5-stringer in this life anymore and I've never ever played in a band that needed a low B sound.