Our issues sound similar. Mine is the leather and chrome type. The outer knob is definitely spinning on an inner component that does not allow it to turn off the shaft. I am replacing the old leather chrome knob with a wood burl one that matches my trim. So I don’t care if the old knob is destroyed. I think I may be catching the inner component and that is preventing me from prying the outer knob off. Once the outer knob is off I’m home free. I probably just have to get a better means of prying the outer knob.

 

Peculiar. Mine is easily removed as per these instructions