This reminds me of the early Christian battle regarding food and circumcision. The folks in Jerusalem insisted that new converts must conform to the Jewish laws on these matters, but Paul told them that these little things don't matter. He told them that if they felt it was a sin to eat unclean food or not to be circumcised, then to them it was sin, but don't try to impose it on others, because to them it is not sin.
But here is my concession to reality. Religion is about appearance. Beards, hair, vestments, crosses, jewelry, no jewelry, makeup, no makeup, burkas, hijab... appearance is what defines many religious people. Deviation from what defines them is an attack on who they are. Others can't see what is in their heart, but their apparel is there for all to see. So they say that their appearance reflects what is in their heart. Does it? The Amish didn't want to change their looks because in appearing plain they wouldn't be noticed. Now their "plainness" is their pride and makes them a spectacle. Does the modesty of the Taliban make them a beautiful thing? No. Does the wearing of vestments and crosses make anyone more Christian than anyone else? No. Neither apparel nor lack of apparel make a person more Christian, it just makes them more dressed or less dressed. So in my opinion, if not wearing a ring makes one feel more Christian then by all means don't wear the ring. If that is what defines your Christianity, then by all means keep your identity, just like the early Christians who wanted to be defined by their food and circumcision.