Possible conflict between my asylum and my employment based green card

[am8084]

Once you get your green card you can go visit your country safely without worrying about visa difficulties. Unfortunately it's not that simple in my situation. I got my employment based green card few days ago, but I had already been granted asylum (months ago) for fear of persecution in my country on the basis of my religion (by armed groups and not by government). Situation in my country is a little better and I'm less afraid of persecution based on religion and considering visiting my family (a couple family members are sick).

Can I go visit my country or will that be considered to be the basis to accuse me of fraudulent asylum application with the risk of deportation or revoking my green card? Can it be the basis of denying my application for citizenship in the future?