Forgiveness is an act of showing one's superiority over an offence / offender, So if you can't forgive, it simply shows that you are no way better than the offender.
If you are finding it difficult to ask for forgiveness from anyone you 've / might have offended, it implies that you are suffering from the complex of Immaturity because, only the matured have the boldness / courage to approach anyone they offend and ask for forgiveness.
Some say that pride is what makes it hard to ask for forgiveness but the truth is, Pride is a manifestation of immaturity.
Asking for forgiveness is a way of showing your maturity.
What is so hard in "am sorry, forgive me"?.