If a Christian does wrong, it is good that he feels guilty for it shows the Holy Spirit is at work in his life. Jesus said of the Holy Spirit, “… he will convict the world of its sin.”(John 16:8).
Guilt plays a big part in confession. When the prophet Nathan confronted King David about his sin, the king confessed to Nathan, "I have sinned against the Lord" (II Samuel 12:13). Guilt is like a warning light on the dashboard of your car telling you that something is wrong.
When God shows us our wrong, how do we handle the guilt?