When you start an argument with this, you are immediately accusing him of something, implying that he always does something wrong.
That is not the way to have a healthy conversation and it will only make him choose the defensive route.
Instead of making sweeping accusations, focus on the thing in question and the present. If he has to defend himself all the time, he’s just not going to take it that much longer.