You BOTH need to speak to someone who is impartial - perhaps a counselor.
I do not believe that people say something drunk that they didn't think of when they were sober. Inhibitions drop, but thoughts don't come out of nowhere.
I'd be filing for divorce, but that's just me.
Oh, wait, I did divorce a husband who was a drunk who said painful, harsh, unfair things to me and other people when he was drinking. My fear was always that we would have a child, and that child would be exposed to his abusive behavior. I got out of the marriage before that could happen.