I agree, it's a horrible feeling. Half of our sessions end that way right now, for lots of reasons: we are learning how to communicate effectively with each other, she's struggling with initial defensiveness in response to my pain from the EAs, and I'm still dealing with feelings of anger from time to time. I'm sure there are other reasons, too.
We have our sessions scheduled in the evenings with a babysitter so that we can go on a dinner date after. We continue to talk about it after a cooling down period in which we get into the car and decide where to go to dinner. Sometimes we are quiet half the way there, but eventually we start talking about it again.
I think it's to be expected, even though it feels crappy. I had grand expectations of R and WSs capability for demonstrating remorse due to the wonderfully humble WSs found on the website. She's just not there yet and things will be rocky until then, as I will continue to struggle with trusting her until she is.
Good luck. I agree with the pp. I wonder if giving in and running to "make up" is part of the maladaptive pattern in your relationship that needs to change. If that's true, the odds of him coming to you to do so are nil at this point. A continued discussion about the conflict, however, is not giving in. It's being respectful, genuine, honest and authentic in your interactions. Though it is difficult and you will have arguments as you adapt to this way of being, it will be worth it in the long run.
...or that's what I keep telling myself.