What Clouds said. He's not concerned that any of this caused you pain: he's concerned because your response/loss is causing HIM pain. All the flowers and the offer of financial help and grand gestures are just tools to use, not from the heart. I'm sorry but he needs tons of IC before he's going to be in a place where he can actually have remorse (I'm sorry what I did hurt you so bad, you are a person with intrinsic value who did not deserve this) as opposed to regret (I'm sorry the consequences of my actions are hurting ME so bad, you are a prized object and the loss of you causes me discomfort and pain. And possibly loss of status and respect from my wider social circle. And a hit to my income/lifestyle. Poor me!) One is about actually valuing you as a person and wanting to make amends and (crucially) would accept if the best outcome for you would be to divorce and start your life fresh. The other is about valuing your utility to their lives plus some emotional attachment, but not enough to prioritize your best interests over their emotional, financial, and social needs. If it causes you further pain and suffering to stay together, if it's something you can't get over, that's ok: it would ease theirs so they're willing (for you) to make that sacrifice.
At the same time, don't think that he isn't experiencing genuine, urgent, intensely painful emotional distress over this. It will look like he's suffering and genuinely wants you back. That's because he is, and he does. But that's not enough. Why he wants you back and if he's willing to let you go so you can heal are what matter. If you don't have that, you don't have a safe relationship. It will just happen again, because he has not evolved to the point that he sees not hurting you as more important that not hurting himself or really, giving himself pleasure at your expense. Treating his well being as paramount and yours as unimportant. That's why the 180 is crucial. That's why so many WS advise other WS they have to be ready to lose the marriage, and do the work anyway. One, because they will just mess up future relationships if they don't, and two, it's the best way not only to in reality be able to prioritize your BS's well-being over your own needs to maintain the relationship at their cost, but also to make them FEEL that this is real and a fact.
I know you've seen him in the past prioritize your well-being over things he may have wanted in the past. You wouldn't have married him if you hadn't. But clearly, he's not capable of it when he really, really wants something, and you would be seriously hurt by it. And the thing was sex. Don't let him blame-shift. That's what it comes down to. He could have broken up with you. But it sounds like that was never on the table; he just wanted what he wanted, when he wanted it. He now has to prove to you that he understands why he would decide that was ok to do to the person he claims to love and married, and that he has worked through it to point it won't happen again. He also has to prove to you that he loves you to the point that he would let you go if that was what was healthier for you, because he loves you as a person. Otherwise, you're right back to "well, I do love you, but not enough to not go for something I desperately want at your expense. You can deal with being a little broken if it makes me whole again."
Instead of sex, this time it's you staying in the relationship. But it's the same damn thing. And that's what you would pattern as what a relationship is for your kid. I know you don't want that.
He can only prove to you he's worth risking more years of your life on with time and actions, not words or flowers. Right now the flowers and constant disregard of your boundaries are showing that he's NOT safe, that he doesn't care that reminders of him and contact with him are hurtful to you. You've told him what you want and he said OK and just did what made HIM feel better anyway, even though you made it clear it comes at a cost to you and you don't want top pay that cost. And nothing you have asked for is wrong, or bad, or unreasonable. You're newly pregnant, under a pressure cooker of stress, and emotionally devastated. You are entitled to all the time you need, and anything else that reduces the stress on you. He doesn't get to prioritize the distress and uncertainty he feels right now over your mental and physical health. He is F'ing up right now and violating your legitimate and valid boundaries to make himself feel better, just like he had sex with someone else because he wanted to feel good. Same motivation, selfishness. HIs reaction to temptation is selfishness and his reaction to emotional distress is selfishness too. Maybe with time and intense counseling he will make himself into a good person (not a nice one, a good one! Who respects your mental and emotional health to the point he won't damage them to make himself feel better/pleasure!)) and become a safe partner. It's far more likely that he won't, because it's a monumental task. The best thing you can do for yourself is to hope for the best, but prepare for the worst. Let you and your IC come up with good solid boundaries for how much contact (which can be no contact for a good while) you can have with him. Let him know that you know that his violations of your boundaries will each be a nail in the coffin of divorce, because you see them for what they are - selfishness, prioritizing his emotional needs over damage to your own. Remember you taking care of yourself is not taking actions to hurt him. It's the consequences of his own actions hurting him. If he beat you and put you in the hospital the bills wouldn't be you trying to punish him financially. You needing time and space to heal your emotional wounds and help reduce stress so you body can grow a healthy baby are not any different.
All my love and support to you. Put yourself and your baby first in this, always.