All this is true, and of course we already know you're not factoring in risk, as you could also lose money in your stock account. But the kicker to me is that I have a 6 month fully-funded emergency fund. And yes, I've done the Dave Ramsey plan, in case you haven't already picked up on that.
But again, I don't care whether you pay your house off or put your money in the stocks.....the key here is that you are budgeting that number for an asset....whatever it is. And not spending it. It's all just a math equation. I'm just a little more on the conservative side, and I want that house payment gone. Especially since at this point, it's also an investment for me (I got mine at a great deal and see no way it can ever go below where I got it again).
ETA: And also.....with a paid for house......whenever you make that final payment, will be the last time you'll ever have to worry about living at that level. Assuming you are using that 14K or whatever your payment used to be, and sticking that in assets, with no payments, your wealth will take off exponentially. It's a slow slog to get there, but once you do, it's worth it. That's why we should be telling our kids to do these things early, so they get the best of both worlds. Or better yet, invest in real estate early and house hack. In college, early life, whenever.