All of the above is what you should have led with. Those are way more valid reasons why it is difficult to win there than how many times they have to play Bama.
It all plays into the same story. You are right -- I was guilty of attempting to oversimplify initially. I just don't think anyone looks at them as an elite job. Or if you want to say, since they are technically a legitimate national title contender on paper and historically that they are an elite job, they are the worst/most difficult job in that category nationally. There's a reason no one avenges losses to Saban in that title game. Also a reason no one has managed to make it back to see them again since he built the Bama monster. He ends their seasons before they get that chance.
Maybe they actually pull off this Gruden thing how some of their message boards are claiming, he brings Tee Martin back, they recruit their asses off nationally, and can compete. But personally I also wouldn't be surprised to see them actually end up drastically overpaying for a "sexy" hire that is actually less proven than Butch Jones was at the time. That job has been turned down by a metric 17 ton of coaches from schools traditionally considered worse jobs than UT in the last decade. I can think of a half dozen or more off the top of my head and I didn't follow those searches particularly closely.