Rhaegar, Tywin, and Doran Martell were plotting to overthrow him together.
Steffon Baratheon, Rickard Stark, Hoster Tully, and Jon Arryn were also plotting to overthrow him together. They had arranged intermarriages between all of their great houses on purpose. Read "Southron Ambitions" theory.
Peace could have been had so easily so many different ways there...