You're right, this could basically spark a civil war inside King's Landing.

It's worth noting that after the War of the Five Kings in book 3 (seasons 3 and 4 of the show) the events leading up to Cersei's walk of shame are in a book called "A Feast for Crows".

Basically referring to all the players left after the war, trying to grab for themselves whatever scraps are remaining. Hence Cersei's shortsightedness in rearming the Faith Militant inside King's Landing's walls.