The Gov. is powerful and all, but would you actually go into war with this man you just met? He convinces them when they remain there and collects his new camp, they will perish. The people that ruined live in a penitentiary, he says, and of course since he is dead, Martinez is not there to correct him. The penitentiary has fences, walls, they may live there if they are willing to take it. We see how he got Michonne and Hershel. “They are the key.” They will help them take the penitentiary without firing a shot.