Liverpool went down to 10 men and came from behind to win the game for the first time since football was invented the Premier League was formed 24 years ago.
For the first hour of the game the reds were hapless and deserved to go a goal down early in the second half.
James Milner received his marching orders shortly after the hour for a second yellow card - shortly after Klopp had made a tactical substitution moving Milner to right back.
10 minutes later Firmino was on hand to profit from a defensive error by Palace and late substitute Benteke was brought down for a penalty in the 4th minute of injury time.
He duly despatched the penalty to give the reds an unexpected but welcome 3 points that give Liverpool their 3rd consecutive Premier League win - the first time in just under a year.
Milner will miss the next game for the reds but that won't arrive until after the next two games - home and away against Manchester United in the last 16 round of the Europa League.