"It will have to be sufficient for me," said Palace manager Roy Hodgson. "The referee gave the penalty and I am just happy we scored it this time - the rules of the game prevent people from pulling shirts."
It probably should have been all three points for the home side who had more than 62% of the possession and 14 shots in the second half.
Palace had also started strongly and Newcastle goalkeeper Karl Darlow had to be alert to save from Wilfried Zaha and Benteke in the first half.
Crystal Palace have been awarded six penalties in the Premier League this season, with five of those coming at Selhurst Park
But they will have to settle for a three-point cushion above the relegation zone with 12 games remaining after Milivojevic's fourth goal in eight games.
"The way we were playing in the first half I would certainly have taken (a point). After watching the second half and watching us dominate the game as we did, I feel we were unlucky not to have got all three points in the end because our second half performance merited that," said Palace manager Hodgson.
"It's a point that doesn't help either of us enormously but we've just got to make sure that when we can't win, we draw."
Palace have lost only three of their 17 league matches since a 1-0 defeat at Newcastle in the reverse fixture in October.
And with that turnaround coinciding with Hodgson's arrival in September, it is the first time Palace have enjoyed clear daylight above the bottom three.