Teenager Sullay Kaikai had fired a last-gasp equaliser on his senior Palace debut to deny Newcastle victory in normal time.
Palace's 19-year-old academy graduate poked home in second-half injury time to stun the visitors, only for full-back Dummett to wrestle victory for the Tynesiders.
Emmanuel Riviere's coolly-taken brace put Newcastle on the brink of victory, before Kaikai made the scores 2-2 on the brink of full-time.
Dwight Gayle's accurate penalty handed Palace a first-half lead, but Riviere then notched the first goals of his Magpies career to end his six-game drought.
Manchester City scored seven times in the second half as they began their defence of the Capital One Cup by thrashing Sheffield Wednesday 7-0.
Boss Manuel Pellegrini named a strong team but City were frustrated by a spirited performance from the visitors in a goalless first half as they sought a first win in four games.
The contest turned in a frenetic 15-minute period at the start of the second half; Frank Lampard, Edin Dzeko and Jesus Navas netting before Yaya Toure scored from the spot following a foul on Lampard that earned Kamil Zayatte a red card.
Dzeko added the fifth 13 minutes from time and teenager Juan Angel Pozo marked his debut with a late sixth before man-of-the-match Lampard rounded off a 7-0 win.
Premier League leaders Chelsea made hard work of Bolton but they advance to the Capital One Cup fourth round thanks to Oscar's second-half strike in a 2-1 win at Stamford Bridge.
Goalkeeper Petr Cech was captain, starting for the first time this season after being displaced as first choice by Thibaut Courtois, but his services were barely required as the Blues dominated.
Andre Schurrle spurned a host of chances and Bolton goalkeeper Andrew Lonergan was in fine form to deny the hosts.
French defender Kurt Zouma, on his debut, struck from a corner after 25 minutes to break the deadlock but the anticipated flood of goals did not materialise.
Chelsea's profligacy was punished and their defensive frailties exposed when Bolton captain Matt Mills headed in an equaliser six minutes later with their only real opportunity.
It denied Cech a clean sheet and leaves the normally miserly Blues with one shutout in seven matches this season.
Oscar saved his side's blushes by striking from 25 yards out after 55 minutes for what proved to be the winner in a contest which should have been far more comfortable for Jose Mourinho's men.
Tottenham edged the battle of the B teams with Nottingham Forest, sneaking into the fourth round after Ryan Mason's wonderful long-range effort inspired a late comeback victory.
A combined total of 20 changes were made by Mauricio Pochettino and Stuart Pearce at White Hart Lane, with this weekend's north London derby and Forest's promotion push the clear priority.
It had looked like Forest would leave victorious after Jorge Grant's second-half effort, although a wonderful strike from Mason was followed up by a poacher's goal from Roberto Soldado and a fine Harry Kane attempt as Spurs ran out 3-1 victors.
West Brom's late double blast stunned Hull to send the Baggies through.
Gareth McAuley and Saido Berahino scored in the final three minutes to rescue Alan Irvine's men and steal it 3-2.
Robbie Brady and Tom Ince looked to have won it for the Tigers after the visitors had hit back with their own quickfire goals either side of the break.
But Berahino's winner two minutes from time handed Albion a trip to Bournemouth in the last 16.
Brown Ideye had given the Baggies the lead, the £10million man's first goal for Albion, in an unremarkable game before a thrilling finish at The Hawthorns.
Brighton breezed into the fourth round of the Capital One Cup with a 3-0 victory that ended Burton's giantkillking run.
The visitors ended the game with 10 men after skipper Inigo Calderon received his marching orders for a dangerous high challenge on home midfielder Robbie Weir in the 88th minute, but the game had long slipped away from the League Two side.
The hosts had dumped out Wigan and QPR but goals from Rohan Ince, Kazenga Lua Lua and Craig Mackail-Smith justified Seagulls boss Sami Hyypia's decision to make nine changes to the side held to a goalless draw by Championship strugglers Blackpool on Saturday.