Ayoze Perez and Georginio Wijnaldum had left the reigning Barclays Premier League champions looking at a fourth successive defeat at St James' Park, but the Brazilian duo came off the bench to claim a 2-2 draw as Steve McClaren came within four minutes of a first league win of his reign.
It was tough on the Magpies, who turned in their best performance of the campaign to date, but even the most one-eyed of home fans could not deny that the visitors deserved something for their efforts after launching a concerted fightback.
Nevertheless, Jose Mourinho's men find themselves eight points adrift of new leaders Manchester United after an uncharacteristically mixed start to the season.
Harry Kane ended his goal drought to make it 3-1 to Spurs
Manchester United replaced Manchester City at the Barclays Premier League summit on a day of contrasting fortunes for the rivals.
City slumped to a third straight defeat in all competitions - and to second in the league - as Tottenham enjoyed a handsome 4-1 at White Hart Lane.
Manuel Pellegrini's side, with Joe Hart only judged fit enough for a place on the bench, took the lead through Kevin de Bruyne but then saw Eric Dier, Toby Alderweireld, Harry Kane and Erik Lamela net for the hosts.
Barclays Prem
Liverpool 3 - 2 A Villa
Leicester 2 - 5 Arsenal
Tottenham 4 - 1 Man City
Man Utd 3 - 0 S'land
So'ton 3 - 1 Swansea
Newcastle 2 - 2 Chelsea
West Ham 2 - 2 Norwich
Stoke 2 - 1 B'mouth
Barclays Prem Results
For Kane it was the end of a 748-minute duck in club football.
Barclays Prem
1 Man Utd 16
2 Man City 15
3 West Ham 13
4 Arsenal 13
5 Tottenham 12
6 Leicester 12
7 Liverpool 11
8 Everton 9
9 So'ton 9
10 C Palace 9
11 Swansea 9
12 Watford 9
13 Norwich 9
14 W Brom 8
15 Chelsea 8
16 B'mouth 7
17 Stoke 6
18 A Villa 4
19 Newcastle 3
20 S'land 2
United had a much more straightforward afternoon against Sunderland at Old Trafford, leapfrogging City by a point with a 3-0 win.
The decisive goals came either side of the half-time whistle, Memphis Depay putting his side ahead at the end of the first period and Wayne Rooney bundling home from close range shortly after the restart.
England captain Rooney had gone 1,000 Premier League minutes without a goal, but he now sits level third alongside Denis Law on United's all-time list.
Juan Mata later added the gloss.
Under-pressure Liverpool boss Brendan Rodgers had Daniel Sturridge to thank for a much-needed 3-2 win over Aston Villa at Anfield.
James Milner eased the nerves with a second-minute opener before Sturridge's fine volley made it 2-0 in his second start after a lengthy injury lay-off.
Sturridge grabbed a second in the 67th minute, with Villa's Rudy Gestede scoring two of his own either side of that to make Rodgers sweat over the points.
Arsenal ended Leicester's unbeaten start to the campaign to go third with a 5-2 win, Chilean forward Alexis Sanchez opening his account for the season with a hat-trick.
Claudio Ranieri's Foxes took the lead at the King Power Stadium through Jamie Vardy's 13th-minute effort, only for Theo Walcott to level within five minutes.
Sanchez took centre-stage with goals in 33rd, 58th and 81st minutes before the in-form Vardy and substitute Olivier Giroud netted late on.
Stoke picked up a first win of the season at home to Bournemouth, Mame Biram Diouf settling a 2-1 victory in the 83rd minute after goals from Dan Gosling and Jon Walters had cancelled each other out.
Cherries striker Callum Wilson suffered a nasty first-half injury.
Southampton saw off visiting Swansea 3-1, as Virgil van Dijk, Dusan Tadic and Sadio Mane established a healthy advantage in little more than an hour.
Gylfi Sigurdsson pulled one back from the penalty spot, Swans' first goal in September.
At the Boleyn Ground, Cheikhou Kouyate stole a 2-2 draw for West Ham in the dying seconds against Norwich.
Nathan Redmond thought he had won it for the Canaries seven minutes from time after strikes from Robbie Brady and Diafra Sakho.