Newcastle showed their defensive quality by holding out against a Chelsea side who had 70% possession during the game and created the vast majority of the chances in freezing conditions on Tyneside.
If there was a touch of fortune to the result, Bruce will not care.
It was another "backs against the wall" performance, and until the 94th minute I'd have been happy with a point!
Bruce's side has been struck by a succession of injuries this season, and he will be delighted that the returning Saint-Maximin created the goal which lifts his side to 12th place in the table.
The result also earned a fourth game unbeaten for the first time this season, albeit with two of those games coming against Rochdale in the FA Cup.
Despite all of Chelsea's endeavour, it was Newcastle that created the game's other best chance when Joelinton crashed a header against the crossbar.
With the £40m summer signing breaking a 20-game goal drought by scoring against Rochdale, Saint-Maximin offering trickery on the wing after a hamstring injury and Miguel Almiron scoring in three of the last four games, Bruce may feel his luck is turning.
The cheer that greeted the final whistle underlined how elated and surprised the home crowd were as the hosts earned their first Premier League win since December 21.
Chelsea came into the match unbeaten since Boxing Day but it was another game where Lampard's young side showed their inexperience in front of goal.
Abraham, who has excelled in his first season at Chelsea with 15 goals, was wasteful at times, but there were also missed chances for Mason Mount, who headed over, plus Willian and Emerson, who could not find the target.
Despite their links with a striker during the transfer window and the potential departure of Olivier Giroud to Inter Milan, Lampard has said nothing is "imminent".
But he showed his frustration on the touchlines as his side failed to take advantage, much as they did in a 1-1 draw at Brighton earlier this month.
Lampard's faith in youth has drawn praise this season, and while home losses to Bournemouth and Southampton have been countered by away wins against Tottenham and Arsenal, this result will offer the new Chelsea boss plenty of food for thought.
His strict adherence to playing one lone striker may also be something he will consider in games where teams sit behind the ball and try to break as Newcastle did so successfully.
And there may be further questions about Arrizabalaga, who has had a mixed season, and looked like he could have done more to prevent the winner going in.