Chelsea struggled to hold on to the ball in the first half and might have gifted a goal to Newcastle.
But Dwight Gayle's indecision reprieved Davide Zappacosta after the Italian's poor pass was intercepted. Willy Caballero then saved a Jonjo Shelvey volley.
Chelsea had half-chances, but their accuracy in the final third was perhaps best summed up by a Danny Drinkwater shot which went out for a throw in.
That changed when Antonio Rudiger fed Pedro down the right and the Spaniard threaded a cross-field pass 40 yards to Eden Hazard, turning the Magpies defence.
Hazard waited and teed-up the on-rushing Alonso, who had the presence of mind to play the ball on for Batshuayi to tuck in. There may have been a suspicion of offside, but without the Video Assistant Referee, there was no review and the goal stood.
Rudiger curled a left-footed shot from 25 yards around the far post before Chelsea, and Batshuayi, enjoyed further good fortune.
Pedro and Hazard combined to find the striker, whose shot ballooned up off the leg of Jamaal Lascelles and in.
Caballero saved from Shelvey and Chancel Mbemba in quick succession before Chelsea resumed on the attack.
Pedro fired just over and Alonso had a volley saved by Karl Darlow.
Alonso had his seventh goal of the season when he curled in a 25-yard free-kick after Zappacosta was fouled.
Darlow denied Batshuayi in stoppage time as the Blues eased into the next round. Perhaps it was his final chance to score a Chelsea hat-trick.
Manchester City were available at 1000/1 with Sky Bet to win the Premier League, FA Cup, League Cup and Champions League at the start of the season but only 80 punters took that price on the unprecedented feat.
But now it's 'just' 20/1, with the club holding a 12-point lead in the title race, into the fifth round of the FA Cup, the final of the League Cup and the last 16 of Europe's premier competition.
As far as Sunday's game was concerned, Kevin De Bruyne and Raheem Sterling scored on the day that a new attendance record of 32,339 was set for a club game at the Cardiff City Stadium.
De Bruyne's eighth-minute free-kick was a candidate for the cheekiest goal of the season as he sent the ball under the Cardiff wall, while Sterling's header gave City a two-goal cushion before half-time.
Cardiff were committed throughout - too much at times with defender Joe Bennett sent off in stoppage time - but the gap between the top of the Premier League and third in the Sky Bet Championship never looked like being bridged.
City booked their place in the final of the Carabao Cup in midweek by beating Bristol City and hopes of an unprecedented quadruple remain very much alive after this impressive showing.
Michy Batshuayi scored twice as Chelsea marched on in the FA Cup with a 3-0 win over Newcastle at Stamford Bridge.
Defeat by Arsenal last Wednesday saw the Blues eliminated from the Carabao Cup at the semi-final stage.
But Chelsea avoided elimination from a second cup competition in a week as Batshuayi's first-half double and a brilliant Marcos Alonso free-kick ensured progression to Monday's fifth-round draw.
Batshuayi could still be allowed to leave before Wednesday's transfer deadline, particularly if a rumoured deal for Roma's Edin Dzeko materialises.
Peterborough 1-5 Leicester
Huddersfield 1-1 Birmingham
Hull 2-1 Nottingham Forest
MK Dons 0-1 Coventry
Middlesbrough 0-1 Brighton
Millwall 2-2 Rochdale
Notts County 1-1 Swansea
Sheffield United 1-0 Preston
Southampton 1-0 Watford
Wigan 2-0 West Ham
Newport 1-1 Tottenham
Liverpool 2-3 West Brom
Liverpool were dumped out of the FA Cup with a 3-2 defeat at home to West Brom in a match littered with Video Assistant Referee referrals which caused confusion on the pitch and in the stands.
Three major incidents in the first half alone required the intervention of technology - including the first time a match referee has used a pitchside television replay.
And although it eventually led to the correct decisions being made the time it took, with almost four minutes elapsing between Mohamed Salah being fouled and Craig Pawson awarding a penalty which Roberto Firmino missed, left supporters and players on all sides angry and frustrated.
It detracted from an impressive Baggies performance who became the first team since Real Madrid in October 2014 to score three in the first half at Anfield as Jay Rodriguez netted twice in the first 11 minutes following Firmino's opener, with Joel Matip's own goal giving the visitors a three-goal cushion.
Salah's first FA Cup goal, and his 25th of the season, set up a thrilling finale but the Baggies held on.
Tottenham avoided a seismic upset in the FA Cup as Harry Kane rescued a 1-1 draw for the Premier League side at League Two Newport in the fourth round.
Spurs sit 71 places higher than Newport in the league pyramid but they fell behind at Rodney Parade when Padraig Amond rose highest to head home Robbie Willmott's cross in the 38th minute.
An ignominious exit loomed for a reasonably strong Tottenham outfit - with Dele Alli and Son Heung-min introduced from the bench - but Kane tapped in at the back post from an 82nd-minute corner to spare the visitors' blushes and send the tie to a replay.
Will Grigg struck twice as League One leaders Wigan stunned 10-man West Ham.
Grigg opened the scoring after just seven minutes at the DW Stadium and added a second from the spot as Latics, winners of the competition in 2013, triumphed 2-0.
West Ham had Arthur Masuaku sent off early in the second half following an off-the-ball incident involving Nick Powell. There were suggestions he may have spat at the Wigan player, with manager David Moyes later labelling the act 'despicable'.
Jon Stead's goal earned Notts County a deserved replay as they came from behind to draw 1-1 with Swansea.
Luciano Narsingh's strike on the stroke of half-time put the Premier League's bottom club ahead but the underdogs responded with gusto and Stead scored the equaliser just after the hour mark.
With Swansea fighting to preserve their top flight status and Notts County battling for automatic promotion from League Two, a replay is probably what neither wanted but the two teams will do it all again at the Liberty Stadium next month.
Jarrod Bowen scored one and set-up another as Hull progressed with a 2-1 win over Nottingham Forest at the KCOM Stadium.
Forest had dumped holders Arsenal out in round three, but looked a shadow of that side as they put in a lacklustre display following manager Aitor Karanka's decision to make 10 changes from the side which won at high-flying Wolves last time out.
The hosts made eight changes of their own, but Bowen was one of those to keep his place and showed why he has been linked with several Premier League clubs in a classy display, grabbing his 12th goal of the season before assisting Nouha Dicko.
Apostolos Vellios struck two minutes from time but it was too little, too late for two-time winners Forest.
Glenn Murray came off the bench at Middlesbrough to fire Premier League Brighton into the fifth round without knowing a great deal about it.
The striker, who was this week arrested along with his wife as part of a £1.1million tax investigation, was in the right place at the right time for defender George Friend to blast a 90th-minute clearance straight at him and look on as the ball flew into the Boro net.
It was tough luck on the Sky Bet Championship side, who had hit the post before the break through Adama Traore, although the much-changed visitors were significantly better after that to clinch a 1-0 win.
Lukas Jutkiewicz's second-half equaliser earned Birmingham a deserved replay as tie at Huddersfield ended 1-1.
Jutkiewicz arrowed home a low shot nine minutes into the second period after Steve Mounie's emphatic header had given the Premier League side an interval lead.
Another bite at the cherry was just reward for Steve Cotterill's Sky Bet Championship strugglers, who comfortably held their own against Huddersfield, who still look out of sorts after their recent top-flight struggles.
Captain Billy Sharp sent Sheffield United into the fifth round of the FA Cup for the first time in three seasons, as they emerged 1-0 victors over Preston in a tight contest at Bramall Lane.
The Blades haven't progressed past round four since their run to the semi-finals in the 2013/14 season but Sharp's 80th-minute penalty saw them move into the next round with consecutive 1-0 wins.
Meanwhile, it was disappointment for the Lilywhites, as they came into the game with only one loss in their last seven FA cup ties away from home but instead suffered their first defeat to United in the FA Cup since 1936.
Jack Stephens marked his 24th birthday with his first goal for Southampton as they progressed with a 1-0 home win over Watford.
The visitors' new head coach Javi Gracia was overseeing his first fixture since his arrival to succeed the sacked Marco Silva, but their form remained unconvincing as they fell to a ninth loss in 13 games.
After Stephens' early goal they played with the same lack of conviction that had contributed to Silva's dismissal, finishing fortunate not to lose by a wider scoreline to a team also struggling, and who in that same period have won only twice.
Ben Thompson's 90th-minute equaliser salvaged a 2-2 draw for Millwall against League One strugglers Rochdale in what was a gripping tie.
The Lions, quarter-finalists in last season's competition, were on the brink of being the victims of a shock, with the Dale making light of the 32 league places between the two teams.
Matt Done's goal put them agonisingly close to reaching the fifth round for just the third time in their history, but they will get a second crack in 10 days' time.
Maxime Biamou's second-half winner saw 1987 FA Cup winners Coventry cause a minor upset and progress into the fifth round with a 1-0 win over League One strugglers MK Dons.
Fast becoming one of this season's memorable FA Cup stories, the Sky Blues saw off Premier League Stoke to make it to this stage and they were backed by almost 8,000 travelling supporters, as well as Sporting Life tipster Nick Hext who made them his best bet of the afternoon.
New signing Fousseni Diabate and Kelechi Iheanacho both struck twice as Leicester advanced with a 5-1 win at Peterborough in another result which was favourable for our followers.
Onyinye Ndidi also got on the scoresheet in the latter stages as Claude Puel's side brushed aside their League One opponents at the ABAX Stadium, Matt Brocklebank having weighed in with a hefty bet on over 2.5 goals - which was paid out after half an hour.
Diabate made an instant debut impression by opening the scoring after just nine minutes and Iheanacho, who also grabbed a double against Fleetwood the third round, added two more before the interval.
Andrew Hughes shot through a crowded area to give Posh a glimmer of hope just before the hour but Diabate and Ndidi put the gloss on Leicester's performance in the closing minutes.