Brendan Rodgers' men, who have lost three of their opening five Premier League fixtures, were far from their best again on Tuesday as they drew 2-2 after extra time in a pulsating encounter.
And the incredible action continued into the spot-kicks, with some players required to take second penalties after 20 of the first 22 were converted. A further seven followed suit until Middlesbrough's Albert Adomah missed the target to secure the Liverpool's progress.
It appeared a comfortable night was in the offing for Liverpool when teenage debutant Jordan Rossiter struck in the 10th minute, but Adam Reach headed the Championship side level midway through the second half to force extra time.
Suso's first Liverpool goal looked to have put his side through, but Patrick Bamford equalised from the spot in injury time of extra time before the Merseyside club finally progressed to the fourth round.
A sublime strike from Nathaniel Clyne dumped Arsenal out as Southampton won 2-1 at the Emirates Stadium.
Right-back Clyne fired home from 30 yards out five minutes before half-time to ensure that Ronald Koeman's impressive start to his Southampton tenure continued with a victory over an Arsenal side that included 10 changes from the starting line-up at Aston Villa on Saturday.
Alexis Sanchez had put the home side in front with a stunning free-kick, but Arsene Wenger's wait to win this competition for the first time goes on after Dusan Tadic equalised with his first goal for the club from the penalty spot and Clyne's thunderbolt put the south-coast club through.
Nathan Dyer continued his early-season goalscoring form as Swansea City defeated Everton 3-0.
The diminutive winger had netted three Premier League goals in five games coming into this clash at the Liberty Stadium, and he made it four for the campaign with a composed strike in the first half.
Gylfi Sigurdsson then added a second after the interval as he reacted first to Sylvain Distin's wayward header coming back off the crossbar before substitute Marvin Emnes fired in a third with three minutes remaining.
Marc Muniesa scored a brace as Stoke City came from behind to progress with a 2-1 win at Sunderland.
The defender fired home in each half to score his first goals for the club and give Mark Hughes' men their first victory at the Stadium of Light after Jozy Altidore had put Sunderland ahead.
League Two side Shrewsbury Town claimed another scalp as Norwich City were beaten 1-0 at Greenhous Meadow, with James Collins scoring the only goal of the game.
Micky Mellon's men won at Premier League side Leicester City in the second round and they march on into the last 16 after pulling off another shock victory.
Benik Afobe helped himself to another double as MK Dons beat League One rivals Bradford City 2-0.
The on-loan Arsenal striker scored twice in a stunning 4-0 hammering of Manchester United in the second round and he was at it again at the expense of 2013 finalists Bradford.
Bournemouth were 3-0 winners at managerless Cardiff City, while Derby County beat Reading 2-0 in the other all-Championship tie and Fulham saw off Doncaster Rovers 2-1 for their first victory under caretaker boss Kit Symons.
Sheffield United progressed with a 1-0 triumph at fellow League One side Leyton Orient.