The CAF Champions League winners won 5-4 in the shootout after the teams were tied at 2-2 at the end of 90 minutes.
And Madoui, whose side suffered a shock defeat to Auckland City in the previous round, was proud of his players' resolve.
"Today you could see that my team performed differently to the way they did in the quarter-final," he said. "Sydney are a good side but we were determined and well-prepared psychologically.
"We made a few mistakes in the first half but after the break we created several chances.
"My players were absolutely determined to win this watch and I'd like to congratulate them because they deserved the victory."
Western Sydney moved into a 1-0 lead courtesy of Romeo Castelen's fifth-minute strike, before a Daniel Mullen own goal and an Abdelmalek Ziaya effort turned the match on its head.
Vitor Saba forced the match to penalty shootout by scoring two minutes from time, but it was Setif who came out on top, with Toufik Zerara scoring the decisive penalty.
"We played well," remarked Wanderers coach Tony Popovic. "We made a few changes and gave a lot of young players the chance to play and they made the most of it.
"Penalty shootouts always come down to luck. We were by far the better team in the first half but were unable to score more goals.
"In the second half we allowed our opponents back into the game by making a few defensive mistakes.
"The result is disappointing but we played good football and can take a lot of positives home with us."