The hosts, chasing 101 to win the opening Test of the three-match series, slipped to 0-3 in the fifth over before Mahmudullah (28) and the captain rescued their run chase to record victory.
It was left to Taijul Islam (15 not out) to hit the winning runs to complete a dramatic match for the 22-year-old, having already claimed Test-best figures by a Bangladesh bowler of 8-39.
Mushfiqur highlighted his side's resilience to complete the victory, but admitted they still have plenty of improvements to make in the remainder of the series.
"We are not where we want to be but the boys played very well and a win is a win," he said in his post-match interview. "There's a lot of areas to improve on, both our batting and bowling.
"Taijul has been great for the last couple of matches, Shakib [Al Hasan] also bowled really well and Jubair [Hossain] too.
"It's a total team and all-round effort."
Zimbabwe skipper Brendan Taylor was disappointed with his side's showing, though, admonishing their "weak" mentality in throwing away a winning position.
"Credit to Bangladesh, we were mentally weak with the bat, we can't win Tests like that," he said.
"Our bowlers showed a lot of character. You have to have the mindset that you can win from any situation. We started well and had them three down but dropped two chances.
"They played better cricket and we'll come back stronger in the second Test."
Both teams move on to Khulna for the second Test of the series that begins on November 3.