Carroll has endured a difficult time at Upton Park since signing permanently in June 2013, having been on loan during the previous campaign.
The forward has been beset by injury problems and faces another spell on the sidelines as he requires surgery on an ankle complaint.
He made just 15 Premier League appearances last term and Sullivan, who has previously confessed to having made a mistake in signing Carroll, knows reinforcements are needed in his absence.
Speaking to talkSPORT, he said: "We have to bring in another striker and we're working on it now to buy one, or loan one, or both.
"You can never have enough strikers and really with Andy gone we haven't got enough."
On Carroll's purchase, which cost the club a reported £15million, he added: "It was desperate last year. We probably made a mistake by putting all our eggs in one basket.
"Had we known what we know now we wouldn't have signed the player. But it's easy to be wise after the event.
"We were assured by our physio - who has since left the club - that the player would be available for the start of the season.
"He wasn't available until January or February. A mistake was made and we were given the wrong information. We based our decision and our purchase of the player on what the medical department told us."
West Ham have recruited attackers Enner Valencia and Mauro Zarate in the close-season.