The American world number one avoided an upset loss, thrashing countrywoman Vania King 6-1 6-0 in the second round.
But, despite already being a favourite to win the grand slam, Williams' half of the draw opened up.
Estonian Kaia Kanepi upset Stosur in three sets, while Ivanovic was comfortably beaten by Czech Karolina Pliskova.
Williams was surprised by those losses, but said she was unfazed as she aims to go beyond the fourth round at a grand slam for the first time this year.
"I know they had a wonderful summer, so that's surprising," she said.
"But, you know, again, I promise you, I'm trying to make it through a few rounds in a grand slam this year.
"I'm really not too much concerned about the later seeds because I haven't made it there this year. So hopefully this year I can do something."
Williams is aiming for her 18th grand slam title, which would draw her level with Chris Evert and Martina Navratilova.
But the 32-year-old said she never dreamed of even winning 17.
"I never even thought I would have as many as I even have. Who am I? I'm just an average person. I never thought this would be me. So for me it's incredible," Williams said.
"I never really had a number. Obviously now that things are reachable, I do think of different things. But in the big picture, it's just all an amazing, amazing ride."