The Italian took the lead early in the race when Kenny Roberts's Suzuki fell victim to the wet conditions and he quickly built a five-second lead at the Shanghai International Circuit.
France's Olivier Jacque, a replacement rider for Kawasaki and a former world 250cc champion, ate into Rossi's lead over the final laps but had to settle for a surprising second at the end of the 22-lap, 116-km race.
Italy's Marco Melandri on a Honda was third after overtaking Spanish team mate Sete Gibernau, whose back tyre shredded, on the last lap.
The win extends Rossi's lead in the championship after he won the first race of the season in Spain and was second in Portugal.
Rossi, who finished in 50 minutes 2.463 seconds on Sunday, has 70 points to Melandri's 45 and 43 for Brazilian Alex Barros.
Rossi, sixth on the starting grid, said he had not expected to triumph in the wet. "I think it was the biggest surprise of my career to win today."
The 125cc race was delayed by 30 minutes because of the heavy rain and visibility was very poor when the MotoGP
started.
Gibernau, a wet weather specialist, was on pole position but he lost the lead on the first rain-soaked corner by going wide. The riders took it easy over the early laps although Carlos Checa and Troy Bayliss fell.
Once Rossi had taken the lead, there was going to be only one winner and interest was centred on the performance of Jacque who started 15th on the grid.
The 31-year-old was called up for the Chinese race after German Alex Hofmann cracked his wrist in a demonstration ride last month. Jacque showed extraordinary skill on a new bike on a wet track to finish just 1.7 seconds behind Rossi.
"It's like a dream, it's a big surprise," said Jacque. "It's very strange to be back after one year. I feel like a hair in a soup, or rather a fish in water."
More from rediff