With deep research into historical gardens as well as the existing ones in both China and Japan, this article compares the gardens' origins and evolution in the two countries, and draws the conclusion that the gardens in each country emerged and developed naturally, and classical Chinese gardens originated earlier than those in Japan, but made their progress in a much slower way.