Aim The aim of this study is to clarify the role of eye movements in the reduction of physiological and psychological metrics of stress during Japanese garden viewing. Methods We chose the well-structured Murin-an garden as a test site and a garden with similar visual elements but less well-maintained as a control site. We measured pulse rates and eye movements to monitor physiological responses. Psychological responses were tracked with the POMS2 Brief form and a short questionnaire. Results We found that the Murin-an garden was more effective in decreasing pulse rate and improving mood. Also, in the Murin-an garden the participants’ gaze ranged more broadly across the visual field and moved more rapidly. Contrary to our expectations, in neither garden did pulse rate rise or fall based on the particular object a participant was viewing. Conclusion Visual stimuli of a well-designed garden can elicit significant stress reduction. Our data suggest that the composition of the elements and the attention to maintenance of a garden result in viewers shifting their gaze more frequently and more quickly. These appear to be the key drivers behind the stress reduction effect. Although we had hypothesized that specific visual elements in the garden would be responsible for reducing measures of stress, our data instead suggest that it is the overall pattern of rapid horizontal eye movements, induced by the garden design, that drives the observed stress reduction. We draw an analogy between our results and the technique known as EMDR (eye movement desensitization and reprocessing) whose practitioners use rapid gaze shifts to elicit stress reduction.