Yellen, in testimony before a congressional committee on Wednesday said there are good reasons to believe the U.S. will stay on a path of moderate growth that will allow the Fed to pursue "gradual" adjustments to monetary policy.
But she also acknowledged risks facing the U.S. economy from tightening financial conditions driven by falling stock prices and uncertainty over China.
USD/JPY fell to lows of 111.00 overnight, the weakest level since October 2014 and was last at 111.33, down 1.75% for the day.
The dollar's steep drop against the yen was amplified by thin trading conditions, with markets in Japan and China closed for public holidays.
The dollar is now down almost 8% against the yen from the six-week high of 121.68 reached on January 29, following the Bank of Japan's shock decision to adopt negative interest rates.
The greenback fell to three-and-a-half month lows against a basket of the other major currencies.
The U.S. dollar index, which measures the greenback's strength against a trade-weighted basket of six major currencies, hit lows of 95.40, the weakest since October 22 and was last at 95.61.
The low-yielding euro was higher against the dollar, with EUR/USD at 1.1327, not far from three-and-a-half month highs of 1.1354 set earlier.
But the single currency fell to its lowest level in more than two-and-a-half years against the stronger yen, with EUR/JPY down 1.5% at 126.07.
The dollar was at four-month lows against the Swiss franc, which tends to be bought by investors in times of risk aversion, with USD/CHF down 0.67% at 0.9672.