The U.S. dollar index, which measures the greenback's strength against a trade-weighted basket of six major currencies, was down 0.33% at 94.86, the lowest since March 18.
The index ended the previous session down 0.85%, the largest one day decline in almost two weeks.
In a speech at the Economic Club of New York, Yellen said global risks to the U.S. economy, including low oil prices and uncertainty over China justified taking a cautious approach to tightening monetary policy.
The comments contrasted with recent remarks by some Fed officials who indicated that the bank could act as soon as next month to raise interest rates.
The dollar weakened against the euro, with EUR/USD rising 0.3% to 1.1327.
The dollar was also lower against the yen, despite weak economic data out of Japan, with USD/JPY down 0.51% at 112.11.
Data on Wednesday showed that Japan's factory output posted the largest drop in February since a massive earthquake and tsunami in 2011 hit the supply chain, adding to fears that the economy is at risk of falling into a recession.
The pound also advanced, with GBP/USD rising 0.46% to 1.4449 as Brexit fears eased.
The commodity linked currencies rallied against the greenback, with the New Zealand dollar scaling five-month highs of 0.6955.
AUD/USD jumped 0.85% to 0.7690 and USD/CAD was down 0.475 at 1.3011.
The commodity currencies were also underpinned as oil prices rebounded on a smaller than anticipated build in crude oil stockpiles last week.
The U.S. was to release the ADP report on private sector jobs growth later in the day, ahead of Friday's government nonfarm payrolls report.
The nonfarm payrolls report is viewed as the clearest indicator of how the U.S. economy is performing.