Yarborough was the founder and music director of the American Chamber Orchestra, which first performed in 1980, using National Symphony musicians. From 1986 to 1994, the ACO performed its season at the Kennedy Center.
His first appointment was as music director of the American Symphony Orchestra in Paris, when he was 19. He conducted all over the United States and Europe, including the Royal Philharmonic Orchestra in London, the St. Cecilia Orchestra in Rome, and the Boston Symphony's Berkshire Orchestra. He was, before moving to Washington DC in 1972, the music director of Virginia's Lynchburg Symphony.
Yarborough retired in 1996.