When the quartet was formed in 1946, under Juilliard president William Schuman, Koff was its second violinist; it also included first violinist Robert Mann, violist Raphael Hillyer, and cellist Arthur Winograd. The ensemble was the first quartet in residence in an American conservatory, and was renowned for its support of contemporary music.

Koff made a number of recordings with the quartet, as well as teaching at Juilliard, before leaving in 1958 to take up the position of director of performance activities at Brandeis University. He later became chair of the music department there, and played in the Brandeis String Quartet for ten years. He retired from the faculty in 1983.

Born in Los Angeles, Koff graduated from the Oberlin Conservatory and while in the Juilliard graduate program studied with Hans Letz. He also taught at Aspen and Tanglewood during their summer sessions, and at Tel Aviv University and Harvard.