As a child, Zoller was taught classical violin by his father, a professional violinist. In his teens, he switched to flügelhorn, then jazz bass, and finally guitar. Zoller quit school during the Russian occupation of Hungary following World War II and began playing professionally in Budapest jazz clubs. He escaped Hungary in 1948 just before the permanent Soviet blockade of the country, hiking across the mountains to Austria, carrying just his guitar and a few articles of clothing. Zoller became an Austrian citizen, settling in Vienna, where he formed a jazz group with the accordionist Vera Auer. Zoller moved to Germany in the '50s, where he played with pianist Jutta Hipp and saxophonist Hans Koller. Visiting American musicians (notably Oscar Pettiford and Lee Konitz) admired Zoller's work and urged him to move to the U.S., which he did in 1959 after winning a scholarship to the Lenox School of Jazz. There he studied with Jim Hall and roomed with Ornette Coleman, whose influence sparked Zoller's interest in free jazz. Zoller played in drummer Chico Hamilton's group in 1960 and with flutist Herbie Mann from 1962-1965. In 1965, Zoller began leading a free jazz-influenced group with the pianist Don Friedman, and in 1968 co-led a group with Konitz and trombonist Albert Mangelsdorff. Zoller never turned his back on more traditional forms of jazz, playing swing with vibist Red Norvo and clarinetist Benny Goodman, and bop with saxophonist Stan Getz, among many others. In 1974, Zoller founded the Attila Zoller Jazz Clinics in Vermont (Zoller incorporated in 1985 and the clinics became the Vermont Jazz Center). He remained active as a performer -- in the U.S. and overseas -- until the end of his life. During the '80s and '90s, he recorded several albums for the Enja label with such artists as vibist Wolfgang Lackerschmidt and guitarist Jimmy Raney, as well as longtime collaborators Konitz and Friedman.

Zoller was also a designer of electronic instruments; he patented a bi-directional pickup for guitars in 1971 and helped design his own signature line of guitars with the Framus and Hoefner companies. He also designed a line of strings for the LaBella company. Zoller's final gig was at New York's Zinc Bar with pianist Tommy Flanagan and bassist George Mraz on January 6, 1998; the trio recorded the next day. Zoller died of colon cancer less than three weeks later.