There are two types of common toolholder defects: toolholder run-out (eccentric error) and toolholder fretting. Toolholder run-out is caused by either manufacturing error or assembly error and can be easily measured and corrected. On the other hand, toolholder fretting may be caused by the relative vibration between the machine spindle and the toolholder. It is much less understood and an effective correction method remains an engineering challenge. In this paper, the dynamics of the toolholders are investigated by means of mathematical modeling, finite element simulation, as well as modal analysis. The investigations consistently indicate that toolholder fretting is caused by high frequency vibration of the toolholder and it may be corrected by changing the toolholder design (e.g., extending the tool overhang and/or increasing the diameter of the toolholder).