We study the dynamics of targeted energy transfers in suppressing chatter instability in a single-degree-of-freedom (SDOF) machine tool system. The nonlinear regenerative (time-delayed) cutting force is a main source of machine tool vibrations (chatter). We introduce an ungrounded nonlinear energy sink (NES) coupled to the tool, by which energy transfers from the tool to the NES and efficient dissipation can be realized during chatter. Studying variations of a transition curve with respect to the NES parameters, we analytically show that the location of the Hopf bifurcation point is influenced only by the NES mass and damping coefficient. We demonstrate that application of a well-designed NES renders the subcritical limit cycle oscillations (LCOs) into supercritical ones, followed by Neimark–Sacker and saddle-node bifurcations, which help to increase the stability margin in machining. Numerical and asymptotic bifurcation analyses are performed and three suppression mechanisms are identified. The asymptotic stability analysis is performed to study the domains of attraction for these suppression mechanisms which exhibit good agreement with the bifurcations sets obtained from the numerical continuation methods. The results will help to design nonlinear energy sinks for passive control of regenerative instabilities in machining.