Considering the viscoelastic property of railway track structure, it is suggested in this paper to include the influence of track dynamics in calculation of lateral stability of railway vehicles running on tracks. A direct numerical method based on the stability of time histories of vehicle components is proposed to ascertain the nonlinear critical hunting speed of railway vehicles. The time histories of the vehicle system are calculated by use of the three-dimensional vehicle-track coupled models, which are capable of taking into account the influence of dynamic properties of track structures on vehicle dynamics. The effect of track system properties such as the rail fastener stiffness and the rail profile on the lateral hunting stability of the vehicle is investigated. Differences of the critical hunting speeds of vehicles on rigid track model and on elastic track model are found out. Generally, the rigid track model overestimates the critical hunting speed by 5–10% for different types of vehicles, which implies that the classical vehicle dynamics predicts higher lateral stability of a vehicle than the vehicle-track coupled dynamics does. This conclusion is of significance to the safety design of railway vehicles and should be taken notice in the design stage. The reason that causes the differences is discussed. A full-scale field experiment was carried out to investigate the hunting behavior of a freight car with three-piece bogies on a straight track. Very obvious hunting motion was observed in the experiment when the unloaded vehicle ran at the speed of 75 km/h, which verified the theoretical calculation results.