This paper reveals the dynamical behaviors of a bidirectional neural network consisting of four neurons with delayed nearest-neighbor and shortcut connections. The criterion of the global asymptotic stability of the trivial equilibrium of the network is derived by means of a suitable Lyapunov functional. The local stability of the trivial equilibrium is investigated by analyzing the distributions of roots of the associated characteristic equation. The sufficient conditions for the existence of nontrivial synchronous and asynchronous equilibria and periodic oscillations arising from codimension one bifurcations are obtained. Multistability near the codimension two bifurcation points is presented. Numerical simulations are given to validate the theoretical analysis.