We investigated a model for the neural integrator based on hysteretic units connected by positive feedback. Hysteresis is assumed to emerge from the intrinsic properties of the cells. We consider the recurrent networks containing either bistable or multistable neurons. We apply our analysis to the oculomotor velocity-to-position neural integrator that calculates eye positions using the inputs that carry information about eye angular velocity. By analyzing this system in the parameter space, we show the following. The direction of hysteresis in the neuronal response may be reversed for the system with recurrent connections compared to the case of unconnected neurons. Thus, for the NMDA receptor-based bistability, the firing rates after ON saccades may be higher than after OFF saccades for the same eye position. The reversal of hysteresis occurs in this model only when the size of hysteresis differs from neuron to neuron. We also relate the macroscopic leak time constant of the integrator to the rate of microscopic spontaneous noise-driven transitions in the hysteretic units. Finally, we investigate the conditions under which the hysteretic integrator may have no threshold for integration.