Optical fiber communication systems play an important role in broadband signal transmissions, where the nonlinear dynamics in the long-haul optical fiber transmissions greatly degrade the transmission performance and limit the fiber capacity and reach. This paper analyzes and compensates for the nonlinear dynamics in the long-haul optical fiber transmission with the delay-based optoelectronic reservoir computing (RC) scheme, which provides benefits like time-adaptive tracking, low-complexity and hardware implementation potentials. The generalization of the analysis and compensation schemes from the traditional amplitude-modulated PAM signal to the amplitude-phase-modulated QAM signal is achieved by signal preprocessing. In the numerical study, the proposed RC-based fiber dynamics compensation scheme, with fiber reach from 1400km to 3000km, shows considerable performance with the digital backpropagation scheme, which is always used as the benchmark for nonlinear compensation of long-haul fiber transmissions. The realization complexity is much reduced with the RC-based schemes, which has provided a new dawn for optics-based nonlinear analysis and compensations.