A problem with cellulose-based materials is that they are highly influenced by moisture, leading to reduced strength properties with increasing moisture content. By achieving a more detailed understanding of the water–cellulose interactions, the usage of cellulose-based materials could be better optimized. Two different exchange processes of cellulose hydroxyl/deuteroxyl groups have been monitored by transmission FT-IR spectroscopy. By using line-shape-assisted deconvolution of the changing intensities, we have been able to follow the exchange kinetics in a very detailed and controlled manner. The findings reveal a hydrogen exchange that mainly is located at two different kinds of fibril surfaces, where the differences arise from the water accessibility of that specific surface. The slowly accessible regions are proposed to be located between the fibrils inside of the aggregates, and the readily accessible regions are suggested to be at the surfaces of the fibril aggregates. It was also possible to identify the ratio of slowly and readily accessible surfaces, which indicated that the average aggregate of cotton cellulose is built up by approximately three fibrils with an assumed average size of 12 × 12 cellulose chains. Additionally, the experimental setup enabled visualizing and discussing the implications of some of the deviating spectral features that are pronounced when recording FT-IR spectra of deuterium-exchanging cellulose: the insufficient red shift of the stretching vibrations and the vastly decreasing line widths.