Certain ionic liquids are powerful cellulose solvents, but tend to be less effective when small-molecule hydrogen bond donors are present. This is generally attributed to competition with cellulose for hydrogen bonding opportunities to the anion of the ionic liquid. We show that the solubility of cellulose in dimethyl sulfoxide solutions of tetrabutylammonium acetate is less strongly affected by water than by ethanol on a molar basis, contrary to what can be expected based on hydrogen bond stoichiometry. Molecular dynamics simulations indicate that the higher tolerance to water is due to water-cellulose interactions that improves solvation of cellulose and, thereby, marginally favors dissolution. Through Kirkwood-Buff theory we show that water, but not ethanol, improves the solvent quality of DMSO and partly compensates for the loss of acetate-cellulose hydrogen bonds.