The advantages of large output thermovoltage and sustainable constituent materials have generated a rapid growth in research about ionic thermoelectrics. Recently, giant values of ionic Seebeck coefficients up to 10-26 mV K-1 have been reported. However, the fundamental understanding of the ionic thermoelectric effect is still rudimentary and there is a lack of a well-established measurement standard. In this work, we systematically studied the ionic thermoelectric properties of gel electrolytes made of hydroxyethyl cellulose and an ionic liquid. We discovered that the absorbed water from the atmosphere into the cellulose/ionic liquid gel dramatically increases the apparent ionic Seebeck coefficient from 3 to 12.5 mV K-1. We identified the contribution of a hydrovoltaic voltage generated from water concentration difference as the main reason for the enhanced apparent ionic Seebeck coefficient, which depends on the kinetics of water absorption and desorption on the cold and hot side of the device. Finally, we demonstrated that it is possible to harvest electricity and charge a supercapacitor with intermittent temperature gradients by using this combination of ionic Seebeck voltage and hydrovoltaic voltage.