To quickly identify maritime sites polluted by heavy metal contaminants, reductions in the size of instrumentation have made it possible to bring an X-ray fluorescence (XRF) analyzer into the field and in direct contact with various samples. The choice of source-sample-detector geometry plays an important role in minimizing the Compton scattering noise and achieving a better signal-to-noise ratio (SNR) in XRF measurement conditions, especially for analysis of wet sediments. This paper presents the influence of geometrical factors on a prototype, designed for in situ XRF analysis of mercury (Hg) in wet sediments using a 57Co excitation source and an X-ray spectrometer. The unique XRF penetrometer prototype has been constructed and tested for maritime wet sediment. The influence on detection efficiency and SNR of various geometrical arrangements have been investigated using the combination of Monte Carlo simulations and laboratory experiments. Instrument calibration was performed for Hg analysis by means of prepared wet sediments with the XRF prototype. The presented results show that it is possible to detect Hg by K-shell emission, thus enabling XRF analysis for underwater sediments. Consequently, the XRF prototype has the potential to be applied as an environmental screening tool for analysis of polluted sediments with relatively high concentrations (e.g., >2880 ppm for Hg), which would benefit in situ monitoring of maritime pollution caused by heavy metals. © 2022 The Authors