Here, we investigate the effects of O2:N2 (1:1) as ambient gas as compared with pure N2 during activation annealing of Mg as p-type doping in GaN layers grown by MOCVD. The purpose is to understand the impact of O2 on the resulting free hole concentration and hole mobility using SIMS, XRD, STEM, AFM, and Hall effect measurements. Even though the presence of O2 in the ambient gas during annealing is very effective in reducing the H level of the Mg-doped GaN layers, the maximum achievable hole concentration and mobility is still higher with pure N2. The differences are explained by an in-diffusion of O to the GaN layer acting as n-dopant and, thus, giving rise to a compensation effect. The Mg-H complexes at substitutional (MgGa), i.e., the electrically active acceptor sites that provide free holes, are preferentially activated by annealing with N2 only as ambient gas, while annealing with O2:N2 (1:1) also dissociates electrically inactive Mg-H complexes resulting in much less residual H. At the lower growth pressure of 150 mbar compared to higher growth pressure of 300 mbar, an increasing carbon incorporation leads to a compensation effect drastically reducing the free hole concentration while the mobility is unaffected. © 2023 Author(s).