New radio in unlicensed spectrum (NR-U) is an evolutionary extension of the existing unlicensed spectrum technologies, which allows New radio (NR) to operate in the shared and unlicensed frequency bands. However, in such bands, NR-U should coexist with other radio access technologies (RATs) in a commonly shared spectrum. As various RATs possess dissimilar physical and link-layer configurations, NR-U should comply with the requirements for harmonious coexistence with them. For this reason, the majority of the existing studies on NR-U are focused on fair coexistence. In contrast, the efforts on attaining efficiency of the spectrum and fairness concurrently have gained comparatively few interests as they exhibit an adverse feature. Motivated by this limitation, we propose an algorithm called Thompson's sampling-based online gradient ascent (TS-OGA), which jointly considers the fairness between NR-U and incumbents and, at the same time, the efficiency via pertinent idle period adjustment of the incumbents in the operating channel. Because NR-U deals with the two conflicting and competing objectives (i.e., fairness and efficiency), we model it as a multi-objective multi-armed bandit problem using the Generalized Gini Index aggregation function (GGAF). In the proposed scheme, TS-OGA, a Thompson's sampling (TS) policy is employed together with the online gradient ascent to address the multi-objective optimization problem. Through simulation results, we show that TS-OGA can significantly enhance overall channel throughput, while maintaining fairness. Further, TS-OGA provides the best performance compared to three different baseline algorithms such as greedy, upper confidence bound, and pure TS.