Context:. Drowsiness affects the driver’s cognitive abilities, which are all important for safe driving. Fatigue detection is a critical technique to avoid traffic accidents. Data sharing among vehicles can be used to optimize fatigue detection models and ensure driving safety. However, data privacy issues hinder the sharing process. To tackle these challenges, we propose a Federated Learning (FL) approach for fatigue-driving behavior monitoring. However, in the FL system, the privacy information of the drivers might be leaked. In this paper, we propose to combine the concept of differential privacy (DP) with Federated Learning for the fatigue detection application, in which artificial noise is added to parameters at the drivers’ side before aggregating. This approach will ensure the privacy of drivers’ data and the convergence of the federated learning algorithms. In this paper, the privacy level in the system is determined in order to achieve a balance between the noise scale and the model’s accuracy. In addition, we have evaluated our models resistance against a model inversion attack. The effectiveness of the attack is measured by the Mean Squared Error (MSE) between the reconstructed data point and the training data. The proposed approach, compared to the non-DP case, has a 6% accuracy loss while decreasing the effectiveness of the attacks by increasing the MSE from 5.0 to 7.0, so a balance between accuracy and noise scale is achieved.