Data augmentation is an important method for evaluating the robustness of and enhancing the diversity of training datafor natural language processing (NLP) models. In this paper, we present NL-Augmenter, a new participatory Python-based naturallanguage (NL) augmentation framework which supports the creation of transformations (modifications to the data) and filters(data splits according to specific features). We describe the framework and an initial set of117transformations and23filters for avariety of NL tasks annotated with noisy descriptive tags. The transformations incorporate noise, intentional and accidental humanmistakes, socio-linguistic variation, semantically-valid style, syntax changes, as well as artificial constructs that are unambiguousto humans. We demonstrate the efficacy of NL-Augmenter by using its transformations to analyze the robustness of popularlanguage models. We find different models to be differently challenged on different tasks, with quasi-systematic score decreases.The infrastructure, datacards, and robustness evaluation results are publicly available onGitHubfor the benefit of researchersworking on paraphrase generation, robustness analysis, and low-resource NLP.