Cooperating in Virtual organizations requires trust between the constituting organizations. SLA contracts are put in place in order to specify the quality of service of services offered. For these contracts to be effective they also need to be monitored effectively. In a Service Oriented Architecture this often means monitoring Web service invocations and evaluating if the service fulfills the obligations in its SLA. In this paper we present an implementation of a rule engine based SLA Evaluator specifically designed for the needs of a virtual organization. The evaluator fits in the context of a virtual organization through the use of open XML-based standards and a loosely coupled, event-driven architecture.