The use of vector-based models of information for the purpose of semantic analysis is an area of research that has gained substantial recognition over the last decade. However, the application of high-dimensional vector representations to linguistic data has, to a large extent, remained exclusively statistical and consequently paid minimal or no attention to the linguistic structures of the data used in the experiments. In this paper, we show that the performance of vector based semantic analysis can be improved by considering basic linguistic structures - e.g. morphology - in the data.