Software process reference models (such as Capability Ma- turity Model Integration – CMMI–DEV) have been used for years for software process evaluation and improvement. However, when a team uses agile methods for software development, these models hinder sus- taining agility in higher maturity levels. This is the reason why some agile maturity models have been proposed in the last years. Although there are some models suggested in literature, few studies actually eval- uate these models with real teams. The objective of this study is thus to evaluate an agile maturity model – the Agile Compass – to contribute with empirical results for agile teams in the process improvement field. We conducted this research with two field studies in two different agile teams: an ethnographic study and a focus group. Our findings confirmed the need for empirical validation of academically–proposed models. The Agile Compass was effective in creating a maturity picture in the teams, but both teams seemed to prefer a more “objective” evaluation.