Although simulated patients are increasingly used in medical education, little research has been carried out on their validity. Validity in this case defines the relationship between performance with a simulated patient and performance with a real patient. One of the objectives of this study was to determine the validity of the use of simulated patients in assessing the consultation skills of trainees in vocational training at the Department of General Practice, University of Utrecht, The Netherlands. A check-list with a rating scale was used to assess the consultation skills of trainees at the department with simulated patients as well as in their training practices with real patients. The simulated and the selected practice cases were patients with complex multi-conditional problems like low back pain, headache and chest pain. The consultation skills were subdivided into four groups: the patient-centered approach, the non-somatic approach, communication skills and interpersonal skills. The measurement of skills, in particular of consultation skills, is very difficult. A description is given of the way the research group solved this problem. The analysis was performed by determining the sensitivity and predictive value of the assessment of a simulated encounter with a routine practice encounter. A difference existed in the assessed level of consultation skills in the simulated encounter compared to the level in the training practice. In simulation the level of consultation skills was higher than in day-to-day practice. This difference can reflect the difference between competence and performance. Competence is defined as what a doctor is capable of doing and performance as what a doctor actually does in day-to-day practice.(ABSTRACT TRUNCATED AT 250 WORDS)