The level of cognitive performance that should be mastered at the end of a course or program is usually described and classified using taxonomy levels (University of Zurich n.d.).  An example of such taxonomy levels is shown in Exhibit 2.  At the lower levels of this system, the focus is on imparting lexical knowledge, while at the upper levels, knowledge is related and synthesized, evaluated and selected for its usefulness (Evaluating), and adapted for new application contexts or used to gain entirely new insights (Creating/Innovating) (Kramp 2020).  The upper levels of the taxonomy thus address the formation of competencies, which Weinert (2001, pp. 27–28 as cited in Klieme et al. 2023) defines as the cognitive abilities and skills that individuals possess or can learn to possess to solve certain problems, as well as the associated motivational, volitional, and social readiness and abilities to use the problem solutions successfully and responsibly in variable situations.  It is obvious that competencies in this sense are required to solve business problems and manage projects successfully. Examinations should provide information about the achievement of learning objectives at the different levels of the taxonomy by allowing examinees to demonstrate their abilities.

At top-tier consultancies, training is aligned with the requirements of the role typically taken on a project at the current or next career level, or with individual development needs.  As a result, consultants are required to apply what they have learned to specific project situations very quickly.  Their performance is evaluated on a regular basis.  This corresponds to Taxonomy Level 3 (Applying, see Exhibit 2).  In addition, the following skills are typically evaluated at the (senior) consultant level, i.e., before the project manager level, within the main category of analysis/problem solving skills:

• Structuring problems and breaking them down into individual aspects
  (Taxonomy Level 4: Analyzing)
• Performing complex and sophisticated (data) analysis
  (Taxonomy Level 4: Analyzing)
• Interpreting and evaluating the results of the analysis and deriving recommendations for action
  (Taxonomy Level 5: Synthesizing)
• Initiating and driving change within the client organization
  (Taxonomy Levels 5 and 6: Synthesizing and Evaluating)
• Extending one’s knowledge by integrating and generating new insights
  (Taxonomy Levels 5 and 6: Synthesizing and Evaluating)

For project managers, the following skills are also included in the assessment:

• Adjusting the approach or focus of the solution
  (Taxonomy Levels 6 and 7: Evaluating and Creating/Innovating)
• Developing creative and innovative recommendations while considering the client’s ability to implement them
  (Taxonomy Levels 6 and 7: Evaluating and Creating/Innovating)
• Developing new and creative conceptual frameworks and analytical approaches
  (Taxonomy Level 7: Creating/Innovating)
• Contributing to the development of new knowledge and methodologies
  (Taxonomy Level 7: Creating/Innovating)

For the top-tier management consultancies, the evaluation of learning success is always based on the ultimate question of how well employees perform in day-to-day projects and how well they generate new consulting knowledge.  The latter is, after all, a key business driver for consulting service providers.  In this context, cognitive requirements are often placed at the highest level of the taxonomy.

If a top-level consultant receives a poor or very poor project review, he or she is immediately placed on a watch list.  If he fails in the next project, he is made aware that he will not be promoted to the next career level and is encouraged to leave the consulting firm.  Based on this strict up-or-out policy (e.g., Berndt 2013), outsiders who have no insight into internal performance evaluations can assume that consultants who have worked for a top consulting firm for a long time and have even been promoted one or more times have met high performance standards in their projects.  Otherwise, they would not have stayed with the firm for so long, let alone been promoted to the next career level.  Against this backdrop, promotion to project manager (called Engagement Manager at McKinsey & Company) and subsequent work in that role is often seen as a seal of quality that can be verified externally through a job reference.

The Master in Project Management program at Colorado State University Global (CSU Global) emphasizes the writing of seminar papers to demonstrate mastery of the material covered in each course.  It is not possible to assign a specific taxonomy level based on the type of assessment alone, as the content requirements are important.  Unfortunately, no information is available on the specific tasks for the seminar papers to be written in the program examined here as an example.  However, the preparation of a disposition for a seminar paper as a planning process can already be assigned to the Taxonomy Level 7 (Creating/Innovating) (Gröblinghoff 2015, p. 5 with reference to Anderson & Krathwohl 2001 and Bloom 1956).  A seminar paper is optimally successful if it leads the reader to new insights in clear and unambiguously comprehensible steps (University of Karlsruhe 2007, p. 3); according to Exhibit 2, the generation of new insights is at least at Taxonomy Level 5 (Synthesizing).  Seminar papers of the basic type “Problem Overview” (compilation) usually require the development of evaluation or selection criteria regarding the applicability of the respective theories to specific problems (University of Karlsruhe 2007, p. 5) and thus include an evaluation (Taxonomy Level 6 in Exhibit 2).  The basic type “Exemplary Study” can be based on the transfer of a theory or methodology to an object, a question or an exemplary target group that has not yet been considered or the exemplary exploration (of something new) of a topic that is still largely unknown (University of Karlsruhe 2007, p. 6); this corresponds to the requirements of the spectrum between Taxonomy Levels 3 (Applying) and 5 (Synthesizing) according to Exhibit 2.  The Capstone course, a compulsory part of the Master in Project Management program at CSU Global, involves applying or transferring the learned content to a specific project of the student’s own choice (CSU Global 2023), which corresponds to Taxonomy Level 3 (Applying).  Overall, it can therefore be assumed that the assessments in the degree program under review typically correspond to at least Taxonomy Level 3 (Applying).

When taking a test to obtain a project management certificate, such as the widely recognized PMI-PMP^®, the restrictions in terms of response time and question format prevent the application of knowledge to a (fictitious) new practical situation, such as developing a meaningful project schedule based on the objectives to be achieved and the available capacities.  This is akin to having to pass only the theory part of a driving test, while neglecting the practical driving component.  For question 1 in Exhibit 1 of blog post no. 19 entitled “With the PMI-PMP^®, the Proof of Knowledge and Competence in Project Management is Demonstrated by Means of a Written Online Exam,” it is only necessary to know the elements of the Project Scope Statement and to identify the acceptance criteria (answer A) as such.  The Project Scope Statement’s elements and the acceptance criteria for meaningful acceptance criteria (Taxonomy Level 2: Understanding) are not relevant for this question.  Neither is the development of acceptance criteria specifically tailored to a (fictitious) project, or the evaluation (fictitious) project results based on certain acceptance criteria (Taxonomy Level 3: Applying).  The higher Taxonomy Levels 4 to 7 (Analyzing, Synthesizing, Evaluating, and Creating/Innovating) are not considered when classifying question 1.  This means that for question 1, the certificate candidate must recognize and reproduce learned knowledge (Taxonomy Level 1: Knowing/Remembering) and perform the simplest cognitive task in the present taxonomy system.  The comments on question 1 apply analogously to exam questions 2 to 4, which are reproduced as examples in Exhibit 1 of blog post no. 19.

Questions 5 and 6 (see Exhibit 2 of the blog post no. 19) appear to be application questions (Taxonomy Level 3) at first glance.  However, question 5, for example, is not about showing how one would work with the procurement team or what suggestions one could make to the procurement team.  Rather, it is only about naming a learned procedural step.  In essence, question 5 focuses on reproducing learned knowledge (Taxonomy Level 1) or, at best, a certain understanding of the situation and the appropriate behavior (Taxonomy Level 2).  This is particularly evident in question 6, where the correct answer is to simply delegate the problem that is likely to cause project delays back to management for resolution.  This approach contradicts the principles of the PIM-PMP^® certification, which advocates for a more holistic approach to problem-solving.  The PMP-PMP^® certification focuses on a mechanistic understanding of project management, as outlined in the PMBOK^® Guide.  While the PMBOK^® Guide provides a foundational understanding of various project management aspects, it does not offer specific guidance on how to achieve desired results, and it covers some trivial details.  Consequently, the exam focuses on lower-level cognitive skills, predominantly at Taxonomy Level 1 (Knowing/Remembering) and to a lesser extent at Taxonomy Level 2 (Understanding).  Given that the catalog of exam questions can be memorized in advance, all questions in the PMI-PMP^® exam are at the lowest Taxonomy Level 1.

Certificates of achievement that merely test memorized facts and only partially test comprehension of the content cannot provide a positive prognosis for the successful application of a method (which may not be explained in detail in the context of the training), for the selection of a suitable method based on an assessment, or for the independent development of a new approach that meets specific requirements.  This is especially true since the corresponding skills are not systematically taught as part of the training program before the exam.

When providers of project management education and training programs select one or more assessments, they should consider several key aspects during the design and development process.  These assessments should be process-oriented in the sense of project administration, as typically performed by a project management office (PMO), and should also comprehensively cover the development of solutions, as is relevant for well-managed projects.  Ultimately, project management has a supporting function for the content work.  Most of the tasks should be at Taxonomy Level 3 (Applying) or higher in terms of difficulty, and the more realistic the exam tasks are, the better.  For example, case studies with a very specifically described realistic problem are useful.