An 8 question challenge

As one of the ones on the "inner circle" I am often asked to participate in upcoming ITIL exams, as a Pilot Tester.

The good news is that I get asked and I am one of the first to ever see these new exams. The bad news is that I never get to see my result and I still have to take the test myself, when the actual exams are finally released.

Over the last 4 weeks I have taken 5 of the upcoming ITIL Service Lifecycle and Service Capability exams. The expected format of the exams is well publicized. Eight (8) complex multiple choice exams, to be answered in 90 minutes.

You may be thinking ONLY 8 ! and 90 MINUTES !... I'm here to tell you not to get lulled into a sense of security. Some of the questions are - I wouldn't use the work complex - I'd use the terms of LONG, TIRING TO READ, TRICKY TO UNDERSTAND and SUBJECTIVE.

The answers have different points. So it isn't even a case of one RIGHT and three WRONG. There are grades of RIGHT and WRONG. This is what makes the format hard for me to like.

If you get three of four experienced managers together and ask them to write an answer to an issue you will get COMPLETELY different answers. This is a result of experience, knowledge and even how you felt at the time.

Then you have an independant person say what is the most right answer etc.. Just doesn't sit well with me. I have my views and opinions and because they may differ to the people who write the questions and answers I could fail the exam.

There is no doubt that it is an enormous challenge, but I wonder if the examination panel would be better suited opting for simply MORE simple multiple choice questions and avoid the issues that are simply inevitable with the complex format.

Just when you thought it was safe.... New ITIL exams

The latest set of draft syllabus for the ITIL v3 Intermediate courses have been published by APMG.

You may have heard that the Intermediate courses which encompass the five Service Lifecycle, four Service Capability and Managing Across the Lifecycle courses will be released in two waves.

Wave 1 - October 1st 2008 sees exams available to the public for Service Transition, Service Operation, Release, Control and Validation, Operational Support and Analysis and Service Offerings and Agreements.

Having seen the syllabi I can report that there are some changes from the draft syllabi that were produced several weeks ago. The most notable and probably of greatest interest is the exam format in all of the courses.

8 multiple choice questions !

So you are thinking that can't be too bad; but if you consider that the exam is 90 minutes long you get the idea that these are not simple questions.

In fact the questions are scenario based, each with 4 possible answers and (get this) there are varying degrees of "correctness" for each of the 4 answers. Correct, no longer is one question right and the others wrong. Now there is a really right one that is worth 5 marks, a not quite so right one that is worth 3 marks, a answer that earns 1 mark and a "distracter" worth 0.

Love it or hate it, the pass mark for the exam is 65% or 26 marks out of a possible 40 marks. You can earn yourself a distinction by getting 30 marks (75%).

So that is the breaking news on the new exam format for ITIL v3 Intermediate courses.

Personally, the format does not appeal to me. Having a gradient scoring system is very subjective. In real life there are multiple ways to achieve a result, so in this regard it makes sense to have different "correct" answers. However, my view of what is the right way to deal with a situation will be different from someone else and the choice is dependant on many more factors than can be documented in an exam (e.g. emotions, experiences).

The v3 Multiple Choice approach

APMG have recently published draft syllabi for all the Service Capabilty and Service Lifecycle programs, along with details regarding the Managing across the Lifecycle capping course.

It is clear from the syllabi that ALL exams for ITIL v3 will be multiple choice.

It raises the question of why depart from the traditional model of multiple choice for the entry (Foundation) and intermediate (practitioner) levels and the challenge of the written response for the advanced (managers) level.

Multiple choice questions support the Blooms Taxonomy that is at the heart of how APMG are pitching the different levels of qualification for ITIL v3. Perhaps this is the reason for going to multi-choice.. OR ... perhaps in an attempt to make all levels of certification more appealing to the mass IT community it is a clever move.

There is no doubt that the ITSM community shudders at the thought of the current exam style ITIL v2 Managers exams. The exam is hard, the marking is hard and the failure rate is high.

On the flip side, the ability to earn points from the markers is there, provided you can demonstrate an understanding, at an advanced level of the theory and its application.

The move to multiple choice really negates the ability of the exam taker to "show their style" as a manager. Now, it is a case of learning the theory in order to give yourself the best chance of passing.

APMG have not helped the industry in the way the new course syllabus support the theory buff. Why not have an element of in course assessment that training organizations can use to ensure that if someone is going to earn the top level certification, that they can in fact communicate effectively? (a must for any Service Managment professional worth their salt).