The CISSP’s Switch to CAT
(Computer Adaptive Testing)
In early October 2017, we shared the news that at Security Congress, (ISC)² CEO, David Shearer, announced that the CISSP exam would soon be introducing a CAT (computer adaptive testing) format. In December 2017, the CAT exam format change took effect. Though beneficial in many ways, preparing for a CAT is different than a traditional linear exam. We’ll help you understand what the change means for you and how we can help you best prepare for your test day experience.
What is a CAT?
CAT stands for “computerized adaptive testing” or test. Although the CISSP has been computer-based since June of 2012, this new adaptive aspect of the exam will create a drastically different test-taking experience. Given our relationship with sister company, Kaplan, we know a great deal about CAT examinations, how it adapts to your performance as you go and how to prepare.
What you need to know.
How does Computer Adaptive Testing work?
- Every time you answer a question, the computer evaluates your ability to get the next question right based on your previous submissions and the difficulty of the questions.
- As you get answers right, the computer delivers more difficult questions and increases its estimate of your ability – you get things right, it gets harder, if you get things wrong then the computer serves up an easier question.
- As you answer more questions, the computer’s estimate of your ability gets more precise.
- Each question affects the next question so there is no going back to change your answer to a previously answered question.
- If you reach a point where it is no longer possible to earn a passing score, the exam will terminate prior to the completion of all of the questions.
Why Is It changing?
So why is (ISC)² making such a significant change to their exam? During the announcement, (ISC)² focused on the fact that the exam would be “more effective” and “more efficient.” Here are the benefits of the CAT exam:
Reliability: Provides a more precise and efficient measurement of cybersecurity managerial competence because it caters the exam to the aptitude of each individual test-taker.
Efficiency: The average time taken to complete the test (pass or fail) will be significantly reduced, providing more opportunities for individuals to sit for the CISSP exam.
Increased Question Security: CAT reduces question exposure and the security risks that come with it. By switching to a CAT, (ISC)² can limit the number of times certain questions are revealed. For instance, someone who fails on the first attempt will not see all 250 questions, and thus will not have the benefit of experiencing some questions if they reappear on a retake of the exam at a later date.
Will CAT be more difficult?
Let’s go back to how a CAT works to answer this question. The computer analyzes your answers and uses that data to decide what question to feed you next, all in real-time. The computer decides if you pass or fail by using a rating called a competence level, which is basically your score range. The competence level gets more and more specific as the test goes along. The more difficult questions you answer correctly, the higher your competence level grows. So, it’s a good thing if you find yourself answering difficult questions.
Will my chances of passing the exam change?
Yes and no. Overall, the passing rate of the exam is unlikely to change. However, it is difficult to say if the same individuals who would pass in the current, traditional exam format will have the same result in the CAT format. The psychology of taking the exam may impact some more than others. The bottom line – be prepared and know your stuff.
Will I be able to review or "flag" questions?
Unfortunately not. In a CAT exam, you can move in one direction: forward. Test-takers will be unable to skip, flag, or revisit questions.
What are the "cons" of the CAT format?
As mentioned, different test-takers will react in different ways to the change in format. However, these are the general cons or “downsides” for test takers.
- The format can be unfamiliar. Knowing that the test ‘adapts’ to your correct and incorrect choices as you complete it, you may start second guessing yourself. This can lead to some anguish if you haven’t exposed yourself to the adaptive format while preparing.
- You can’t accrue knowledge throughout the testing session. In long exams, the ability to review questions is particularly useful intrinsically (we’re allowed to change our minds or ignore an incorrect gut response). Additionally, by reading all 250 questions on the exam and using all 6 hours of time, additional context or data contained within certain questions and answers might prove useful in responding to a previous question. In the CAT exam, since you can’t go back to previous questions as mentioned in the previous FAQ, you can only benefit from recall clues from exam content in one direction: forward.
- It’s a shorter test. You would think a shorter test is a good thing, but remember that you still have to show aptitude in the 8 domains on the CISSP. That means having to answer a multitude of questions correctly in each domain. A ‘good’ test session, in that regard, is one that keeps you going through each domain until you pass. Conversely, a ‘bad’ test session is one that ends quickly, because they’ve already decided you don’t know enough of one or more domains.
How is this change different from the previously announced upcoming test change in April for 2018?
The April 2018 exam update revolves around content. Some content will be added, subtracted, and/or re-organized in April 2018. The December 2017 CAT update is just an exam format update. The CAT exam will cover the same content as the current, traditional CISSP exam. Once the new content rolls out in April 2018, the exam will remain in the CAT exam format.
Is the content covered on the exam (the 8 Domains) changing?
No. The CAT exam will, until April 2018, cover the same content as the current exam.
How many questions will I receive on the CISSP CAT exam?
Each test taker will be presented with a minimum of 100 questions. To receive a pass or a fail, you must answer a minimum of 100 questions and no more than 125 questions. Please note that there are 25 unscored or experimental questions that aren’t counted in these totals. (You won’t be able to distinguish between the scored and unscored questions so don’t try.)
Will I have a choice in whether to take a CAT or the current format?
You will not have a choice in what version of the CISSP version to take. As on December 18, 2017, (the day the CISSP CAT is implemented), all test takers must take the CAT version of the exam. You cannot take the CAT version of the exam earlier than December 18, 2017.
Will the CAT exam include experimental questions like the previous exam?
Yes, the exam will include up to 25 experimental questions that do have no impact on a passing (or failing) score.
Exam Scoring
What score do I need to pass the test?
The passing standard for the CAT exam is the same as the traditional exam. While the total percentage of questions answered correctly in order to pass will likely change, the scaled passing score (700) will not change.
What is the exam retake policy with this exam?
The exam retake policy will not change with the CAT update.
You can take the CISSP exam up to three times within a 12-month period. For both the CISSP CAT and linear examinations:
• If you don’t pass the exam the first time, you can retest after 30 days.
• If a you don’t pass the exam on the second attempt, you can retest after an additional 90 days.
• If the third time still isn’t the charm, you can retest after 180 test-free days from their most recent exam attempt.Will I still be able to see my score if I fail the exam? If I fail, what kind of reporting will I receive on my test performance?
Each test taker will be presented with a minimum of 100 questions. To receive a pass or a fail, you must answer a minimum of 100 questions and no more than 150 questions. Please note that there are 25 unscored or experimental questions that aren’t counted in these totals. (You won’t be able to distinguish between the scored and unscored questions so don’t try.) These experimental questions appear in the first 100 questions.
Does a longer session mean I'm doing well and close to passing?
Not necessarily. The computer is adapting to your answers and determining your competence level.
Do all of the questions count equally towards my score?
Technically, no. However, your competence level (your score range) is shaped significantly by early questions. If you don’t do well in the early questions, then your competence level will go below the passing level and it will be hard to recover your competence level to above the passing threshold.
That’s why it’s important to get comfortable early in the exam.
Exam Timing
Will I be able to take breaks?
Test-takers will be able to take breaks during the CISSP exam in the new CAT format; however, the clock of the exam will continue to count down. In other words, breaks will not pause the exam from reaching its time limit – which is 3 hours.
When will I receive my results?
Right away! If you don’t pass, then you will receive diagnostic feedback showing the domains in which you struggled.
How long is the CAT exam? How will the length of the exam change?
In terms of time, the CAT exam can last up to three (3) hours. If you need more time due to medical conditions, you will need to get pre-approval from (ISC)². There is no minimum exam time limit. In terms of the number of questions, each test taker will be presented with a minimum of 100 questions. To receive a pass or a fail, you must answer a minimum of 100 questions and no more than 150 questions.
Are there time limits for each question?
No. But keep in mind that you want the exam to be able to evaluate your competence level. So, at a minimum, you need to get through at least 100 questions in three hours. But the exam may need up to 50 more questions to evaluate your competence so you should plan to answer 150 questions in three hours
Is the exam one continuous session or is it broken down into different sections?
No. The CAT exam will feel like one, continuous exam and is not broken down into mini CATs. This is significant because there is no guarantee that all content on the CISSP exam is covered. So, like with most tests, you might not see content from certain sub domains.
What is the shortest and longest time I can expect to sit for the exam?
You can prepare and expect to sit for the exam for three hours. There is no minimum exam time, but you do know the minimum number of questions: 100.
How does the CAT exam end?
The CAT exam will end in one of three different ways:
- Confidence Interval Rule – At any point from questions 100-149 this rule can be invoked. This rule simply means the the exam determines based on statistical data that you have either passed or failed the exam.
- Maximum Length – This rule is invoked when you hit the maximum number of questions, 150. At this point, to determine if you passed or failed, the exam will look back at your last 75 questions. In order to pass, your Confidence Interval had to be above the passing threshold at all times. If, at any point, your Interval dips below the passing threshold, the exam results in a fail.
- Run Out of Time (R.O.O.T.) – This rule is invoked when the three hour time limit runs out. If the time runs out before you reach 100 questions, you automatically fail. If you make it past question 100, and the time runs out, the exam will look back at your last 75 questions. In order to pass, your Confidence Interval had to be above the passing threshold at all times. If, at any point, your Interval dips below the passing threshold, the exam results in a fail. Sound familiar? The Maximum Length rule discussed above is identical to the R.O.O.T rule. That’s part of what makes the CISSP exam so challenging. That’s why it’s important to do well early to ensure you are well above the passing threshold when entering the homestretch of the exam.
Exam Environment
Will the testing sites change with the test change?
Effective December 18, 2017, the CISSP CAT exam will be available exclusively through authorized Pearson VUE test centers. This means (ISC)² is cutting down on third party “mobile” testing facilities, so it’s more important than ever to schedule your exam well in advance.
What now?
Don’t worry! CAT exams are nothing new to us, and many of our team members have been preparing students for these types of exams for years. We’re happy to help you in your preparation to earn your CISSP credential.
Think of this upcoming test change as an opportunity. We’re happy to help you through the process. Check out our latest promotions for the CISSP and other certifications here. We’ll see you in class.