When Visa and MasterCard trotted out their security standards back in 2002 and 2003, the large eCommerce merchants that got to see them complained that they were too much. Fast forward more than a decade and we still hear complaints that the PCI standards are too much. Well if you are still complaining, things are about to get worse with version 3. And the ever more consistent rumor is that business as usual (BAU) will be coming in v4. If that comes to pass, I know some people that will likely jump out of windows as they did in the 1929 stock market crash.
So how is the PCI DSS getting more rigorous?
I spent some time analyzing the PCI DSS v3 as I did with v2. From an analysis of v3 to v2, here are some of my findings.
- There is an overall 11% increase in the number of tests in v3 versus v2.
- Tests requiring some form of documentation have increased a whopping 83%. Not that 83% more documents will be required, just that there are 83% more tests where documentation is reviewed. I will have more on this later in the post.
- The number tests requiring interviews is up 48%. Again, not necessarily involving more people, just more questions to be asked and answered.
- Tests requiring an observation of a process or activity are up 31%. As with the others, this is not a wholesale jump in new observations, but more an increase in things that must be observed.
- Tests involving sampling are up 33%. This actually is an increase in the number of things sampled, but not all of the 33% increase are new samples. This increase is the result of more clarifications from the Council to have QSAs explain what was sampled as it was implied in v2, but not explicitly requested.
Speaking of sampling, not only are the number of tests involving sampling increasing but the PCI SSC has told all of the QSAs that the days of “poor” or “inappropriate” sampling are over. I have seen Reports On Compliance where QSAs have literally used a sample of one out of thousands under the rationale of “they are all configured the same”. If you only tested one, how can you even draw the conclusion that the remaining thousands truly are the same? You cannot and that is a big reason why the Council is getting picky on sampling.
The Council are also tired of incomplete samples. The example most often quoted is there are 100 servers, half are Windows-based and half are Red Hat Linux. A lot of QSAs were stopping there and sampling say five of each and calling their work complete. Wrong!
What the Council is pointing out is that the QSA must go deeper in some cases when choosing their samples. In the example above, the QSA needs to know the function of those servers so that they sample them based on their function such as database server, directory server, application server, etc. In addition, the Council is also saying that it may be necessary to consider the applications involved as well to ensure that sampling provides a more complete picture of the environment. In an assessment involving multiple applications, it might be necessary to sample database and application servers used by each application and not just a random sample of servers.
Finally, sampling might be higher for an entity’s first assessment or the first assessment by a QSA after a prior QSA. The reason is that a higher sample size is warranted because all might not be as it is represented and minimal sampling would likely not reveal any issues. This is common in the financial audit industry in situations where a new auditor is coming into the organization or the operations of the organization have been under increased scrutiny by regulators, banks or their prior auditors.
I earlier stated that documentation testing was up 83% and that was related to more testing of the same documents already being collected. That is not to say that the amount of documentation is not increasing. Regarding the amount of documentation required for v3 versus v2, I am estimating a conservative increase of around 100%. I have been hearing horror stories regarding the amount of documentation being requested for v3. I would not be shocked if the amount of documentation a QSA requires is up by 150% to 200% in some instances, particularly those situations where the QSA was not necessarily collecting all of the relevant documentation they should have been collecting. A lot of this increase is that document counts now include observations which were considered separately in v2.
Based on this information, you should not be shocked if your QSAC increases the fees they are charging you for assessing your PCI compliance under v3. Someone has to conduct all of those tests and review all of the extra documentation generated. Even QSACs that have been doing the right thing all along are seeing impacts in the increases in testing required by v3. But it has been definitely worse for those QSACs that were doing as little as possible to get an assessment done. They are seeing the most impact from these changes and will likely find them highly onerous and difficult to justify the huge increases in professional fees required to cover their higher costs. As a result, I would not be surprised if a number of QSACs stop doing PCI assessments because of the new requirements put on them.
But why are the changes occurring?
The primary reason is to minimize the “wiggle room” QSAs have in their testing so that assessments from one QSA to another are more consistent. There has to be flexibility given to a QSA because organizations are never alike. In addition what is compliant to one QSA can be non-compliant to another even within the same QSAC. That occurs because every individual has their own sense of risk acceptance and avoidance. This issue should be able to be taken out of the equation through discussion of the issue with the QSA and their superiors and, if necessary, development of mitigation strategies.
Under v2, a QSA that had a high risk tolerance could deem an organization compliant when the evidence would indicate that the organization is not compliant. Or a QSA with a low risk tolerance could say one or more requirements are not in place in the same situation. The new Reporting Template is an attempt to take the extremes out and reduce the wide swings in what is and is not compliant. However, the new version of the PCI DSS does still allow some wiggle room for QSA/ISA judgment.
In addition to taking extremes in risk acceptance out of the assessment process, the Council is also trying to address the issue with QSAs that are judging organizations as PCI compliant when the QSA’s documentation does not support such a claim. While the majority of QSAs thought this issue was addressed with the Reporting Instructions in v2, based on what the Council is telling us is that it apparently was not. So the Council is getting stricter and stricter on their guidance as to what is acceptable through the language in the Reporting Template/Instructions as well as through their QSA training.
Another reason for the rigor is the breaches that keep occurring. Each breach supplies information that might need to be incorporated into the PCI DSS. One of the best examples of this is requirement 8.5.1:
“Service providers with remote access to customer premises (for example, for support of POS systems or servers) must use a unique authentication credential (such as a password/phrase) for each customer.”
This new requirement is in response to the significant number of breaches where the attacker gained access to a merchant’s cardholder data by knowing the remote access credentials of a vendor that is supporting the merchant such as those vendors that support point of sale (POS) solutions or card transaction processing.
Finally, the changes are also an attempt to circumvent some of the “legal” arguments that occur between the QSA and their client. I am not the only QSA that has encountered clients that come up with very legal-like arguments and interpretations of what a particular test requires. As a result, the Council has attempted to use wording in the tests and related testing guidance that reduces or even eliminates such interpretation arguments. However, in my experience, clients that take this “legal” approach to their assessment are not going to stop. They are not interested in security, they are interested in “checking a box”. But the Council does no one any favors by only allowing QSAs and ISAs to read and have copies of the Reporting Template/Instructions until the client goes through their first PCI assessment under the new testing. The Reporting Template should be a public document not one that only QSAs and ISAs have access.
