As businesses attempt to improve their development processes by accelerating their release schedules, there can often be a detrimental knock-on effect to the security of the application. Whether the application is web-based or compiled, internal or external, this pruning of the development cycle to rush out the latest software solution makes for easy pickings by security consultants.
The reasons behind this lie with the limited access to "top-drawer" security consultants and the advanced scheduling of their time. For many penetration testing companies, dates for conducting security work must be agreed a week or two in advance – for some organizations, this might even be a month or two. So any changes or slippage in the development cycle will probably affect when the security work can be carried out, and consequently the "go live" date and – invariably – the project manager's financial bonus.
Instead of rescheduling the security work for the next available slot (such as two or three weeks later), an increasing number of organizations choose to carry out the work anyway; against systems and application components that are already in place.
In many cases, security work that was originally scoped for the final "pre-live" environment must now be conducted against a partial or incomplete "test" environment. As a result, the consultants find many bugs and configuration flaws that, while critical or high risk from a standalone vulnerability perspective, do not actually relate back to what will eventually be the live application. Instead of getting valuable security advice and analysis that will help secure the business-critical application, the client receives a report detailing configuration errors, lack of functionality, usability bugs, lax permission settings and scalability issues that provide very little insight into the real application's eventual security.
Invariably, if any relevant vulnerabilities are discovered that would be applicable to the live environment, their value is diminished among the confusion of what the real differences between the environment tested and the actual live environment will be. In some cases, the consultants are forced to evaluate the security of the "pre-live" hosting infrastructure without even part of the application running in it.
Apart from having completed a milestone on some middle manager's project development schedule, I have to question both the security benefit and financial expenditure of carrying out this particular piece of work. The clients' security team understood the worthlessness of the application assessment, but their hands were tied – they had a schedule to keep. Unfortunately for them, they then had to spend several hours in meetings with system developers and architects reviewing the findings of the final report – probably followed by hours of violent disagreement with all concerned that the findings should not apply to what would eventually constitute the final application deployment.
A lot of organizations place too great an emphasis on assessing the security of applications before they are deployed – regardless of their state. When possible, I would sooner be flexible and work with the client to slot the security assessment into a scheduled period where the results would be most beneficial. In almost every case, the client would be better off waiting until the application goes live before testing, rather than testing an incomplete ensemble of code and components.
While I am only too familiar with the fact that security consultancy is a "growth area" within IT services, the man-day rates for using penetration testing companies with access to the most experienced security consultants is still an expensive affair. Many small-to-medium sized companies find it a bit of a shock to discover that access to technical security consultancy skills is more expensive than paying for external accountants.
However, these consultancy rates shouldn't be used as the battering ram for carrying on regardless and assessing the security of something that is incomplete. Most professional penetration testing companies are very flexible in rescheduling and can easily accommodate changes given a few days' notice. In the worst case scenario, they might seek to charge the client for some of the lost time – but a little negotiation will usually keep this figure very low (or dropped if you hint at extra work in the future). It is certainly better to agree on a timeline that sees the complete application being assessed, rather than paying for a worthless report for an application that still has to be security tested. Believe me, even the penetration testers carrying out the work would sooner work on the real application.