|
|
Feel Good About Your Network
|
|
IDsec Limited
31-33 College Road
Harrow, Middlesex
HA1 1EJ
United Kingdom
(Map)
T: 020 8861 2001
F: 020 8861 3433
www.idsec.co.uk
Copyright © 2008
IDsec Ltd
5.08
|
|
|
|
|
|
|
|
|
 |
Automated scanning over the Internet is no substitute for
manual penetration tests but it can be of considerable use to hard-pressed
IT security managers.
Stephen Bishop looks at the issues.
|
|
|
|
|
Regular network security testing is more or less a given for any
organisation with an Internet presence. But, as in many other
technical areas, there can be a conflict between automated and manual
approaches. Few would deny that a full manual audit by a team of
experienced professionals will give the best results, but this can be
very time consuming and expensive; an automated scan might be seen as
second best by the purists but it can be much easier to fit into
budgets.
In fact, one of the issues with allocating budgets for penetration
tests is in deciding whether to carry out one intensive test per year or
commission a series of smaller checks throughout the year, run
monthly or even weekly.
This leads on to the difficult issue of timeliness: commissioning
full manual tests takes considerable effort, and many organisations
find it difficult to keep to a fixed schedule of, say, six-monthly
tests - especially when the target systems themselves may be
undergoing changes to meet business demands.
|
|
|
A typical penetration test is carried out by a small team with a
carefully defined brief, usually following an established
methodology, even if this in not formally documented. The task may be
may split into several phases of exploration, vulnerability testing,
analysis and review. Most practitioners will use a number of tools
plus various manual techniques for further exploration of potential
flaws. At the end of the project the client will expect a written
report, with a short summary and various levels of detail.
Manual testing is undoubtedly more thorough than an automated probe
and can be tailored to clients' needs. Experienced practitioners can
draw on their own experience and change the emphasis of their testing
according to the type of system under test: in other words, they can
follow their nose to a certain extent even within a defined
methodology.
This is particularly true for application-level testing. Automation
can help in mapping web sites in great detail, and carrying out some
of the repetitive manipulation, but uncovering intrinsic flaws in the
specific application generally relies on intensive manual effort.
|
|
|
There are various forms of automated testing services available, but
they are typically offered via subscription, are run remotely across
the Internet and their findings are presented in a series of
management and technical reports. The client can specify the target
set and the timing of the test runs, but otherwise has little
influence over the process.
Such tests usually carried out much more frequently than manual
testing, perhaps monthly rather than yearly. This, of course, means
that any unforeseen changes in services offered to the outside world
will be picked up sooner.
However, the absence of human cross-checking brings with it a risk of
false positives appearing in the results: in other words, flagging
apparent vulnerabilities that are not actually present. The
developers of automated scanning systems put a lot of effort into
mechanisms that cross-check any flaws found, and will usually have a
quality control process to make sure that any anomalies are kept out
of future runs, but individual reports from automated scans must
still be read with some caution.
On a related note, an automated system is less likely to be able to
give useful threat levels for any vulnerabilities found than a
skilled penetration tester. The latter will always have a better
understanding of the overall context and be able to adjust the output
of tools to match the realities of the service offered.
|
|
|
One way to categorise the various automated scanning offerings is to
distinguish between regular
and on-demand services. The former run on a fixed
schedule, such as 03:00 every Tuesday, while the latter are set in
motion by the client as required.
The regular scan approach provides a form of system monitoring, so
that unforeseen events can be caught. Assuming that successive sets
of results are kept, this also allows for trend analysis and
historical views, possibly helping with forensic analysis going back
months or even years.
This is perhaps of most benefit to security teams and other
specialists who oversee a large address space but have no direct
responsibility for the systems in that space. The appearance or
disappearance of, say, a mail server will be made apparent fairly
rapidly: it may be that this is an unofficial change that would
otherwise have gone unnoticed.
On-demand or user-driven scans are more suitable for specific
investigations, perhaps in support of a wider audit or initiated by a
security scare.
For example, a scan may be run after a remedial configuration change
that does not justify the expense of a repeat penetration test. Or,
perhaps, a significant reworking of an Internet gateway might require
a series of scans run at short notice in order to ensure that any
exposure of vulnerable systems is minimised.
|
|
|
Users can benefit from a system of confidence ratings applied to any
vulnerabilities reported. These show the likelihood that that a
putative hole actually exists, based on the technique used by the
check that caused it to be flagged.
For example, if a check carries out an actual exploit and retrieves,
say, the target system's password file, then the confidence level
would be very high, close to 100%. (It is not possible to award a
full 100% to any test, as there is a small but finite chance that
the target system is a “honey pot” or other device set up to give
deliberately misleading results to intruders.)
On the other hand, a vulnerability deduced from a software version
number or other circumstantial evidence should be given a lower
confidence level, although it is still of value. This is because
many systems get patched for security purposes without a full
version upgrade being applied: the vulnerability disappears but the
version string remains unchanged. In the strictest sense a
confidence level should be based on statistical analysis that
determines how many false positives a check throws up when run
against a significant number of real-life targets; in practice it is
acceptable to use a value like 75% to show that its effectiveness
lies between evens and a dead cert.
Another class of checks are those that send anomalous traffic to a
service and then look to see if it is still active afterwards: if the
service no longer responds this implies that it is vulnerable to a
denial-of-service attack. Unfortunately, this method can often fall
foul of load balancing, intrusion protection systems (IPSs) or even
the inherent unreliability of the Internet itself. These results
should be given a lower rating to indicate that they are worth noting
but need further investigation before any remedial work is set in
motion.
There is, of course, a tricky issue here for any commercial offering.
The user of a service may well expect all its checks to give clear
cut answers all of the time, even if this is somewhat naive. So
putting a confidence level of anything less than 100% against a check
may be viewed negatively, although that check may be using the best
available technique to test for a given vulnerability. In the outside
word, however, we accept such qualification all the time - many
medical screening checks have a relatively low confidence level but
provide a useful means of determining whether a more thorough
examination is needed.
|
|
|
In the longer term, the key to improving confidence levels lies in
quality control. This requires a careful analysis of all false
positives to root out all incorrect assumptions, poor logic, systematic
errors and out-of-date information - not to mention straightforward
bugs in code and misleading documentation.
Quality control is a continuous process of examining successive sets
of results and making appropriate changes, and requires a significant
commitment from the supplier.
Much of this work can be carried out in
isolation - for example, checks that yield variable results when run
against the same target are always suspect - but in some cases the
feedback loop may have to involve the client. This is simply because
a little inside knowledge can make a significant saving in the effort
needed to analyse a system from the outside.
|
|
|
Some services run a number of tools within a scan: they may present a
single set of results, but they rely on a number of sources for the
testing logic.
The most obvious examples of these are systems that explicitly use a
collection of public-domain and commercial probes to carry out the
widest possible set of tests and then consolidate the results into a
common format. But even with a system that uses only dedicated
software there are likely to be significant differences in approach
between the latest developments and checks that have been in the
arsenal for a number of years.
The difficulty for the developers and maintainers of the service is
to correlate the various output streams so that a consistent set of
results can be presented to the user. Although efforts such as the
Common Vulnerabilities and Exposures (CVE) dictionary aim to provide
a single name for each known vulnerability, the developers have to
work hard to avoid reporting the same issue more than once under
different names, with all the confusion and irritation that this may
cause the end user. It is important, however, not to over-simplify
the results by merging together two findings that represent
distinct issues which should therefore be kept apart.
|
|
| Presentation and Reporting |
|
|
|
Most security analysis tools offer a range of reporting styles,
loosely classed as management and technical. The former are
intended to present a very high level view of the security state,
using colourful graphics and steering clear of details; the latter
are aimed at providing evidence and suggestions for remedial work, and
may report all findings down the finest level of detail.
This approach works well in large organisations where there are
clearly defined layers of responsibility: where the “suits” have no
direct involvement in system management and the “techies” are not
expected to concern themselves with the overall security of the
network. It does not always help smaller organisations or operational
sub-divisions where the managerial and technical roles are merged and
the required style of reporting is somewhere in between.
One approach to reporting is to apply the structure of the target to
the report itself, using a hierarchy based on networks, hosts, (IP)
protocols, (TCP and UDP) ports, applications accessed via these ports
and, last but not least, vulnerabilities in these applications. This,
of course, derives from the seven-layer model of networking.
Reports created in this way may appear somewhat austere, but they can
reduce misunderstandings about the nature and value of the results:
for example, in making it clear that although a TCP port can be
opened, the presence of the service normally associated with that
port is not confirmed until the relevant protocol has been use to
elicit a characteristic response.
Users naturally welcome a selection of reports, but these should
present different views of the data obtained by the scan, rather than
versions of the same document that simply vary in the amount of
detail included. It helps if the automated service is based on a
well-defined data model, so that that reporting really is distinct
from the scanning phase.
As well as reports in the form of browsable and printable documents,
some systems provide alarms that alert users to significant changes
in services or vulnerabilities found. These can take the form of
e-mail messages, syslog events or SNMP traps,
although network security considerations may limit the usefulness of
all but the first of these. As with any event reporting, setting
thresholds can be a significant challenge, and the benefits of
issuing alarms can be reduced if users receive so many that real
security threats are obscured.
|
|
|
Automated monitoring is not a substitute for real penetration testing
but a useful aid to system management that can be used alongside and
between manual audits.
There are a number of offerings in this area but it is important for
end users to choose a product that fits their way of working, with
a style of reporting that helps them understand their exposure and
fix any issues that arise.
|
|
|
Stephen Bishop has been working in computer security for more than 20
years, and has carried out many penetration tests for customers in a
range of industry sectors, both public and private.
He has a software development and networking background, and is the
designer of IDsec's Superwalk service.
|
|
