Rob Kroneman contacted me about starting a project related to dashboarding and performance management.

While it gets going it sensible to start it under another focus area, namely this one.

The idea of the project is to;

• Capture and Document Ideas on What a Security Dashboard Should and Could Display
• Collect Existing Dashboard Designs
• Create a Few Reference Dashboards Based on Findings

So over to Rob. You now have write access to the file store.

 Anyone want to share some dashboard screen captures and commentary?

  Hello,

I also have a development project about Risk Management System. (But, more likely Converged Security Management, not RMS exactly.)
This time, I will mention about just Dashboard.

1. Dashboarding

Of course, Risk assessment method or another things are important, but I think that dashboarding is very important.
So, I had researching so many types of dashboards, charts, graphs.
Considering "Intuition" is more important, I think.
It helps Security Administrator to make a dicision what to do first.

(I cannot give you a image right now cause I'm in the office, so I will edit this text again & give you a shot.) 

Very cool. I actually have a nice little collection of them now but not found the time to publish them. Actually I have massive amount of ISM Community things to do ! Yikes!!!!

 Hi all,

dashboard is about Key Indicators, regarding performance (KPI), security (KRI), objective (KGI), etc.

BY a security point of view, the most impotant are KPI and KRI.

I am currently working on a internal project of my company regarding dashboard.

I think there are 3 level of investigation we have to care of:

1. Data Feed and Data Type: this is about data selection needs (e.g. from security device, from risk analysis, etc),  data retrievals (e.g. directly from devices or from risk analysis archive or indirectly from a SIM platform already containing these data) and data types in order to be able to treat data and make calculation about dashboard
2. Indicator Model: hierarchical model for the conception of the indicators. From the raw data to the indicators. Then to the Key Indicators. The main concepts are these:

• Number: the men can easily handle 5, 6 things at a time. So the Key Indicators should be not more than that. For this reason there is the need of making a hierarchy about indicators. Further investigation could be made going in depth on the specific indicators (some thing like the Big Brother 4 interface)
  
• Audience: there are at least two or three level of audience: executive, managers and operational people. Each of these need different kind of indicators. I have single out these three types:

1. Operationl KRIs: these KRIs are derivated directly from any security device. Tipically example are:
   
   • number of false positive in IDS
   • worms detected by AV systems
   • spam identified and blocked by Antispam infrastructure
   • number of unauthorized attempts to access systems
   • number of unauthorized attempts to access networks
2. Management KRIs: these KRIs are derivated from each relevant asset, depending on the security device involved (based on a specific service/security device description matrix). The interested assets are of three kinds:

• Infrastrucutre (like network)
• Basic Services (like E-mail systems, file sharing)
• Applications (like billing, etc)

The source are the same as in the previous KRIs, but there are some filters in place, based on the Business Impact Analysis (in order to identify the most critical infrastructure, service, applications) and Risk Analysis (in order to identify the impact and risk for each critical asset, being infrastructure, service  or application).

1.  KPI: allow to monitor and evaluate the effectiveness of the processes and the achievement of the goals. Usually for the conception of the KPI model it is necessary to use BS (Balanced Scorecard from Kaplan and Norton)
   

1. Presentation: this is the last level, at which it is necessary to display in graph format the indicators. If the data and model are been built in very good manner, these level is simply about representation (e.g. speedometer instead of istogram), access level (e.g. operators seeing Op KRI, executive seeing KPI, etc), views (e.g. by region, by processes, by services), access type (e.g. HR seeing employee relevant information, operation seeing technological things, etc

Sorry, I had some problem formatting and numbering:-(... I was also not able to insert some simple graph that explain better the concepts

Hello All

I toke me a will before I had the time to (re)act.

I see a lot of initiatives for dash boarding the reasons are often very different. That the reasons are different is normal because of the different interest groups.

Without any thought everyone calls them KPI's but often a project is started and quick results are necessary so we start with Performance Indicators and not Key Indicators. This can result in a pitfall because a lot of energy is put into something without giving results that anyone is interested in.

1. If you select Indicators of any kind select the ones that somebody wants to see (is interested in)
2. And very important; are easy to deliver
3. Put the into a process

There is already a lot written on KPI's, KRI's and so on, a good framework good be COBIT, in COBIT a lot is written about Indicators but COBIT is a very big framework covering more than Security. So the scope  (in my opinion) in this community should for the Indicators should be the ISO17799.

The indicators than are part of the PDCA cycle and can help you getting certified against the ISO27001 

I will post again in a short while 

Hi Paolo

I owe you a reply on email to the other mails as well! Sorry just getting caught up.

First off great mail.  

I maybe mis-interpreting or mis-reading the context of your mail but a lot of your sources focus on automated systems or tools. I would think that a large part of the source of collecting data to build dahboards should come from human sources such as "How many people have signed the policy" or "how many applications went live that had subsequent vulnerabilities without a frmal security review"?

Do you have a feeling or a map of all the things you might want to see in a dashboard?

I would like to see a dashboard that follows ISO 2701 categories for instance. Do you think that is realistic?

Look forward to it Rob. I am a little concerned that the security industry seems to swing from one extreme to the other. A few years back no one cared for metrics, now its all the rage. I think there is as much value in well organized and defined dashboards (interpretation of the infomration) as in the numbers themselves. Am I off-base?

 Hi mark and Paolo

I think Mark is right I designed a dashboard for a company and indeed there are also HRM things on the dashboard that are collected by hand.

   * You could think of things like howmany new employees have signed
     the secrecy statement
   * And if your thinking of a cleardesk policy this can be checked by
     security people (responsible for physical security)
   * Hire and fire policies followed by the manager, forms filled in
     assets returned before leaving the company
   * Security meetings held or security on the agenda of the management
   * Security acceptance criteria enforced by the implementation of new
     hardware or business applications


So not all of the world is or could be measured by tools