You asked for dashboards

One of my first conversations with Spyros when I joined Smithy went like this:

😺 “… and we’ll have an awesome web application with custom components, interactive workflows, managing runs, and showing the results in a nice dashboard…”

👾 “Wait wait wait. We don’t want dashboards. We don’t want to build yet
another source of noise. We don’t want to spend a year building something that people would just groan at and then ignore. I have flashbacks of having to look at 6 different places and trawl through all the alerts for the 1 or 2 true positives.”

😺 “But how are we going to show workflow results in a consistent and usable way?”

👾 “Teams already have those, we just need to integrate with them.”

And yet, on every discovery call, every demo, and every conversation, every customer kept asking:

👥 “So when the workflow is done, how do I see the results? Where’s your dashboard?”

Why dashboards can be a tar pit

A “tar pit” in the world of startups is a product or feature idea that looks amazing initially, but when you start working on it, you sink all of your time and effort until it kills you. Ross Haleliuk explains it really well here.

If we build a dashboard without thinking carefully, we could fall into these traps:

1. Self-defeating: Solving the “noise problem” adds more noise
   Colourful graphs and moving charts for their own sake don’t necessarily add value, but they sure add noise. Many tools may report false positives, duplicates, or results that are not well-defined.
   As companies aim to show how many wide-ranging vulnerabilities they can catch, they may be tempted to create dazzling reporting visuals.
2. Time sink: Every user wants something slightly different
   Effective data reporting is different for every user type. A CISO needs to see different results than a SOC team lead, and that’s different from a DevOps specialist or a blue teamer. We can spend six months building a dashboard for one target user, or at least thrice that for all target users. Afterward, every new user would have their own requirements, and they would keep changing forever.
3. Anti-value: Just another “Pain of glass”
   We are building Smithy so users don’t tinker endlessly with cybersecurity tools. If we start posting analysis and alerts, we just add another ToDo list to people’s daily schedules, thus adding more work. Most customers already have a way to get reports in Kibana, Grafana, and whatever ASPM they are using.

Why our dashboards are not a tar pit

Because they aren’t dashboards.

Our customers are asking for a quick way to verify whether their workflows are running successfully. They want to check that with minimal effort, before plugging into their real reporting tool.

If the user is just testing a manual scan, they can get the results on its instance page:

However, a single vulnerability scan does not reflect real world usage. The user may be testing a workflow that runs across their entire company, with an instance for every repository, or an instance for every daily deployment.

In that case, we need to reduce all the noise, before we can show the results. That’s where we’re making use of our new SDK and components, which all speak OCSF. We have built a new data warehouse which keeps all results in that format. It also includes all the magic that intelligently prioritises, enriches, contextualizes and de-duplicates them.

This allows us to fulfil the root needs of the users who ask for dashboards (as a first step):

The purpose of this report is not to be a to-do list (although it can be, if you wish).

It’s a way to check which repositories are the most vulnerable, which ones are covered by your workflows, and which tools are giving you the most value.

It’s a way to analyse your tooling and reduce effort and cost.

Each issue also has further metadata about the specific issue location and how it was found.

Where are my Dashboards?

You won’t see any spider graphs, 3d pie charts, sankey diagrams, or approximate time saving estimations. We aim to show only information that represents the real-world truth, in a clear and usable way.

How we give users only what they need

1. Quiet
   Creating a workflow does not introduce a new source of alerts. It only shows you the vulnerabilities you would get from your existing tools, but de-duplicated and prioritised.
2. Every user gets what they are looking for
   Every user we know wants to see a basic view of the results, immediately. We are only showing that. Every user can filter, group and share the reports, to suit their own needs.
3. No more “Pain of glass”
   Users can still use their old reporting tools, and don’t need this one to maintain their systems.

What do you think?

Of course, this is just scratching the surface. What is your strong opinion about dashboards?

If you disagree, you can join our Discord channel and have a geeky chat with us.

Or if you agree, why not try Smithy out? You can always book some time with us here.