The Veracode platform is comprised of several different application security scan types, each functioning semi-independently. I was the lead designer for Software Composition Analysis (SCA), working closely with two teams of engineers and two Product Managers.
Previously, Veracode had acquired a start up that provided a similar SCA service. It was loosely combined with their existing SCA offering, but the two systems were not well-integrated.
This project was part of the larger on-going effort to determine the future positioning and shape the functionality of our SCA tools to deliver more value and a better user experience.
Most customers aren’t realizing the full value of our product, negatively affecting retention & renewal rates. Many customers who were licensed for this new product weren’t using it at all.
Better understand the most valuable aspects and pain points of each offering to help inform a unified product strategy, identify opportunities for better integration, and remove barriers to adoption.
Users & Needs
The product serves two main user groups: developers working to write code that meets the security standards of their organization, and software security professionals who create those standards and manage their organization's security program.
Broadly speaking, developer users are looking for clear information about what needs to be fixed, why, how, and by when. Security managers need administrative and analytical tools to define their program goals and monitor progress and adherence over time.
Mapping the Current State
The current state offered two parallel paths with similar functionality but significant differences in details & execution along the way.
The goal was to identify the most valuable aspects and pain points of each offering to help inform a unified product strategy, uncover opportunities for better integration, and remove barriers to adoption.
Where can objects and processes be consolidated?
What distinctions are valuable to retain?
What are the barriers to adoption for product B?
We started our research with 1:1 interviews with 8 internal customer support engineers and solution architects who have close contact with customers and how they use our products. This allowed us to better understand customer use cases and get an overview of common problem areas.
We then used these learning and preliminary ideation on possible future states to create a script for in-depth customer interviews. With the help of PMs and account managers, we secured interviews with 7 customers during the fall of 2020, which included a range of organization sizes and a mix of current users of either or both products.
From this research emerged a salient image of a fractured user experience moving between two siloed workflows, loosely bound together as a single product. Users struggled to find effective ways to use these different aspects of the product together and in concert with other products in our platform. The disparate organizational structures & hierarchy, and to a lesser extent, nomenclature and visual presentation, posed a significant learning curve for existing customers or anybody moving between the two systems. While this was not new information, the power of words and anecdotes directly from our customers helped spur action to address these concerns, and the details we gathered provided guidance for how we could do so. Looking at the major areas of dissonance, we identified policy as a good starting point, based on a combination of technical feasibility, our confidence in our understanding of the issues and the proposed solution, potential for positive impact to the customer, and laying necessary groundwork for future initiatives.
Design a unified policy management experience as a first step toward complete unification between the two product experiences, including:
Augmenting the existing SCA policy rules to incorporate all desired functionality
Allowing users to apply those policies in both scan workflows
Reducing duplicative work and upfront configuration
Balancing granular control with ease-of-use
Feature audit of the two systems to be combined
Long-established user difficulty using Controls
Basic discovery, may assume governed by a default policy rather than a different system of rules
Irregular multi-step process to enter editing mode
Vulnerability severity vs. Issue severity caused confusion, poor visibility/summary view
Duplication of work for ranges, with/without vuln method
Goal = provide support to create logical tests, but result was jargony and overly complicated
Legacy constructions that no longer made sense, e.g. “Create Issue” action is only option remaining > eliminate
Unclear how conflicting controls will be resolved
The unified policy designs represents a major increase in usability and efficiency. Instead of needing to translate and rebuild their policy from one system to another, users can now create a single policy for use by both systems.
Significant efficiency gains
Leverages existing patterns to introduce established best practices
Each rule type is self contained with clear logic and purpose
Wizard guides users through the configuration process
Clear high level view of rules & conditions, both during editing and after creation
Visibility into which other applications are using each policy
Augments existing rules to include essential functionality identified from the deprecated system to carry forward
Optional complexity with smart defaults and simple vs. advanced views
Streamlines logic for license handling
Centralizes on single measure of severity to reduce ambiguity and align with the industry standard