BENCHMARKING AND MATURITY ASSESSMENTS: how a small time contribution garners major insights and helps to improve data management

  • 799

    BENCHMARKING AND MATURITY ASSESSMENTS: how a small time contribution garners major insights and helps to improve data management

    With the onslaught of regulatory change programs and corporate realignments, more firms are performing internal current-state assessments of their practices and procedures and reaching out to the industry to benchmark against peers. Often, these initiatives require considerable time and effort, yet once the final reports are delivered, typically only 20% of the findings are acted upon. In this article, Sylvia Darwell and Gavin Kaimowitz explain how taking a slightly different approach can provide the majority of the results in a fraction of the time.

    What is Peer Benchmarking?

    Benchmarking is a continuous, systematic process used to evaluate and compare the capability of the organization as a whole against its peers and/or industry standards in order to optimize processes. Benchmarking can also be utilized to compare certain functions of the organization, such as data management. The analysis builds the foundations that help to create a future vision and enables improvement.

    In order to benchmark itself externally against its peers, a company must first perform an assessment based on an industry-recognized measure, such as a capability maturity model (CMM). During the assessment process, the organization analyzes and uncovers the current state of its processes and areas of improvement, determines the steps to achieve it and defines how to measure progress.

    Benefits of Sentiment-Based Assessments

    A sentiment-based assessment is one type of selfassessment. It captures the opinion, attitude and/or perception of the user and further reinforces the value of the capability maturity exercise, quickly uncovering areas of strength and allowing stakeholders to consider leveraging them to improve the inefficiencies. As a result, it gives benchmarking a greater level of credibility.

    Performing a sentiment-based maturity assessment exercise creates immediate benefits for the firm. Often the perception and the reality vary within the organization. To see the difference between the sentiment (what people think) and the reality (what actually exists), firms must first conduct a current-state analysis, identifying procedures, costs, key stakeholders as well as inefficiencies and overlaps. Bringing together executives at the organizational level helps firms save the time and money that would be needed to garner a greater level of detail, otherwise uncovered only through individual conversations and countless meetings from various groups across different product types and possibly geographies.

    Furthermore, the organization is able to gather and integrate a breadth of information across the entire business. And lastly, by creating a sense of a common goal, the organization generates “buy in” that helps accelerate improvement when establishing and taking the next steps.

    The Capability Maturity Model

    The Capability Maturity Model (CMM) is an industry-recognized measure of maturity and one of the tools used in selfassessments. It was developed between 1986 and 1988 after a study of data collected from organizations that contracted with the US Department of Defense. At the request of the US Air Force, Watts Humphrey of Carnegie Mellon University formalized his Process Maturity Framework in order to create a tool that would help the Department of Defense evaluate the capability of software contractors.

    In its simplest form, a maturity model is a matrix of five levels of maturity that describes how well the practices and processes of the organization function. It is a set of characteristics and indicators that represent the progression and achievement in a particular discipline or process. The “maturity” refers to a degree of formality and optimization of processes, from ad hoc, to formally defined steps, to fully optimized and automated processes. The CMM is a tool firms use to develop and refine their processes. It allows the business to identify its current state, define a taxonomy, or common language, and outline a shared vision. It helps firms develop a framework to prioritize actions and define what improvements mean for the organization.

    The CMM’s application in software development has sometimes been problematic due to the application of multiple models that are not integrated within and across an organization and require costly training and improvement activities. To address the shortcomings of the CMM, Carnegie Mellon University, in conjunction with industry and government representatives, created the Capability Maturity Model Integration (CMMI), which superseded the CMM model. The CMMI continues to be a general theoretical process widely used across various disciplines, including business, commerce and consulting around the world.

    Creating and Running a Capability Maturity Assessment

    Although a maturity assessment is typically used externally, against peers or industry standards, it can also be used internally, whereby a division or business unit evaluates itself against another division or group within the same or different geography. In this instance, it is often easier to adopt the best practices and leverage the experience of an internal group that adheres to the same management and corporate culture, as well as restrictions and budget.

    The basic structure of the maturity model develops critical connections between objectives, assessments and best practices. The model has five levels of maturity (1-initial/ undefined; 2-repeatable/reactive; 3-defined; 4-managed/ measured; 5-optimized/proactive) which enable the comparison between organizations and offer a proven sequence for improvement. Level 1 is the most rudimentary, with an ad hoc process containing many steps that are performed by sheer effort, while Level 5 is the most advanced level or vision. At the highest level, the organization is agile and capable of responding to changes quickly and effectively, with fully automated, adaptive and high-quality data. At this level, the firm deploys a process considered “best in class.”

    While defining each level of maturity and creating process definitions in the context of the organization are the most challenging and time-consuming aspects of the maturity assessment, the time invested yields ample insights into the internal functionality (or lack thereof) of the business.

    Once the CMM matrix is defined and the assessment has gained buy-in from all participants, running the actual exercise is very straightforward. The self-assessment of the current maturity is scored and captured. The future statement or vision is defined and the list of actions that are needed to get the organization to the next level is created. As a result, the small amount of effort put forth to conduct an assessment generates substantial, long-lasting returns for the firm. To achieve the most accurate and comprehensive result, it is critical that key stakeholders across all business lines state their perception of the current state as well as the future goal.

    In the next step, results are scored and used to define measures of success and establish subsequent actions to achieve the business goals. The perception among stakeholders can differ widely; and what some consider to be the current state, others may deem a goal. This finding further highlights the importance of the exercise, which is to get all participants on the same page before moving forward with establishing and achieving future goals. Setting up a future vision can also be challenging. Some participants may consider a Level 5 maturity accessible, while others perceive Level 5 as something that the industry strives toward but cannot really achieve due to various limitations, such as time constraints, size and complexity of the enterprise or budget.

    In the final stage, stakeholders will often define a vision statement that will guide the formation of the foundational layer of information capabilities and responsiveness.

    It is important to note that the goal is not to score a Level 5 for each step that is being evaluated, but rather to recognize the priorities of the enterprise and to identify reasonable expectations. The next phase is to create a roadmap that will help stakeholders navigate from their current state to a state in which they can achieve their goals.

    An Example: Internal CMM for Data Management

    Effective internal assessments need to have actionable results. Achieving them requires a clear definition of the maturity levels and an understanding of the highest priority areas.

    The EDM Council, an industry body specializing in enterprise data management, is on the cusp of qualifying its Data Management Maturity Model into an official CMMI-recognized measurement. This maturity model measures across platform and architecture, data operations, data quality and data strategy. Within each of these areas, there are many granular levels against which firms are assessed. In its current form, firms can leverage the EDM Council’s selfcertification to determine their own level of maturity, utilizing a detailed check list.

    Figure 1: The Five Key Levels of a Data Maturity Model.

    Figure 1 presents an example of the five levels of maturity as defined in the EDM Council’s data management maturity assessment.

    While these levels are useful to help determine the maturity of data management practices within a single department, it can be challenging for multi-national firms to apply them across divisions. Often directives differ between different teams and geographies, making it nearly impossible to apply a single level across the entire firm or translate the insights of a single maturity level into meaningful action.

    Using a sentiment-based approach for data management maturity assessments has proven to minimize some of these challenges, while providing valuable results. For example, one bank conducted an assessment that included several stakeholders across the business (e.g., technology, operations, risk and their internal clients). A simple survey allowed participants to express their opinions, or sentiments, across multiple data management dimensions.

    To add a layer of detail, the “demographics” of the stakeholders were captured. These included the stakeholder’s department, function, job title and geographic location. Gathering this data allows the firm to further apply and benchmark the results against its internal peers with similar responsibilities. Stakeholders were also asked to rank the issues by the level of importance, assisting the business in prioritizing the next steps of improvements. While the individual scores of the stakeholders differed, together they identified a common goal, laying the groundwork for future improvement.

    Figure 2: Individual Maturity Levels across Departments with a Singular External Metric.

    In another example, a different bank wanted to evaluate the current state of its data management activities, including the assessment of data sourcing, data taxonomy and governance, data quality and infrastructure, and create a future design. By developing a capability maturity model, the bank was able to capture the current sentiment and outline a blueprint for the future. To amplify the impact of the exercise, stakeholders took it one step further by defining the required steps and progress measures to achieve their goal.

    Quickly Gaining Insight with a Heat Map

    A useful tool that allows executives to identify the most pressing issues and make quick improvements is the heat map. As shown in Figure 3, a heat map is a graphical representation of data where the values collected are represented in a two-dimensional matrix by a color. When used as a strategic tool, a heat map allows executives to quickly pinpoint the highest priority, most pressing issues or the areas that can be quickly resolved with relatively little effort. Doing so awards them more time to tackle the firm’s bigger and more complex initiatives.

    Figure 3: Sample of a Data Management Maturity Assessment and Heat Map.
    The Authors
    Gavin Kaimowitz

    Gavin Kaimowitz
    is Sapient Global Markets’ Global Data Practice Lead across the capital and commodity sectors. Gavin is responsible for the collation and generation of best practices, thought leadership and strategy. He has a proven track record in the reference and market data domain of solution design, business case definition, strategic roadmap design and in building data management products.

    Sylvia Darwell

    Leave a Comment