Whitepaper written by Daniel Lambert
Structuring an Enterprise or a Business Architecture model without measuring any of its key elements is a pure waste of time. An enterprise architect practice that mostly limits itself to information technology (“IT”) architecture metrics has a very siloed point of view of the organization and will usually not be able to contribute to the success of a business and digital transformation of an organization. This whitepaper examines the need for measurement metrics in all phases of an Agile Strategy Execution. For each phase, effective measurement metrics, including key performance indicators (KPIs), will be identified and set in place to increase your odds of delivering successful transformational roadmaps and initiatives within your enterprise.
The Need for Useful Measurement Metrics
Enterprise and Business Architecture will only contribute to driving IT and business alignment if key business and IT stakeholders encourage a metrics-driven behavior (formulating KPIs and incorporating them into performance assessments). Failure of Enterprise and Business Architecture initiatives and projects is often because C-level management hasn’t taken the time to set the Key Performance Indicators (“KPIs”) appropriately.
Diane Lebeau and Diana Krohn at United Airlines illustrate very well how measurement metrics can be misleading in a presentation entitled “Using Business Capabilities to Make IT Metrics Meaningful”. Their IT reliability metrics were often disconnected from their business reality. Business feedback wasn’t trending with metrics used to measure IT systems and applications. It was unclear how application performance impacted business performance. Their reliability investment methodology was not consistent, and their metrics were not business impact-focused. Only by selecting measurement metrics at the capability and value stream level was United Airlines able to align IT to business.
KPIs can be defined at all levels of a business architecture model, including objectives/strategies, customer journeys, capabilities, value streams, organizations, initiatives, stakeholders, products/services, information, physical and IT assets, requirements, and processes, and especially all the actions and relationships that link the various parts of a business architecture model together.
Some business strategies may truly be ill-advised and extremely risky. Showing the differences between strategies using chosen KPIs is what distinguishes the best enterprise/business architects from the others. Good business architecture will allow identifying the gaps between the current architecture and future state options by comparing architectures against KPIs and quantifying the risks, using impact analyses and decision trees among others.
Measuring All phases of an Agile Strategy Execution
To contribute optimally within its ecosystem, business and enterprise architecture teams need to get involved and contribute to the implementation and use of key performance indicators and measurement metrics in all 4 phases of an Agile Strategy Execution organization, as shown in Figure 1 above.
Business design and strategy is the first phase of an Agile Strategy Execution. It should involve strategies, tactics, but preferably measurable goals and objectives as laid out in a business motivation model. More precisely, the goals and objectives of an organization should include customer-centric, operational, and financial KPIs. While architecting transformation in phase 2, identifying and assessing business capabilities is a necessity. Different measures can be used, like priorities, performance, business complexity, maturity, technical risk, functionality level, reliability, people, and process. Once problematic capabilities have been identified, scenarios of initiatives and roadmaps can be clearly laid out. To choose among these scenarios, financial ratios will be calculated, and risk factors identified for each scenario until it becomes clear which one of the scenarios should be delivered and executed. Delivering an initiative will be done through numerous projects, and sprints. Each project will have a financial budget and timeframe to respect and will target business outcomes linked to goals and objectives.
Phase 1 - Business Design and Strategies
Business architects and enterprise architects are often asked to contribute to the prioritization initiatives and projects. To complete this task, they need their organization’s strategies, goals, tactics, and objectives. Too often, these can be very difficult to gather either because architects have never been aware that they existed or sometimes because they simply and plainly don’t exist. To extrapolate these strategies and objectives, there are numerous business models that are being used for business design, as explained in this article entitled “Business Design and Architecture”. In this whitepaper, we’ll limit ourselves to examine the measurement aspects within Business Motivation Model and Customer Journey Maps.
a- Business Motivation Model
Many enterprise architects that execute any business architecture will often include in their practice the business motivation model. Adopted by the Object Management Group (OMG), the Business Motivation Model (BMM) provides a scheme and structure for developing, communicating, and managing business plans in an organized manner. The core of this methodology resides in Figure 2 below.
Here are the basic definitions. A mission is a short statement of why an organization exists and often specified more precisely with a vision statement to be achieved within a finite time frame and made of a series of long-term goals and short-term objectives. Strategies consist of high–level plan items that also include short-term tactics to achieve an organization’s major goals under conditions of uncertainty. A goal is the desired result that an organization envisions, plans, and commits to achieve within a finite time frame. As for objectives, they are essentially short-term goals whose achievement brings an organization closer to its long-term goals. Objectives are derived from tactics, which are conceptual short-term actions to deliver and execute a strategy.
b- Creating a Customer Journey Map & Measuring Its Success
Examining customer journey maps is becoming the norm within many organizations. A customer journey map is a visual representation of every experience your customers may have with your organization. It helps to tell the story of a customer's experience with your products, services, and brand from their original engagement and into possibly a long-term relationship. The whitepaper entitled “Architecting and Delivering Optimal Customer Journeys” describes in detail how enterprise architects can plan ahead to optimize customer journey maps.
The benefits of using customer journey maps are numerous. They bridge the communication gap between sales, marketing, and operations. They provide a better understanding of the customers by building a higher emotional connection with them. They can assist in identifying participating stakeholders and enabling capabilities, as shown in Figure 3 below. Finally, customer journey maps also enable the identification of holes, where initiatives need to be planned to fill them with effective touchpoints. Journey maps do not need to be limited to customers. Journeys are not just limited to customers, they can also apply to partners, vendors, regulators, internal stakeholders, etc.
Customer experience measurement metrics should be taken at each step of a customer journey map to examine the possible breakdowns and disjoints of each step. At a minimum, customer journey maps will show the typical customer emotion measured for each stage of a customer journey map. Several other customer-centric measurement metrics can be used at each stage of a customer journey, including the 19 mentioned in Figure 4 below .
Phase 2 – Architecting Transformation
Architecting transformation can involve many domains. In this whitepaper, we’ll limit ourselves to architecting and assessing business capabilities, since business capabilities are considered by most enterprise architects at the heart of their business architecture and that they have relationships with most other business domains in an organization as shown in Figure 5 below . A capability is “an ability that an organization, person, or system possesses, ” according to the Open Group. Capabilities are far more stable in time than org charts, processes, products, or applications. Capabilities can simplify a lot your enterprise planning, delivery, and execution. Enabling business capabilities is what is needed by an organization to create value for its customers.
A business capability enables a stage of a customer journey (or value stream), among others. Measuring current and new customer-centric enabling capabilities, as shown in Figure 6 below, according to their priority, performance, and business complexity, for example, will allow building strategic initiatives that focus foremost on the capabilities that will provide the most value to both the customer and the organization. In this example, enabling business capabilities of a customer journey (or value stream) are sorted first by their priority, performance, and complexity. It enables to identify the most problematic business capabilities, indicated here by a red arrow. These are of very high or critical priority and performing at a low level. In case there are too many problematic capabilities, starting initiatives on the less business complex initiatives will enable quicker delivery and execution of improved business capabilities.
Other measurement metrics can be used to assess business capabilities. Figure 7 below, for example, shows 17 types of business capability measurement that can be used within an organization by business and enterprise architects.
Often because quantitative measurement metrics are not available, business architects and enterprise architects will rely instead on surveys from subject matter experts, middle or high-level management to assess the enabling capabilities of customer journeys based on a set of questions like the ones mentioned in figure 43 of this book entitled “Practical Guide to Agile Strategy Execution: Design, Architect, Prioritize, and Deliver your Corporate Future Successfully”, where answers will afterward be examined and weighted.
Phase 3- Initiative Planning
Once enterprise architects have assisted management in selecting a roadmap and/or initiatives to deliver and execute based on identified problematic capabilities, scenarios need to be identified and measured financially and in terms of risks factors in collaboration with financial analysts.
Customer-driven digital transformation and new product and services projects should not just examine costs, but also revenue and profitability while a financial and risk analysis is being performed. Enterprise architects must perform this task with their collaborators within their organization. Revenue assumptions should be fixed with marketing, sales, and finance stakeholders. Cost assumptions should ideally be based on past experiences or proposals received from several vendors and finance stakeholders. Be aware that long-term versus short-term views, may affect decision-making drastically. Finally, it should be noted that choosing scenarios based on risk factors alone, without crunching the numbers is far from the optimal approach to portfolio management.
Let’s examine the financial planned cash flow of scenarios 1 and 2, as shown in Figure 8 above. For further definitions and explanations about some of the financial terms used in this section, read section 6.1 in this book entitled “Practical Guide to Agile Strategy Execution: Design, Architect, Prioritize, and Deliver your Corporate Future Successfully”. First, note that it is not a profit and loss table. It’s the total amount of money being transferred into and out of the organization regarding each one of the two scenarios, especially as affecting liquidity. It should also be noted that it is laid out by quarter and that it represents approximate amounts of dollars. Although scenarios 1 and 2 have a duration of 12 and 18 months, respectively, the length of the examined cash flow period is 20 quarters or 60 months. It could easily be shorter than this, and sometimes longer depending on the size of the initiative. This cash flow also includes two types of cost savings for each scenario: fewer loan managers and lower bad loan provisions. This cash flow also consists of the cost involved in each scenario. They can be salaries, fees, expenses, and necessary assets for the completion of each scenario.
Already observations can be made regarding the planned cash flow of scenarios 1 and 2, as shown in Figure 8 above. They are as followed:
- 1. Scenario 1 is profitable by Q5, and its cumulative loss stops in Q12,
- 2. Scenario 2 is profitable by Q7, and its cumulative loss ends in Q12,
- 3. Scenario 2’s gains after five years after the beginning of the initiative are 82% higher than in Scenario 1 (or $5,1 million higher- $6,200,000 for scenario 1 versus $11,300,000 for scenario 2), and finally
- 4. Scenario 2’s cost is $3.4 million higher than in Scenario 1 ($4.4 million for scenario 1 versus $7.8 million for scenario 2).
Based on the cash flow projections made for scenarios 1 and 2 shown in Figure 8, financial ratios can be calculated, as shown in Figure 9 above. There are seven financial ratios mentioned here. They are capital investment required, time to profitability, cash flow after three years, cost savings per quarter, internal rate of return (IRR) over three, four, and five years. The net present value financial ratio could also have been used. It is clear here, that scenario 1 is preferable to scenario 2 for most of the ratios mentioned in Figure 9, except for cost savings per quarter and IRR over 5 years.
The Monte Carlo method and other financial simulation methods can be used to simulate the financial cash flow of substantial and long initiatives often involving infrastructure or non-IT assets. This kind of initiative may have results and a timeframe of high statistical variance like scenario 2 (the blockchain project). Variables with high variance may typically include 1- new types of revenues, 2- the quarterly growth of income, 3- the timing on the delivery of a strategic initiative (it could drag), 4- the high cost of sub-initiatives that have never been executed in the past, 5- the volatile cost of scarce resources, etc.
Enterprise architects often consider multiple risk factors to evaluate scenarios. Many of them are not financial, as shown in Figure 10 above. The strategic value, the need for change, the market opportunity, the standard conformity, the reliability enhancement, the innovation, and the complexity reduction in Figure 10 are not financial risk factors, for example. Contrarily to financial risk factors, other types of risk factors are often more subjective and less factual. If we consider the diamond diagram shown in the top section of Figure 10, scenario 2 covers more surfaces, than scenario 1. Examining the table at the bottom section, six out of eight risk factors are in favor of scenario 2, and therefore, only two favor scenario 1. Contrarily to the financial ratios shown in Figure 9 above, considering the eight risk factors mentioned in Figure 10 would lead to the selection of scenario 2, which implies the use of blockchain technology to automate further the personal loan delivery process.
Most organizations do not limit themselves to only one strategic initiative (and their intrinsic scenarios). They have several of them, as shown in Figure 11 below. Here, there are five initiatives, including scenario 1 from our previous example, that are evaluated according to six different factors, the capital investment required, time to profitability, IRR of five years, strategic value, standard conformity, and complexity reduction. The organization has a total budget of $8,000,000 for strategic initiatives. It cannot select all five initiatives that require a total capital investment of $14,854,000. Which ones should be chosen? Should our example, described in the previous section, which consists of increasing the automation of personal loans using current applications, be selected?
To answer these questions, let’s pick the top two initiatives for each one of the six different evaluating factors. The first top initiative for each factor is in a red rectangle. As for the second top initiative, it is shown with a golden rectangle. Based on this method of evaluation, initiatives 1 and 3 could be selected. These two initiatives alone require a total of $ in capital investment, leaving a free balance of $878,000. This is n enough to select initiative 5 that requires $889,000 in capital investment. It should also be noted that initiative 2 involves a lot of capital and is never among the top 2 based on any of the six evaluating factors.
It should be noted that selecting the top initiatives to be delivered and executed should be done at regular intervals, at least once a year. Some organizations do this exercise every six months. It is possible that longer partly initiated initiatives may not make the cut the second time. In this case, it is not uncommon to allocate the resources that were dedicated to the selected/unselected initiative to another one that made it through the top strategic initiatives to be delivered.
Phase 4- Agile Delivery & Execution
Finally, once a scenario is decided for an initiative, capability-based initiatives need to be broken into multiple projects, sub-projects, and sprints, laying out their nomenclature and GANTT Chart. Each project and sub-project should include appropriate budgeted human and capital resources that should be used in a specific timeframe and should finally aim at a precise and unique milestone or business outcome, as shown in Figure 12 below. Setting a definition of a milestone may only be technical, but each business outcome should be derived from precise and detailed goals and objectives defined in phase 1 of an Agile Strategy Execution.
This whitepaper has examined the necessity for measurement metrics in all phases of an Agile Strategy Execution. For each phase, effective measurement metrics, including key performance indicators (KPIs), need to be identified and put in place to increase your odds of delivering successful transformational initiatives within your enterprise. An architect practice that mostly limits itself to information technology (“IT”) architecture metrics has a very siloed point of view of the organization and will usually fail to contribute to the success of a business and digital transformation of an organization.
 Slide deck entitled “Using Business Capabilities to Make IT Metrics Meaningful” was made by Diane Lebeau and Diana Krohn at United Airlines in March 2017.
 Based on the definition of the Business Motivation Model on Wikipedia.
 Figures 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 are extracted from this book entitled “Practical Guide to Agile Strategy Execution: Design, Architect, Prioritize, and Deliver your Corporate Future Successfully”.
 Figure 5 is extracted from this video entitled “Capabilities – the Heart of your Business and Enterprise Architecture”.
 Definition extracted from this Open Group webpage.