AI Project Management – Measure Phase

In the last section we talked about the Define phase in Lean Six Sigma. As the result we will have a set of documents which answer following questions:

  • Does the A.I. project will financially benefit my company
  • Will the A.I. Project make a certain process faster, better or cheaper?
  • What do customers, employees and my business will consider a successful A.I. project. Which are the clear needs of each of the three groups
  • Define Process Metrics / KPI for customers, employees and company
  • Defince Success Metrics for the A.I. project – What values exactly have to be reached to make it a success or failure.

As Lean Six Sigma is data and math oriented, we try to put our project or problem in an equation:

X1 + X2 + X3 +… Xn = Y.

Each X resembles a cause for the problem and Y is the Effect.

Imagine your customer satisfaction is low, which is problematic for you. And you want to use Machine Learning to help fixing it.

While the Define Phase sets the frame for solving a specific problem, the Measure and Analyse Phase try to answer the question: “What causes my problem?”.

In your equation “Customer dissatisfaction” would be your Y. The reasons for a low customer satisfaction on the other hand would be resembled by Xs. Reasons could be “phone system always breaks down, when customers call”, “headsets are of bad quality”, “Customer agents are un-trained”, “Sold products are of low quality”…In the end, this would look something like this:

Bad ServiceTec + Untrained Customeragents + Low-quality products = Customer Dissatisfaction.

That already looks like something, your A.I. outsourcing partner could start to understand, doesn`t it? Of course, in a later stage you need to know exactly how to measure each X and define it in a data collection plan, but this equation already condensed the complete complexity of your company’s processes into a nice equation, which you can easily show to external partners for easy and quick understanding.

But before we reach this lovely equation, you will have to walk down a thorny path. And that is covered within the Measure and Analyse Phase. 

The main purpose of the Measure Phase is to find all relevant causes for your perceived problem. For that reasons the Measure Phase is equipped with a set of tools and structured meetings which can be used to engage all relevant stakeholders in your company to “discuss and extract” potential relevant causes.

In the following section we will discuss a few of the major important tools of the Measure Phase.

Tools and methods of the Measure Phase

Yesterday we talked about that the measure phase helps to find and define causes for your perceived problem. Today we are going to look into a few tools and methods which are used to achieve your goal.

The first step in finding the reasons for your problem is having brainstorming sessions with all potentially relevant stakeholders involved in the problem. Sticking with our problem of high customer dissatisfaction, we would involve service desk employees, their team lead, their head of division but also customer facing employees like account managers, sales managers and also product managers and technical personal, because customers mostly complain about products which do not work. Do not keep the group of people in your brainstorming too small, because you want to include all customer-touch points. Each person interacting with customers and interacting with the product might hold data and information which we can later use.

In the brainstorming sessions, stakeholders together fill out a so-called fishbone-diagram. Fishbone diagrams break down problem sources into six pre-defined groups (like people, machines, methods, material) and talk potential causes in those groups or categories. This pre-definition helps to lead group discussions and help people to stay focused on one topic. The result of the brainstorming sessions will be a long list of potential causes for the problem.

After that all processes involving the named causes from the brainstorming sessions will be mapped. This process discovery is one of the most important overlooked steps in A.I. projects. It sets potential causes in relation to each other and displays how people cause perceived problems while interacting across departments, teams and time. After developing a set of more and more detailed process maps, which we will look at in later chapters, the process maps will be converted in so called value stream maps where each process step will need to show which data points will be created there and what value it creates. These process step parameters will be later used by your A.I. outsourcing partner to fuel their models. As you see, your initial business problem was first embedded in your business processes and the process steps now are converted to data producing units. If you are at this point, your outsourcing partner will be able to much easier bridge the gap between your business and its ability to create data models.

In a next step, you will again come together with your brainstorming brain trust and reduce the potential causes for your problem. Why would you do that? Well, imagine you came up with 100 reasons for bad customer experience. Fixing all of these problems would take a long time and cost a lot of money. However, you should always focus on the main reasons for your problems. This will always yield the highest ROI. The following example will make it clearer: Imagine 9 out of 10 customers complain about products being broken after having been shipped and only 3 customers complain about Jim in customer support. What do you do? Well, it is obvious: fire Jim 😉 Joke aside, of course you would look at a new packaging solution, which in this case does not involve any A.I. at all.

Goldblum's Services