Imagine yourself as an explorer, an Indiana Jones or a Laura Croft, seeking out buried treasure, map gripped tightly in one hand. As you take your next step, the flagstone beneath your foot sinks slightly. Odd, you think. There’s nothing shown on your map, but deep in the gloom ahead you can hear the rumbling of something large and heavy picking up speed and rolling in your direction…
When you embark on a digital transformation, perhaps your most valuable asset is an accurate and exhaustive description of the as-is state of your processes and their metrics. At BusinessOptix, we think it’s a critical success factor. In a previous blog on Process Mining we talked about ways of turning data from the mining process, into information and then into knowledge and insight, allowing you to build maps of your current processes and identify opportunities for improvement. The trouble with maps, as our intrepid explorer above found out, is the devil’s in the detail. Process mining gives you much, sometimes even most, of the detail you need but it rarely gives you all of it. Why? For the same reason you’re transforming in the first place – most organizations are hybrid environments, a mix of digital, legacy, and manual processes.
The challenges associated with mining are many and varied. Sometimes there’s too much data in the logs and you can’t see the wood for the trees, although this form of data distraction is becoming less of an issue with modern miners. Sometimes systems perform batch logging, where all activities are logged all at once at the end of a defined period of time, and as a result, the transaction order can be lost (as can the record of the actual change, in some cases). Sometimes events have missing attributes, which means the mining process can’t extract any value from them.
In some cases, the challenge stems from a disconnect between business and IT that is literally hard coded into the system. Maybe the event names in the transaction logs aren’t logical or don’t match those used by the business. These unintelligible semantics can require a time-consuming and costly exercise to reconcile them back to the relevant business process. Another kind of translation error occurs when is the mined data is at a different level to that by which the business understands the process. Even when all the data is available, it might be difficult to follow a transaction through a process because its case ID changes depending on where it is in that process. For example, in a P2P process, the transaction might start with a purchase scenario reference, then a purchase order ID, then an invoice number, and finally a payment ID. Same transaction; lots of different guises; insufficient correlation.
Perhaps the biggest challenge is that most processes don’t go straight through a system (which is probably the reason you’re trying to automate and transform in the first place). There are often branches and breaks off to different, unconnected systems, or to a manual intervention where the next activity – which might be a complex decision – occurs off-line. Manual processes are a particular problem because they’re invisible to the mining software. Legacy systems (yes, they still exist, even today), which often don’t produce the kinds of logs miners can interrogate, represent another opaque source of confusion.
None of this means mining is bad, just the opposite. On average mining can give you 60 – 90% of the answer depending on how much your environment is impacted by the kinds of issues listed above, and that’s true of any mining platform. Think about that for a minute – somewhere between ‘more than half’ and ‘almost all’ of what you previously had to do manually can now be automated. The moral of this story isn’t that mining is bad, it’s that you shouldn’t allow yourself to be seduced into thinking it’ll give you everything. That’s why process modeling still has a role and why we strongly recommend it as part of your transformation process. It allows you to fill in the blanks, which is vital because you won’t know how important those blanks are until you know what they are.
In a previous blog, Process Modeling vs Process Mining, Travis Bristow explains how these two approaches complement each other. He refers to process modeling as “the ‘de jure’ model of process discovery, relying on human perception and input to build a picture of how a business process works, or at the very least how it should work.” Modeling is, by its nature, subjective because it is the users’ perception of how the process works – the view of the surface of the pond, rather than what’s happening in the murky depths. It does, however, address many, if not all, of the challenges we discussed earlier that give data mining tools a problem and result in an incomplete process map. Modeling also gives you valuable insights into the why of a process, whereas mining focuses exclusively on the what.
You can model processes in two ways. You can adopt the traditional method, physically drawing them in a modeling tool using BPMN notation based on data gathered from workshops, interviews, and reviews of procedural manuals. The BusinessOptix platform uses mining data to auto-create process diagrams which an analyst can then tune in this way. On the other hand, you could use rapid process capture which is, simplistically, a way of crowd sourcing information using online surveys. You identify all the people involved in the operation you want to study (say, the accounts payable department) and send them a questionnaire via a link, asking them about the work they do – the inputs, the outputs, and the steps they take to turn one into the other. The beauty of asking the crowd (let’s say there’s one hundred people in our accounts payable department) is you get depth and breadth of coverage – multiple answers about the same process which helps to capture all the exceptions and boundary conditions and smooths out any inconsistencies or contradictions you might get with smaller sample sizes. It’s more efficient too because you’re using a small slice of everyone’s time rather than tying people up in lengthy set-piece workshops.
BusinessOptix advises its clients to use a combination of process mining and process modeling to develop the most complete and accurate view of their as-is processes and hence, the firmest foundation on which to build their transformation. In fact, with our platform, you can either use the built-in mining capability or use another process miner and use our modeling functionality (whether traditional or rapid) to fill in the blanks. Mining and modeling complement each other perfectly and enable you to build on actual behavior and performance metrics, rather than incomplete information or assumptions, which reduces risk and gives you better outcomes quicker.
