Process Mining - Practical Uses

Process mining is a family of techniques that support the analysis of business processes based on event logs. During process mining, specialized data mining algorithms are applied to event log data in order to identify trends, patterns and details contained in event logs, to auto-discover and paint process definitions of processes. Process mining can be used to discover processes from events or improve process efficiency by identifying bottlenecks and exceptions

By itself process mining has several applications in large enterprises, to discover or validate business processes (in finance, manufacturing and retail) merely from events and timestamps, but the exciting thing is that the same technique can be used to get insights into user/customer behaviors on your website and also aid in customer segmentation.

Consider a simplistic apache web log (though much more detailed application logs can also be used)
127.0.0.1 - peter [9/Feb/2017:10:34:12 -0700] "GET /login.html HTTP/2" 200 1479
127.0.0.1 - peter [9/Feb/2017:10:34:12 -0700] "GET /catalog.html HTTP/2" 200 1479
127.0.0.1 - peter [9/Feb/2017:10:34:12 -0700] "GET /checkout.html HTTP/2" 200 1479
127.0.0.1 - peter [9/Feb/2017:10:34:12 -0700] "GET /checkstatus.html HTTP/2" 200 1479

The important pieces of information are the user id "peter", the date timestamp and the URL visited which can proxy up for the user's intended action. The above log can be easily transformed into

Sample CSV file

<user>,<action>,<action date>, <user attribute 1>, <user attribute 2>, <user attribute n>
UserId,Action,StartTimeStamp,EndTimeStamp,PremiumCustomer,UserLocation,UserOccupation
Peter,Logged In,12-20-2017  10:20:00 AM,12-20-2017  10:25:00 AM,premium_customer,US,architect
Bob,Logged In,12-20-2017  10:20:00 AM,12-21-2017  10:25:00 AM,premium_customer,US,architect
Peter,Browsed Catalog,12-20-2017  10:35:00 AM,12-20-2017  10:40:00 AM,premium_customer,US,architect
Alice,Browsed Catalog,12-21-2017  10:35:00 AM,12-21-2017  10:40:00 AM,premium_customer,US,architect
Bob,Browsed Catalog,12-21-2017  10:35:00 AM,12-21-2017  10:40:00 AM,premium_customer,US,architect

The above CSV data can now to be fed into a process mining tool to generate the "user/customer's" interaction process, as he navigates and uses the website. This can give many insights into how customers are using our websites and what we can do to make things more convenient for them.

Statistics like what time duration is spent by each user in any of the activities can also be readily obtained as a report from the tool. The major deviations in the main process flow can also be denoted. Animations of the process flow, over a period of months can be played out within minutes to get insight into the activities that are bottle-necking the overall process.

Instead of discovering processes for individual users we can generalize processes for say "platinum customers" by filtering the data within the process mining tool itself.

Process Mining deals with Petrinet and BPMN notations of process definitions and some popular algorithms for process mining are alpha miner, inductive visual miner and fuzzy miner.

Notes on Enterprise Application Development Strategy

At the end of the day, technology is an enabler for extracting business value. Some sure fire ways to burn money needlessly, during enterprise application development are noted below. Use it as a checklist to guard against, in your own enterprise development. As always, use your own context, while weighing-in, any generic advice.

• Don't care why and what the end user actually wants. Its more important to impress top management, with a good-looking demo
• Using a particular tech because it is popular at google, netflix or facebook, irrespective of its applicability within your own context, especially NFR context
• No clarity about the WHY, obsessed with only the HOW
• Not able to contain scope creep and needless escalation of complexity
• First thought that comes up becomes the definitive design, no choices, no alternatives considered
• Resume driven or Ego driven, development by tech leads
• Why use a suitable tech, when a tougher, riskier, newer tech is at hand, no matter how irrelevant
• Underestimating integration and dependencies
• Committing to technology choices without understanding the full scope of functionality and the non-functional requirements(NFR) in detail
• Forgetting that no application is an island. Its important that an application can "fit into" the enterprise landscape in terms of security standards, IDAM (identity and access management), audit standards, deployment, reliability and business continuity standards
• No thoughts given for migration path from existing application/s
• Disproportionate emphasis on UI look and feel rather than UX and information models
• Re-invent the wheel, under the guise of tech bravado
• Dev Teaming mistakes
• Thinking that employing 10 mediocre developers, instead of 3 crack devs, will be better
• Believe the myth that application devs, still write framework level code
• Never acknowledge, that app devs now, require to use proper frameworks and APIs, rather than belt out algorithms and complex programs