Jan Bosch describes the challenges and enablers of the Automation 2.0 stage where AI is unleashed at individual steps in a business process that AI could not automate without the help of modern AI agents or LLM.
In the second stage, called “Automation 2.0,” business process owners are trying to use AI to automate individual steps in a process that cannot be automated with traditional approaches. In many cases, this means that there is a need for a certain degree of pattern matching and human interpretable generation that could not be achieved by traditional means. Examples mentioned in interview studies include selection and prioritization of candidates in HR based on application and resume, transaction classification in financial and product planning tasks. Not all tasks are fully automated, but AI can provide important help and reduce human effort by several orders of magnitude.
Of course, nothing is as simple as it appears to be a high level, so we start by discussing the challenges a company experiences and the enablers that it needs to be implemented to achieve the desired outcome. The first challenge is, surprisingly, many companies use the process. This is largely implicit and not clearly defined in the process model. Basic processes are often explained at a certain level of high level, but often involve a significant amount of ambiguity and the need to understand the context that only humans can do to properly track the process. Obviously, when you’re trying to integrate AI, you need an explicit process model that captures most, if not all cases.
The second challenge is that replacing or substantial support for humans at specific process steps often requires considerable integration with existing systems and processes. The complexity of this integration can be enormous as it may require interfaces with CRM solutions, databases, and those far from the obvious extent to which data semantics in these systems can be missing or limited APIs.
Furthermore, building and operating these systems, especially in the context of regulatory compliance, requires a lot of skills, as it is important to understand how AI agents have reached specific outcomes. This is because you can sign off on the outcome and cause serious harm. This could lead to important cultural resistance that AI is simply not trusted.
The third issue concerns validating the results. How can you know that AI Agent or LLM results are good? Anyway, what does “good” mean? In many contexts, KPIs are simply not explicitly defined and, if so, may not match AI-driven contexts. For example, customer requests response times make sense when human support is running, but this metric is pointless when using AI agents. If you are unsure of what constitutes “good” or “sufficient”, it becomes difficult to evaluate the system and trust the solution it offers.
The three main enablers are tool integration, rags and human loops
To overcome these challenges, businesses must implement enablers. First, they need to invest in better tool integration. This often involves developing a richer API. Even existing AI solutions show that more and more ways to build integrations between agents and other systems, such as Openai function calls, Microsoft Copilot integration, and tools like Langchain. Furthermore, LLMS can be interpreted and acted more responsively than traditional software integration, so it is extremely useful to use AI models not only for inference but also for dynamically interpreting unstructured data from other tools.
The second enabler is to extensively use the retrieved generation (RAG) to provide considerable context for LLMS or AI agents that are not specifically trained in use cases. This allows these models to provide domain-specific responses rather than general boiler plates. As part of this, companies usually need to improve their data pipelines to ensure that data semantics are clearly defined and that the quality of the data is at a level that avoids bad results from the model.
The third enabler is to keep humans in a loop. Especially when output quality is important, for example, in regulatory compliance or high-risk situations, keeping people in the loop can go a long way in reducing cultural resistance and building trust in the solution. There are at least two patterns here. The first is that humans are the main operators and AI acts as supervisors. This is useful if two people were previously involved and one person was examining the other’s results. A more typical case is where AI is the operator and humans act as supervisors. In this case, there is often a continuous front and back between the two, and the model can learn and improve over time.
The second step in the maturity model is Automation 2.0. Here we automate or significantly support individual steps in business processes that could not be automated without the help of modern AI agents or LLM. There are usually at least three main challenges. The lack of explicit process models, the complexity of integration, and the validation. To address these, the three main enablers are tool integration and APIs, which use RAG to provide more domain-specific results and in-loop approaches. Ultimately, to quote Tom Preston-Werner, we either create automations or are those who are automated.
