by Daniel D. Gutierrez, Principal Analyst & Resident Data Scientist
A logic processor like LogicGem is a valuable programming tool for unraveling sections of code implementing intense business logic. Deep networks of if-statements can be sources of bugs that are very difficult to solve, and a logic processor with its ability to render guaranteed “perfect logic” can offer valuable peace-of-mind to software engineers. Using decision tables, LogicGem empowers coders to find any missing, ambiguous, redundant or contradictory rules and easily fix them. These so-called “logic tools” become invaluable in situations where the complexity of business logic becomes overwhelming.
Many developers, however, find it challenging to come up with an initial decision table containing all the required conditions, actions and rules upon which the logic is based. There is this early period in the development process where the programmer engages a new project, and the desire is to just get started with a first-cut decision table. It doesn’t need to be complete, and maybe you’re missing some conditions and/or actions, and maybe the developer hasn’t completely thought through all the rules. This is where LLMs can kickstart the process and help initiate the first version of the decision table.
Prompt Engineering
In terms of the prompt engineering required for an LLM to get a good start with building a decision table, a Chain-of-Thought (CoT) strategy should be used. CoT leads the LLM to break the task down into smaller chunks and solve each one step-by-step. The LLM then combines these intermediate steps to produce the final answer. This kind of prompting will direct the LLM to properly build the initial decision table for further handling by the logic processor. For this use-case example, I used the free version of the Claude 3.5 Sonnet LLM by Anthropic.
Below is a sample prompt sequence for implementing a Customer Ordering example. You start by describing the layout of the desired decision table to match the structure that the logic processor uses. In this case, we’ll use the structure used with LogicGem. This is an important first step since without guidance, the LLM may come up with one of many alternative structures. It’s important to specify the use of limited-entry decision tables where the rules can only have Y or N values (whereas extended-entry decision tables can have other categorical values such as High, Medium, and Low).
You then proceed by prompting all the conditions, actions and rules you can think of. Most business problems include a short list of top priority items. The remaining items often account for most of the work because you typically have to think harder to define them. Start with the easily identified items to allow the LLM to create a first-cut version of the decision table. The following is one possible prompt sequence for this problem:
Create a limited-entry decision table where the top rows represent the conditions and the bottom rows represent the actions. The rules are represented by the columns.
Use the following requirements for the decision table for a customer ordering use case. This information should be used to come up with the conditions, actions and rules for the decision table.
if the order total > $1,000 and the customer is a member and the item is in stock then Apply 10% discount; Offer free shipping; Process order immediately.
if the order total is not > $1,000 and the customer is not a member, and the item is in stock, then process the order immediately.
if the order total is not > $1,000 and the customer is not a member, and the item is not in stock, then Place on backorder; and Notify customer service
The LLM responds with an explanation of the resulting decision table. This material includes definitions of items appearing in the decision table (like Y/N values in the rules), along with an accounting of all the rules it was able to glean from the information you provided in the prompts.
In this decision table:
- Y means “Yes” (condition is true)
- N means “No” (condition is false)
- X indicates that the action should be taken
The table contains three rules (R1, R2, and R3) based on the given requirements:
R1: If the order total > $1,000 AND the customer is a member AND the item is in stock
R2: If the order total is not > $1,000 AND the customer is not a member AND the item is in stock
R3: If the order total is not > $1,000 AND the customer is not a member AND the item is not in stock
LLM Generated Decision Table
Next, the LLM produces an actual image of the decision table containing the conditions, actions, and rules that were recognized from your prompts (see Figure 1). Once you review the decision table, you can prompt the LLM to add more conditions, actions and rules, or you can just enter the LLM’s first-cut decision table as-is into the logic processor in order to complete the job. The completeness of the LLM generated decision table depends on how complete your prompts were in describing the required logic. At this early stage in the project, it’s unlikely that the LLM produces a decision table in final form, that’s what a logic processor is for.
Using LogicGem’s Logic Tools
Once the LLM generated decision table is inside of LogicGem, the first thing you should do is check the Completion Ratio to check whether the logic represented in the decision table is complete. This is a wonderful feature of LogicGem that instantly computes whether or not the logic is complete or whether it needs more elements. LogicGem tells us that the completion ratio is 8:3 (see indicator at the bottom of the screen) which means that the model includes a total of 8 rules but only 3 have been accounted for. This incomplete decision table is par for the course when using an LLM as a front-end to a logic processor. The idea is to consult with the LLM in order to kickstart the initial version of the decision table.
Now, we can use LogicGem’s logic tools to finish the job. First, you might use the Ambiguity Check tool to see if the LLM produced any ambiguous rules (and if so, you’d use the Disambiguate tool to remedy this situation). In this case, the rules are unambiguous. Next, you can use the Missing logic tool to fill out the decision table with all the missing rules along with a completion ratio of 8:8 as shown below.
At this point, you’d use combinations of LogicGem’s logic tools, along with your own domain knowledge of the problem being solved to refine the remaining rules in a compact and optimized form as shown in Figure 3. The goal is to obtain a balanced completion ratio.
From here, LogicGem can translate the complete logic in the decision table to one of many supported programming languages, and natural languages.
Conclusion
Working together, an LLM coupled with a logic processor like LogicGem can yield very useful results. An LLM offers the ability to kickstart the logic building process by listening to your first-thought ideas for conditions, actions, and rules representing complex business logic, and then to construct a corresponding decision table. Getting started is often the most challenging part of using a logic processor, and an LLM is able to close this gap. As we’ve seen in this article, a leading logic processor like LogicGem can then be used to complete the process. The end result is logically complete code that can become part of a software application.
See Also
This article was originally posted on August 20, 2024, as Use Case: Leveraging an LLM as a Front-End for a Logic Processor for Radical Data Science.