Implementing Rule Execution Recording in a database
Corticon Decision Service execution is stateless, where all state is maintained in the message payloads. A single incoming message payload seeds the engine's working memory after which rules processing commences. Once rules processing is complete, the final state of the engine's working memory is returned as a response. The response message payload includes two discrete sets of data: data payload and posted messages that can each include data values as well internal information about the sequence of rules firing. While some data might have been stored in a database through an Enterprise Data Connection, the complete message payload is flushed from memory.
There are several reasons to want to retain payloads and rule messages:
Auditors might be assigned to determine how certain results were derived from a rule processing. When the exact request and its response payload can be accessed, the auditors can precisely reconstruct a decision.
Rules always evolve. Developers of rules can replay a 'real' request to see how rule changes impact responses as well as performance.
Rule modelers can check rule coverage in a large set of 'real' rule messages to see if rules considered obsolete are really not getting any use.
Rule modelers can also track down common sequences of rule activities to reveal race conditions or re-entrant rules.
Corticon's Rule Execution Recording feature records all input payloads sent to a Decision Service, all the rule messages produced processing the payloads, and the returned payloads. This provides documentation of the activity of a Decision Service. When setup and activated at the server level, each Decision Service deployed on the server must opt in to use the server's rule execution recording mechanism.
Overview of schema for storing execution payloads and rule messages
The general pattern of the schema for the necessary tables in a database you create are as follows:
Within the schema, each Decision Service version execution is timestamped with its time interval, and provided a unique primary key. Payloads and rule messages are in separate tables, each with a unique primary as well as the foreign key to link it to its execution identifier.
Consider:
The database you use for rule execution recording could be, and typically should be, distinct from the enterprise database of record. Rule execution recording can be used independent of Corticon EDC.
Multiple Corticon Servers could connect to and record execution in the same database.
Where rule tracing has been implemented, rule messages can be prepended with the metadata of the rule that fired. You enable this feature by setting a line in brms.properties ascom.corticon.reactor.rulestatement.metadata=true
The process is asynchronous "fire-and-forget" from Corticon Server. Data that accrues from this feature in your database is entirely your responsibility. You must provide adequate performance, storage, backup, rollover, and reporting mechanisms. Correspondingly, Corticon Servers and Studio does not read from the execution recording database at any time.
This feature is not supported on Studio's embedded server. When a remote server is used as the Studio's test subject, that server could be performing execution recording.
Creating the database schema for Rule Execution Recording
The first step is to setup the schema in your preferred database. Corticon Studio provides a wizard that lets you:
Define and test the connection to the target database
Create the default schema for the database brand in the target database
Define the properties and encrypted credentials that a server will add to its brms.properties file to connect to that database
Note: Your database administrator might prefer to create the schema by using (or adapting) the SQL script samples supplied in a Corticon Server installation at [CORTICON_HOME]/Server/src/sql. The database connection parameters can also be created entirely in a server's brms.properties file, although the credentials would have to be in plain text as the decryption algorithm requires that the credentials (either or both username and password) are encrypted by the algorithm applied in Studio.
To test and generate the connection parameters
1. In the target database's administrative tool, locate or create the named database instance for recording executions.
2. In Studio, choose the menu command Project > Create Execution Recording Schema, as shown:
3. In the Create Execution Recording Schema dialog, choose the Database Server brand and version brand from the dropdown list. Then edit the Database URL that is displayed to change <server>:nnnn to your server host:port, and <database_name> to the name you created. Then enter appropriate username and password credentials for that database.
4. Click Test Connect to ensure that the information achieves connection.
5. Click Show Encrypted Values to get the connection properties and values that you will move to each participating server.
6. Click Copy to put the connection information on the clipboard, click OK. Paste the information into a text file that will be transferred to Server locations for inclusion in their brms.properties.
Note: This completes the testing of the connection information and creation of encrypted credentials. You can continue to actually create the schema using the default scripts for the specified database.
2. Restart Studio, and then choose the menu command Project > Create Execution Recording Schema, as shown:
3. Enter and test the connection information as described in the previous procedure.
4. Click Test Connect to ensure that the information achieves connection.
Note: If the schema exists, its tables and existing data will be deleted, and then the tables recreated.
To create the schema, click Finish.
6. It is a good idea to shut off the ability to recreate the schema once it has been created. To do that, edit Studio's brms.properties to change the line you added to:
Note: Either or both the username and password could be entered as plain text instead of encrypted values.
Turning message recording on for individual Decision Services
With the schema set up in the target database, the intent to record enabled on the server, and the database connection tested, all that needs to be done is to set the properties on each Decision Service that will participate in this feature. By default, no Decision Service will use this feature. There are two ways to enable a Decision Service to enable execution recording:
Execution properties in a manually edited CDD file - Add the following line to the options in a decisionservice section of the CDD file: <option name="PROPERTY_EXECUTION_RECORDING_SERVICE_ENABLED" value="true"
CcServer API methods - Use setDecisionServicePropertyValue for the DecisionServiceName to set the static property PROPERTY_EXECUTION_RECORDING_SERVICE_ENABLED and the property value true.
Once established on a Decision Service, you can turn off execution recording by changing the setting to false.
Settings for Rule Execution Recording
These are the complete rule execution recording settings that you specify in the brms.properties file to control and impact rule execution recording behaviors:
This property will define how many executions will written to the database while wrapped in a single database transaction. The larger the number, the less times the CcServer has to make a connection to the database to establish a transaction. Better performance can be attained by increasing this number. Default value is 10
This property defines how many execution transactions can be queued up before the next transaction is rejected. The executions associated with that transaction will not be written to the database. A log message will be written to the log informing the user that data was not inserted into the database. Default value is 10
Auditors might be assigned to determine how certain results were derived from a rule processing. When the exact request and its response payload can be accessed, the auditors can precisely reconstruct a decision.
