Try OpenEdge Now
skip to main content
OpenEdge Change Data Capture Guide
Change Data Capture : CDC Policies : Policy life cycle
 

Policy life cycle

CDC policies guide the production of change data capture records. In the schema section, the Cdc-Table-Policy and Cdc-Field-Policy tables were defined. The policy designer manages these policies. The OpenEdge Management policy tool allows the policy designer to manage activities such as schema changes and temporary deactivation of change capture activity for database maintenance. The following sections detail the activity surrounding the creation, deactivation, reactivation and disablement of policies by the tool.

Creating policies

Creating a CDC Table and/or its Field policies is when the policy designer establishes user defined indexes and the defines the capture columns of interest in the Change Table. This is also the time the designer indicates the level of capture desired. Creating a policy helps with the transition from one table definition to another due to schema changes. Once the policy set has been established, a certain number of schema change operations are allowed. Other actions allowed for a policy set is to be deactivated, reactivated, or disabled.
Consider how creating a policy on a new table differs from an existing table in terms of captured record operations. The type of record operation captured could affect the initial value of a field in the external data source. Creation of a CDC policy for a new table is different only in the captured field values to start the process are different. In the case of the new table, generating change records could be captured as insert operations in a Change record. CDC capture on an existing table might be captured as update operations in a Change record. The update operations could be of two types: old and new column values in separate Change records. These types of Change records can be used to initialize the external data source during ETL and may yield different results.

Policy Instances

Policies are managed by the value of the _Policy-Instance field in the Table Policy. The table below explains the instance values.
_Policy-Instance Field
Values and their meaning
0
0 means this is a current instance of the policy and a Change table may be in existence for this policy depending on policy level. This is the policy that is loaded to the CDC cache and used during data capture by the database triggers.
1
1 means this is a previous instance of the policy.
If the policy level of the policy required a Change Table, it remains in existence even if the current policy is not adding to it.
A previous policy can represent historical data capture.
2
2 means this is a pending instance of the policy. The pending policy occurs when a user has not finished editing a policy and will continue at some other point in time.
Only one current instance is in use by CDC triggers for purposes of change capture. Many previous”instances can be in use by the ETL application(s). The ABL language and the Policy Tool will restrict that only one instance of the policy will be a current instance. The previous policies will exist to support data extraction. Purging of previous policy change records is at the discretion of the policy designer. The current policy is used immediately to generate change records through CDC triggers as current CUD operations occur.

Deactivating and reactivating policies

There may be times that particular CDC enabled tables need capture activity suspended temporarily, such as during a time-consuming database maintenance activity. The activity can be temporarily suspended by deactivating a table policy through the policy management tool. Deactivation and reactivation activities apply to policies in active or inactive states (_Policy-State).

Deleting a Policy Set(s)

The deletion of a policy means the policy designer has decided the policy or policies designating that a table is enabled for CDC are no longer needed. This means the object state or instance for each Table policy related to the Source table does not matter. The action will occur on an active or inactive policy set. The action of deleting a policy has the effect of removing the related Change Table and Policy Tables. In addition, the CDC cache entries for these policies are invalidated.

Policies and schema changes

If a source Table is altered by a schema change, a new Table policy may or may not need to be added depending on the need of the policy designer and the intended external data source. When a new field is added to the Source Table, the current policy does not know about the new field, so the policy designer must create a new Table and Field policy set to include this field, if capture is desired. Until the new policy is created, it is as if the field does not exist for the database triggers. When the new field is added and a new Table policy is created, the policy set will now include the new field policy.
A schema change involving a field deletion will cause the current policy to no longer record changes to this column. A data capture for the deleted field will record no value in the change record.
Below are common Source schema changes and how they are supported.
Source Table Schema Changes Supported
Action Required
Change to the Table Policy with an instance of “current” - Level field can be altered to specify either more or less detail in the captured data.
Commit change and it becomes immediately in effect.
Add a field to an existing Source Table.
When the new field is added to the Source Table a Change Tracking record with an operation code “add field” is created. In addition, the Change Table’s field map will reflect the newly added fields.
Commit schema change. In order to capture the new field, add a new Table/Field policy set that includes the new field. Upon creation, the new policy becomes the current instance and the old policy becomes the previous instance. As soon as this happens, the database triggers will begin to capture data against the new policy and use the existing Change Table.
Delete a field to an existing Source Table.
When the new field is added to the Source Table a Change Tracking record with an operation code “add field” is created. In addition, the Change Table’s field map will reflect the newly added fields.
Commit change schema change. From the point the field is removed from the Source Table, the deleted field in the change record will no longer contain values.
If a source field participates in an identifying field in the Change Table, it cannot be deleted. The policy(s) containing this field must be deleted (as well as the Change Table itself) before this field can be dropped.
Table Rename of Source Table.
A change of a table name does not require a change to the Table policies because the Source Table recid is stored in the policy, rather than the table name.
Field Rename of field in Source Table that is being tracked in a field policy.
Field rename is allowed, the field policy does not store field names. The Change Table does use the field name to name the Change fields, so field renames must cascade through the Change Table schema.
This action is restricted to an offline operation because it requires a Database Schema lock.
Source table is dropped by Language Drop syntax.
This operation requires that the policies are already deleted. Act of dropping table does not remove records in the Change Tracking Table.
Source partition is dropped
No effect on data capture activity. The capture is based on the Source table and not a specific partition so capture proceeds as usual.
Source partition is renamed.
Both the languages and a utility can rename a partition. The act of renaming a partition does not change the partitionId, so this has no effect on CDC capture activities.
The following table describes supported changes to Table and Field policies.
Table 2. Supported changes to Table and Field policies
Change
Action Required
Change to the Table Policy with an instance of “current” - Level field can be altered to specify either more or less detail in the captured data.
Commit change and it becomes immediate.
If a Table Policy is marked as in the “Pending” instance and the _Object-Id field is set to zero, the table policy can be altered.
Change the table policy. Add/delete field policies. This may occur when a policy was being constructed and was not “finished” and made “current”. This policy can be revisited until it is made to be the current policy.
A field policy may be added, deleted, or changed so long as the parent Table Policy is in the “Pending” instance and its _Object-Id field is set to zero. An object Id of zero means the Change Table associated with the policies have not been created yet.
Change field policy.
Deleting Table/Field Policy Sets.
Note: The intention for this operation is that it is a cascading delete operation. All table and field policies and the Change Table will be deleted at once.
Can occur on a Table policy regardless of state or instance. Does not delete related records in the Change Tracking Table.
The following table describes changes to Table and Field policies that are NOT supported.
Table 3. Changes Not Supported for Table and Field Policies
Change
Action Required
If a Change Table was created without an Identifying Field Index, a new Identifying Index cannot be added to the Change Table.
Existing Policy Sets cannot be altered to add a new index. A new policy is required for this table and the old one must be retired.
If a Change Table was created with an Identifying Field Index, the Identifying Field Index cannot be deleted from the Change Table.
Existing Policy Sets cannot be altered to delete an index. A new policy is required for this table and the old one must be retired. An alternative to this situation would be to deactivate the index.
Identifying fields (which are components of the Identify Field Index) cannot be altered or added to an existing Table and Field policy set.
A new policy is required for this table and the old one must be retired.