The Purge Scheduler and Purge Modules

The Purge Scheduler controls the timings of when the Database Purger executes and deletes unnecessary data rows from the database. In doing so, the Purge Scheduler assists in keeping the size of the Connect database manageable and retaining only recent and relevant information. The Database Purger is made up of several purge modules that each purge specific areas of data. The Database Purger runs within a global purge window controlled by the Purge Time Window system parameter: "Start purge during this time period" (Database | Advanced). For more information on managing system parameters, please see Managing System Parameters. The time windows for each individual purge module can be controlled in the Purge Scheduler screen.

Note: No purge module will run outside of the global purge time window, regardless of the scheduled settings.

Because the Purge Scheduler is date-driven, the system parameters associated with each purge module should be verified to ensure that the retention duration is appropriate for your organization's data life cycle. You should verify that the date range for each purge module is appropriate for the retention needs for your organization.

Purge Modules

There are several modules available that allow you to purge specific data rows that are no longer relevant or necessary. The timing of the purges are scheduled. Each module is managed by a system parameter that regulates the retention window for the specific data. That is, a retention system parameter that is set to '60' would retain all data within 60 days of the scheduled purge. All data rows specific to the scope of the purge outside of the 60 day retention window would be deleted. In this manner, regular use of the Purge Scheduler would maintain data for the most recent 60 days.

The system parameter values are able to be edited in the System Parameters module using a Category of 'Database' and Access Level of 'Advanced Access'.

The system parameters associated with each purge module is listed below:

Purge Module Descriptions

Each Purge Module contains the following:

Class name
Range of tables from which it will purge rows
Condition that regulates the information to be purged
Retention system parameter that regulates the data that is retained in days

The purge modules in control of purging all qualifying row entries are as follows:

Purge ETL Errors: Error messages occurring during ETL processes
Purge Click History: Customer click history in messages
Purge Counter History: Counter history
Purge Critical Errors: Critical error message entries from the error history log
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeErrorHistory
- Purged Tables: ERROR_HISTORY
- Purge Condition:
  ERROR_ID between 3001 and 4000 -> ErrorTypes.minCritical,ErrorTypes.maxCritical
  
  INSERT_DATETIME <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainErrorHistoryCritical (com.kana.connect.common.sysparam.dbm.RetainErrorHistoryCritical)
- Default Parameter Value: 90 days
- Minimum Parameter Value: 30 days
- Maximum Parameter Value: 365 days
Purge Customer Attributes: All customer attributes
Purge Customer Attributes History: All customer attribute historical data
Purge Customer Interest Stats: All statistical data relating to customer interests in link activity
Purge Customer Monthly Stats: Customer's monthly link activity statistics records
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeCustMonthStats
- Purged Tables: LINK_MASTER
- Purge Condition: STATUS_CODE != 9999 AND LINK_ID > 101
- Retain System Parameter (class name): DBM.RetainLinkInfo (com.kana.connect.common.sysparam.dbm.RetainLinkInfo)
- Default Parameter Value: 0 days (i.e. keeps the link active indefinitely)
- Minimum Parameter Value: 90 days
- Maximum Parameter Value: 99999 days
Purge Customer Preferences: Customer preference data
Purge Customer Queue: Customer queue data
Purge Customer Subscriptions: Customer subscription data
Purge Customer Weekly Stats: Customer's weekly activity statistics records
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeCustWeekStats
- Purged Tables: CUST_WEEK_STATS
- Purge Condition: STAT_WEEK <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainCustWeekStats (com.kana.connect.common.sysparam.dbm.RetainCustWeekStats)
- Default Parameter Value: 30 days
- Minimum Parameter Value: 14 days
- Maximum Parameter Value: 90 days
Purge Diagnostic Errors: Diagnostic error messages from the error history log
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeErrorHistory
- Purged Tables: ERROR_HISTORY
- Purge Condition:
  ERROR_ID between 1 and 1000 -> ErrorTypes.minDiagnostic,ErrorTypes.maxDiagnostic
  
  INSERT_DATETIME <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainErrorHistoryDiagnostic (com.kana.connect.common.sysparam.dbm.RetainErrorHistoryDiagnostic)
- Default Parameter Value: 7 days
- Minimum Parameter Value: 7 days
- Maximum Parameter Value: 365 days
Purge domain day agg: Domain daily statistical aggregate records
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeDomainDayAgg
- Purged Tables: DOMAIN_DAY_AGG
- Purge Condition: UPDATE_DATETIME <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainDomainAggStats (com.kana.connect.common.sysparam.dbm.RetainDomainAggStats)
- Default Parameter Value: 180 days
- Minimum Parameter Value: 90 days
- Maximum Parameter Value: 99999 days
Purge Error Errors: Error message entries from the error history log
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeErrorHistory
- Purged Tables: ERROR_HISTORY
- Purge Condition:
  ERROR_ID between 2001 and 3000 -> ErrorTypes.minError,ErrorTypes.maxError
  
  INSERT_DATETIME <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainErrorHistoryError (com.kana.connect.common.sysparam.dbm.RetainErrorHistoryError)
- Default Parameter Value: 90 days
- Minimum Parameter Value: 30 days
- Maximum Parameter Value: 365 days
Purge Event Queue: Event queue row entries that trigger events from event-triggered campaigns
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeEvents
- Purged Tables:
  - EVENT_QUEUE_DETAIL
  - EVENT_QUEUE_XML
  - EVENT_QUEUE_ATTACHMENT
  - EVENT_QUEUE
- Purge Condition:
  STATUS_CODE in (3,99) - HANDLED, EXPIRED
  
  INSERT_DATETIME <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainEvents (com.kana.connect.common.sysparam.dbm.RetainEvents)
- Default Parameter Value: 60 days
- Minimum Parameter Value: 5 days
- Maximum Parameter Value: 365 days
Purge Link Master: Master link data
Purge Message History: Purges all customer message history records
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeMessageHistory
- Purged Tables:
  - MESSAGE_HISTORY
  - MESSAGE_HISTORY_CONTENT
- Purge Condition:
  SEND_DATETIME <= sysdate - retention
  
  STATUS_CODE > 0
- Retain System Parameter (class name): DBM.RetainMessageHistory (com.kana.connect.common.sysparam.dbm.RetainMessageHistory)
- Default Parameter Value: 90 days
- Minimum Parameter Value: 30 days
- Maximum Parameter Value: 99999 days
Purge Msg Demo Seg Agg: Purges all message segmentation aggregate statistical information
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeMsgDemoSegAgg
- Purged Tables: MSG_DEMO_SEG_AGG
- Purge Condition:
  UPDATE_DATETIME < sysdate - retention
- Retain System Parameter (class name): DBM.RetainDemoAggStats (com.kana.connect.common.sysparam.dbm.RetainDemoAggStats)
- Default Parameter Value: 180 days
- Minimum Parameter Value: 90 days
- Maximum Parameter Value: 99999 days
Purge Msg Seg Link Demo Agg: Purges all message segment link aggregate statistical information
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeMsgSegLinkDemoAgg
- Purged Tables: MSG_SEG_LINK_DEMO_AGG
- Purge Condition: UPDATE_DATETIME < sysdate - retention
- Retain System Parameter (class name): DBM.RetainDemoAggStats (com.kana.connect.common.sysparam.dbm.RetainDemoAggStats)
- Default Parameter Value: 180 days
- Minimum Parameter Value: 90 days
- Maximum Parameter Value: 99999 days
Purge Reply History: Purges all of the reply entries from the reply history logs
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeReplyHistory
- Purged Tables: REPLY_HISTORY
- Purge Condition:
  REPLY_DATETIME < sysdate - retention
- Retain System Parameter (class name): DBM.RetainMessageHistory (com.kana.connect.common.sysparam.dbm.RetainMessageHistory)
- Default Parameter Value: 90 days
- Minimum Parameter Value: 30 days
- Maximum Parameter Value: 99999 days
Purge Storage History: Purges all storage history records
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeStorageHistory
- Purged Tables: STORAGE_HISTORY
- Purge Condition:
  UPDATE_DATETIME < sysdate - retention
  
  STATUS_CODE in (30,40,50) - PROCESSED, BAD, DROPPED
- Retain System Parameter (class name): DBM.RetainStorageHistory (com.kana.connect.common.sysparam.dbm.RetainStorageHistory)
- Default Parameter Value: 60 days
- Minimum Parameter Value: 3 days
- Maximum Parameter Value: 730 days
Purge Warning Errors: Purges all warning message entries from the error history log
- Purge Module Class Name: com.kana.connect.server.dbm.PurgeErrorHistory
- Purged Tables: ERROR_HISTORY
- Purge Condition:
  ERROR_ID between 1001 and 2000 -> ErrorTypes.minWarning,ErrorTypes.maxWarning
  
  INSERT_DATETIME <= sysdate - retention
- Retain System Parameter (class name): DBM.RetainErrorHistoryWarning (com.kana.connect.common.sysparam.dbm.RetainErrorHistoryWarning)
- Default Parameter Value: 30 days
- Minimum Parameter Value: 14 days
- Maximum Parameter Value: 365 days