Configuring Database and File System State Persistence¶

This section explains how to prevent the loss of data that can result from a system failure by persisting the state of WSO2 SI periodically either into a database system or into the file system.

Prerequisites¶

Before configuring RDBMS database persistence, the following prerequisites must be completed.

One or more Siddhi Applications must be running in the WSO2 SI server.
A working RDBMS instance that can be used for data persistence must exist.
The requirements of the datasource must be already defined.
Database persistence involves updating the databases connected to WSO2 Streaming Integrator with the latest information relating to the events that are being processed by WSO2 SI at a given time.

Configuring database system persistence¶

The supported databases are H2, MySQL, Postgres, MSSQL and Oracle. The relevant jdbc driver jar should be downloaded and added to the <SI_HOME>/lib directory to prior to using database system persistence.

To configure periodic data persistence, update the <SI_HOME>/conf/server/deployment.yaml file under state.persistence as follows:

Parameter	Purpose	Required Value
`enabled`	This enables data persistence.	`true`
`intervalInMin`	The time interval in minutes that defines the interval in which state of Siddhi applications should be persisted	1
`revisionsToKeep`	The number of revisions to keep in the system. When a new persist takes place, the old revisions are removed.	3
`persistenceStore`	The persistence store .	`org.wso2.carbon.streaming.integrator.core.persistence.DBPersistenceStore`
`config > datasource`	The datasource to be used in persisting the state. The provided datasource should be properly defined in the deployment.yaml. For detailed instructions of how to configure a datasource, see Configuring Datasources .	`WSO2_PERSISTENCE_DB (Datasource with this name should be defined in wso2.datasources)`
`config > table`	The table that should be created and used for the persisting of the state.	`PERSISTENCE_TABLE`

The following is a sample segment of the required configurations in the <SI_HOME>/conf/server/deployment.yaml file to configure file system persistence.

Sample deployment.yaml segment

    state.persistence:
      enabled: true
      intervalInMin: 1
      revisionsToKeep: 3
      persistenceStore: org.wso2.carbon.streaming.integrator.core.persistence.DBPersistenceStore
      config:
        datasource: <DATASOURCE NAME>   # A datasource with this name should be defined in wso2.datasources namespace
        table: <TABLE NAME>

Configuring file system persistence¶

This section explains how to persist the states of Siddhi applications during a required time interval in the file system in order to maintain back-ups. To configure state persistence, update the <SI_HOME>/conf/server/deployment.yaml file under state.persistence as follows:

Parameter	Purpose	Required Value
`enabled`	This enables data persistence.	`true`
`intervalInMin`	The time interval in minutes that defines the interval in which state of Siddhi applications should be persisted	`1`
`revisionsToKeep`	The number of revisions to keep in the system. When a new persist takes place, the old revisions are removed.	`3`
`persistenceStore`	The persistence store.	`org.wso2.carbon.streaming.integrator.core.persistence.FileSystemPersistenceStore`
`config > location`	A fully qualified folder location to where the revision files should be persisted.	`siddhi-app-persistence`

The following is a sample segment of the required configurations in the <SI_HOME>/conf/server/deployment.yaml file to configure file system persistence.

Sample deployment.yaml segment

    state.persistence:
      enabled: true
      intervalInMin: 1
      revisionsToKeep: 2
      persistenceStore: org.wso2.carbon.streaming.integrator.core.persistence.FileSystemPersistenceStore
      config:
        location: siddhi-app-persistence