General Data Protection Regulation (GDPR) for WSO2 API Manager¶
The Forget-Me Tool, also known as the Identity Anonymization Tool, can be used to obfuscate the identities of an external user who was deleted based on the request of the system administrator. This tool removes the user identities stored in the database and in log files in order to meet the GDPR requirements.
Removing the references of the deleted user identities¶
Follow the instructions below to remove the references of the deleted user identities stored in WSO2 product databases and log files:
Step 1 - Build the Forget-Me Tool¶
Info
- This tool can be used to delete the users' identities across multiple WSO2 products simultaneously.
Follow the instructions below to build the Forget-Me Tool:
-
Download the
identity-anonymization-tool
repository to a preferred location. -
Navigate to the
identity-anonymization-tool
directory.cd identity-anonymization-tool
-
Build the Forget-Me Tool.
Run the following Maven command to build the standalone Forget-Me Tool.
mvn clean install
This downloads all the dependencies and builds the tool in your local repository. You can find the
org.wso2.carbon.privacy.forgetme.tool-SNAPSHOT.zip
file created in theidentity-anonymization-tool/components/org.wso2.carbon.privacy.forgetme.tool/target
directory. -
Unzip the
org.wso2.carbon.privacy.forgetme.tool-x.x.x-SNAPSHOT.zip
file. Wherex.x.x
refers to the Forget-Me Tool version.This creates the
identity-anonymization-tool-x.x.x-SNAPSHOT
directory with a directory. The path to theidentity-anonymization-tool-x.x.x-SNAPSHOT
directory will be referred to as<TOOL_HOME>
throughout this section.
Step 2 - Change the default configurations of the tool¶
All configurations related to this tool can be found inside the <TOOL_HOME>/conf
directory.
More information on the configuration related directories and files in the Forget-Me Tool
The following table describes the purpose of the most important configuration related directories and files of the tool, which are in the <TOOL_HOME>/conf
directory.
Description/File Name | Purpose |
---|---|
config.json | This is the master configuration file. You can configure this file depending on the Metadata database tables, access logs, audit logs, or any other log files on which you want the Forget-Me Tool to run. For information on how to configure this file, see Configure the master configuration file. |
datasources | This is the default directory where configured datasources are searched for when you run the Forget-Me Tool. If necessary, you can define your own datasource configurations depending on the databases that you want to connect to, and specify the defined datasource configuration location using the command line arguments. |
log-config/patterns.xml | This file should contain all the Regex patterns that can be used to find and replace the references to the deleted user identities in the log file entries. |
sql | This directory should include all the SQL files that contain the required queries to replace or delete the references to the deleted user identities. |
Configure the master configuration file
The following is a sample master configuration (config.json
) file. You can configure this file depending on the Metadata database tables, access logs, audit logs, or any other log files on which you want the Forget-Me Tool to run.
{
"processors" : [
"log-file", "rdbms"
],
"directories": [
{
"dir": "log-config",
"type": "log-file",
"processor" : "log-file",
"log-file-path" : "logs",
"log-file-name-regex" : "wso2carbon.log"
},
{
"dir": "sql",
"type": "rdbms",
"processor" : "rdbms"
}
],
"extensions": [
{
"dir": "datasources",
"type": "datasource",
"processor" : "rdbms",
"properties" : [
{"identity": "WSO2_CARBON_DB"}
]
}
]
}
You can configure the following in the config.json
file based on your requirement:
processors
- A list of processors on which you want the tool run. The processors that you can specify are predefined. Possible values areRDBMS
andlog-file
.-
directories
- The definitions of directories on which you want the tool to run. When you specify a directory definition, be sure to either specify the directory path relative to the location of theconfig.json
file or specify the absolute path to the directory.Example code snippet to define multiple directories in the Forget-Me Tool
"directories": [ { "dir": "log-config", "type": "log-file", "processor" : "log-file", "log-file-path" : "<APIM_HOME>/repository/logs", "log-file-name-regex" : "(.)*(log|out)" }, { "dir": "sql", "type": "rdbms", "processor" : "rdbms" }, { "dir": "log-config", "type": "log-file", "processor" : "log-file", "log-file-path" : "<IOT_HOME>/repository/logs", "log-file-name-regex" : "(.)*(log|out)" }, { "dir": "sql", "type": "rdbms", "processor" : "rdbms" } ],
-
processor
- The type of processor to use to process the instructions in the corresponding directory. extensions
- The extensions to be initialized before starting a processor.
The default configurations are set up as follows:
-
Read Logs:
The default read logs are available in
<TOOL_HOME>/logs
(Relevant to the standalone Forget-Me Tool) directory.You can change this location in the
directories
→log-file-path
section of the log-file processor. -
Read Datasource :
A datasource is used to establish a connection to a database. You can configure the datasources in the
<TOOL_HOME>/conf/datasources/
directory.You can point to the latter mentioned datasources in the
config.json
file using theextensions
section as shown below:As shown in the above code snippet, you need to specify the absolute path of the datasources that you defined in the"extensions": [ { "dir": "datasources", "type": "datasource", "processor" : "rdbms", "properties" : [ {"identity": "WSO2_CARBON_DB"} ] } ]
dir
section.Define the following in the
properties
section of theconfig.json
file as shown below:-
<key>
- Define the name of the directory in which the database scripts reside. By default, the database scripts reside in the sub-folders that are available in<TOOL_HOME>/conf/sql
(Relevant to the standalone Forget-Me Tool) directories.If you wish, you can add any additional database scripts in the relevant existing sub-folders itself or you can add them in a new sub-folder within the
sql
directory. -
<value>
- Define the database name on which you need to execute those scripts.
-
-
Default datasource name:
WSO2AM_DB, WSO2_CARBON_DB
-
Log file name regex:
(.)*(log|out)
Step 3 - Run the tool¶
Run the Forget-Me tool based on the WSO2 products that you have in your deployment.
Note
Before you begin...
- This tool is designed to run in offline mode (i.e., the server should be shut down or run on another machine) in order to prevent unnecessary load to the server. If this tool runs in online mode (i.e., when the server is running), DB lock situations on the H2 databases may occur. This DB lock may happen if at least one of your databases point to H2. For example, if you have the User, REG, and AM databases pointing to MySQL, but your Carbon DB is in H2, then also you can get a DB lock error when running in online mode.
-
If you have configured a database other than the default H2 database, copy the relevant driver to the following directory based on your deployment.
<TOOL_HOME>/lib
-
The tool is designed to replace all references to a deleted user's identity with either a randomly generated UUID value, or a pseudonym that you specify when you run the tool. Therefore, you need to manually delete the user and then use this tool to clear the residuals in tables.
Run the Forget-Me Tool¶
Follow the instructions below to run the Forget-Me Tool in standalone mode:
-
Open a new terminal window and navigate to the
<TOOL_HOME>/bin
directory. -
Execute one of the following commands depending on your operating system:
- On Linux/Mac OS:
./forgetme.sh -U <username>
- On Windows:
forgetme.bat -U <username>
Info
The command specified above uses only the
-U <username>
option, which is the only mandatory option to run the tool. There are several other optional command-line options that you can specify based on your requirement. For more information, see Supported command-line options when running the Forget-Me Tool - On Linux/Mac OS:
-
All references to the user are removed from the respective WSO2 products. You can view the generated reports inside the
<TOOL_HOME>/conf
directory. Reports will be generated with the naming convention ofReport-<PROCESSOR>-<TIMESTAMP>.txt
in your current working directory.Example:
Report-log-file-1598483873677.txt
Supported command-line options when running the Forget-Me Tool¶
The following is the list of all the command-line options that can be used with this command.
Command-line option | Description | Required | Default Value | Sample value |
---|---|---|---|---|
U | The name of the user whose identity references you want to remove. | Yes | -U alex.doe |
|
d | The configuration directory to use when the tool is run. | No | <TOOL_HOME>/conf |
-d /users/alex/forgetme/config |
T | The tenant domain of the user whose identity references you want to remove | No | carbon.super |
-T example-company
|
TID | The tenant ID of the user whose identity references you want to remove. | No | -TID 1234 |
|
D | The userstore domain | No | PRIMARY |
-D Finance-domain |
pu | The pseudonym with which the username should be replaced. Note If you run the tool to replace all references to a particular deleted user’s identity, then you add the same user back to the system for some reason, and later you want to delete the user again and replace all the references to the user with the same pseudonym that was used the first time, you need to specify a pseudonym when you run the tool the first time and ensure that you use that same pseudonym when you run the tool the second time. |
No | A random UUID value is generated as the pseudonym. | -pu “123-343-435-545-dfd-4” |
carbon | Define the absolute path of the WSO2 product in which you need to remove the references of the deleted user identities.
This value will replace the variable |
No | -carbon “usr/bin/wso2am/wso2am4.0.0” |