

Production Deployment Guidelines¶

The requirements for deploying WSO2 products can be changed based on the deployment scenario and pattern. The recommendations under this topic are for general production use, assuming moderate load conditions. For situations where a high volume of traffic is expected and if there are large deployments, these guidelines may not be sufficient. See Troubleshooting in Production Environments for information on how to obtain and analyze information to solve production issues. The following are the topics addressed in this section.

Installation Prerequisites
Installing the WSO2 Product
Running the Product
Tuning Parameters
Common Guidelines and Checklist
Backup and Recovery Recommendations

Installation Prerequisites¶

Prior to installing any WSO2 Carbon-based product, it is necessary to have the appropriate hardware and software for running the product.

System Requirements¶

Physical	3 GHz Dual-core Xeon/Opteron (or latest) 4 GB RAM (2 GB for JVM and 2 GB for the operating system) 10 GB free disk space ~ Recommended minimum - 2 Cores. For high concurrencies and better performances - 4 Cores. Disk space is based on the expected storage requirements that are calculated by considering the file uploads and the backup policies. For example, if three WSO2 product instances are running in a single machine, it requires a 4 GHz CPU, 8 GB RAM (2 GB for the operating system and 6 GB (2 GB for each WSO2 product instance)) and 30 GB of free space.
Virtual Machine (VM)	2 compute units minimum (each unit having 1.0-1.2 GHz Opteron/Xeon processor) 4 GB RAM 10 GB free disk space One CPU unit for the operating system and one for JVM. Three WSO2 product instances running would require VM of 4 compute units, 8 GB RAM, and 30 GBfree space. ~ 512 MB heap size. This is generally sufficient to process typical SOAP messages but the requirements vary with larger message sizes and the number of messages processed concurrently.
EC2	One c5.large instance to run one WSO2 product instance.
Cassandra data nodes	4 core processors 8 GB RAM For more information, see the Cassandra documentation on hardware recommendations for enterprise implementations .

Environment compatibility¶

If you are using the product installer to install the product, by default, WSO2 API-M is installed with OpenJDK, which allows you to run the product as soon as it is installed. If you are using the product's binary distribution to install the product (instead of the product installer), install JDK. Make sure your JDK version is compatible with the WSO2 product.

Tip

To use a different JDK, point the JAVA_HOME environment variable to the new JDK. Make sure your JDK version is compatible with the WSO2 product .

All WSO2 products are generally compatible with most common DBMSs. The embedded H2 database is suitable for development, testing, and some production environments. For most enterprise production environments, however, we recommend you use an industry-standard RDBMS such as Oracle, PostgreSQL, MySQL, MS SQL, etc. For more information, see Working with Databases in the Administration Guide. Also, we do not recommend the H2 database as a user store.
It is not recommended to use Apache DS in a production environment due to scalability issues. Instead, use an LDAP like OpenLDAP for user management.
On a production deployment, it is recommended that WSO2 products are installed on latest releases of RedHat Enterprise Linux or Ubuntu Server LTS.
For environments that WSO2 products are tested with, see Compatibility of WSO2 Products .
If you have difficulty in setting up any WSO2 product in a specific platform or database, contact us .

Installing the WSO2 Product¶

Given below is how to install a WSO2 product:

Download and Install the Product¶

If the installation prerequisites are satisfied, follow the steps below:

Go to the product page and download the product installer (click Installer pkg ).

Tip

Note that there are several options for installing the product in various environments. Use the available links for more information on each option.

Double-click to open the installation wizard, which will guide you through the installation. When you finish, the product will be installed and ready for use.

Access the HOME Directory¶

Let's call the installation location of your product as the <PRODUCT_HOME> directory. This is located in a place specific to your OS as shown below:

OS	Home directory
Mac OS	`/Library/WSO2/<PRODUCT_NAME>/<VERSION>`
Windows	`C:\Program Files\WSO2\<PRODUCT_NAME>/<VERSION> \`
Ubuntu	`/usr/lib/wso2/<PRODUCT_NAME>/<VERSION>`
CentOS	`/usr/lib64/<PRODUCT_NAME>/<VERSION>`

Uninstalling the Product¶

To remove an already installed product, follow the instructions below:

OS	Instructions
Mac OS	Open a terminal and run the following command as the root user: `sudo bash /Library/WSO2/<PRODUCT_NAME>/<VERSION>/uninstall.sh`
Windows	Go to the Start Menu -> Programs -> WSO2 -> Uninstall <PRODUCT_NAME_VERSION> or search Uninstall <PRODUCT_NAME_VERSION> and click the shortcut icon. This will uninstall the product from your computer.
Ubuntu	Open a terminal and run the following command: `sudo apt-get purge <PRODUCT_DISTRIBUTION_NAME>`
CentOS	Open a terminal and run the following command: `sudo yum remove <PRODUCT_DISTRIBUTION_NAME>-x86_64`

Setting System Properties¶

If you need to set additional system properties when the server starts, you can take the following approaches:

Set the properties from a script . Setting your system properties in the product's startup script (wso2server.sh for Linux and wso2server.bat for Windows), is ideal because it ensures that you set the properties every time you start the server. To avoid having to modify the script each time you upgrade, the best approach is to create your own startup script that wraps the WSO2 startup script and adds the properties you want to set, rather than editing the WSO2 startup script directly.

Note

Be sure to set the system properties org.opensaml.httpclient.https.disableHostnameVerification and httpclient.hostnameVerifier in the the product's startup script file to as follows. This setting will enable hostname verification of HTTP requests and responses in the Carbon server, and thereby avoid security issues in production environments. For more information, see Enabling HostName Verification.

    -Dorg.opensaml.httpclient.https.disableHostnameVerification=false \
    -Dhttpclient.hostnameVerifier=Strict \

Set the properties from an external registry

If you want to access properties from an external registry, you could create Java code that reads the properties at runtime from that registry. Be sure to store sensitive data such as username and password to connect to the registry in a property file instead of in the Java code and secure the properties file with the secure vault .

Note

When using SUSE Linux, it ignores /etc/resolv.conf and only looks at the /etc/hosts file. This means that the server will throw an exception on startup if you have not specified anything besides localhost. To avoid this error, add the following line above 127.0.0.1 localhost in the /etc/hosts file: <ip_address><machine_name> localhost

You are now ready to run the product.

Running the product¶

To run WSO2 products, you start the product server at the command line. You can then run the Management Console application to configure and manage the product.

Before you begin¶

When you move into a production environment, it is recommended to grant restricted access to the management console. See Securing Carbon Applications for instructions.
The config-validation.xml file in the <PRODUCT_HOME>/repository/conf/etc directory contains a list of recommended system parameters, which are validated against your system when the server starts. See Configuring config-validation.xml for details on modifying these parameters before starting the server.
The Management Console uses the default HTTP-NIO transport, of which the configurations are reflected in the catalina-server.xml file in the <PRODUCT_HOME>/repository/conf/tomcat directory. According to the new depolyment.toml based configuration model, to change the configurations in this file, please add the below configurations with appropriate values in <PRODUCT_HOME>/repository/conf/deployment.toml file.

Click to view the configurations

[transport.http.properties]
port = 9763
redirectPort = 9443
bindOnInit = false
maxHttpHeaderSize = 8192
acceptorThreadCount = 2
maxThreads = 250
minSpareThreads = 50
disableUploadTimeout = false
connectionUploadTimeout = 120000
maxKeepAliveRequests = 200
acceptCount = 200
server = "WSO2 Carbon Server"
compression = "on"
compressionMinSize = 2048
noCompressionUserAgents = "gozilla, traviata"
compressibleMimeType = "text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/pg,image/jpeg"
URIEncoding = "UTF-8"

[transport.https.properties]
port = 9443
bindOnInit = false
maxHttpHeaderSize = 8192
acceptorThreadCount = 2
maxThreads = 250
minSpareThreads = 50
disableUploadTimeout = false
enableLookups = false
connectionUploadTimeout = 120000
maxKeepAliveRequests = 200
acceptCount = 200
server = "WSO2 Carbon Server"
compression = "on"
scheme = "https"
secure = true
SSLEnabled = true
compressionMinSize = 2048
noCompressionUserAgents = "gozilla, traviata"
compressibleMimeType = "text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
URIEncoding = "UTF-8"

[transport.https.sslHostConfig.properties]
protocols = "+TLSv1,+TLSv1.1,+TLSv1.2"
sslProtocol = "TLS"
certificateVerification = false

These configurations has to be set properly for the management console to be accessible.

As explained in the installation prerequisites , the default product installation uses OpenJDK. Therefore, you don't require a separate JDK. However, if you have set up Oracle JDK or IBM JDK, be sure to apply the following settings to your product distribution.
- Oracle JDK
- IBM JDK
Some updates of JDK 1.8 (for example, JDK1.8.0_151 ) are affected by a known issue related to GZIP decoding. Until this issue is fixed, we recommend that you disable GZIP decoding for your product by following the steps given below. This will ensure that your product is not affected by the known issue .
1. Open the deployment.toml file from the <PRODUCT_HOME>/repository/conf/ directory.
2. Set the compression parameter (under each of the above described HTTP-NIO transport connector configurations: [transport.http.properties] and [transport.https.properties]) to false as shown below:
```
[transport.http.properties]
compression="off"
```
```
[transport.http.properties]
compression="off"
```
3. Restart the server.
If you are using IBM JDK 1.8, change the value of the org.owasp.csrfguard.PRNG.Provider property to ' IBMJCE ' in the Owasp.CsrfGuard.Carbon.properties file. This file is stored in the <PRODUCT_HOME>/repository/conf/security / directory.

Starting the Product Profiles¶

When a WSO2 product starts, it starts all components, features and related artifacts bundled with it. Multi-profile support allows you to run the product on a selected profile so that only the features specific to that profile along with common features start up with the server. For more on product-profiles please refer Product Profiles.

Stopping the Server¶

To stop the server, press Ctrl+C in the command window, or click the Shutdown/Restart link in the navigation pane in the Management Console.

Running Recommendations for Security¶

The following are security related recommendations to be followed when running the product.

Running as a different user : For security reasons, it's recommended to run the product as an unprivileged user. After adding a user to the system, apply your organizational security policies to that user.
Running on a different port : If you want to run on a different port, like port 80, the recommended way is to add a port forwarding rule from your firewall.
Running as a Unix daemon : You have the option of running each product as a standard Unix service. You can start, stop, and restart the WSO2 product instances as follows. # sh bin/wso2server.sh [start | stop | restart]

Tuning Parameters¶

The latency numbers (~50ms) are based on a two datacenter setup with a high-speed network connection. With the default configuration, you might notice intermittent behavior, so it is important to tune the system.
It is not recommended to use Metrics for system monitoring (JVM, CPU, etc.) in a production deployment. You can use external monitoring tools for this purpose.

Common Guidelines and Checklist¶

The following table lists out the common guidelines and details pertaining to them. These are common to all products and are followed for making an installed WSO2 product ready for production.

Guideline	Details
Security hardening	Guidelines for hardening the security of a WSO2 deployment in a production environment can be discussed under three high-level categories: Product-level security OS-level security Network-level security Related links See Security Guidelines for Production Deployment for the detailed list of security-related recommendations.
Hostname	By default, WSO2 products identify the hostname of the current machine through the Java API. However, this value sometimes yields erroneous results on some environments. Therefore, users are recommended to configure the hostname by setting the `hostname` parameter in the `<PRODUCT_HOME>/repository/conf/deployment.toml` file. `[server] hostname = "test.wso2.com"` Related links See the topics on changing hostnames shown below: Setting up hostnames and ports Changing the hostname See Working with Transports for information on transports in WSO2 products.
Registry and governance	All WSO2 products make use of an instance of a registry to store configurations. The registry uses a database as the persistent storage. By default, the registry uses an embedded H2 database. This embedded database might yield a lower performance and is less reliable compared to a standard database like MySQL when there are a large number of deployed artifacts. Hence, you should look at associated trade-offs, and we recommend that you switch to a database like Oracle, MySQL or MSSQL. Moreover, it is worth noting that the default setup does not include database backup procedures. The production setup should obviously need to have regular database backup procedures configured. When the registry database is pointed to a remote database, multiple running instances of the same product can boot up and run against the same configuration stored in the registry. This, in turn, helps with governance. Related links See here for more information on sharing a registry space across multiple WSO2 product instances.
User stores	WSO2 products offer three choices to store user details: Using a database Using an LDAP server Using an Active Directory service The default is to use the embedded H2 database, with the user store schema. Like in the case of the registry database, you can switch to a database like Oracle, MySQL or MSSQL. You can point to an existing LDAP or an Active Directory to make use of existing user bases and grant access privileges for WSO2 products based on those user stores. Related links See Configuring Primary User Store and Configuring Secondary User Stores for more information on user stores, how they work, and how to configure them.
Monitoring with JMX	WSO2 Products supportJMXformonitoring. By default, JMX uses port 9999. You can configure this to the desired port by adding the JMX port parameter in the `<PRODUCT_HOME>/repository/conf/deployment.toml` file as below `[monitoring.jmx] rmi_registry_port = 9999 rmi_server_port = 11111` Related links Please refer JMX-Based Monitoring for information on monitoring WSO2 products using JMX.
Tuning WSO2 products	Most of the performance tuning recommendations are common to all WSO2 products. However, each WSO2 product may have additional guidelines for optimizing the performance of product-specific features. Related links See Performance Tuning for the WSO2 API Manager related guidelines. For performance tuning guidelines that are specific to each product, go to the product documentation for each product listed below and search for performance tuning guidelines. Listed below are the main WSO2 products: API Manager Data Analytics Server Enterprise Integrator profiles ESB profile Business Process Management profile Message Broker profile Analytics profile Micro Integrator profile IOT Server Identity Server Enterprise Service Bus The following are now legacy products of WSO2: Application Server Business Rules Server Enterprise Mobility Manager Enterprise Store Governance Registry
Firewalls	The following ports must be accessed when operating within a firewall. Please note that these configurations are required to be configured in `<PRODUCT_HOME>/repository/conf/deployment.toml` file, eventhought they are reflected in the below mentioned config files, according to the new configuration model introduced with APIM 3.0 .0 onwards. 9443 - Used by the management console and services that use the servlet transport, and is reflected in the `<PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml` file. 9763 - Used by the services that use servlet transport, and is reflected in the `<PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml` file. 9999 - Used for JMX monitoring, reflected in the `<PRODUCT_HOME>/repository/conf/carbon.xml` file. 8280 - Default HTTP port used by ESB for proxy services, and is reflected in the `<PRODUCT_HOME>/repository/conf/axis2/axis2.xml` file. 8243 - Default HTTPS port used by ESB for proxy services, and is reflected in the `<PRODUCT_HOME>/repository/conf/axis2/axis2.xml` file. Related links See Default Product Ports for a list of common and product-specific ports used by WSO2 products.
Proxy servers	If the product is hosted behind a proxy such as ApacheHTTPD, users can configure products to use the proxy server by providing the following system properties at the start-up. `-Dhttp.proxyHost=xxxx -Dhttp.proxyPort=xxxx`
High availability	For high availability, WSO2 API Manager can be run on a cluster. Hazelcast clustering is disabled in WSO2 API Manager (WSO2 APIM) by default, because you can successfully deploy WSO2 API Manager without Hazelcast. Related links See Working with Hazelcast Clustering for more details on clustering, what it is, how it helps and other related information. See Separating the Worker and Manager Nodes for information on clustering WSO2 products by separating worker and manager concerns between the nodes.
Data backup and archiving	For data backup and for archiving of data, use the functionality provided by the RDBMS.

Backup and Recovery Recommendations¶

None of the WSO2 products persist data in the file systems or retain or generate artifacts. By default, we only store log files in the file system and data and artifacts in the databases and the repository.

What You Should Backup¶

Database backups :
- Back up of all the databases defined in <PRODUCT_HOME>/repository/conf/datasources/master-datasources.xml .
- Back up any other databases configured in any files in the <PRODUCT_HOME>/repository/conf/datasources directory.
Artifact backups : This includes hot-deployment artifacts, web applications, synapse files, tenant directories, etc. Back up of the <PRODUCT_HOME>/repository directory periodically. The frequency of the back ups depends on your usage. For example, if you are creating or updating APIs daily, take this backup daily.
WSO2 product instance backups : A one-time-only backup that you take of the entire server directory. This includes all the configuration files, logs, server extensions, and deployment artifacts for both tenants and super tenants. This back up is ideally done when the server is ready to be deployed in a production environment.

Backup recommendations¶

We recommend that you use a proper artifact management system such as Puppet to back up and manage your artifacts before deploying them in the WSO2 Carbon runtime. Also, use the WSO2 Update Manager (WUM) tool , which is a command-line utility that allows you to get the latest updates ( bug fixes and security fixes ) of a particular product release. Diagram : managing your artifacts using a configuration management system

Recovery Recommendations¶

Be sure to determine the following depending on your business-continuity requirements:

Recovery Point Objective (RPO) : Up to what points are you to recover. This is determined by the latest, known, good point.
Recovery Time Objective (RTO) : How long does it take to recover to the RPO.
Backup Frequency : How frequently you should take backups. If your RPO is one day, your backup frequency should be daily.
Disaster Recovery Site : The place where the latest copy of your backup is. This can be from a different shelf in your data center to a completely different geographical location.

We also recommend the following:

Align your artifact deployment and recovery processes.
Schedule disaster recovery drills to test the recoverability of the system.
Test your artifacts in an environment that is identical to the production environment before deploying them into production.

Recovery Strategy¶

The following steps include how to recover your setup using the backups:

Recover the hot-deployment artifacts by replacing the <PRODUCT_HOME>/repository directory with the backed up copy.
Recover the entire WSO2 product by directly replacing the existing WSO2 server directory in the production setup with the backup server directory. This will ensure that all the files, logs, and configurations made to the product do not need to be redone.
To recover the databases, follow the recovery strategy recommended by the databases you are using. For information on supported and tested databases, see Tested Database Management Systems .

Top