Production Deployment Guidelines¶
The requirements for deploying WSO2 products can change based on the deployment scenario and pattern. The recommendations in this topic are for general production use, assuming moderate load conditions. For situations where a high volume of traffic is expected and if there are large deployments, these guidelines may not be sufficient. See Troubleshooting in Production Environments for information on how to obtain and analyze information to solve production issues. The following are the topics addressed in this section.
[ Installation prerequisites ] [ System requirements ] [ Installing the WSO2 product ] [ Download and install the product ] [ Access the HOME directory ] [ Uninstalling the product ] [ Running the product ] [ Tuning parameters ] [ Hazelcast properties ] [ Common guidelines and checklist ] [ Backup and recovery recommendations ]
Prior to installing any WSO2 Carbon-based product, it is necessary to have the appropriate hardware and software for running the product.
Disk space is based on the expected storage requirements that are calculated by considering the file uploads and the backup policies. For example, if three WSO2 product instances are running in a single machine, it requires a 4 GHz CPU, 8 GB RAM (2 GB for the operating system and 6 GB (2 GB for each WSO2 product instance)) and 30 GB of free space.
|Virtual Machine (VM)||
Three WSO2 product instances running would require VM of 4 compute units, 8 GB RAM, and 30 GBfree space.
|Cassandra data nodes||
For more information, see the Cassandra documentation on hardware recommendations for enterprise implementations .
By default, WSO2 products are installed with OpenJDK , which allows you to run the product as soon as it is installed.
To use a different JDK, point the
JAVA_HOMEenvironment variable to the new JDK. Make sure your JDK version is compatible with the WSO2 product .
All WSO2 products are generally compatible with most common DBMSs. The embedded H2 database is suitable for development, testing, and some production environments. For most enterprise production environments, however, we recommend you use an industry-standard RDBMS such as Oracle, PostgreSQL, MySQL, MS SQL, etc. For more information, see Working with Databases in the Administration Guide. Also, we do not recommend the H2 database as a user store.
- It is not recommended to use Apache DS in a production environment due to scalability issues. Instead, use an LDAP like OpenLDAP for user management.
- On a production deployment, it is recommended that WSO2 products are installed on latest releases of RedHat Enterprise Linux or Ubuntu Server LTS.
- For environments that WSO2 products are tested with, see Compatibility of WSO2 Products .
- If you have difficulty in setting up any WSO2 product in a specific platform or database, contact us .
Installing the WSO2 product¶
Given below is how to install a WSO2 product:
Download and install the product¶
If the installation prerequisites are satisfied, follow the steps below:
Go to the product page and download the product installer (click Installer pkg ).
Note that there are several options for installing the product in various environments. Use the available links for more information on each option.
Double-click to open the installation wizard, which will guide you through the installation. When you finish, the product will be installed and ready for use.
Access the HOME directory¶
Let's call the installation location of your product as the <PRODUCT_HOME> directory. This is located in a place specific to your OS as shown below:
Uninstalling the product¶
To remove an already installed product, follow the instructions below:
|Mac OS||Open a terminal and run the following command as the root user:|
sudo bash /Library/WSO2/<PRODUCT_NAME>/<VERSION>/uninstall.sh|
| Windows | Go to the Start Menu -> Programs -> WSO2 -> Uninstall <PRODUCT_NAME_VERSION> or search Uninstall <PRODUCT_NAME_VERSION> and click the shortcut icon. This will uninstall the product from your computer. |
| Ubuntu | Open a terminal and run the following command:
sudo apt-get purge <PRODUCT_DISTRIBUTION_NAME>|
| CentOS | Open a terminal and run the following command:
sudo yum remove <PRODUCT_DISTRIBUTION_NAME>-x86_64|
Setting system properties¶
If you need to set additional system properties when the server starts, you can take the following approaches:
Set the properties from a script . Setting your system properties in the startup script (i.e. the
<PRODUCT_HOME>/bin/wso2server.sh file), is ideal because it ensures that you set the properties every time you start the server. To avoid having to modify the script each time you upgrade, the best approach is to create your own startup script that wraps the WSO2 startup script and adds the properties you want to set, rather than editing the WSO2 startup script directly.
Be sure to set the
org.wso2.ignoreHostnameVerificationsystem property in the
<PRODUCT_HOME>/bin/wso2server.sh file to
This setting will enable hostname verification of HTTP requests and responses in the Carbon server, and thereby avoid security issues in production environments.
Set the properties from an external registry . If you want to access properties from an external registry, you could create Java code that reads the properties at runtime from that registry. Be sure to store sensitive data such as username and password to connect to the registry in a property file instead of in the Java code and secure the properties file with the secure vault .
Note : When using SUSE Linux, it ignores
/etc/resolv.conf and only looks at the
/etc/hosts file. This means that the server will throw an exception on startup if you have not specified anything besides localhost. To avoid this error, add the following line above
127.0.0.1 localhost in the
You are now ready to run the product.
Running the product¶
To run WSO2 products, you start the product server at the command line. You can then run the Management Console application to configure and manage the product.
Before you begin¶
- When you move into a production environment, it is recommended to grant restricted access to the management console. See Securing Carbon Applications for instructions.
config-validation.xmlfile in the
<PRODUCT_HOME>/repository/conf/etcdirectory contains a list of recommended system parameters, which are validated against your system when the server starts. See Configuring config-validation.xml for details on modifying these parameters before starting the server.
- The Management Console uses the default HTTP-NIO transport , which is configured in the
catalina-server.xmlfile in the
<PRODUCT_HOME>/repository/conf/tomcatdirectory. This transport must be properly configured in this file for the management console to be accessible.
As explained in the installation prerequisites , the default product installation uses OpenJDK. Therefore, you don't require a different JDK. However, if you have set up Oracle JDK or IBM JDK, be sure to apply the following settings to your product distribution.
Some updates of JDK 1.8 (for example, JDK1.8.0_151 ) are affected by a known issue related to GZIP decoding. Until this issue is fixed, we recommend that you disable GZIP decoding for your product by following the steps given below. This will ensure that your product is not affected by the known issue .
- Open the
catalina-server.xmlfile from the
compressionparameter (under each of the connector configurations) to false as shown below:
Restart the server.
If you are using IBM JDK 1.8, change the value of the
org.owasp.csrfguard.PRNG.Providerproperty to '
IBMJCE' in the
Owasp.CsrfGuard.Carbon.propertiesfile. This file is stored in the
- Open the
Starting the product profiles¶
Open a command prompt and execute the name of the product distribution (For example,
Stopping the server¶
To stop the server, press Ctrl+C in the command window, or click the Shutdown/Restart link in the navigation pane in the Management Console.
Running recommendations for security¶
The following are security related recommendations to be followed when running the product.
- Running as a different user : For security reasons, it's recommended to run the product as an unprivileged user. After adding a user to the system, apply your organizational security policies to that user.
- Running on a different port : If you want to run on a different port, like port 80, the recommended way is to add a port forwarding rule from your firewall.
- Running as a Unix daemon : You have the option of running each product as a standard Unix service. You can start, stop, and restart the WSO2 product instances as follows.
# sh bin/wso2server.sh [start | stop | restart]
- The latency numbers (~50ms) are based on a two datacenter setup with a high-speed network connection. With the default configuration, you might notice intermittent behavior, so it is important to tune the system.
- It is not recommended to use Metrics for system monitoring (JVM, CPU, etc.) in a production deployment. You can use external monitoring tools for this purpose.
WSO2 products use Hazelcast as its default clustering engine. The following configuration must be placed in the
<PRODUCT_HOME>/repository/conf/hazelcast.properties file. Create this file if it does not exist.
The above configurations are explained below.
- Hazelcast shutdown hook: This configuration disables the shutdown hook in hazelcast, which ensures that the hazelcast instance shuts down gracefully whenever the product node shuts down. If the hazelcast shutdown hook is enabled (which is the default behavior of a product), you will see errors such as " Hazelcast instance is not active! " at the time of shutting down the product node: This is because the hazelcast instance shuts down too early when the shutdown hook is enabled.
- Hazelcast logging type: This configuration sets the hazelcast logging type to log4j, which allows hazelcast logs to be written to the
Once you enable log4j for hazelcast as explained above, add
log4j.logger.com.hazelcast=INFO to the
<PRODUCT_HOME>/repository/conf/log4j.properties file. For more information on logging, see Monitoring Logs .
Additionally, Hazelcast indicates that if all members are not mentioned in the well-known member list, there can be a split-brain (network partition) situation. If the cluster spans across data centers, it is important to add all the members to the well-known members list in the
Common guidelines and checklist¶
The following table lists out the common guidelines and details pertaining to them. These are common to all products and are followed for making an installed WSO2 product ready for production.
Guidelines for hardening the security of a WSO2 deployment in a production environment can be discussed under three high-level categories:
By default, WSO2 products identify the hostname of the current machine through the Java API. However, this value sometimes yields erroneous results on some environments. Therefore, users are recommended to configure the hostname by setting the
To configure hostnames for WSDLs and endpoints, users are recommended to add the following parameter in the <transportReceiver> section in the
|Registry and governance||
All WSO2 products make use of an instance of a registry to store configurations. The registry uses a database as the persistent storage. By default, the registry uses an embedded H2 database.
This embedded database might yield a lower performance and is less reliable compared to a standard database like MySQL when there are a large number of deployed artifacts. Hence, you should look at associated trade-offs, and we recommend that you switch to a database like Oracle, MySQL or MSSQL.
Moreover, it is worth noting that the default setup does not include database backup procedures. The production setup should obviously need to have regular database backup procedures configured.
When the registry database is pointed to a remote database, multiple running instances of the same product can boot up and run against the same configuration stored in the registry. This, in turn, helps with governance.
WSO2 products offer three choices to store user details:
The default is to use the embedded H2 database, with the user store schema. Like in the case of the registry database, you can switch to a database like Oracle, MySQL or MSSQL. You can point to an existing LDAP or an Active Directory to make use of existing user bases and grant access privileges for WSO2 products based on those user stores.
|Monitoring with JMX||
WSO2 Products supportJMXformonitoring. By default, JMX uses port 9999. You can configure this to the desired port by setting the JMX port parameter in the
|Tuning WSO2 products||
Most of the performance tuning recommendations are common to all WSO2 products. However, each WSO2 product may have additional guidelines for optimizing the performance of product-specific features.
The following ports must be accessed when operating within a firewall.
If the product is hosted behind a proxy such as ApacheHTTPD, users can configure products to use the proxy server by providing the following system properties at the start-up.
Alternatively, this can be done by adding the following configurations in the
For high availability, WSO2 products must run on a cluster . This enables the WSO2 products to still work in the case of failover. Databases used for the repository, user management, and business activity monitoring can also be configured in a cluster or can use replication management provided by the RDBMS.
|Data backup and archiving||For data backup and for archiving of data, use the functionality provided by the RDBMS.|
Backup and recovery recommendations¶
None of the WSO2 products persist data in the file systems or retain or generate artifacts. By default, we only store log files in the file system and data and artifacts in the databases and the repository.
What you should back up¶
- Database backups :
- Back up of all the databases defined in
- Back up any other databases configured in any files in the
- Back up of all the databases defined in
- Artifact backups :
T his includes hot-deployment artifacts, web applications, synapse files, tenant directories, etc. Back up of the
<PRODUCT_HOME>/repositorydirectory periodically. The frequency of the back ups depends on your usage. For example, if you are creating or updating APIs daily, take this backup daily.
- WSO2 product instance backups : A one-time-only backup that you take of the entire server directory. This includes all the configuration files, logs, server extensions, and deployment artifacts for both tenants and super tenants. This back up is ideally done when the server is ready to be deployed in a production environment.
We recommend that you use a proper artifact management system such as Puppet to back up and manage your artifacts before deploying them in the WSO2 Carbon runtime. Also, use the WSO2 Update Manager (WUM) tool , which is a command-line utility that allows you to get the latest updates ( bug fixes and security fixes ) of a particular product release.
Diagram : managing your artifacts using a configuration management system
Be sure to determine the following depending on your business-continuity requirements:
- Recovery Point Objective (RPO) : Up to what points are you to recover. This is determined by the latest, known, good point.
- Recovery Time Objective (RTO) : How long does it take to recover to the RPO.
- Backup Frequency : How frequently you should take backups. If your RPO is one day, your backup frequency should be daily.
- Disaster Recovery Site : The place where the latest copy of your backup is. This can be from a different shelf in your data center to a completely different geographical location.
We also recommend the following:
- Align your artifact deployment and recovery processes.
- Schedule disaster recovery drills to test the recoverability of the system.
- Test your artifacts in an environment that is identical to the production environment before deploying them into production.
The following steps include how to recover your setup using the backups:
- Recover the hot-deployment artifacts by replacing the
<PRODUCT_HOME>/repositorydirectory with the backed up copy.
- Recover the entire WSO2 product by directly replacing the existing WSO2 server directory in the production setup with the backup server directory. This will ensure that all the files, logs, and configurations made to the product do not need to be redone.
- To recover the databases, follow the recovery strategy recommended by the databases you are using. For information on supported and tested databases, see Tested Database Management Systems .