Administering WebLogic Clusters
This document describes how to configure a WebLogic Cluster, and how to start a WebLogic Server so that it joins a cluster.
There are a number of constraints placed on the networking environment and on the cluster configuration.
To avoid a single point of failure when using a shared file system, the administrator must ensure that the file system is stored on fault tolerant hardware. There are a variety of approaches to fault tolerance for disks:
Off-the-shelf file servers can provide fault tolerance out-of-the-box, with operating systems such as Solaris and Windows NT providing the software support. Consult with your hardware supplier for further information on installing and running such a system.
All WebLogic Servers in a cluster must be the same version.
Configure the Servers in your cluster depending on the particular mix of services that you are offering.
In practice, this often results in WebLogic Servers in the cluster being identically configured in all areas to do with WebLogic services, class files, and external resources (such as databases).
Configuration is the most complicated part of setting up a cluster. This document contains step-by-step instructions to help you set up your file system and the various properties files.
There are several configurations of clustered WebLogic Servers supported. All of these configurations use the same basic structure: a series of WebLogic Servers all accessible over the same LAN, and accessible to a set of clients - browser servlet clients and Java application clients - over one of WebLogic's various protocols (HTTP/S, TCP, etc.).
The supported configurations are:
A WebLogic Cluster serving servlets will provide failover only if session persistence is enabled (either JDBC or in-memory replication).
This scenario is a likely configuration for load balancing and failover of services across a set of WebLogic Servers. Each server handles client communications as well as the execution of requests. The three-tier cluster also provides the ability to "route" applet requests between the server with which the applet is communicating and the server hosting the service object (impl) that the applet wants to use.
For environments where HTTP tunneling is a requirement, clients can communicate either directly with one of the WebLogic Servers, or via the WebLogic Netscape (proxy) plug-in.
In general, before you start a WebLogic Server that can join a cluster, you need to carry out the following steps to set up the environment:
In this section, we cover each of these steps in greater detail.
Setting up the Network
The DNS name of the cluster and the IP addresses of the servers in the cluster are used by clients to connect to the cluster. The multicast address is used by the individual servers in the cluster to talk to each other. Note that these IP addresses must be fixed (that is, you cannot use dynamically assigned IP addresses in a cluster).
For information on DNS, there are many excellent online resources. A search on any of the popular Internet search engines will provide many useful links. One of the most highly regarded works on DNS is DNS and BIND, published by O'Reilly & Associates, Inc.
You should configure DNS so that a single host name directs incoming requests to a cluster of servers, rather than directing incoming requests to a single server. Cluster-wide load balancing and failover both depend upon DNS round-robining. You should configure:
For example (from the DNS BIND database):
guava.mydomain.com. IN A 18.104.22.168 # mycluster solaris
lemon.mydomain.com. IN A 22.214.171.124 # mycluster NT
quince.mydomain.com. IN A 126.96.36.199 # mycluster alpha
mycluster IN A 188.8.131.52
mycluster IN A 184.108.40.206
mycluster IN A 220.127.116.11
Make sure that there is no multicast conflict with another application. If the servers use the same IP multicast address and port as another application on the same LAN, the cluster will not work properly.
Whatever way you choose to resolve the multicast address, make sure that all of the servers in the cluster use the same multicast address. Each server will use the same port for the multicast address as it does for its normal IP address.
Setting up the Shared File System
Here we assume that you are using a shared file system, which greatly simplifies initial configuration. On the shared file system, set up these directories:
Note that you can name the directory whatever you want, but if the directory is not named weblogic/ you will not be able to take advantage of all the startup defaults. In this document, we refer to this directory exclusively as weblogic/.
Here is an example of this directory structure that shows the classes/, the standard myserver/ directory, and the lib/ directories that were installed with the distribution, as well as the mycluster/ directory that you have created:
Note that the name "mycluster" is a standard default. You can name the mycluster/ directory something else and change your configuration parameters accordingly, but here we use the default for simplicity. We suggest that you follow this naming convention during your initial configuration until you are familiar with how the configuration details relate to each other.
Note that as you start servers that join this cluster, WebLogic will look in the mycluster/ directory for a per-server directory for each server that joins the cluster. If the per-server directory does not exist, one will be created in the mycluster/ directory. The default naming scheme for the individual servers is myserver/<last three numbers of the IP address>. The per-server directory contains per-server administrative files like the HTTP access logs, the weblogic.log, and the JDBC log, and it may contain a per-server properties file.
Here is an example of one cluster configuration used for testing at WebLogic consisting of 5 Solaris machines, most of which are multihomed, and all of which have DNS names set up.
The computers used in the cluster include:
The common file system is resident on node sunglass, and nfs-mounted to the other machines in the cluster. WebLogic Server has been installed onto this shared file system.
Netscape Enterprise Server with the WebLogic plugin is installed on node Mellen as the Web Server.
BEA recommends that you use a similar setup for your cluster, although there is no reason to believe that other configurations could not work equally well.
You have completed the first step of cluster setup. The next step is configuration, which is done with a set of weblogic.properties files.
Setting up the weblogic.properties Files
With clustering, another level of control is added to the properties hierarchy. There are now three levels of properties files: global, cluster, and per-server. The global properties file is read first; its properties apply to all WebLogic Servers. The cluster properties is read next, and its properties apply to servers in a particular cluster. The per-server properties file is read last; its properties apply only to the server that is starting.
What happens when you start a server that will join a cluster? If no other server in the cluster is running, the server you start will become the first member of the cluster.
The server will go through the following steps:
If the directory is found, the Server will look for a per-server properties file and will read that. Any properties set in the per-server properties file will override the same properties that were set from the cluster or global properties files.
An identical property set in a properties file will override the setting for the same property in any properties file read earlier in the startup sequence. For example, properties in each per-server properties file will override identical properties already set in the global or cluster properties files. Finally, any property set on the command line will override all other settings for that property.
The properties files are nested:
Creating a per-server directory for each server in the cluster is not required, unless you want the per-server directory to be read upon startup. Each server's logfiles will also be created in its per-server directory as weblogic.log for general WebLogic logging and (default name unless configured in the per-server properties file) access.log for HTTP request logs. The default file names may be changed using the weblogic.system.name property.
To set up the properties file hierarchy:
Here is a typical configuration, with the locations shown of the various properties files:
Default and global properties
All paths that you configure in the cluster properties file, such as servlet classpath, document root, etc., should point to the correct directory on the shared file system.
Note that if a per-server properties file exists in any of the server directories beneath the cluster directory (mycluster/), it should not contain any properties that exist in the global or cluster properties files, except those properties that you want to override settings established in the global or cluster properties files. Make sure you delete such properties from the per-server properties file, since the presence of a property in the per-server property overrides the identical property in global and cluster properties files.
For instructions on setting your CLASSPATH, see the Administrators Guide, Setting classpath.
Setting up HTTP servlets
WebLogic Server supports clustering servlets across a WebLogic Cluster. This section describes the issues involved and how to set up a servlet cluster.
Initial HTTP servlet requests are passed to an arbitrary server node in a cluster. When a servlet request is part of an ongoing session, subsequent requests must have access to the HTTP session data. You set up your WebLogic Server Cluster to accomplish this using either in-memory state replication or JDBC session persistence. .
Using In-Memory State Replication with a Proxy Server
This feature provides better performance, adequate failover, and does not use a persistent store. It (unlike JDBC session persistence) does require a WebLogic Server Cluster license.
When using in-memory state replication, your WebLogic Server Cluster must live behind one or more proxy servers. The proxy servers are smart enough to send servlet requests, belonging to the same HTTP session, back to the same server in the cluster that holds the session data. This eliminates the need to write the session data to a shared database. You may use the Netscape Server, MS-IIS or WebLogic Server as a proxy server.
Failover is provided by replicating each session on a secondary node in the cluster. If the primary node fails, the secondary node takes over, becoming the new primary node for the session data. Another node is selected as the secondary node, and a backup of the session is made. If Weblogic Server is used as the proxy server, failover will occur up until the results are sent to the browser. However, if Netscape Server or MS-IIS is used as the proxy server, failover will only occur up until a successful request is accepted by a clustered server.
It is important to keep failover in mind when writing servlets that will be executed in a WebLobic Cluster. If any exception occurs during the proxying of a request, a WebLogic Server proxy will attempt to re-issue the request to a different server in the cluster. It is possible that some of the request may have processed before the exception, and may be processed again. For this reason, it is a good idea to write idempotent servlets (servlets that end with the same result whether they are executed one time or N times in a session).
If a servlet session request for a cluster doing in-memory replication goes to the wrong server (i.e. not the primary or secondary for that session), the WebLogic Servers do not recognize that they have received a misdirected request for in-memory replication. The Cluster relies on the proxy server to direct the request to the correct primary or secondary. If the node in the WebLogic Cluster receives a misdirected request, it will return an error. Any data that must absolutely survive failures should be kept in a database (either using database persistance for the session or an entity bean).
Setting up the Proxy Server
For details on setting up a Netscape Server as a proxy server, see the Administrators Guide Installing the Netscape-WebLogic Server Plugin.
For details on setting up a Microsoft Internet Information Server as a proxy server, see the Administrators Guide Installing the MS-IIS-WebLogic Server Plugin.
This section shows you how to set up a WebLogic Server as a proxy server for clusters.
Requests intended for the WebLogic Cluster (for example JSP pages and servlets) must be handled by the HttpClusterServlet on the proxy server. The HttpClusterServlet knows how to proxy the request to the appropriate server in the WebLogic Cluster. Set up your DNS to point to the proxy server, not directly to a server in the WebLogic Cluster.
There are two ways to configure the proxy server: you can set the default servlet as the file servlet, or the cluster servlet.
In the first option, the proxy server acts as the file server for all static content, and only registered dynamic content is handled by the cluster. This configuration is clear and direct, because only JSP files and registered servlets are passed to the cluster servlet, while html pages, images, and other static content are delivered by the proxy server's file servlet.
The following properties in the proxyserver.weblogic.properties file set up the FileServlet as the default servlet.
Requests that are intended for the WebLogic Cluster can be registered with the cluster servlet as shown below:
The initialization parameter defaultServers lists the addresses of servers in the WebLogic Cluster.
In the second option, the proxy server's default is the cluster servlet and static requests can be handled by the file servlet. The main reason for choosing this configuration over the first option would be a large number of dynamic requests with different file extensions. Setting the cluster servlet as the default servlet in the proxy eliminates the need to individually register servlets, which may have arbitrary file extensions. Instead, static content should be registered.
The following properties set HttpClusterServlet as the default servlet.
Requests that are intended for the WebLogic FileServlet can be registered as shown below for the .html, .jpg, and .gif MIME types. Other MIME file extensions can be added in the same manner.
We recommend that you remove all other servlet registrations from the proxy server's weblogic.properties file, except the chosen default servlet (FileServlet in option 1, and ClusterServlet in option 2). That is, remove all other properties that begin with
This will avoid confusion where the proxy server may attempt to directly respond to a request if it matches a registration, or where the proxy servlet may fail due to an error in the cluster.
Setting up the WebLogic Cluster
In addition to setting up the proxy server, you need to set up a WebLogic Cluster as described below, including the weblogic.properties files. In addition to setting the normal properties for a clustered server and an HTTP servlet, you must set the following properties in the cluster-wide <my_cluster>weblogic.properties file to enable clustered HTTP sessions using in-memory replication:
Using JDBC Session Persistence
Note: JDBC session persistence (unlike in-memory state replication) does not require a WebLogic Cluster license.
Each cluster node accesses servlet session data via a shared JDBC persistent store. For details on setting up persistent storage for servlet sessions, see the Developers Guide Using WebLogic HTTP Servlets: Making sessions persistent. WebLogic supports servlet clustering with EJB entity bean-managed persistence so the frequency of physical writes to disk can be carefully controlled.
Each time you add or change data in the HttpSession the change is written immediately to the persistent store database, allowing any other servers access to the updated data. This method provides good failover, but each change to the session involves a write to the database, which can be a substantial performance reduction compared to using no persistence. We recommend instead that you use in-memory state replication.
Note that persistence is only for failover; previous session data is available only if you use a database or an entity EJBean to make it persistent beyond the session.
Follow these steps to set up servlet clustering using JDBC session persistence:
Setting Up Enterprise JavaBeans
In addition to the information on Enterprise JavaBeans provided in Deploying EJBs Using DeployerTool,and in the General Requirements section of this chapter, to use the EJB hot deployment feature in a cluster you must:
This keeps the version of the bean present in the cluster the same at all times. There is a window where bean operations can fail and beans are unavailable, causing transactions to roll back or bean homes not to be found. You should write your clients to retry the EJBean in the event it can not be found to cover this case, as described in our Tech Support Guide Troubleshooting your WebLogic EJB application. For more information on hot deploy, see Using WebLogic Server hot deploy.
Starting the WebLogic Server and Joining a Cluster
You should have completed the items listed in General Requirements before you start a server that will join a cluster.
Starting with Defaults
For general information on starting a WebLogic Server, see the Administrators Guide Setting up and starting the WebLogic Server. We assume that you have read this and have started a WebLogic Server in a single server mode and are familiar with the use of the supplied scripts.
In the examples below on starting the Server, we have left out much of the command, and only shown the commands that are relevant to the cluster.
You can start a cluster using defaults and a couple of command-line options. The defaults assume that the host is not multihomed. The simplest way to start a WebLogic server that will join a cluster is to set the property weblogic.cluster.enable to true on the command line, as shown here:
$ java -Dweblogic.cluster.enable=true
Using this property alone on the command line will invoke a server with the following default settings:
Cluster name defaults to "mycluster" (and matches the name of the directory that contains your properties file and security files) IP address for this WebLogic Server defaults to its assigned IP address Listen port defaults to "7001" Server name defaults to the pattern "server" plus the last three digits of the IP address, as in "server154" Multicast address defaults to 18.104.22.168
Details on Starting WebLogic in a Clustered Environment
This section provides more details on the properties you can set for starting the Server in a Cluster, on the command line, in the weblogic.properties file, the cluster-wide properties file and the per-server properties file.
Other standard startup options are detailed in the Administrators Guide document, Setting up and starting the WebLogic Server, which details options for single-server configuration.
Command Line Properties
When you start each server that will join the cluster, you must set certain properties on the command line - with the -D option - unless you want to use the defaults. Here is a complete list of command line properties that affect server configuration, their defaults, and dependencies. Cluster-specific properties that can be specified in the cluster-wide properties and the per-server propertiesfile are included.
weblogic.cluster.name=clustername, where clustername is the name of the cluster this WebLogic Server should join on startup, and matches the name of the "mycluster/" directory. If you do not supply this command-line property, the cluster will start with the "mycluster" name, and will look in the weblogic/mycluster/ directory for its cluster property file, security-related files, and its document root. The cluster name property must be the same for every server in the cluster and must be set only on the command line.
$ java -Dweblogic.cluster.name=megacluster
weblogic.system.name, to specify the server name. If you do not use the standard pattern for establishing the server name ("server" + last 3 digits of the IP address), you must set the server name on the command line. Of course, the server name must be unique for each server in the cluster.
$ java -Dweblogic.system.name=bizserver
weblogic.cluster.enable should be set to true. This property must always be supplied on the command line.
weblogic.system.listenPort. This property must be the same for every server in a cluster. If you do not specify a listen port, the default used is "7001". If you specify the listen port for one server, you must specify the same listen port for every other server in the cluster. (Note that you cannot set up a cluster by running multiple servers on the same IP address listening at different ports; all servers must be listening on different IP addresses at the same port.) The listen port can be set on the command line or in the cluster-wide properties file.
weblogic.system.bindAddr, if the machine is multihomed and you want to specify the IP address for this server process. If you do not specify an IP address when you start WebLogic on a multihomed machine, WebLogic will choose one of its host's IP addresses; the algorithm for determining which IP address is chosen is operating-system dependent. You should set the bindAddr property if the machine is multihomed to avoid unpredictable results. You can either set this property on the command line or - if you have specified the server name, as described above - in the appropriate per-server properties file.
$ java -Dweblogic.system.bindAddr=22.214.171.124 (or DNS name)
Here is a sample of properties to be set for the WebLogic Server "server154" to join "mycluster". The multicast address for this cluster is 126.96.36.199, and the machine is multihomed, with IP addresses 188.8.131.52 and 184.108.40.206, and the WebLogic Server would be running on the 220.127.116.11 address.
In addition to the above properties, you can set these properties in the cluster-wide properties file:
weblogic.cluster.multicastAddress, to specify the multicast address that the servers in a cluster will use to communicate with each other. If you do not specify an address, WebLogic will attempt a DNS lookup of the name of the cluster plus "Multicast", such as "clusternameMulticast". If no DNS entry is found, WebLogic will use the default multicast address "18.104.22.168". The multicast address must be the same for every server that joins the cluster. Consequently, it is best if you set it only in the cluster-wide properties file.
weblogic.cluster.multicastTTL, the time-to-live (number of hops) for the cluster's multicast messages. The default is 1, with a range of 1-255.
weblogic.cluster.defaultLoadAlgorithm, the default load balancing algorithm to be used between replicated services if none was specified for that particular service. The default is "round-robin", with acceptable values of "random", "round-robin", and "weight-based". Load balancing is explained in Writing and Deploying Clustered Objects.
There are also cluster-related properties that apply only to a specific server; you set these properties in the per-server properties file:
weblogic.system.weight, the weight of the individual server for the weight-based load-balancing. The default is 100, with a range of 0-100. Larger numbers increase the amount of traffic routed to the server.
Stopping a Server in a Cluster
Use the Admin SHUTDOWN command to halt a WebLogic Server in a cluster, just as you would a WebLogic Server running in stand-alone mode. The other members of the Cluster will determine that the halted server is no longer available and the cluster-wide JNDI tree will be updated accordingly.
Special Considerations for Clients of a Cluster
Performance of your JAVA client can be degraded when the number of socket readers is lower than the number of WebLogic Servers in your cluster. You can improve the performance by setting the following properties from the command line while starting your client:
$ java -Dweblogic.system.executeThreadCount=<number of threads>
$ java -Dweblogic.system.percentSocketReaders=<percentage of threads to be used as socket readers>
Setting the number of threads and the percentage of socket readers to ensure an equal number of socket readers and instances of WebLogic Server will help to maintain high performance levels.
Copyright © 2000 BEA Systems, Inc. All rights reserved.