BEA Logo BEA WebLogic Server Release 1.1

  Corporate Info  |  News  |  Solutions  |  Products  |  Partners  |  Services  |  Events  |  Download  |  How To Buy

   Using WebLogic Server Clusters:   Previous topic   |   Next topic   |   Contents   |  Index

 

Administering WebLogic Clusters

 

This document describes how to configure a WebLogic Cluster, and how to start a WebLogic Server so that it joins a cluster.

General Requirements

Typical Configurations

Configuration
Setting up the Network
Setting up the Shared File System
Setting up the weblogic.properties Files
Setting CLASSPATH
Setting up HTTP Servlets
Using In-Memory State Replication with a Proxy Server
Setting up the Proxy Server
Setting up the WebLogic Cluster
Using JDBC Session Persistence
Setting up Enterprise JavaBeans

Starting the WebLogic Server and Joining a Cluster
Starting with Defaults
Details on Starting WebLogic in a Clustered Environment
Command Line Properties
Advanced Properties
Per-Server Properties

Stopping a Server in a Cluster

Special Considerations for Clients of Clusters

General Requirements

There are a number of constraints placed on the networking environment and on the cluster configuration.

  1. To run a WebLogic Cluster, you need a special license. The standard download of WebLogic Server does not include the clustering feature. Contact your sales representative.

  2. The machine(s) you will be using as WebLogic hosts for the cluster must have permanently assigned, static IP addresses. You cannot use dynamically-assigned IP addresses in a clustering environment. If the servers are behind a firewall and the clients are in front of the firewall, each server must have a "public" static IP address to be reached by the clients.

  3. The WebLogic Servers in a cluster must all be located on the same local area network (LAN) and must be reachable via IP multicast.

  4. For ease of configuration and initial setup, we highly recommend that you use a shared file system that all the servers in the cluster can access. The shared file system will store the configuration files and the shared objects that the cluster serves. All the instructions in this document are predicated on the assumption that you are using a shared file system.

    To avoid a single point of failure when using a shared file system, the administrator must ensure that the file system is stored on fault tolerant hardware. There are a variety of approaches to fault tolerance for disks:

Typical Configurations

There are several configurations of clustered WebLogic Servers supported. All of these configurations use the same basic structure: a series of WebLogic Servers all accessible over the same LAN, and accessible to a set of clients - browser servlet clients and Java application clients - over one of WebLogic's various protocols (HTTP/S, TCP, etc.).

The supported configurations are:

  1. Servlet Cluster: a three-tier cluster that serves servlets. Servlet clustering provides load balancing or load balancing plus failover for servlet sessions. Servlet session clustering configuration and setup is documented in Setting up WebLogic as an HTTP server and Using WebLogic HTTP servlets.

    A WebLogic Cluster serving servlets will provide failover only if session persistence is enabled (either JDBC or in-memory replication).

  2. EJB Cluster: Three-tier cluster that serves objects. This type of cluster is the most typical scenario for "thicker," object-based client applications, with Java/VB/C++ code rather than HTML in the client.

    This scenario is a likely configuration for load balancing and failover of services across a set of WebLogic Servers. Each server handles client communications as well as the execution of requests. The three-tier cluster also provides the ability to "route" applet requests between the server with which the applet is communicating and the server hosting the service object (impl) that the applet wants to use.

    For environments where HTTP tunneling is a requirement, clients can communicate either directly with one of the WebLogic Servers, or via the WebLogic Netscape (proxy) plug-in.

  3. Servlet and EJB Cluster: a multi-tier cluster of mixed objects and servlets, such as a mixture of HTML browser servlets and EJBeans. One of WebLogic's strengths is its integration of standard APIs to support all of the Enterprise services, including RMI, EJB, JDBC, and HTTP servlets, in a single application server framework.

  4. Four-tier Web Application Cluster: Details in this document are chiefly for configurations 2 and 3, for clustering EJB and RMI objects and clustering objects in a mixed environment. More information about servlet session clustering is available in a companion Administrators Guide, Setting up WebLogic as an HTTP server.

Configuration

In general, before you start a WebLogic Server that can join a cluster, you need to carry out the following steps to set up the environment:

  1. Acquire a set of addresses and configure DNS. You need an IP address for each WebLogic Server and a multicast address for the cluster. Note that you cannot set up a cluster on a single machine with one IP address by running multiple servers listening on different ports; all servers must be listening on different IP addresses at the same port.

  2. Set up a shared file system for the cluster

  3. Locate classes and register classes in the properties files

  4. Specify deployent and startup class properties in the properties files

  5. Set the CLASSPATH for each WebLogic host

In this section, we cover each of these steps in greater detail.

Setting up the Network

  1. Acquire the following addresses:

Setting up the Shared File System

Here we assume that you are using a shared file system, which greatly simplifies initial configuration. On the shared file system, set up these directories:

  1. Create weblogic/. Create a common WebLogic home directory on a shared network file system that is reachable by all members of the cluster and name it weblogic/.

    Note that you can name the directory whatever you want, but if the directory is not named weblogic/ you will not be able to take advantage of all the startup defaults. In this document, we refer to this directory exclusively as weblogic/.

  2. Install the WebLogic distribution in the weblogic/. Your file system will now appear something like this:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/myserver/

  3. Create a directory named mycluster/ in the weblogic/ directory. If you have used WebLogic before, you can think of the mycluster/ directory as analogous to the myserver/ directory, except it supports a cluster of WebLogic Servers rather than a single server.

    Here is an example of this directory structure that shows the classes/, the standard myserver/ directory, and the lib/ directories that were installed with the distribution, as well as the mycluster/ directory that you have created:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/mycluster/
    /weblogic/myserver/

    Note that the name "mycluster" is a standard default. You can name the mycluster/ directory something else and change your configuration parameters accordingly, but here we use the default for simplicity. We suggest that you follow this naming convention during your initial configuration until you are familiar with how the configuration details relate to each other.

  4. Copy the public_html/ directory and all of its contents from the weblogic/myserver/ directory into the weblogic/mycluster directory. Your file system should now appear as:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/mycluster/
    /weblogic/mycluster/public_html/
    /weblogic/myserver/

  5. Copy any administrative files (like security certificates) that all cluster members will share into the mycluster/ directory. Here is an example:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/mycluster/
    /weblogic/mycluster/ca.der
    /weblogic/mycluster/ca.pem
    /weblogic/mycluster/democert.pem
    /weblogic/mycluster/demokey.pem
    /weblogic/mycluster/public_html/
    /weblogic/myserver/

Note that as you start servers that join this cluster, WebLogic will look in the mycluster/ directory for a per-server directory for each server that joins the cluster. If the per-server directory does not exist, one will be created in the mycluster/ directory. The default naming scheme for the individual servers is myserver/<last three numbers of the IP address>. The per-server directory contains per-server administrative files like the HTTP access logs, the weblogic.log, and the JDBC log, and it may contain a per-server properties file.

Here is an example of one cluster configuration used for testing at WebLogic consisting of 5 Solaris machines, most of which are multihomed, and all of which have DNS names set up.

The computers used in the cluster include:

The common file system is resident on node sunglass, and nfs-mounted to the other machines in the cluster. WebLogic Server has been installed onto this shared file system.

Netscape Enterprise Server with the WebLogic plugin is installed on node Mellen as the Web Server.

BEA recommends that you use a similar setup for your cluster, although there is no reason to believe that other configurations could not work equally well.

You have completed the first step of cluster setup. The next step is configuration, which is done with a set of weblogic.properties files.

Setting up the weblogic.properties Files

With clustering, another level of control is added to the properties hierarchy. There are now three levels of properties files: global, cluster, and per-server. The global properties file is read first; its properties apply to all WebLogic Servers. The cluster properties is read next, and its properties apply to servers in a particular cluster. The per-server properties file is read last; its properties apply only to the server that is starting.

What happens when you start a server that will join a cluster? If no other server in the cluster is running, the server you start will become the first member of the cluster.

The server will go through the following steps:

  1. Open and read the global properties file, located in the WebLogic home directory (installed by default as weblogic/).

  2. Open and read the cluster properties file (installed by default as weblogic/mycluster). Any properties set in the cluster properties file will override the same properties that were set from the global properties file.

  3. Look for a subdirectory whose name matches the name you assign to the Server when you start it, by setting the weblogic.system.name property on the command line. The server name installed by default is weblogic/>clusterdir>/server><last three digits of the IP address>. If the directory is not found, the Server will quit, with an appropriate error message.

    If the directory is found, the Server will look for a per-server properties file and will read that. Any properties set in the per-server properties file will override the same properties that were set from the cluster or global properties files.

  4. Load all of its startup classes and deployments.

  5. Start listening for login requests.

An identical property set in a properties file will override the setting for the same property in any properties file read earlier in the startup sequence. For example, properties in each per-server properties file will override identical properties already set in the global or cluster properties files. Finally, any property set on the command line will override all other settings for that property.

The properties files are nested:

To set up the properties file hierarchy:

  1. Copy the weblogic.properties file that was installed from the distribution in the weblogic/ directory.

  2. Place a copy in the <mycluster>/ directory.

Here is a typical configuration, with the locations shown of the various properties files:

Default and global properties

 /weblogic/weblogic.properties

Cluster properties

 /weblogic/<mycluster>/weblogic.properties

All paths that you configure in the cluster properties file, such as servlet classpath, document root, etc., should point to the correct directory on the shared file system.

Per-server properties

 /weblogic/<mycluster>/server154/weblogic.properties

Note that if a per-server properties file exists in any of the server directories beneath the cluster directory (mycluster/), it should not contain any properties that exist in the global or cluster properties files, except those properties that you want to override settings established in the global or cluster properties files. Make sure you delete such properties from the per-server properties file, since the presence of a property in the per-server property overrides the identical property in global and cluster properties files.

Setting CLASSPATH

For instructions on setting your CLASSPATH, see the Administrators Guide, Setting classpath.

Setting up HTTP servlets

WebLogic Server supports clustering servlets across a WebLogic Cluster. This section describes the issues involved and how to set up a servlet cluster.

Initial HTTP servlet requests are passed to an arbitrary server node in a cluster. When a servlet request is part of an ongoing session, subsequent requests must have access to the HTTP session data. You set up your WebLogic Server Cluster to accomplish this using either in-memory state replication or JDBC session persistence. .

Using In-Memory State Replication with a Proxy Server

This feature provides better performance, adequate failover, and does not use a persistent store. It (unlike JDBC session persistence) does require a WebLogic Server Cluster license.

When using in-memory state replication, your WebLogic Server Cluster must live behind one or more proxy servers. The proxy servers are smart enough to send servlet requests, belonging to the same HTTP session, back to the same server in the cluster that holds the session data. This eliminates the need to write the session data to a shared database. You may use the Netscape Server, MS-IIS or WebLogic Server as a proxy server.

Failover is provided by replicating each session on a secondary node in the cluster. If the primary node fails, the secondary node takes over, becoming the new primary node for the session data. Another node is selected as the secondary node, and a backup of the session is made. If Weblogic Server is used as the proxy server, failover will occur up until the results are sent to the browser. However, if Netscape Server or MS-IIS is used as the proxy server, failover will only occur up until a successful request is accepted by a clustered server.

For example:
The browser's request goes to the proxy.
The proxy picks a server from the cluster. (If the first
server in the cluster is not available, proxy servers will failover
to an available server.)
The proxy server sends the request to the back-end server.
(After this request is accepted, Netscape and MS-IIS will not failover.
Instead, if the request fails, an error message will be sent to the
browser.)
The back-end server sends the results back to the proxy.
(Because a WebLogic Server proxy does not assume idempotence, failover will occur up until the
results are sent back to the proxy.)
The proxy sends the results to the browser.

It is important to keep failover in mind when writing servlets that will be executed in a WebLobic Cluster. If any exception occurs during the proxying of a request, a WebLogic Server proxy will attempt to re-issue the request to a different server in the cluster. It is possible that some of the request may have processed before the exception, and may be processed again. For this reason, it is a good idea to write idempotent servlets (servlets that end with the same result whether they are executed one time or N times in a session).

If a servlet session request for a cluster doing in-memory replication goes to the wrong server (i.e. not the primary or secondary for that session), the WebLogic Servers do not recognize that they have received a misdirected request for in-memory replication. The Cluster relies on the proxy server to direct the request to the correct primary or secondary. If the node in the WebLogic Cluster receives a misdirected request, it will return an error. Any data that must absolutely survive failures should be kept in a database (either using database persistance for the session or an entity bean).

Setting up the Proxy Server

For details on setting up a Netscape Server as a proxy server, see the Administrators Guide Installing the Netscape-WebLogic Server Plugin.

For details on setting up a Microsoft Internet Information Server as a proxy server, see the Administrators Guide Installing the MS-IIS-WebLogic Server Plugin.

This section shows you how to set up a WebLogic Server as a proxy server for clusters.

Requests intended for the WebLogic Cluster (for example JSP pages and servlets) must be handled by the HttpClusterServlet on the proxy server. The HttpClusterServlet knows how to proxy the request to the appropriate server in the WebLogic Cluster. Set up your DNS to point to the proxy server, not directly to a server in the WebLogic Cluster.

There are two ways to configure the proxy server: you can set the default servlet as the file servlet, or the cluster servlet.

In the first option, the proxy server acts as the file server for all static content, and only registered dynamic content is handled by the cluster. This configuration is clear and direct, because only JSP files and registered servlets are passed to the cluster servlet, while html pages, images, and other static content are delivered by the proxy server's file servlet.

The following properties in the proxyserver.weblogic.properties file set up the FileServlet as the default servlet.

  weblogic.allow.execute.weblogic.servlet=everyone
weblogic.httpd.register.file=\
weblogic.servlet.FileServlet
weblogic.http.defaultServlet=file

Requests that are intended for the WebLogic Cluster can be registered with the cluster servlet as shown below:


weblogic.httpd.register.*.jsp=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.*.jsp=\
defaultServers=server1:9001|server2:9001
weblogic.httpd.register.*.servlet=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.*.servlet=\
defaultServers=server1:9001|server2:9001
weblogic.httpd.register.myfavoriteservlet=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.myfavoriteservlet=\
defaultServers=server1:9001|server2:9001

The initialization parameter defaultServers lists the addresses of servers in the WebLogic Cluster.

In the second option, the proxy server's default is the cluster servlet and static requests can be handled by the file servlet. The main reason for choosing this configuration over the first option would be a large number of dynamic requests with different file extensions. Setting the cluster servlet as the default servlet in the proxy eliminates the need to individually register servlets, which may have arbitrary file extensions. Instead, static content should be registered.

The following properties set HttpClusterServlet as the default servlet.


weblogic.allow.execute.weblogic.servlet=everyone
weblogic.httpd.register.cluster=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.cluster=\
defaultServers=server1:9001|server2:9001
weblogic.httpd.defaultServlet=cluster

Requests that are intended for the WebLogic FileServlet can be registered as shown below for the .html, .jpg, and .gif MIME types. Other MIME file extensions can be added in the same manner.


weblogic.httpd.register.*.html=\
weblogic.servlet.FileServlet
weblogic.httpd.register.*.jpg=\
weblogic.servlet.FileServlet
weblogic.httpd.register.*.gif=\
weblogic.servlet.FileServlet

We recommend that you remove all other servlet registrations from the proxy server's weblogic.properties file, except the chosen default servlet (FileServlet in option 1, and ClusterServlet in option 2). That is, remove all other properties that begin with

  weblogic.httpd.register ...

This will avoid confusion where the proxy server may attempt to directly respond to a request if it matches a registration, or where the proxy servlet may fail due to an error in the cluster.

Setting up the WebLogic Cluster

In addition to setting up the proxy server, you need to set up a WebLogic Cluster as described below, including the weblogic.properties files. In addition to setting the normal properties for a clustered server and an HTTP servlet, you must set the following properties in the cluster-wide <my_cluster>weblogic.properties file to enable clustered HTTP sessions using in-memory replication:

  weblogic.httpd.clustering.enable=true
weblogic.httpd.session.persistence=true
weblogic.httpd.session.persistentStoreType=replicated

Using JDBC Session Persistence

Note: JDBC session persistence (unlike in-memory state replication) does not require a WebLogic Cluster license.

Each cluster node accesses servlet session data via a shared JDBC persistent store. For details on setting up persistent storage for servlet sessions, see the Developers Guide Using WebLogic HTTP Servlets: Making sessions persistent. WebLogic supports servlet clustering with EJB entity bean-managed persistence so the frequency of physical writes to disk can be carefully controlled.

Each time you add or change data in the HttpSession the change is written immediately to the persistent store database, allowing any other servers access to the updated data. This method provides good failover, but each change to the session involves a write to the database, which can be a substantial performance reduction compared to using no persistence. We recommend instead that you use in-memory state replication.

Note that persistence is only for failover; previous session data is available only if you use a database or an entity EJBean to make it persistent beyond the session.

Follow these steps to set up servlet clustering using JDBC session persistence:

  1. As described earlier, set up the DNS hosts file for your LAN so that there is a single IP entry that points to all the server hosts that make up the cluster.

  2. Set up separate physical machines, each installed with the same version of WebLogic Server and the same directory structure. Note that you cannot run different versions of WebLogic in any kind of interdependent environment.

  3. Set up the same connection pools for each WebLogic Server to support JDBC-based persistence.

  4. Make sure the servlet-related properties (for sessions, connection pools, and ACLs) are set to the same values in all of the properties files.

  5. Install your servlets on each machine, each in the proper directory structure.

  6. Start the WebLogic Servers.

  7. Your clients should request servlets using the cluster IP address rather than an IP address of an individual host.

Setting Up Enterprise JavaBeans

In addition to the information on Enterprise JavaBeans provided in Deploying EJBs Using DeployerTool,and in the General Requirements section of this chapter, to use the EJB hot deployment feature in a cluster you must:

  1. Undeploy the jar unit on all machines in the cluster

  2. Deploy the new jar unit on all machines in the cluster

This keeps the version of the bean present in the cluster the same at all times. There is a window where bean operations can fail and beans are unavailable, causing transactions to roll back or bean homes not to be found. You should write your clients to retry the EJBean in the event it can not be found to cover this case, as described in our Tech Support Guide Troubleshooting your WebLogic EJB application. For more information on hot deploy, see Using WebLogic Server hot deploy.

Starting the WebLogic Server and Joining a Cluster

You should have completed the items listed in General Requirements before you start a server that will join a cluster.

Starting with Defaults

For general information on starting a WebLogic Server, see the Administrators Guide Setting up and starting the WebLogic Server. We assume that you have read this and have started a WebLogic Server in a single server mode and are familiar with the use of the supplied scripts.

In the examples below on starting the Server, we have left out much of the command, and only shown the commands that are relevant to the cluster.

You can start a cluster using defaults and a couple of command-line options. The defaults assume that the host is not multihomed. The simplest way to start a WebLogic server that will join a cluster is to set the property weblogic.cluster.enable to true on the command line, as shown here:

 $ java -Dweblogic.cluster.enable=true
... weblogic.Server

Using this property alone on the command line will invoke a server with the following default settings:

Cluster name defaults to "mycluster" (and matches the name of the directory that contains your properties file and security files) IP address for this WebLogic Server defaults to its assigned IP address Listen port defaults to "7001" Server name defaults to the pattern "server" plus the last three digits of the IP address, as in "server154" Multicast address defaults to 237.0.0.1

Details on Starting WebLogic in a Clustered Environment

This section provides more details on the properties you can set for starting the Server in a Cluster, on the command line, in the weblogic.properties file, the cluster-wide properties file and the per-server properties file.

Other standard startup options are detailed in the Administrators Guide document, Setting up and starting the WebLogic Server, which details options for single-server configuration.

Command Line Properties

When you start each server that will join the cluster, you must set certain properties on the command line - with the -D option - unless you want to use the defaults. Here is a complete list of command line properties that affect server configuration, their defaults, and dependencies. Cluster-specific properties that can be specified in the cluster-wide properties and the per-server propertiesfile are included.

weblogic.cluster.name=clustername, where clustername is the name of the cluster this WebLogic Server should join on startup, and matches the name of the "mycluster/" directory. If you do not supply this command-line property, the cluster will start with the "mycluster" name, and will look in the weblogic/mycluster/ directory for its cluster property file, security-related files, and its document root. The cluster name property must be the same for every server in the cluster and must be set only on the command line.

For example:

 $ java -Dweblogic.cluster.name=megacluster
... weblogic.Server

weblogic.system.name, to specify the server name. If you do not use the standard pattern for establishing the server name ("server" + last 3 digits of the IP address), you must set the server name on the command line. Of course, the server name must be unique for each server in the cluster.

For example:

 $ java -Dweblogic.system.name=bizserver
... weblogic.Server

weblogic.cluster.enable should be set to true. This property must always be supplied on the command line.

weblogic.system.listenPort. This property must be the same for every server in a cluster. If you do not specify a listen port, the default used is "7001". If you specify the listen port for one server, you must specify the same listen port for every other server in the cluster. (Note that you cannot set up a cluster by running multiple servers on the same IP address listening at different ports; all servers must be listening on different IP addresses at the same port.) The listen port can be set on the command line or in the cluster-wide properties file.

weblogic.system.bindAddr, if the machine is multihomed and you want to specify the IP address for this server process. If you do not specify an IP address when you start WebLogic on a multihomed machine, WebLogic will choose one of its host's IP addresses; the algorithm for determining which IP address is chosen is operating-system dependent. You should set the bindAddr property if the machine is multihomed to avoid unpredictable results. You can either set this property on the command line or - if you have specified the server name, as described above - in the appropriate per-server properties file.

For example:

 $ java -Dweblogic.system.bindAddr=237.0.0.5 (or DNS name)
... weblogic.Server

Here is a sample of properties to be set for the WebLogic Server "server154" to join "mycluster". The multicast address for this cluster is 237.5.6.7, and the machine is multihomed, with IP addresses 227.15.15.154 and 204.198.18.6, and the WebLogic Server would be running on the 227.15.15.154 address.

  weblogic.cluster.name=mycluster
weblogic.cluster.multicastAddress=237.0.0.5
weblogic.system.bindAddr=227.15.15.154
weblogic.system.listenPort=7001
weblogic.cluster.enable=true

Advanced Properties

In addition to the above properties, you can set these properties in the cluster-wide properties file:

weblogic.cluster.multicastAddress, to specify the multicast address that the servers in a cluster will use to communicate with each other. If you do not specify an address, WebLogic will attempt a DNS lookup of the name of the cluster plus "Multicast", such as "clusternameMulticast". If no DNS entry is found, WebLogic will use the default multicast address "237.0.0.1". The multicast address must be the same for every server that joins the cluster. Consequently, it is best if you set it only in the cluster-wide properties file.

weblogic.cluster.multicastTTL, the time-to-live (number of hops) for the cluster's multicast messages. The default is 1, with a range of 1-255.

weblogic.cluster.defaultLoadAlgorithm, the default load balancing algorithm to be used between replicated services if none was specified for that particular service. The default is "round-robin", with acceptable values of "random", "round-robin", and "weight-based". Load balancing is explained in Writing and Deploying Clustered Objects.

Per-Server Properties

There are also cluster-related properties that apply only to a specific server; you set these properties in the per-server properties file:

weblogic.system.weight, the weight of the individual server for the weight-based load-balancing. The default is 100, with a range of 0-100. Larger numbers increase the amount of traffic routed to the server.

Stopping a Server in a Cluster

Use the Admin SHUTDOWN command to halt a WebLogic Server in a cluster, just as you would a WebLogic Server running in stand-alone mode. The other members of the Cluster will determine that the halted server is no longer available and the cluster-wide JNDI tree will be updated accordingly.

Special Considerations for Clients of a Cluster

Performance of your JAVA client can be degraded when the number of socket readers is lower than the number of WebLogic Servers in your cluster. You can improve the performance by setting the following properties from the command line while starting your client:

 $ java -Dweblogic.system.executeThreadCount=<number of threads>  

$ java -Dweblogic.system.percentSocketReaders=<percentage of threads to be used as socket readers>

Setting the number of threads and the percentage of socket readers to ensure an equal number of socket readers and instances of WebLogic Server will help to maintain high performance levels.