Home » Clustering, Glassfish, Java, Liferay

How to install and configure a Glassfish cluster

26 August 2008 11 Comments

If you are looking for Glassfish 3.1 Cluster installation then go here: Installing a Glassfish 3.1 Cluster on Debian 6.0 only from command line

In the last days I tried to install and configure a Glassfish and a Liferay cluster for a client. From what I see in Internet is a lot of “old” documentation for Glassfish / Liferay clustering and nothing related to latest versions of Glasfish / Liferay (glassfish v2ur2 and Liferay 5.1.1), and I know this documentation also will become old in some months 🙂 . So I intend to write (as best as I can) a very simple documentation about how I manage this to work and what problems I found trying to do this. The software configuration of the my machines is:

Debian 4.0r3
Glassfish v2UR2
Liferay 5.1.1
Mysql 6.1.25
JDK 1.6.0_07


In the first part of this article we will discus about Glassfish cluster and it’s problems and in the second part about Liferay.

First of all we need to know the configuration for our cluster:

1. How many nodes do we have in cluster configuration ? (2)
2. What are the host names of the nodes ? (node0, node1, nodeX)
3. What are the ip’s of the nodes ? (10.0.0.1, 10.0.0.2, 10.0.0.X)

First think what you need to do, when you start configuring the cluster, is to set a correct DNS server or to write the hosts file for your cluster nodes.

IF YOUR DNS IS NOT CONFIGURED WELL YOU CAN GET A LOT OF “STUPID” PROBLEMS.

In my case I configured “hosts” file in this way:

file: /etc/hosts from node0

127.0.0.1 localhost
10.0.0.1 node0
10.0.0.2 node1
#here add more node if is the case

Now upload /etc/hosts in all your nodes (I used scp to achieve this). Also I used to add the same addresses on my windows workstation (C:\Windows\System32\drivers\etc\hosts if I remember well) to easy access the cluster from my workstation, but if you are using a DNS server then you don’t need to do that 🙂

When you are working in a cluster environment is a good idea to setup some ssh keys for all your nodes. (more details to come)

Installing java

I downloaded JDK 1.6.0_7 from http://java.sun.com i586 version, but if you’re machines are x86_64 get the right version. After download I put the archive in /opt and as root user I install it:

cd /opt
sh ./jdk-6u7-linux-i586.bin
ln -s jdk1.6.0_07 java

And that it. You have java installed on your system.

Configuring environment:

For all my applications I use a normal user to run them and NOT the “root” user. So I create a new user with adduser command:

# adduser gfish
Adding user `gfish’ …
Adding new group `gfish’ (1013) …
Adding new user `gfish’ (1013) with group `gfish’ …
Creating home directory `/home/gfish’ …
Copying files from `/etc/skel’ …
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for gfish
Enter the new value, or press ENTER for the default
Full Name []: Gfish
Room Number []: 13
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [y/N] y

Do that on all your nodes and put a strong password, if this node can be accessed from outside the world or if some other users have minimal access to the nodes you can have security problems.

Today, with a good Nvidia card (with CUDA support), brute forcing can process ~270 millions hash’s per second and that means several seconds for a 6 small letter password and several hours for a 8 small letter password. Just to know for a 12 character password with signs and all the stuff the time for bruteforceing is somewhere at several millions years (something like that 4n$ 4r3 M3r#!)

.

To run java and glassfish without adding path in front of the commands each time (or changing directory) is good to add to PATH environment path’s for java and glassfish:

edit /home/gfish/.bash_profile on node0 and add to the end of file:

PATH=/opt/java/bin:~gfish/glassfish/bin:”${PATH}”

Copy .bash_profile in all your nodes in /home/gfish/
Now just logout and login again as gfish.

Installing Glassfish:

1. Be sure you are in the right place (our case /home/gfish)
2. Extract Glassfish – java -Xmx256M -jar glassfish-dist.jar
3. Edit details in /home/gfish/glassfish/setup-cluster.xml (At least change the admin password)
4. cd /home/gfish/glassfish
5. put execute attribute on ant: chmod +x lib/ant/bin/ant
6. lib/ant/bin/ant -f setup-cluster.xml

If you get a “BUILD SUCCESSFULY” the you are ready to configure the cluster

Starting setup the cluster:

gfish@node0:~/glassfish$ asadmin create-node-agent –host node0 –port 4848 ws-agent-0
gfish@node0:~/glassfish$ asadmin create-cluster –host node0 –port 4848 ws-cluster
gfish@node0:~/glassfish$ asadmin create-instance –host node0 –port 4848 –nodeagent ws-agent-0 –cluster ws-cluster ws-instance-0
gfish@node0:~/glassfish$ asadmin start-node-agent –syncinstances=true ws-agent-0

Now you will need to configure Glassfish JDBC resources to can be accessed by ws-cluster also an not only by Stand-Alone Instances. JDBC resources what you need to add ws-cluster on target are:

jdbc/__CallFlowPool
jdbc/__default
jdbc/__TimerPool

I will just give you an example for jdbc/__CallFlowPool and for the rest are the same steps.

Login on administration console http://node0:4848/ and Go to (in the left menu)
Resources ->JDBC -> JDBC Resources -> jdbc/__CallFlowPool and here choose Target -> Manage Targets . Now chose from the left listbox “Available Targets” ws-cluster and click “Add”. Now should appear on the right listbox “Selected Targets:”. Press OK.

The main problem is on jdbc/__TimerPool. When you try to add ws-cluster as Target I get this error:

An error has occurred
Creation or deletion of the system resource reference jdbc/__TimerPool is not allowed.

The solution is to add a new JDBC resources (jdbc/NewClusterTimer) with the Pool Name __TimerPool and set the target to ws-cluster.

After all this steps just restart the domain and the node:

asadmin stop-node-agent
asadmin stop-domain
asadmin start-domain
asadmin start-node-agent –syncinstances=true ws-agent-0

Now if you go to http://node0:38080 you will find the default GlassFish page.

Adding a new node in cluster:

Login on node1

1. Be sure you are in the right place (our case /home/gfish)
2. Extract Glassfish – java -Xmx256M -jar glassfish-dist.jar
3. Edit details in /home/gfish/glassfish/setup-cluster.xml (At least change the admin password)
4. cd /home/gfish/glassfish/
5. put execute attribute on ant: chmod +x lib/ant/bin/ant
6. Setup Glassfish – lib/ant/bin/ant -f setup-cluster.xml

If you get a “BUILD SUCCESSFULY” the you are ready to configure the node
Starting setup the node

gfish@node1:~/glassfish$ asadmin create-node-agent –host node0 –port 4848 ws-agent-1
gfish@node1:~/glassfish$ asadmin create-instance –host node0 –port 4848 –nodeagent ws-agent-1 –cluster ws-cluster ws-instance-1
gfish@node1:~/glassfish$ asadmin start-node-agent –syncinstances=true ws-agent-1

Now just go on node0:4848 (http://node0:4848/) and on Cluster-> Applications -> Deploy to deploy your application

After deploying if you get OutOfMemory errors when you are trying to start nodes (that happens for large applications) then you need to increase the heap for that nodes. From node0 just run this command

asadmin set domain.node-agent.nodeX.property.INSTANCE-SYNC-JVM-OPTIONS=”-Xmx256m”

where nodeX is the node with low heap.

Now the cluster is ready and you can access it on http://node0:38080 or http://node1:38080.

Good luck with your round robin DNS or balancing software 🙂

Sorry for my grammar or other mistakes. If you find a very nasty mistake just write me and I will correct it asap.


11 Comments »

  • shepherdching (author) said:

    This article is very useful.
    Thank you.

  • sacx (author) said:

    I hope it will be useful 🙂

    Regards

  • kingpk said:

    I wish it could work for me as well 🙂

  • admin said:

    But what problem do you have ? Installing just a simple cluster is easy … Installing liferay over a glassfish cluster is pain in the ass … 🙂

    Regards

  • Install Glassfish 2.1 with HADB | Random Bugs (author) said:

    […] NOT start again to show you how to configure a Glassfish cluster, because this is already made in this article. Today I will present how to add HADB functionality to an existent Glassfish […]

  • loadbalancing in glassfish cluster said:

    I write about Glassfish Cluster Loadbalancer with apache.

    Take care, also, the ejb timer in glassfish cluster, because the embedded derby db can’t manage two or more instance for the ejb timer features.

    You can use other dbms, or an external derby.
    Glassfish cluster and ejb timer

  • admin said:

    The ejb timer is still a problem even in 2.1.1. Anyway thanks for the tip.

    Regards

  • edh said:

    Hi, I followed up your manual. But I can’t start the cluster because the instance ws-instance-1 isn’t running, it says that the node agente ws-agent-1 isn’t running but i check and it is.

    Can you please help me?.

    Thanks in advance.

  • admin said:

    Ok. Probably is a zombie 🙂 Try to kill the process with kill command, kill and if still not working try kill -9 . After that try to start again the node.

    R.

  • Neeraj said:

    Thanks a lotttttttt.

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.