Home » Clustering, Featured, Glassfish, Headline, How-to, Java, Linux, Shell

Installing a Glassfish 3.1 Cluster on Debian 6.0 only from command line

4 May 2011 3 Comments

Glassfish Logo

Glassfish Logo


Finlay is here. For me, Glassfish 3.1, was one of most awaited piece of software, for all it’s new features and clustering support. Glassfish 3.1 slightly changed the way to install cluster, in a much better and integrated way. This version come with a lot of new functionality, but also with improved parts related to Glassfish 2.1.

The most interesting features of Glassfish 3.1 are: Clustering (100 instances per domain), SSH provisioning, Centralized management (with the help of ssh), High Availability Session Persistence, High Availability Java Message Service, RMI-IIOP Load Balancing and Failover.

In this tutorial I will show you how to install Glassfish 3.1 with cluster support only from command line.

Settings for all machines from cluster.

1. Install ssh on all nodes, probably you already made that, but because my tutorial depends on that, I want it to remind you again.

# apt-get install ssh

2. create a special user for all your nodes. Don’t install your cluster as root. Is a big security problem if you install it as root user.

To add a user just type:

# adduser user

3. Glassfish 3.1 should have the same Java on DAS and nodes. The best way is to install the same OS for all.

If you have multiple Java version installed you can check this post to see how to change it to the correct one: Change the default Java Runtime on Ubuntu (works for Debian also)

# update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).

Selection Path Priority Status
------------------------------------------------------------
0 /usr/lib/jvm/jdk1.6.0_24/bin/java 100 auto mode
* 1 /usr/lib/jvm/java-6-sun/jre/bin/java 73 manual mode

Press enter to keep the current choice[*], or type selection number:

4. Configure all nodes from cluster (including DAS) with the correct hostnames

# vi /etc/hosts

and add your hosts. Example:

172.16.0.11 node-1.domain.com node-1
172.16.0.12 node-2.domain.com node-2
172.16.0.1 node-das.domain.com node-das

5. To have access to asadmin utility without writing all path add Glassfish bin folder to your system PATH

# export PATH=${PATH}:/home/user/glassfis/bin

6. Take a look at file descriptors (Tuning file descriptors limits on Linux) under Linux if you deploy this configuration on a production environment.

7. Also after installation don’t forget to increase Xmx and Xms values.

Installing Glassfish Cluster

1. Glassfish is dependent on CURL. With curl, Glassfish 3.1, will upload the installation files on the rest of nodes from cluster.

2. Install WX widgets … probably only if you want a graphical installation. Under Debian I didn’t need it, but under Ubunt, install-node command complain about it.

3. Cluster Installation

Become Glassfish [user]

# su – user

or

$ssh user@node-das

Download Glassfish 3.1, as glassfish user in /home/user run the following command:

$ sh glassfish-3.1-unix.sh -s
Extracting the installer archive...
Extracting the installer runtime...
Extracting the installer resources...
Extracting the installer metadata...

Welcome to GlassFish V3 installer

Using the user defined JAVA_HOME : /usr
Entering setup...

SETTING UP DOMAIN FOR SILENT INSTALL...
Executing /home/user/glassfish3/glassfish/bin/asadmin --user admin --passwordfile /tmp/asadminTmp2627469239751504419.tmp create-domain --savelogin --checkports=false --adminport 4848 --instanceport 8080 --domainproperties=jms.port=7676:domain.jmxPort=8686:orb.listener.port=3700:http.ssl.port=8181:orb.ssl.port=3820:orb.mutualauth.port=3920 domain1
/home/user/glassfish3/glassfish/bin/asadmin --user admin --passwordfile /tmp/asadminTmp3780145047492184623.tmp create-domain --savelogin --checkports=false --adminport 4848 --instanceport 8080 --domainproperties=jms.port=7676:domain.jmxPort=8686:orb.listener.port=3700:http.ssl.port=8181:orb.ssl.port=3820:orb.mutualauth.port=3920 domain1Using port 4848 for Admin.
Using port 8080 for HTTP Instance.
Using port 7676 for JMS.
Using port 3700 for IIOP.
Using port 8181 for HTTP_SSL.
Using port 3820 for IIOP_SSL.
Using port 3920 for IIOP_MUTUALAUTH.
Using port 8686 for JMX_ADMIN.
Using default port 6666 for OSGI_SHELL.
Using default port 9009 for JAVA_DEBUGGER.
Distinguished Name of the self-signed X.509 Server Certificate is:
[CN=node-das.domain.com,OU=GlassFish,O=Oracle Corporation,L=Santa Clara,ST=California,C=US]
Distinguished Name of the self-signed X.509 Server Certificate is:
[CN=node-das.domain.com-instance,OU=GlassFish,O=Oracle Corporation,L=Santa Clara,ST=California,C=US]
No domain initializers found, bypassing customization step
Domain domain1 created.
Domain domain1 admin port is 4848.
Domain domain1 allows admin login as user "admin" with no password.
Login information relevant to admin user name [admin]
for this domain [domain1] stored at
[/home/user/.asadminpass] successfully.
Make sure that this file remains protected.
Information stored in this file will be used by
asadmin commands to manage this domain.
Command create-domain executed successfully.

Now we should configure a easy access way to this nodes. This version of Glassfish already have implemented some SSH specific commands (SSH Provisioning) what will help us to generate ssh keys for the machines what we want to install in our cluster.

$ asadmin setup-ssh node-1
Would you like to generate a SSH key pair (without a key passphrase) for user to access [node-1]? [y/n]: y
Enter SSH password for user@node-1>
Copied keyfile /home/user/.ssh/id_rsa.pub to user@node-1
Successfully connected to user@node-1 using keyfile /home/user/.ssh/id_rsa
Command setup-ssh executed successfully.

$ asadmin setup-ssh node-2

[ … ]

After ssh-setup ends successfully we need to install Glassfish on all nodes. This process was improved from version 2.1 and with just 1 command you can install everything without donwload, install the entire software on the node.

$ asadmin install-node node-1
Created installation zip /home/user/glassfish4887107989978147510.zip
Successfully connected to user@node-1 using keyfile /home/user/.ssh/id_rsa
Copying /home/user/glassfish4887107989978147510.zip (108666394 bytes) to node-1:/home/user/glassfish3
Installing glassfish4887107989978147510.zip into node-1:/home/user/glassfish3
Removing node-1:/home/user/glassfish3/glassfish4887107989978147510.zip
Fixing file permissions of all files under node-1:/home/user/glassfish3/bin
Command install-node executed successfully.

$ asadmin install-node node-2
[ ... ]

Now we need to start the domain administration server, because we need to create cluster and instances.

$ asadmin start-domain domain1
Waiting for domain1 to start .............
Successfully started the domain : domain1
domain Location: /home/user/glassfish3/glassfish/domains/domain1
Log File: /home/user/glassfish3/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.

Optionally you can enable secure administration for your DAS. Now is the right moment to do it, because after you will lose control to your nodes if you enable it.
Also some users reported the following if you don’t enable secure administration:

Failed to rendezvous with DAS on node-1:4848. Please check if this server is running, that the host and port are correct, and that this server is configured to allow remote access.

$ asadmin enable-secure-admin
Command enable-secure-admin executed successfully.

If you enable-secure-admin then you need to restart the domain again

$ asadmin restart-domain domain1

Now we create the node for each instance.

$ asadmin create-node-ssh --nodehost node-1 node-1
Command create-node-ssh executed successfully.

$ asadmin create-node-ssh --nodehost node-2 node-2
Command create-node-ssh executed successfully.

And now we create the cluster:

$ asadmin create-cluster cluster1
Command create-cluster executed successfully.

Now we add the instances to the corresponding nodes
$ asadmin create-instance --cluster cluster1 --node node-1 instance-1
Command _create-instance-filesystem executed successfully.
Port Assignments for server instance instance-1:
JMX_SYSTEM_CONNECTOR_PORT=28686
JMS_PROVIDER_PORT=27676
HTTP_LISTENER_PORT=28080
ASADMIN_LISTENER_PORT=24848
JAVA_DEBUGGER_PORT=29009
IIOP_SSL_LISTENER_PORT=23820
IIOP_LISTENER_PORT=23700
OSGI_SHELL_TELNET_PORT=26666
HTTP_SSL_LISTENER_PORT=28181
IIOP_SSL_MUTUALAUTH_PORT=23920
The instance, instance-1, was created on host node-1
Command create-instance executed successfully.

$ asadmin create-instance --cluster cluster1 --node node-1 instance-2

$ asadmin start-instance instance-1
CLI801 Instance is already synchronized
Waiting for instance-1 to start ..................
Successfully started the instance: instance-1
instance Location: /home/user/glassfish3/glassfish/nodes/node-1/instance-1
Log File: /home/user/glassfish3/glassfish/nodes/node-1/instance-1/logs/server.log
Admin Port: 24848
Command start-local-instance executed successfully.
The instance, instance-1, was started on host node-1
Command start-instance executed successfully

# asadmin start-instance instance-2

Also to stop an instance run (on DAS server):

# asadmin stop-instance instance-1

If you are on the instance host you should use stop-local-instance and start-local-instance

The last step is to change administrator and master password

$asadmin change-admin-password
Enter admin user name [default: admin]> admin
Enter admin password>
Enter new admin password>
Enter new admin password again>
Command change-admin-password executed successfully.

$ asadmin change-master-password
Enter the current master password>
Enter the new master password>
Enter the new master password again>
Command change-master-password executed successfully.

Now go to http://node-das:4848 and take a tour through Glassfish Interface.

Useful Commands

1. Listing running instances

$asadmin list-instances -l
NAME HOST PORT PID CLUSTER STATE
instance-1 node-1 24848 13976 cluster1 running
instance-2 node-2 24848 19374 cluster1 running
Command list-instances executed successfully.

2. Collecting log files from all instances

$asadmin collect-log-files --target cluster1
Log files are downloaded for instance-1.
Log files are downloaded for instance-2.
Created Zip file under /home/user/glassfish3/glassfish/domains/domain1/collected-logs/log_2011-05-04_06-29-10.zip.
Command collect-log-files executed successfully.

3. Stooping local instance

$ asadmin stop-local-instance instance-1
Waiting for the instance to stop ...
Command stop-local-instance executed successfully.

4. Starting local instance
$ asadmin start-local-instance instance-1
Warning: Synchronization with DAS failed, continuing startup...
Waiting for instance-1 to start ..................
Successfully started the instance: instance-1
instance Location: /home/user/glassfish3/glassfish/nodes/node-1/instance-1
Log File: /home/user/glassfish3/glassfish/nodes/node-1/instance-1/logs/server.log
Admin Port: 24848
Command start-local-instance executed successfully.

5. Get cluster health
$ asadmin get-health cluster1
instance-1 started since Wed May 04 06:10:03 UTC 2011
instance-2 started since Tue May 03 13:52:40 UTC 2011
Command get-health executed successfully.
I hope this will be a enough for a fresh start on Glassfish clustering.

Good Luck!


3 Comments »

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.