Installing Glassfish 4.1 on Ubuntu 14.04 LTS

Before you start doing anything you should think about a security concept. A detailed security concept is out of scope for this tutorial. Very important from security point of view is not to run your Glassfish server as root. This means you need to create a user with restricted rights which you can use for running Glassfish. Once you have added a new user, let's say 'glassfish', you might also want to add a new group called 'glassfishadm'. You can use this group for all users that shall be allowed to "administer" your Glassfish in full depth. In full depth means also modifying different files in the Glassfish home directory. Below you find user and group related commands that you might want to use.

#Add a new user called glassfish
sudo adduser --home /home/glassfish --system --shell /bin/bash glassfish

#add a new group for glassfish administration
sudo groupadd glassfishadm

#add your users that shall be Glassfish adminstrators
sudo usermod -a -G glassfishadm $myAdminUser

#in case you want to delete a group some time later (ignore warnings):
#delgroup glassfishadm

Glassfish allows some of the configuration tasks to be managed via a web based Administration GUI. We will simply call it AdminGUI from now on. You can reach the AdminGUI by visiting http://www.yourserver.com:4848/ in your browser (please replace www.yourserver.com with localhost or where ever your Glassfish server is). As you can see port 4848 is used. Of course, we don't want anyone to access our AdminGUI. Therefore we have to restrict access to the AdimnGUI. A way do this is to block port 4848 via the firewall. Anything you can do via AdminGUI is also available via the asadmin tool that ships with Glassfish. So you don't have to worry about not being able to configure Glassfish if you block the AdminGUI.

Usually you want to run Glassfish on port 80 (everything below port 1024 has to run as root). But since we don't suggest to run Glassfish as root we cannot run Glassfish on port 80. But there are still ways to run Glassfish as a non-root user and still receive http requests on port 80. One option could be mod_jk, but this would only be another component that needs to be managed. An easy way is to use a simple iptables redirection rule, that redirects requests on port 80 to port 8080 (http) and requests on port 443 to port 8181 (https).

You should make sure that you do not block other important ports, for example your ssh port which usually runs on port 22 (else you will be locked out). Changing the default ssh port to some other is actually a good idea, but for now we will simply assume your ssh port is 22. Another helpfull iptables rule related to your ssh port 22 is to slow down connection tries from an ip if they fail 3 times. I found a rule for that on the web and added it below. Although I will not mention it here you should also use other techniques and tools to secure your ssh port. Unfortunately, I did not get the time to post a tutorial about that.

Now we have created a lot of rules. You could enter them always one by one, but we don't want this kind of effort. I suggest to enter the following iptables rules in a separate file which contains all of our iptables related ideas we discussed so far:

#!/bin/bash

# ATTENTION: flush/delete all existing rules
iptables -F

################################################################
# set the default policy for each of the pre-defined chains
################################################################
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD DROP

# allow establishment of connections initialised by my outgoing packets
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

# accept anything on localhost
iptables -A INPUT -i lo -j ACCEPT

################################################################
#individual ports tcp
################################################################
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
iptables -A INPUT -p tcp --dport 8181 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT
#uncomment next line to enable AdminGUI on port 4848:
#iptables -A INPUT -p tcp --dport 4848 -j ACCEPT

################################################################
#slow down the amount of ssh connections by the same ip address:
#wait 60 seconds if 3 times failed to connect
################################################################
iptables -I INPUT -p tcp -i eth0 --dport 22 -m state --state NEW -m recent --name sshprobe --set -j ACCEPT
iptables -I INPUT -p tcp -i eth0 --dport 22 -m state --state NEW -m recent --name sshprobe --update --seconds 60 --hitcount 3 --rttl -j DROP

#drop everything else
iptables -A INPUT -j DROP

################################################################
#Redirection Rules
################################################################
#1. redirection rules (allowing forwarding from localhost)
iptables -t nat -A OUTPUT -o lo -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A OUTPUT -o lo -p tcp --dport 443 -j REDIRECT --to-port 8181

#2. redirection http
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080

#3. redirection https
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8181


################################################################
#save the rules somewhere and make sure
#our rules get loaded if the ubuntu server is restarted
################################################################
iptables-save > /etc/my-iptables.rules
iptables-restore < /etc/my-iptables.rules

#List Rules to see what we have now
iptables -L

I suggest to create a file called iptables.DISABLE_4848.rules which contains exactly everything from the code box above. Then you could also create a file called iptables.ENABLE_4848.rules which has line 28 uncommented (everything else is just the same). Of course, you have to make both files executable with the command chmod +x $filename (see below). Then you can simply run one of the scripts when ever you want to disable or enable the AdminGUI on port 4848, i.e. sudo ./iptables.DISABLE_4848.rules

chmod +x iptables.DISABLE_4848.rules
chmod +x iptables.ENABLE_4848.rules

Please also do not forget that all your iptables rules should also be activated if your Ubuntu server is restarted. Otherwise you would have to remember to run your iptables rules manually after each restart. If you forget to run them all manually, or if you have simply forgotten that your server has been restarted, then your firewall is open for everyone. If you are lucky nothing will happen, if not you might get some successful instrusion attacks. Lines 58 and 59 will help you to make sure your rules are automatically loaded after each restart. But this is not everything for iptables configuration on startup. You also need to create a file at /etc/network/if-pre-up.d/iptablesload and one at /etc/network/if-post-down.d/iptablessave. For more information please have a look at the official Ubuntu help sites for iptables. The following code boxes show the content of our two files and where they have to be created. As you can see in both the latter two code boxes, line 2 is refering to the file /etc/my-iptables.rules, which we have defined in line 58 and 59 of our files iptables.DISABLE_4848.rules and iptables.ENABLE_4848.rules respectively. I have added /sbin/ in front of the iptables commands (see below) because I was facing the problem that iptables commands without /sbin/ could not be found at the time when the files iptablesload or iptablessave were executed during the Ubuntu server startup process.

# first we have to create these two files:
sudo touch /etc/network/if-pre-up.d/iptablesload
sudo touch /etc/network/if-post-down.d/iptablessave

#!/bin/sh
/sbin/iptables-restore < /etc/my-iptables.rules
exit 0

#!/bin/sh
/sbin/iptables-save -c > /etc/my-iptables.rules
if [ -f /etc/iptables.downrules ]; then
   /sbin/iptables-restore < /etc/iptables.downrules
fi
exit 0

Finally you have to make sure that both files are executable. For that you only need to execute the following commands once.

sudo chmod +x /etc/network/if-post-down.d/iptablessave
sudo chmod +x /etc/network/if-pre-up.d/iptablesload

At this point you can try what happens if you reboot your Ubuntu server (sudo reboot). After Ubuntu has restarted just try sudo iptables -L on the shell. It should show you the rules we have defined. You should see something like this if you hit sudo ./iptables.DISABLE_4848.rules before rebooting:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
DROP       tcp  --  anywhere             anywhere            tcp dpt:ssh state NEW recent: UPDATE seconds: 60 hit_count: 3 TTL-Match name: sshprobe side: source
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh state NEW recent: SET name: sshprobe side: source
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:www
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:http-alt
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:8181
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https
DROP       all  --  anywhere             anywhere

Chain FORWARD (policy DROP)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Your firewall settings are loaded automatically whenever Ubuntu is starting up. We can continue with the next steps. Please do not forget that these are only some minimum firewall settings. For maximum security you might need to add your own iptables rules.

Now we can download Glassfish. I suggest to switch the user now to glassfish, which we have created in the first step. We want to download the Glassfish zip installation file to /home/glassfish/downloads/. Afterwards the zip file has to be extracted and the content can be moved to /home/glassfish/ - this is everything needed for installing Glassfish. Usually the zip file is extracted to a directory called ./glassfish4/. Make sure to move the content of ./glassfish4/ and not ./glassfish4 itself to /home/glassfish/.

#if you dont't have "unzip" installed run this here first
sudo apt-get install unzip

#now switch user to the glassfish user we created (see step 1)
sudo su glassfish

#change to home dir of glassfish
cd /home/glassfish/

#create new directory if not already available
mkdir downloads

#go to the directory we created
cd /home/glassfish/downloads/

#download Glassfish and unzip
#Hint: get the correct link from  https://glassfish.java.net/download.html if wget fails
wget http://download.java.net/glassfish/4.1/release/glassfish-4.1.zip
unzip glassfish-4.1.zip


#move the relevant content to home directory
mv /home/glassfish/downloads/glassfish4/* /home/glassfish/
#if something has not been moved, then move it manually, i.e.:
mv /home/glassfish/downloads/glassfish4/.org.opensolaris,pkg /home/glassfish/.org.opensolaris,pkg

#exit from glassfish user
exit

#change group of glassfish home directory to glassfishadm
sudo chgrp -R glassfishadm /home/glassfish

#just to make sure: change owner of glassfish home directory to glassfish
sudo chown -R glassfish /home/glassfish

#make sure the relevant files are executable/modifyable/readable for owner and group
sudo chmod -R ug+rwx /home/glassfish/bin/
sudo chmod -R ug+rwx /home/glassfish/glassfish/bin/

#others are not allowed to execute/modify/read them
sudo chmod -R o-rwx /home/glassfish/bin/
sudo chmod -R o-rwx /home/glassfish/glassfish/bin/

Let's create an init script now. It helps you to start, stop and restart your Glassfish easily. We also need this to make Glassfish start up automatically whenever Ubuntu is rebooting. The file we need to create is /etc/init.d/glassfish. For starting and stopping Glassfish we will use the asadmin tool that ships with Glassfish (we used it a little in the previous step).

#create and edit file
sudo vi /etc/init.d/glassfish

#(paste the lines below into the file and save it...):

#! /bin/sh

#to prevent some possible problems
export AS_JAVA=/usr/lib/jvm/jdk1.8.0

GLASSFISHPATH=/home/glassfish/bin

case "$1" in
start)
echo "starting glassfish from $GLASSFISHPATH"
sudo -u glassfish $GLASSFISHPATH/asadmin start-domain domain1
;;
restart)
$0 stop
$0 start
;;
stop)
echo "stopping glassfish from $GLASSFISHPATH"
sudo -u glassfish $GLASSFISHPATH/asadmin stop-domain domain1
;;
*)
echo $"usage: $0 {start|stop|restart}"
exit 3
;;
esac
:

The init script is set up. Now we can add it to the default run levels. This way our Glassfish will startup whenever Ubuntu is restarted.

#make the init script file executable
sudo chmod a+x /etc/init.d/glassfish

#configure Glassfish for autostart on ubuntu boot
sudo update-rc.d glassfish defaults

#if apache2 is installed:
#stopping apache2
sudo /etc/init.d/apache2 stop
#removing apache2 from autostart
update-rc.d -f apache2 remove

Even now we should not really use Glassfish in production. We will now begin the configuration of Glassfish itself. You should always run these steps, for example changing the default passwords, enabling https, changing the default ssl certificate to be used for https etc. We will also put our attention on Glassfish obfuscation.

Our first step is to change the master password. Glassfish uses it to protect the domain-encrypted files from unauthorized access, i.e. the certificate store which contains the certificates for https communication. When Glassfish is starting up it tries to read such "secured" files - for exactly this purpose Glassfish needs to be provided with the master password either in an intertactive way or in a non-interactive way. I will choose the non-interactive way because we want our Glassfish to start up on Ubuntu reboot as a deamon (in the Windows world this would be called a service). This is necessary so that the start-domain command can start the server without having to prompt the user. To accomplish this we need to set the savemasterpassword option to true. This option indicates whether the master password should be written to the file system. The file is called master-password and can be found at /home/glassfish/glassfish/domains/domain1/master-password. To change the master password you have to ensure that Glassfish is not running - only then you can call the command change-master-password which will interactivly ask you for the new password.

#switch user to glassfish (stay with this user for complete Step 6!)
sudo su glassfish

#change master password, default=changeit
/home/glassfish/bin/asadmin change-master-password --savemasterpassword=true
#prompt: choose your new master password ==> myMasterPwd
# ==> stores file to /home/glassfish/glassfish/domains/domain1/master-password

The next step is to change the administration password with change-admin-password. Because this command is a remote command we need to ensure that Glassfish is running before we can execute the command. Since we want "automatic login" we will create an admin password file allowing us to login without being asked for credetials (it will be stored at /home/glassfish/.gfclient/pass).

#change admin password
/home/glassfish/bin/asadmin change-admin-password
#1. enter "admin" for user (default)
#2. hit enter because default pwd is empty
#3. choose you new pwd ==> myAdminPwd

#now we have to start Glassfish
/home/glassfish/bin/asadmin start-domain domain1

#login for automatic login...
/home/glassfish/bin/asadmin login
#prompt:
#user = admin
#password = myAdminPwd
#==> stores file to /home/glassfish/.gfclient/pass

#now stop Glassfish
/home/glassfish/bin/asadmin stop-domain domain1

Glassfish is coming with a pre-configured certificate which is used for ssl (https). You can see it in the keystore.jks file if you check for the alias s1as. Since Glassfish 3.1 there is even another preconfigured certificate available: glassfish-instance. But that also means that everybody else can get these two certificates, the public keys, private keys, etc. With that information you could never be safe because "others" could "read" your data sent to Glassfish via https. That means you should always make sure to replace the pre-configured s1as and glassfish-instance entries in your keystore. But you should not delete them as long as the alias "s1as" and "glassfish-instance" are still in use (and it is by default in use for https...). I faced some strange behaviour as I did not think of that at the beginning when I simply deleted s1as - learn from my mistake and do not delete it for now... But we can help us with generating a new alias first (myAlias) and when ever needed or wanted we could change each occurrence of s1as to myAlias (i.e. via admin console) and then we could finally delete that s1as. The same has to be done also for glassfish-instance.

The following code box shows you the commands we need for modifying our Glassfish keystore. As you can see we first delete our pre-configured s1as entry (Glassfish mustn't be running!). Later a new s1as entry is generated - it is now unique for us! Similar steps have to be executed also for our second certificate (glassfish-instance).

#create new certs
cd /home/glassfish/glassfish/domains/domain1/config/
keytool -list -keystore keystore.jks -storepass myMasterPwd
keytool -delete -alias s1as -keystore keystore.jks -storepass myMasterPwd
keytool -delete -alias glassfish-instance -keystore keystore.jks -storepass myMasterPwd
keytool -keysize 4096 -genkey -alias myAlias -keyalg RSA -dname "CN=nabisoft,O=nabisoft,L=Mannheim,S=Germany,C=DE" -validity 3650 -keypass myMasterPwd -storepass myMasterPwd -keystore keystore.jks
keytool -keysize 4096 -genkey -alias s1as -keyalg RSA -dname "CN=nabisoft,O=nabisoft,L=Mannheim,S=Germany,C=DE" -validity 3650 -keypass myMasterPwd -storepass myMasterPwd -keystore keystore.jks
keytool -keysize 4096 -genkey -alias glassfish-instance -keyalg RSA -dname "CN=nabisoft,O=nabisoft,L=Mannheim,S=Germany,C=DE" -validity 3650 -keypass myMasterPwd -storepass myMasterPwd -keystore keystore.jks
keytool -list -keystore keystore.jks -storepass myMasterPwd

#make sure that keystore.jks and cacerts.jks stay in sync, so let's update cacerts.jks:
#1. export the two certificates we just created to files: 
keytool -export -alias glassfish-instance -file glassfish-instance.cert -keystore keystore.jks -storepass myMasterPwd
keytool -export -alias s1as -file s1as.cert -keystore keystore.jks -storepass myMasterPwd
#2. now delete the old/existing aliases initially shipped with glassfish:
keytool -delete -alias glassfish-instance -keystore cacerts.jks -storepass myMasterPwd
keytool -delete -alias s1as -keystore cacerts.jks -storepass myMasterPwd
#3. now import the two certificates we stored into files:
keytool -import -alias glassfish-instance -file glassfish-instance.cert -keystore cacerts.jks -storepass myMasterPwd
keytool -import -alias s1as -file s1as.cert -keystore cacerts.jks -storepass myMasterPwd

#we can delete the two cert files now
rm glassfish-instance.cert s1as.cert

Now we want to enable https for the admin console. Once we have done that we can be sure that nobody can listen to our data sent via https because nobody else has our certificates, i.e. nobody can decrypt our password used for entering the admin console via browser (in case someone cought our data packages). However, sometimes clients want to connect via https using SSLv3, which is considered to be vulnerable nowadays (ever heard about "Poodle"?). In other words: disable SSLv3 completely where ever you find it - no discussion about that allowed! Here is a great post about different ways of disabling SSLv3 in Glassfish 4.1. I typically use the command line version (asadmin). Furthermore, it seems that Glassfish 4.1 does not disable TLS client-initiated renegotiation by default, which is considered to open some doors for DoS attacks. The article TLS Renegotiation and Denial of Service Attacks by Ivan Ristic is a good read. For us this means we need to disable client-initiated renegotiation somehow. But this is not all we want to do here. We want to change some of the default JVM Options and we want to make our Glassfish not telling too much ("obfuscation").

The first JVM Option we will change is replacing the -client option with the -server option. I expect the java option -server to be the better choice when it comes to performance. I have also decided to change -Xmx512m (Glassfish default) to a higher value: -Xmx4096m. Additionally, I have added -Xms4096m. Furthermore, we will remove -XX:MaxPermSize=192m and replace it with -XX:MaxMetaspaceSize=512m because -XX:MaxPermSize was deprecated in JDK 8. If you want to find out more about it then have a look at this InfoQ article. For more information about JVM options please check the Oracle's official documentation.
All JVM Options so far are optional. But at least adding -Dproduct.name="" is a good idea for everyone. If you would not add this then each http/https response will contain a header field like this: Server: GlassFish Server Open Source Edition 4.1
This is some great piece of information for hackers - that's why you should disable it. We don't want Glassfish to talk too much for security reasons!

We also don't want Glassfish to send a header similar to X-Powered-By: Servlet/3.1 JSP/2.3 (GlassFish Server Open Source Edition 4.1 Java/Oracle Corporation/1.8) because this is telling everyone we are using a Servlet 3.1 container and that we are (of course) using Java etc. So we have to disable sending x-powered-by in the http/https headers. After executing the commands below our Glassfish will run in silent mode - it is not telling too much any more. Glassfish obfuscation accomplished. SSLv3 and cient-initiated renegotiation will also be disabled, which reduces the surface for attacks.

# the commands here change the file at
# /home/glassfish/glassfish/domains/domain1/config/domain.xml

#first we have to start Glassfish
/home/glassfish/bin/asadmin start-domain domain1

# enable https for remote access to admin console
# requests to http://xxx:4848 are redirected to https://xxx:4848
/home/glassfish/bin/asadmin enable-secure-admin

#change JVM Options
#list current jvm options
/home/glassfish/bin/asadmin list-jvm-options
#now start setting some important jvm settings (change this as you wish)
/home/glassfish/bin/asadmin delete-jvm-options -- -client
/home/glassfish/bin/asadmin create-jvm-options -- -server
#For server deployments, -Xms and -Xmx are often set to the same value
/home/glassfish/bin/asadmin delete-jvm-options -- -Xmx512m
/home/glassfish/bin/asadmin create-jvm-options -- -Xms4096m
/home/glassfish/bin/asadmin create-jvm-options -- -Xmx4096m

#-XX:MaxPermSize was deprecated in JDK 8, and superseded by the -XX:MaxMetaspaceSize option, see http://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html
# see also this good article: http://www.infoq.com/articles/Java-PERMGEN-Removed
/home/glassfish/bin/asadmin delete-jvm-options '-XX\:MaxPermSize=192m'
/home/glassfish/bin/asadmin create-jvm-options -- '-XX\:MaxMetaspaceSize=512m'
/home/glassfish/bin/asadmin create-jvm-options -- '-XX\:MetaspaceSize=512m'

#disable client-initiated renegotiation (to decrease the surface for DoS attacks)
/home/glassfish/bin/asadmin create-jvm-options -Djdk.tls.rejectClientInitiatedRenegotiation=true

#get rid of http header field value "server" (Glassfish obfuscation)
/home/glassfish/bin/asadmin create-jvm-options -Dproduct.name=""

#disable sending x-powered-by in http header (Glassfish obfuscation)
/home/glassfish/bin/asadmin set server.network-config.protocols.protocol.http-listener-1.http.xpowered-by=false
/home/glassfish/bin/asadmin set server.network-config.protocols.protocol.http-listener-2.http.xpowered-by=false
/home/glassfish/bin/asadmin set server.network-config.protocols.protocol.admin-listener.http.xpowered-by=false
/home/glassfish/bin/asadmin set server.network-config.protocols.protocol.sec-admin-listener.http.xpowered-by=false

#optional: let's disable autodeploy (via autodeploy folder)
/home/glassfish/bin/asadmin set server.admin-service.das-config.autodeploy-enabled=false

#disable SSLv3
/home/glassfish/bin/asadmin set server.network-config.protocols.protocol.http-listener-2.ssl.ssl3-enabled=false
/home/glassfish/bin/asadmin set server.network-config.protocols.protocol.sec-admin-listener.ssl.ssl3-enabled=false
/home/glassfish/bin/asadmin set server.iiop-service.iiop-listener.SSL.ssl.ssl3-enabled=false
/home/glassfish/bin/asadmin set server.iiop-service.iiop-listener.SSL_MUTUALAUTH.ssl.ssl3-enabled=false

#restart to take effect
/home/glassfish/bin/asadmin stop-domain domain1
/home/glassfish/bin/asadmin start-domain domain1
#what jvm options are configured now?
/home/glassfish/bin/asadmin list-jvm-options

#we are done with user glassfish
exit

Finally we have come to where we wanted. We have installed, secured and configured our Glassfish installation.

#starting glassfish
sudo /etc/init.d/glassfish start

#remove glassfish from autostart
#update-rc.d -f glassfish remove

If you want a quick and free ssl security audit for your Glassfish server then go to Qualys SSL Labs and check your server configuration (the server must be accessible from the internet). I suggest every server available on the internet should use and even force HTTPS because it makes the web much safer. Even Google has announced to take HTTPS as a ranking signal. The "bad performance" argument is not really relevant anymore: TLS has exactly one performance problem: it is not used widely enough. Google gives you some good hints to Secure your site with HTTPS. Another good read is Ivan Ristić's paper: SSL/TLS Deployment Best Practices.