Controlling class loading in j2ee application

In some cases, we might have to change the order of class loading from different jar. This can be easily controlled for stand alone applications, Change the CLASSPATH to add a jar which need precedence in the first place. For example,

old.jar conatins Function.class file, this has to be modified to use new functionality. So team has developed new.jar with Function.class file. To keep backward compatability we need to maintain both the jars. To make new.jar Function.class precedence add them to class path as below

CLASSPATH=new.jar;old.jar;other.jar

This works great, simple and easy!!!

What happens if we need to follow same in J2EE application? Can you control order of classloading? Yes you can, but need to follow few steps. Before moving on lets consider ear package as below

j2eestandard.ear
          ejb.jar
          web.war
          lib/old.jar
          lib/new.jar
          lib/other.jar

In this sample package, APP server loads calsses in following order

1. First it loads all the classes under lib folder (the jar file order can vary to environment to environment, so we cannot predict old.jar or new.jar gets loaded?). Then appserver refers any manifest:classpath is specified in the jar files under lib folder. If it is specified that will be loaded
2. Second step, ejb jars will be loaded (in this case ejb.jar). If there is any manifest:classpath in ejb.jar then that will also be loaded
3. Finally web app will be loaded (web.war), If there is any manifest:classpath in web.war, that will also be loaded.

So now we have got an overview, now time to tweak a bit to make sure to load new.jar first. As per step 1, all the jars in /lib/ will be loaded first. So keep new.jar under lib folder. Move old.jar to root. Add manifest to ejb.jar and mention about old.jar, as below

j2eemodified.ear
          ejb.jar
                   MANIFEST.MF
                           Manifest-Version: 1.0
                           Class-Path: old.jar
          web.war
          lib/new.jar
          lib/other.jar
          old.jar

The classes will be loaded in below order now,

1. All the jars under lib folder, so new.jar and other.jar
2. ejb.jar, there is manifest info so
3. old.jar will be loaded
4. web.war will be loaded

Its solved now. Happy packaging !!! :)

JDK1.4 & WL10.1 backward compatability

We needed to support JDK1.4 clients connecting to WL 10.1 which is JDK 1.5. But ran into run time error,

java.lang.IncompatibleClassChangeError: Implementing class
java.lang.ClassLoader.defineClass0(Native Method)
java.lang.ClassLoader.defineClass(ClassLoader.java:539)
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:476)
weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:181)
java.lang.ClassLoader.loadClass(ClassLoader.java:289)
java.lang.ClassLoader.loadClass(ClassLoader.java:235)
weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:223)
java.lang.ClassLoader.loadClassInternal(ClassLoader.java:302)
java.lang.ClassLoader.defineClass0(Native Method)
java.lang.ClassLoader.defineClass(ClassLoader.java:539)
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:123)
weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:476)
weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:181)
java.lang.ClassLoader.loadClass(ClassLoader.java:289)
java.lang.ClassLoader.loadClass(ClassLoader.java:235)
weblogic.utils.classloaders.GenericClassLoader.findOrCreateSingleSourceClass(GenericClassLoader.java:640)
weblogic.rmi.internal.StubGenerator.getStubOnClient(StubGenerator.java:782)
weblogic.rmi.internal.StubGenerator.getStubClass(StubGenerator.java:758)
weblogic.rmi.internal.StubGenerator.generateStub(StubGenerator.java:803)
weblogic.rmi.internal.StubGenerator.generateStub(StubGenerator.java:790)
weblogic.rmi.extensions.StubFactory.getStub(StubFactory.java:79)
weblogic.rmi.utils.io.RemoteObjectReplacer.resolveObject(RemoteObjectReplacer.java:237)
weblogic.rmi.internal.StubInfo.readResolve(StubInfo.java:142)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:324)
java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:925)
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1655)
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1274)
java.io.ObjectInputStream.readObject(ObjectInputStream.java:324)
weblogic.common.internal.ChunkedObjectInputStream.readObject(ChunkedObjectInputStream.java:119)
weblogic.rjvm.MsgAbbrevInputStream.readObject(MsgAbbrevInputStream.java:112)
weblogic.rmi.internal.ObjectIO.readObject(ObjectIO.java:56)
weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:159)
weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:285)
weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:244)
weblogic.jndi.internal.ServerNamingNode_812_WLStub.lookup(Unknown Source)
weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:343)
weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:336)
javax.naming.InitialContext.lookup(InitialContext.java:347)

There was not any documentation about backward compatability in WL. Then found that weblogic.jar of WL8.1SP2 has this problem. But with webloigc.jar WL8.1SP4 just worked fine. So in case needed just play with different version of weblogic jars :)

initial context - OOM

Creating intial context and not closing the context can lead to OOM condition based on the load. Either this needs to be cached or closed immediately after usage. Caching may not be possible when application connects to multiple App Server. Even though this is the better solution, some time forced to use close option. InitialContext.close makes that object GC collectable immediately. So it does not move tenured space, will be collected from new generation space.

How to find HP system version

uname -a

provides version of HP unix

getconf KERNEL_BITS

to find out 32 / 64 bit processor

Relook logic

Its good to put some of the problems/issues for a small gap (say 5 to 7 days) and take a look again, it sure gives solution.

Weblogic server status check - UNIX command

To automate weblogic managed server startup steps, need to check Admin server status first. Then start managed server. Check managed server status before continuing next steps like bringing up services deployed. Found to be useful to use weblogic.Admin option to achieve this easily,

Here is the step to check Admin server status

. setEnv.sh
$JAVA_HOME/bin/java weblogic.Admin -url $admin_server PING 1 >/dev/null 2>&1
admin_server_status=$?

Step to check Managed server status

ms_server_status=`$JAVA_HOME/bin/java weblogic.Admin -url $admin_server GETSTATE $app_name grep -c RUNNING `

write small script to simplify deployment/bounce

Its good to automate all the process related to component bounce, application server bounce, deployment steps etc. It may look like a simple steps and need not require automation. But when we work on the same component for quite a long time we would have spent more time for just copy/pasting comments.

Making it as a simple script has several advanatges,
1. we dont need to remember each and individual commands and dont need to remember authentication details
2. all typo can be avoided
3. In the long run this can save lot of time
4. anyone can easily manage

example of scripting managed weblogic server:

startDemoApp1.sh

umask 002
WLS_USER=admin
WLS_PW=admin123
export WLS_USER
export WLS_PW
nohup ./startManagedWebLogic.sh demoapp1 t3://localhost:7200 $* > demoapp1.out 2>&1 &

stopDemoApp1.sh

./stopWebLogic.sh admin admin123 demoapp1 t3://localhost:7200

UML Tools

Not all the freeware for UML is great. Each has its own advantages and disadvantages. So we are force to use Low cost product. I have been using enterprise architect tool for past 7 days and its cool. It has got all the basic feature and greatly integrated with svn. With the basic desktop version could share the project and separate each modules into separate file.

It has got great reporting option to create both rtf and html versions. For the kind of price it is pretty good tool.

Script to check OpenDB

I am mostly working on automating all the steps of deployment and testing. Most of the time we face open DB issue. Somewhere in the server, there can be code issue which leads to holding database connection. This happens mainly with the old release codes. This needs to be constantly monitored across the system. The better approach would be poling DB to check no of database connection which is idle for more than specified time (say 1 hour). This is simple in case of Oracle, because this information can be derived from v$session table.

It has to be implemented with some scripting language (like PERL), but decided to use ant because this has been used before. Here is the sample target used,

<target name="db1" if="oracle1.url" >
 <!-- Connect to database to find idle connections -->
 <echo message="......................... ${db1.name} .........................."/>
 <sql
     driver="${oracle1.driver}"
     url="${oracle1.url}"
     userid="${oracle1.user}"
     password="${oracle1.password}"
     print="yes"
  showheaders="false"
  showtrailers="false"
  onerror="continue"
  output="${session.log}"
  keepformat="yes"
     >
 <classpath refid="common.classpath"/><![CDATA[
  select     rpad(username,15 ,' ')
      rpad(MACHINE, 30,' ')
   rpad(process,15, ' ')
   lpad(last_call_et,15, ' ') '   '
   rpad(osuser, 15,' ')
  from V$session where
   last_call_et > ${oracle1.min.idle.time}
   and last_call_et < ${oracle1.max.idle.time}
   and username = UPPER('${oracle1.user}')
   and (
    machine in ( ${oracle1.machine.name} ) OR
    osuser in ( ${oracle1.os.user} )
   );
  ]]>
 </sql> 
</target>

All the parameter can be configured and email the result file session.log

Now this can be set as crontab and run every one hour.

Miscellaneous Tasks - ant - SQL

I was going through misc tasks of ant while working on build script for one our release. I found SQL task and was wondering why do we need this?

But today i could use this for one of our daily verification script, used this to get failure count from one of the table. what a great usage :)

Link

Sample:

<target name="queue" if="oracle.url" >
<sql
driver="${oracle.driver}"
url="${oracle.url}"
userid="${oracle.user}"
password="${oracle.password}"
print="yes"
showheaders="false"
showtrailers="false"
onerror="continue"
>
<classpath refid="common.classpath"/><![CDATA[

select 'No of item to be processed : ' || count(sqno) 
from Queue where processedtime is null ;

select 'Seq number : ' || sqno || CHR(10) || 
'Total time in queue : ' || numtodsinterval(sysdate-queuecreated,'day') || CHR(10) ||
' (day hour:minute:seconds.milliseconds) ' || CHR(10) ||
'Total number of failure(s) : ' || FAILURECOUNT 
from QUEUE where SQNO =
( select min(SQNO) from Queue 
where processedtime is null );

]]>
</sql>
</target>

Reversion - steps to be considered

All enterprise applications should have reversion process before deploying them into production. This is followed by most of the online applications; to take care of addressing unavoidable issues (revert back to old release, if there is no workaround). Configuration changes are key in this process. Important things to take care,

* Don't depend on back up configuration files: Always change the parameters in current configuration files. This is because even though we take back up of configuration file before deploying current release, there is a possibility that it will be changed in production environment for some other components/reasons. When we revert the release, the changes done after the release will be gone if we use back up file.

* Don't overlook any configuration changes: Do not leave any configuration change assuming that it will not affect old release. If we overlook one parameter and that can even bring down entire environment. We had such a situation while testing in Dev, one parameter has been changed from milliseconds to minutes (frequency between run). So as part of release we changed from 60000 to 1. Everything went well. We have done reversion and missed this parameter, old application assumes this as 1 and runs. Because it is 1 millisecond and it went into loop. Ultimately it brought down entire database, oops. So learned from mistake

ebay buyers feel pride to win product

Past 3 weeks i am watching ebay bidding to buy zune 30GB (i blew black friday deal 76$ :( ). Used zune sold for more than 150$ with shipping around 20$. I am wondering, same model new zune is 159$ (even went down to 149$) in amazon.com. This offer comes with free shipping, why dont they buy this instead of ebay for more.

It may be because winning product gives more joy than buying a fixed price item or I see some customers are relisting item because some body just bid for fun :)

Anyway i may not buy this in ebay if it is more than 60$.

Verifying batch program failure

We have automated ant process which runs every day to check health check of particular component, this has been configured through crontab. It failed to complete the process and hung in between, so there is no mail generated. To close this hole, have written small script which will check the log file and look for 'BUILD SUCCESSFUL' if it is not found error mail will be generated. This is configured to run 1 hour after the verification script. This works well :)

Here is the sample script which i used

ErrorCheck.sh:

#!/bin/ksh
log_file=$1
subject=$2
shift
shift
mailaddrs=$*
## checks for (BUILD SUCCESSFUL). If not present, sends email.
grep_result=`grep -c "BUILD SUCCESSFUL" $log_file`
if [ "$grep_result" = "1" ]
then
echo "NO PROBLEM"
else
echo "PROBLEM"
mailx -s"$subject" $mailaddrs < $log_file
fi

How to Execute:

./ErrorCheck.sh {logfilename} {Email subject} {email addresses separated by space}

./ErrorCheck.sh junitresult.log "Dev Test failed" test@test.com test1@test.com