Avri Blog

Blog à l'avricot - Lord Of Castle, java, nosql, Utomia, AvriChat, javascript, Json, Css, Mootools, ajax, php...

Aller au contenu | Aller au menu | Aller à la recherche

Stop Bubble | Start

jeudi, juin 30 2011

hibernate @Any mapping, just can't fetch the "any" object org.hibernate.type.AnyType cannot be cast to org.hibernate.type.ComponentType

Using hibernate @Any mapping to map multiple entities in the same, I haven't be able to build a request to fetch all the objects of a specific class.
Hibernate doesn't seems to be capable to handle this kind of request.
My application doesn't start and throw the following error :

Invocation of init method failed; nested exception is java.lang.ClassCastException: org.hibernate.type.AnyType cannot be cast to org.hibernate.type.ComponentType

removing the part "left join fetch ic.item i" solve this error.

@Entity
@NamedQueries({
@NamedQuery(name = "GET_ITEMS", query = "from ItemContainer ic left join fetch ic.item i where ic.id=:containerId and ic.class=:clazz")})
public class ItemContainer {
.... 
    @Any(metaColumn = @Column(name = "TYPE"), fetch = FetchType.LAZY)
    @AnyMetaDef(idType = "long", metaType = "string", metaValues = { @MetaValue(targetEntity = Item1.class, value = "I1") })
    @JoinColumn(name = "TYPE_ID")
    private IItem item;
...
}

(Item1 implements IItem)

@Override
@SuppressWarnings("unchecked")
public List<ItemContainer> getItem1s(final Long containerId) {
return getHibernateTemplate().findByNamedQueryAndNamedParam("GET_ITEMS", new String[] { "containerId", "clazz" },
		new Object[] { containerId, Item1.class });
}

:/

Any ideas ?

vendredi, février 18 2011

Java and tomcat class loader, priority & hierarchy

It's always hard to remember tomcat class loader order !
You'll find here a complete resume :)

Java class loader hierarchy

We have 3 main class loader :

  • Bootstrap class loader, called primordial class loader. It's a part of the JVM., and is responsible for loading classes from the core Java package (java.lang, java.util etc.) These classes are found in JAVA_HOME/jre/lib/rt.jar.
  • Extension class loader. It monitors the JAVA_HOME/jre/lib/ext folder and load the existing jars.
  • System class loader. It monitors the folders and Jars which can be find in the classpath.

An important point is that a class loader can only see the class locations that are above it in the hierarchy. So a class loaded from Extension class loader can't see a class from the class path.

Tomcat / java EE hierarchy

For security reason, each web app must use its very own class loader (you don't want to share any class with another malicious application)
During startup, tomcat neutralize the System class loder by clearing the classpath.
It also changes the endorsed directory to point to CATALINA_HOME/endorsed.
Doing so, the class-loader hierarchy is reversed:

  • Tomcat internal class loader. Loads classes for tomcat internal code. Web applications can't use this class loader.
  • Web application class loader. Loads classes from the WEB-INF/classes (first) and WEB-INF/lib for a webapp
  • Shared class loader. Loads classes from the folder specified in catalina.properties. Theses classes are visible only to all web apps.
  • Common class loader. Loads classes from CATALINA_HOME/lib (all web apps + tomcat: servlet-api.jar, jasper.jar, coyote.jar, jsp-api.jar etc.)
  • System class loader: Loads classes from the classpath, but the classpath is truncated by tomcat, so it doesn't do anything except looking for CATALINA_HOME/bin/bootstrap.jar (tomcat startup), tomcat-juli.jar (logging) and tools.jar (for the jsp compiler).
  • Extention class loader (JAVA_HOME/jre/lib/ext)
  • Bootstrap class loader
  • Endorsed Standards Override Mechanism (the Endorsed Standards Override Mechanism lets you place overrides to certain classes (CORBA and JAXP classes) in the JAVA_HOME/lib/ endorsed folder. The Bootstrap loader will then load these preferentially over any classes that it might otherwise find. For details on this mechanism, see http://java.sun.com/j2se/1.5.0/docs/guide/standards/, Source : Tomcat 6 developer's Guide, by Damodar Chetty)

When you try to load a class in a tomcat webapp, your default classloader first delegate to the standard class loader hierarchy :

  • the bootstrap serves up core classes.
  • extension loader : classes held within any installed extensions jar.
  • system class loader : catalina startup classes as well as supported classes.

If the class can't be found, the webapp looks within its own repositories

  • from /WEB-INF/classes folder
  • from /WEB-INF/LIB/*.jar

If the class is still not found, the webapp delegate to it's parent classloaser :

  • from %CATALINA_HOME%/lib/*.jar (common to all webapps, and internal tomcat classes)
  • the shared class loader : serves up classes that are common across all web appart within a given host.

jeudi, octobre 7 2010

Java Calendar timestamp and Jdk 1.5.015 buggy

Let's have some fun with java's Date...
I want the number of seconds from now to midnight...
Easy ? That's what I thought !
Doing the following just doesn't work

public static void main(String[] args){
GregorianCalendar c = new GregorianCalendar(Locale.FRANCE);
int timeStampActuel = (int) (c.getTimeInMillis()/1000);
System.out.println(timeStampActuel);
c.set(Calendar.HOUR_OF_DAY, c.getActualMaximum(Calendar.HOUR_OF_DAY);
c.set(Calendar.MINUTE, c.getActualMaximum(Calendar.MINUTE));
String theDate = DateUtils.dateToString(c.getTime()); //DateUtils convert the Date to a String with a default formater
int timeStampAMinuit = (int) (c.getTimeInMillis()/1000);
System.out.println(timeStampAMinuit);
}

Basically, timeStampActuel is the correct value, theDate display the correct midnight value, but timeStampAMinuit is definitely wrong (02:59:23, not 23:59:23 as expected... )
That was just because of my jvm which was buggy ( 1.5.015, I know it's an old one...). just update to a newer version :)

mercredi, septembre 22 2010

Fast export csv writer

Just a little java cvs writer, I had to write. Nothing special, but it can be usefull !

package com.atlas.ui.bo.modules.exportProspectionCommande;
 
import javax.management.RuntimeErrorException;
 
import com.cardiweb.cardiboxv6.framework.utils.FormatUtils;
 
/**
 *  Permet d'ecrire facilement dans un fichier csv
 *  utilisation:
 *  CsvWriter aCsvWriter = new CsvWriter.Builder("exportProspectionCommande").setEndOfLigneCharacter("|").set[...].build() ;
 *  aCsvWriter.appendValue("ceQueVousVoulez");
 *  aCsvWriter.newLine();
 *  Valeur par defaut utilisées:
 *  separatorCharacter = ",";
 *	endOfLigneCharacter = ";";
 *	delimitorValue = "\"";
 *	createNewLine = "\n";		
 *	escapeCharacter = "\\";
 * @author qambard
 */
public class CsvWriter  {
	public static final String KEY_CONTENT_TYPE_HEADER = "Content-Type";
	public static final String KEY_CONTENT_DISPOSITION_HEADER = "Content-Disposition";
	public static final String KEY_CONTENT_LENGTH_HEADER = "Content-length";
	private String separatorCharacter ;
	private String delimitorValue ;
	private String endOfLigneCharacter ;
	private String createNewLine ;		
	private String fileName ;
	private String escapeCharacter;
	private String contentType;
	private String contentDisposition ;
	private boolean beginOfLine = true;
	private boolean cachedLengthUpToDate = false ;
	private boolean toStringCalled = false ;
	private int cachedLength = 0 ;
	StringBuilder sb = new StringBuilder ();
	private CsvWriter (Builder builder) {
		separatorCharacter = builder.separatorCharacter;
		endOfLigneCharacter = builder.endOfLigneCharacter;
		delimitorValue = builder.delimitorValue;
		createNewLine = builder.createNewLine;
		fileName = builder.fileName;
		contentType = builder.contentType;
		contentDisposition = "attachment; filename="+fileName+".csv";
		escapeCharacter = builder.escapeCharacter;		
	}
	/**
	 * invalid the cache of the 
	 */
	private void invalidCached() {
		cachedLengthUpToDate = false ;
		toStringCalled = false ;
	}
	/**
	 * Add a value to the last line, escape=true
	 * @param aValue
	 */
	public void appendValue(Object aValue) {
		appendValue(aValue, true);
	}
	/**
	 * Add a value to the last line
	 * @param aValue value to add
	 * @param escape is you want the value to be escaped
	 */
	public void appendValue(Object aValue, boolean escape) {
		invalidCached();
		if (!beginOfLine) {
			sb.append(separatorCharacter);
		} else {
			beginOfLine = false ;
		}
		sb.append(delimitorValue);
		if (aValue != null) {
			if (aValue instanceof java.util.Date) {
				sb.append(FormatUtils.getDateFormatUtils().getDateFormat("yyyy-MM-dd").format((java.util.Date) aValue));
			} else {
				String valueString = aValue.toString();
				if (escape) {
					valueString.replace(separatorCharacter, escapeCharacter+separatorCharacter);
				}
				sb.append(valueString);
			}
		}
		sb.append(delimitorValue);
	}
	/**
	 * Create a new line in the file
	 */
	public void newLine () {
		invalidCached();
		beginOfLine = true ;
		sb.append(endOfLigneCharacter);
		sb.append(createNewLine);
	}
 
	/**
	 * Return the file as a String 
	 */
	public String toString () {
		if (!toStringCalled) {
			toStringCalled = true ;
			sb.append(endOfLigneCharacter);
		}
		return sb.toString();
	}
 
	/**
	 * @return the file as byte[]
	 */
	public byte[] toBytes () {
		byte[] b = toString().getBytes("UTF8") ;
		if (!cachedLengthUpToDate) {
			cachedLength = b.length ;
			cachedLengthUpToDate = true ;
		}
		return toString().getBytes();
	}
 
	/**
	 * the length of the file is cached. Doesn't cost anything.
	 * @return the length of the file.
	 */ 
	public String getContentLength() {
		if (!cachedLengthUpToDate) {
			throw new RuntimeErrorException(new Error("The lenght of the content is probably invalid. You have to invoke a toBytes() before getContentLength() to clear the cache"));
		}
		return Integer.toString(cachedLength) ;
	}
 
	public String getSeparatorCharacter() {
		return separatorCharacter;
	}
	public void setSeparatorCharacter(String separatorCharacter) {
		this.separatorCharacter = separatorCharacter;
	}
	public String getDelimitorValue() {
		return delimitorValue;
	}
	public void setDelimitorValue(String delimitorValue) {
		this.delimitorValue = delimitorValue;
	}
	public String getEndOfLigneCharacter() {
		return endOfLigneCharacter;
	}
	public void setEndOfLigneCharacter(String endOfLigneCharacter) {
		this.endOfLigneCharacter = endOfLigneCharacter;
	}
	public String getCreateNewLine() {
		return createNewLine;
	}
	public void setCreateNewLine(String createNewLine) {
		this.createNewLine = createNewLine;
	}
	public String getFileName() {
		return fileName;
	}
	public void setFileName(String fileName) {
		this.fileName = fileName;
	}
	public String getEscapeCharacter() {
		return escapeCharacter;
	}
	public void setEscapeCharacter(String escapeCharacter) {
		this.escapeCharacter = escapeCharacter;
	}
	public String getContentType() {
		return contentType;
	}
	public void setContentType(String contentType) {
		this.contentType = contentType;
	}
	public String getContentDisposition() {
		return contentDisposition;
	}
	public void setContentDisposition(String contentDisposition) {
		this.contentDisposition = contentDisposition;
	}
 
	public static class Builder {
		private String separatorCharacter = ",";
		private String endOfLigneCharacter = ";";
		private String delimitorValue = "\"";
		private String createNewLine = "\n";		
		private String escapeCharacter = "\\";
		private String fileName ;
		private String contentType = "text/csv" ;
		public Builder (String aFileName) {
			fileName = aFileName;
		}
		public Builder setContentType(String contentType) {
			this.contentType = contentType; return this;
		}
		public Builder setSeparatorCharacter(String separatorCharacter) {
			this.separatorCharacter = separatorCharacter; return this;
		}
		public Builder setEscapeCharacter(String escapeCharacter) {
			this.escapeCharacter = escapeCharacter; return this;
		}
		public Builder setEndOfLigneCharacter(String endOfLigneCharacter) {
			this.endOfLigneCharacter = endOfLigneCharacter; return this;
		}
		public Builder setCreateNewLine(String createNewLine) {
			this.createNewLine = createNewLine;  return this;
		}
		public Builder setDelimitorValue(String delimitorValue) {
			this.delimitorValue = delimitorValue;  return this;
		}
		public CsvWriter build () {
			return new CsvWriter(this);
		}
	}
}

mardi, mai 11 2010

Java - MultiThread singleton - LocalThread vs Double Check Locked (dcl) vs synchronized vs static vs volatil

There are 4 ways to create a thread-safe singleton in a multithread environment.

  1. Using a LocalThread
  2. Using a full synchronized method
  3. Using a static singleton
  4. Using DCL (not a safe solution)
  5. Using volatil keyword


But wich solution is the better ?...

LocalThread

Probably the most complicated solution.
Basicly, LocalThread is an unique attribut wich won't be shared between the thread.
You can see it as a volatil attribute.
This implementation suxs before JVM 1.4 (because LocalThread was a synchronized object).
But with the new JVM, it has a better implementation and is faster.

public class LocaThreadSingleton extends Singleton {
  private static ThreadLocal initHolder = new ThreadLocal();
  public static Singleton getInstance () {
    if (initHolder.get() == null){
      synchronized (LocaThreadSingleton.class){
      if (instance== null)
        instance = new LocaThreadSingleton();
        initHolder.set(Boolean.TRUE);
      }
    }
    return instance;
  }
}

The Static solution

This is the best - and the more efficient solution.
You should use it in almost all cases (It doesn't fit if you need to create the Singleton with class attributes/variables).

public class StaticSingleton extends Singleton {
	private static StaticSingleton instance = new StaticSingleton() ;
	public static Singleton getInstance () {
	    return instance;
	}
}

Full synchronized solution

Probably the worst solution, after DCL : It's totally inefficient because you will synchronized the whole method and lock it every single call.

public class SynchronizedSingleton extends Singleton{
	public static synchronized Singleton getInstance () {
		if (instance == null)
			instance = new SynchronizedSingleton();
		return instance ;
	}
}

Double Check solution

The worst solution. It just doesn't work because of the out of order memory write.
Basically, the jvm could do the following :

mem = allocate() ;
instance = mem ;
constructorSingleton(instance)

Because of this behaviour, you could get an "instance" object !=null but also != new DCLSingleton();
This may happen in a very exceptional circumstance, but you have to keep in mind that it is not safe.
Here is the wrong implementation, don't use it :

public class DCLSingleton extends Singleton{
	public static Singleton getInstance () {
		if (instance==null){
			synchronized (DCLSingleton.class){
				if(instance==null)	
					instance=new DCLSingleton();
			}
		}
		return instance;			
	}
}

Volatil solution

Clean and efficient !

public class VolatilSingleton extends Singleton{
	private volatile static Singleton singleton;
	public static Singleton getInstance () {
	     if(singleton==null) {
	    	 synchronized(Singleton.class){
		    	   if(singleton==null)
		        	  singleton= new VolatilSingleton();
		       }
	     }
	     return singleton;
	}
}

I made a couple of bench, trying to find-out wich solution is the better one.
The bench I made throw at the same moment 5 trheads (that's because I wanted to avoid any kind JVM optimisation due to the fact singletons was created from the same object/thread).
Every single threads will execute X other threads (lets say 50 000) wich will try to get an instance of the type of singleton you have chosen at the start of the bench.

The cost of the singleton creation is +- 500ms (you can choose it)

The first result is that the full solution "synchronized" is really bad.
That's not because of the cost of the "lock" operation (and I think it's important to notice that).
The matter is that too many thread wich are kept in the "queue", even after the instanciacion of your singleton.

So it's clear that :

  1. DCL is bad (it's not safe)
  2. full synchronized solution is bad
  3. Static method is the better solution, when you can implement it.



Now what about volatil vs LocalThread ?
Let's run our bench ! (average of 6time for each solution)

Here are the results.
bench singleton thread
Volatil seems to be the fastest solution.
LocalThread is slower than volatil, and more complicated.
My point is :

  1. Use the static solution for all your implementation, because it's easy and safe.
  2. If you can't or if you really look after optimization and you are sure your application will run with 1.6 Jvm, think about the volatil implementation.



Find bench Source & .jar here
You can monitor your bench using jconsole (you will find it into your jdk).
You need to lauch the jar with the following command :

java -Dcom.sun.management.jmxremote -jar bench.jar



More information about singleton here

lundi, mai 3 2010

Get started with java JVM memory (heap, stack, -xss -xms -xmx -xmn...)

JVM maximum size (32bit/64bit)

First of all, let's have a global look about JVM Memory.
If we are using a 32bit OS, then the jvm can't have more than 4GB of RAM
Over this 4Gig, Os have their own needs. For exemple Windows needs 2GB for its kernel usage. So it left 2GB for our JVM.
But JVM implementation needs to have a continous memory region for the heap.

Here is the explanation :

The reason we need a contiguous memory region for the heap is that we have a bunch of side data structures that are indexed by (scaled) offsets from the start of the heap. For example, we track object reference updates with a "card mark array" that has one byte for each 512 bytes of heap. When we store a reference in the heap we have to mark the corresponding byte in the card mark array. We right shift the destination address of the store and use that to index the card mark array. Fun addressing arithmetic games you can't do in Java that you get to (have to :-) play in C++.
Usually we don't have trouble getting modest contiguous regions (up to about 1.5GB on Windohs, up to about 3.8GB on Solaris. YMMV.). On Windohs, the problem is mostly that there are some libraries that get loaded before the JVM starts up that break up the address space. Using the /3GB switch won't rebase those libraries, so they are still a problem for us.
We know how to make chunked heaps, but there would be some overhead to using them. We have more requests for faster storage management than we do for larger heaps in the 32-bit JVM. If you really want large heaps, switch to the 64-bit JVM. We still need contiguous memory, but it's much easier to get in a 64-bit address space.


So usually, JVM heap cannot be bigger than 1.2-1.6BG.

JVM memory overview

We could resume Java process heap with the following schema :


Java Heap & GC tuning

Java "Heap" is a continous memory region where all Objects data will be stored (by data, we mean instance of class, primitive and references). It's a big part of the process heap.
It can be configured using the following parameters :
  • -Xmx : max heap size (ex: -Xmx1024)
  • -Xms : min heap size. Having -Xms = 1.8GB (32bit) can be bad, because you don't let memory for anything else.
  • -Xmn : the size of the heap for the young generation
    Young generation represents all the objects which have a short life of time. Young generation objects are in a specific location into the heap, where the garbage collector will pass often. All new objects are created into the young generation region (called "eden"). When an object survive is still "alive" after more than 2-3 gc cleaning, then it will be swap has an "old generation" : they are "survivor" .
    Good size is 33%
  • -XX:NewRatio : the same as Wmn, but using a % (dynamic fs static -Xmn option). -XX:NewRatio=3 means that the ratio between the old and young generation is 1:3
  • -XX:NewSize - Size of the young generation at JVM init. Calculated automatically if you specify -XX:NewRatio
  • -XX:MaxNewSize - The largest size the young generation can grow to (unlimited if this value is not specified at command line)
  • -XX:SurvivorRatio : "old generation" called tenured generation, ratio, in %. For example, -XX:SurvivorRatio=6 sets the ratio between each survivor space and eden to be 1:6 (eden is where new objects are created)
  • -XX:MinHeapFreeRatio: default is 40%. JVM will allocate memory to always have as minimum 40% of free memory. When -Xmx = -Xms, it's useless.
  • -XX:MaxHeapFreeRatio: default is 70%. The same as Min, to avoid unecessary memory allocation.

More informationabout Sun HotSpot GC tuning here and general gc tuning here
At this point, we get a problem. In a multithread application, object can be created at the same time. Thread could try to write data into the same heap location at the same time.
To avoid this problem, we allow each thread to have a private piece into the eden space.
jdk 1.5> uses dynamic sizing algorithm, specific for each thread. (otherwise, parameter such as -XX:UseTLAB or -XX:TLABSize can be used to tune this parameter)

And EveryThing Else...

Every thing else is...
  • Permanent Space : It's the third part of the memory. Here are stored classes, methods etc.
    • -XX:PermSize: initial value
    • -XX:MaxPermSize: max value
  • Code generation : Converted byte code into native code. Shouldn't cause troubles.
  • Socket Buffer (contains the 2 buffers for each sockets: receive/send)
  • Thread Stacks: Each thread has its own stack. It makes possible to get your methods thread-safe.
    • -Xss: change the space of a thread stack. 2048 could be a write value. It can cause a java.lang.stackOverFlow Error
    • If you get a "java.lang.OutOfMemoryError : unable to create new native Thread, you can decrease -Xss or decrease the java heap using -Xmx/-Xms (to increase the thread stack space)
  • Direct memory space (ability to let Java developers map memory outside the Java Object Heap. Can be adjusted using -XX:MaxDirectMemory=
  • JNI Code, if you use it
  • Garbage Collection (the GC has its own thread/informations)

A nice gotcha...

Major collection don't run until tenured is full.
This mean that using -Xmx1024, current heap could be 750MB with 500MB of "dead" object. If the JVM is idle, it could stay like that during a very long time => wasting 500MB or RAM for an idle JVM !

More informations about GC memory & monitoring here
You should also read this awesome slideshare from Filip Hanik, Covalent (gotcha, etc... most of this informations come from this slides)

mardi, mai 19 2009

AvriChat : Real time Chat Javascript Comet and Java Server

AvriChat (in development):

related topics :

AvriChat is a real time chat based on mootools.
Server side is based on Java.
Server truly push data to each client (check with firebug !)

AvriChat is based on Moo-Comet and Uvumi Scrollbarr.
(you can use it without Uvumi Scrollbarr)

AvriChat is designed to be used in Lord Of Castle and other projects with Comet.
I will add an Irc connexion very soon.

  • MultiChannel
  • Wips/answer
  • Save x last messages on the channel
  • Admin options (almost done)
  • whois
  • ...

Konwn issue :

  • Little bug with scrollBarr

Download AvriChat

download uncompressed chat sources
download uncompressed avriComet + avriCometd source (cometd wrapper for mootools)
Java source will be released soon.

Exemple :

demo :

Syntax :

  1. // The most easy way :
  2. //we first define the chat
  3. chat = new AvriChat('avriMessage', 'avriMonitor');
  4. //avriMessage : id of the input where the user will type the messages
  5. //avriMonitor : id of the div where the input will be displayed
  6.  
  7. //let's connect !
  8. chat.connect($('name').value, $('password').value);

Options :

  • keepAliveTimer: 20000, //keep-Alive request, in ms. keepAlive's Server has to be at least 2 time this value to avoid disconnections.
  • urlServer: 'http://server.magourex.info/chat/', //url of the server
  • defaultChannel: '', //name of the defaut channel. Server will set his defaut channel if empty
  • autoReconnection: true, //reconnect after a disconnection
  • maxAction: 5, //number of the action saved that you can get with up/down arrows
  • Lots Of Functions(), //all this function are invoked when a data is pushed by the server

//For exemple wisp(_wisp) is invoked to display a wisp on the Monitor.

Main Methods :

  • disconnect () //should be invoked when the user close the current page
  • fireEvent('EveryActionPushedByServer', function (_valueOfTheAction) //throw everytime the server push a data

ChangeLog :

v 0.4 - 01 Jully , 2009

  • Implementation of Bayeux protocol.

v 0.3 - 05 June , 2009

  • Htmlentities used to fix accent bug
  • works with presto.
  • Java code improved
  • Add the typing option which allows you to see what the other users are typing !
  • Use full JSON, parsing of the server's answer really improved (faster)

v 0.2 - 20 May , 2009

  • Avast web shield bug should be really fixed now !
  • I have changed the protocol from Java to javascript. It now use xml and Json inside the xml <data></data>, because Json doesn't fit with so well with this application.

v 0.1 - 19 May , 2009

  • first release