Too many open files in one of my java routine.
- by Irfan Zulfiqar
I have a multithreaded code that has to generated a set of objects and write them to a file. When I run it I sometime get "Too many open files" message in Exception. I have checked the code to make sure that all the file streams are being closed properly. Here is the stack trace.
When I do ulimit -a, open files allowed is set to 1024. We think increasing this number is not a viable option / solution.
[java] java.io.FileNotFoundException: /export/event_1_0.dtd (Too many open files)
[java] at java.io.FileInputStream.open(Native Method)
[java] at java.io.FileInputStream.<init>(FileInputStream.java:106)
[java] at java.io.FileInputStream.<init>(FileInputStream.java:66)
[java] at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
[java] at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
[java] at java.net.URL.openStream(URL.java:1010)
Now what we have identified so far by looking closely at the list of open files is that the VM is opening same class file multiple times.
/export/BaseEvent.class 236
/export/EventType1BaseEvent.class 60
/export/EventType2BaseEvent.class 48
/export/EventType2.class 30
/export/EventType1.class 14
Where BaseEvent is partent of all the classes and EventType1 ant EventType2 inherits EventType1BaseEvent and EventType2BaseEvent respectively. Why would a class loader load the same class file 200+ times. It seems it is opening up the base class as many time it create any child instance.
Is this normal? Can it be handler any other way apart from increasing the number of open files?
