JVM (embarrasingly) parallel processing libraries/tools

Posted by Winterstream on Stack Overflow See other posts from Stack Overflow or by Winterstream
Published on 2010-03-24T15:51:48Z Indexed on 2010/03/24 15:53 UTC
Read the original article Hit count: 495

Filed under:

jvm

|

java

|

parallel-processing

|

embarrassingly-parallel

|

parallel-programming

I am looking for something that will make it easy to run (correctly coded) embarrassingly parallel JVM code on a cluster (so that I can use Clojure + Incanter).

I have used Parallel Python in the past to do this. We have a new PBS cluster and our admin will soon set up IPython nodes that use PBS as the backend. Both of these systems make it almost a no-brainer to run certain types of code in a cluster.

I made the mistake of using Hadoop in the past (Hadoop is just not suited to the kind of data that I use) - the latency made even small runs execute for 1-2 minutes.

Is JPPF or Gridgain better for what I need? Does anyone here have any experience with either? Is there anything else you can recommend?

© Stack Overflow or respective owner

Related posts about jvm

Is there a way to update all Java related alternatives?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Is there a way to quickly switch over all the Java related alternatives using update-alternatives? For instance, if want to switch Java over to 7, I run sudo update-alternatives --config java and select the Java 7 OpenJdk. But if I run update-alternatives --get-selections | grep java I get the following… >>> More
?Pick-Up??????????? JVM ?JRockit????????·??????|WebLogic Channel|??????

as seen on Oracle Blogs - Search for 'Oracle Blogs'
??Oracle JRockit JVM?????????????????????·?????????????????????????JVM??????? ???????????????????????????????????????????????????????????????????????? ??????? ?????????Oracle JRockit???????·??????????????????????????????????????????????????????????? Oracle JRockit JDK???????????????????… >>> More
how can a servlet in one jvm communicate with servlet in another jvm

as seen on Stack Overflow - Search for 'Stack Overflow'
can this be solved simply by specifying the complete url of the servlet in the RequestDispatcher? >>> More
?????????????JVM??????|WebLogic Channel|??????

as seen on Oracle Blogs - Search for 'Oracle Blogs'
JRockit JVM???????????????????????????????????????????????????????????????????????????????????????????? ?JVM???????????????? ?JVM??????????? ???????????????????? ????????? ?JVM??????????????JVM??????????????… >>> More
Apache DS fails to list users

as seen on Stack Overflow - Search for 'Stack Overflow'
Apache ds fails to list the users INFO | jvm 1 | 2012/03/28 15:54:04 | java.lang.Error: ERR_546 CRITICAL: page header magic for block 59 not OK 0 INFO | jvm 1 | 2012/03/28 15:54:04 | at jdbm.recman.PageHeader.(PageHeader.java:95) INFO | jvm 1 | 2012/03/28 15:54:04 | at… >>> More

Related posts about java

Tomcat 6: Access Control Exception?

as seen on Server Fault - Search for 'Server Fault'
I'm trying to setup a tomcat6 server, and I'm trying to match another setup someone else established. However, my deployment (default Ubuntu install) uses a policy.d/ directory structure, and the established server just uses a catalina.policy file. I've tried setting every entry in policy.d to match… >>> More
Problem in creation MDB Queue connection at Jboss StartUp

as seen on Stack Overflow - Search for 'Stack Overflow'
I am not able to create a Queue connection in JBOSS4.2.3GA Version & Java1.5, as I am using MDB as per the below details. I am putting this MDB in a jar file(named utsJar.jar) and copied it in deploy folder of JBOSS, In the test env. this MDB works well but in another env. [ env settings and… >>> More
failing to establish connection between Postgres db and gwt

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am using Postgres and gwt 2.0 for one of my applications. I am facing problem connecting to the database. When I try to connect it gives "ClassNotFoundException". Here is what I get when I try to connect to database: java.lang.ClassNotFoundException: org.postgresql.Driver at java.net… >>> More
failing to establish connection between postgre db and gwt

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, For i am using postgre and gwt 2.0 for one of my applications. I am facing problem connecting to the database. When i try to connect it gives "ClassNotFoundException". Here is what i get when i try to connect to database: java.lang.ClassNotFoundException: org.postgresql.Driver at java.net… >>> More
Migration and deployement problems JBoss 4.2.2.GA to JBoss 6.0.0.M2

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I'm trying to migrate an application running on JBoss 4.2.2.GA to JBoss 6.0.0.M2 I give you some log to explain my problem : boot.log : 2010-03-16 09:59:29,406 ERROR [org.jboss.system.server.profileservice.ProfileServiceBootstrap] (Thread-2) Failed to load profile: Summary of incomplete deployments… >>> More