Hudson 3.1 has some performance optimizations that greatly reduces its memory footprint. Prior to this Hudson used to always hold
the entire data model (all jobs and all builds) in memory which affected scalability. Some installations configured heap sizes in excess of 1GB to counteract this. Hudson 3.1.x maintains an MRU cache and only loads jobs and builds as they are required. Because of
the inability to change existing APIs and be backward compatible with plugins, there were limits to how far we could go with this approach.
Memory optimizations almost always come with a related cost, in this case its additional I/O that has to be performed to load data on request. On a small site that has frequent traffic, this is usually not noticeable since
the MRU cache will usually hold on to all
the data. A large site with infrequent traffic might experience some delays when
the first request hits
the server after a long gap. If you have a large heap and are able to allocate more memory,
the cache settings can be adjusted to take advantage of this and even go back to pre-3.1 behavior.
All
the cache settings can be passed as options to
the JVM container (Tomcat or
the default Jetty container) using
the -D option. There are two caches, independant of each other, one for Jobs and
the other for Builds.
For
the jobs cache:
hudson.jobs.cache.evict_in_seconds ( default=60 )
Seconds from last access (could be because of a servlet request or a background cron thread) a job should be purged from
the cache.
Set this to 0 to never purge based on time.
hudson.jobs.cache.initial_capacity ( default=1024 )
Initial number of jobs
the cache can accomodate. Setting this to
the number of jobs you typically display on your Hudson landing page or home page will speed up consecutive access to that page. If
the default is too large you may consider downsizing and using that memory for
the Builds cache instead.
hudson.jobs.cache.max_entries ( default=1024)
Maximum number of jobs in
the cache.
The default is large enough for most installations, but if you find I/O activity when always accessing
the hudson home page you might consider increasing this, but first verify if
the I/O is caused by frequent eviction (see above), rather than by
the cache not being large enough.
For
the builds cache:
The builds cache is used to store Build objects as they are read from storage. Typically this happens when a user drills down into
the details of a particular Job from
the hudson hom epage.
The cache is shared among builds for different jobs since in most installations all jobs are not accessed with
the same frequency, so a per-job builds cache would be a waste of memory.
hudson.job.builds.cache.evict_in_seconds ( default=60 )
Same as
the equivalent Job cache, applied to Build.
hudson.job.builds.cache.initial_capacity" ( default=512 )
Same as equivalent Job cache setting. Note
the smaller initial size. If your site stores a large number of builds and has frequent access to more builds you might consider bumping this up.
hudson.job.builds.cache.max_entries ( default=10240 )
The default max is large enough for most installations,
the builds cache has bigger sized objects, so be careful about increasing
the upper limit on this. See section on monitoring below.
Sample usage:
java -jar hudson-war-3.1.2-SNAPSHOT.war -Dhudson.jobs.cache.evict_in_seconds=300 \
-Dhudson.job.builds.cache.evict_in_seconds=300
Monitoring cache usage
The 'jmap' tool that comes with
the JDK can be used to monitor cache performance in an indirect way by looking at
the number of Job and Build objects in each cache.
Find
the PID of
the hudson instance and run
$ jmap -histo:live <pid | grep 'hudson.model.*Lazy.*Key$'
Here's a sample output:
num #instances #bytes class name
523: 28 896 hudson.model.RunMap$LazyRunValue$Key
1200: 3 96 hudson.model.LazyTopLevelItem$Key
These are
the keys to
the Jobs (LazyTopLevelItem$Key) and Builds (RunMap$LazyRunValue$Key) in
the caches, so counting
the number of keys is a good indicator of
the number of items in
the cache at any given moment.
The size in bytes can be ignored, they are just
the size of
the keys, not
the actual sizes of
the objects they hold. Those sizes can only be obtained with a profiler. With
the output
above we can conclude that there are 3 jobs and 28 builds in memory.
The 28 builds can all be from 1 job or all 3 jobs. Over time on an idle system, these should get evicted and memory cache should be empty. In practice, because of background cron threads and triggers, jobs rarely fall down to zero. Access of a job or a build by a cron thread resets
the eviction timer.
