xsltproc killed, out of memory

Posted by David Parks on Ask Ubuntu See other posts from Ask Ubuntu or by David Parks
Published on 2012-10-20T02:43:48Z Indexed on 2012/10/20 5:22 UTC
Read the original article Hit count: 370

Filed under:
|
|

I'm trying to split up a 13GB xml file into small ~50MB xml files with this XSLT style sheet.

But this process kills xsltproc after I see it taking up over 1.7GB of memory (that's the total on the system).

Is there any way to deal with huge XML files with xsltproc? Can I change my style sheet? Or should I use a different processor? Or am I just S.O.L.?

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
                xmlns:exsl="http://exslt.org/common"
                extension-element-prefixes="exsl"
                xmlns:fn="http://www.w3.org/2005/xpath-functions">
<xsl:output method="xml" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:param name="block-size" select="75000"/>

<xsl:template match="/">
        <xsl:copy>
                <xsl:apply-templates select="mysqldump/database/table_data/row[position() mod $block-size = 1]" />
        </xsl:copy>
</xsl:template>

<xsl:template match="row">
        <exsl:document href="chunk-{position()}.xml">
        <add>
                <xsl:for-each select=". | following-sibling::row[position() &lt; $block-size]" >
                <doc>
                        <xsl:for-each select="field">
                        <field>
                                <xsl:attribute name="name"><xsl:value-of select="./@name"/></xsl:attribute>
                                <xsl:value-of select="."/>
                        </field>
                        <xsl:text>&#xa;</xsl:text>
                        </xsl:for-each>
                </doc>
                </xsl:for-each>
        </add>
        </exsl:document>
</xsl:template>

© Ask Ubuntu or respective owner

Related posts about Xml

Related posts about xsltproc