Page sequences and memory usage

Anonymous wrote:
How does XSLfast handle large Document (especially does XSLfast generate
multiple page sequences automatically?)

I have to create an adress book, a table of 25000 customer with their
adress and telephone number, in one PDF with arbitrary grouping.

For this use case, FOP has large memory footprint if the XSL-FO
generation is not aware of the data volume.

Do you have information about this ?

XSLfast generates an XSL file, with many pages in every page sequence automatically. However the resulting XSL-FO file can become very large and FOP needs hundreds of megabytes of RAM to generate the PDF.

Sometimes FOP might need more memory and produces an "out of memory" error that gets written into the xslfast.log file.

You can configure XSLfast to give FOP more memory in the following way:

In the same folder as the xslfast.exe file (e.g. <XSLfast folder>/bin), create an xslfast.cfg file with the following contents:

This file configures XSLfast to use 1.2GB of RAM for XSLfast itself and also for FOP.

For 32 bit Java and 32 bit Windows, you may have trouble to get more memory than 1.2 - 1.5GB even you have physically more than 2GB RAM installed.

In 64 bit Environments and with the 64 bit Java 1.6 JRE/JDK, XSLfast can address and use more memory for itself and for FOP.

Best Regards,
Stephan Albers
XSLfast Team

Hello,

Thank you for your quick response !

If I have to embbed the XSL generated by XSLFast in an application server 1,2GB of heap space for 1 report should be explosive !!

Is this "problem" of memory footprint is inherent to FOP ? Do you have advice on other commercial FO engine like XEP or Antenna House Formatter ?

Best Regards,

Grégoire Rolland

This message was edited 3 times. Last update was at 04/09/2009 08:05:09

Unfortunately, a lot of memory is consumed by FOP.

Check the difference for yourself. Launch XSLfast, open your layout with a large XML file and check how much memory is used. Then run FOP and take another look at the memory consumption.

You can use the following heuristics for the memory usage when creating large PDF-files with FOP: multiply your
XSL-FO file size in bytes with 10; that gives you an idea about the memory that FOP needs to render the file.

FOP does the rendering process completely in memory and does not care about the memory consumption.

Other (commercial) renderers like XEP from RenderX or XSLFormatter from Antennahouse can decrease his multiplier to 2 or 3 depending
on the XSL-FO content, especially if you have a lot of images to be printed.

However, when you render a large file with eg. 25.000 customers, this is something where you should always expect that a lot of memory is needed.

In production environments, if you need to print very large PDF files, especially in a web environment, we recommend to use an architecture with a queue of requests and limited worker threads or processes to generate the PDF files asynchronously. Once the PDF is created, you can send an notification about it or simply indicate the status by showing the queue.

You could implement that like this:

a) Instead of generating PDF immediately, collect all necessary files into a queue (eg. by copying the contents in an incoming folder)

b) With separate threads / processes, check the incoming folder periodically

c) If a new request arrived, move the file(s) out of the incoming folder, and do the formatting via FOP

d) When FOP finished the request, move the PDF to an output folder, and return to (b).