Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

Why is my text printed from InDesign (a) fat (b) outlined (c) fuzzy (d) or all of the above?

This article describes the Adobe Support Database Text Is Rasterized When You Print to a RIP from InDesign (2.0 on Windows or Mac OS)

In this example, you can see there has been a Photoshop file placed into a layout:

The Photoshop file on the red-marked layer (named: “photoshop file”) has been masked out of a background image, and saved as a .PSD . A text wrap has also been applied to this Photoshop files alpha channel (or transparency) causing the text in the yellow layer (named: “body text”) to wrap. Nothing too strange about this, however when printed to Postscript and Distilled, the following occurs:

This is a screen dump of the PDF generated from the InDesign 2.0 file above (Print to Postscript as CMYK, using the [High Resolution] Transparency Flattener Style. Distilled using the [Press] .joboption in Distiller 5.0.5)

As you can see, around the marker “A”, the text looks “fatter” and “fuzzier” than the text next to the marker “B”. This is the result of the transparency flattener. (NB: in Acrobat, in Edit>Preferences, Display if you turn on the “Smooth Line Art” option, this fuzziness goes away.)

Lets zoom into the area where the Photoshop file and the body text overlap. You can see in the above image that the red outline of the image overlaps certain lines in the underlying body text. In this instance, the Transparency Flattener has decided to covert the all the text to outlines in the lines that run underneath the image.

The effect we are seeing here is the Transparency Flattener in action. In Postscript, there is no way to have a semi-transparent image (the masked portion of the car) blend into type. Therefore, the flattener converts the relevant text to outlines and “clips” into the outline shape any image information that is required to generate output. The important end goal is to generate output in print that matches the designers intent.

To an average observer, at high resolutions (I have examples at 2400/133lpi Computer-to-Plate output) — its difficult for the naked eye to pick “outlined” vs “normal” type with serif text at low point sizes.

How do you solve the problem?

There are two possible solutions to this problem. One key point I would like to make before I continue is that you must choose one path or the other for the whole job.

Choice 1: Convert All Text to Outlines.

InDesign 2.0, Edit>Transparency Flattener Styles… Create a New Transparency Flattener style that turns on the “Force Text to Outlines” option.

Now when printing using this Flattener Style to the Distiller (ie: same process as above), the end result will look like:

The result is that all the text in the document is converted to outlines. When you compare a page printed (at 2400 dpi/133 lpi) with text converted to outlines side by side with a page where the text is normal, the difference is just noticable to the naked eye.

If you use this flattener style consistently throughout the job, the result will be that all the text looks consistent. The downside is that the text is no longer text – it’s paths – unsearchable and to a trained eye slightly fatter.

Choice 2: Change Layer Ordering

This is my preferred option, and when designing documents in InDesign its best to follow a process where all body text in the topmost layer.

In the InDesign document, I am going to change the order of the layers so the body text sits above the image:

In the above example, you can see that the “photoshop file” layer is underneath the “body text” layer.

QuarkXPress Users: don’t panic! As you would realise, in Xpress, your text wrap is based on the positioning of objects in layers. Images above text pushes the text out of the way: creating text wrap. Not so in InDesign. Text wrap in InDesign is object-to-object based. It doesn’t matter that the image is underneath the text, it will still cause the text above to wrap around.

Prepress operators: don’t panic! Changing layer ordering like this will not cause InDesign 2.0 to re-wrap the text.

InDesign CS2 Update: Note in InDesign CS2, there is a Preference (in the composition area) that effects text flow, too. The setting “Text Wrap Only Effects Text Underneath” may effect how text flows around objects underneath.

What is the result?

In this final result, you can see that the text has not been converted to outlines. This example was printed from InDesign 2.0 using the standard [High Resolution] flattener style.

Use Windows Media Player 9.0

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

Adobe InDesign 2.0 has a feature permitting direct export of Adobe PDF. There have been some designers who have been confused about which version of PDF to send, and why their printers/receivers cannot output a quality result. The idea of this note is to clarify some of the workflow points, and ensure high quality output.

Rather than creating (or printing) to Postscript and using the Acrobat Distiller to make a PDF, the direct export of PDF uses a piece of technology called the PDFLibrary to construct the PDF.

This PDF is no lesser PDF than one created by the Postscript>Distiller method. The differences are noted here: InDesign 2.0: Export or Distill PDFs?

In InDesign 2.0, there are two major revisions of PDF you can export: Acrobat 4.0 (equal to PDF 1.3) and Acrobat 5.0 (equal to PDF 1.4). With each release of Acrobat, Adobe has been adding features to the file format. These features are exploited by InDesign when direct exporting. The graphics operators used in applications such as QuarkXpress and InDesign 1.5 are simple, and can be expressed in PDF 1.3.

Viewing these PDF 1.4 files inside the latest Reader you should not notice any major differences. Inside, however, there is a whole world of difference that dictate how you should output the files.

Please note: not all Acrobat 5.0/PDF 1.4 files you receive will have transparency. With Distiller 5 or 6, using a Print to Postscript and Distill workflow, it is possible to create a PDF1.4 that does not contain transparency. To check for transparency, run the PDF through Pitstop or Acrobat 6.0’s Preflighting tools.

When exporting as an Acrobat 4.0/PDF 1.3, all of the transparency inside the InDesign file is flattened using the Transparency Flattener. This PDF is openable in Acrobat 4.05 or later, and will print/RIP on any Postscript Level 2 or later device.

Exporting as an Acrobat 5.0/PDF 1.4 results in a very different PDF. The transparency remains unflattened. This relies on another process flattening the transparency.

How can you tell?

Open the PDF in Acrobat, and go to File>Document>Document Summary

In the above screen dump there are three key pieces of information:

• Application is InDesign 2.0.x. High likelihood of transparency. Also look for Illustrator 9.0.x or 10.0.x
• PDF Producer is Adobe PDF Library 5.0, indicating a direct export of PDF. High likelihood of transparency/
• PDF version is 1.4, also an indicator of transparency will be contained in the PDF.

How do you produce a quality print result?

Option 1:

Place PDF into InDesign 2.0, and print to Postscript. This will use InDesign’s Transparency Flattener to create a Postscript Level 2/3 file.

Placing multiple-page PDFs into InDesign 2.0 can be time consuming. There are two scripts here that may assist: PDF multipage import
(Win), PDF multipage import (MacOS X) and PDF multipage import (MacOS)

From InDesign 2.0, print the document containing the PDFs to your output device. You can also use this method to Print to Postscript and re-Distill the PDF, or using InDesign 2.0: Printing Output Choices and Flattener Tricks (including force Greyscale export!), change the colour space at output time.

Option 2:

Print from Acrobat 6.0. Why not 5.0? The Transparency Flattener in Acrobat 5.0 and Illustrator 9.0 is “an older generation” – and may not produce the same quality result as InDesign 2.0.

The above illustrates what appears under the Advanced button in Acrobat 6.0’s Print… dialog box. Here, you can set the Transparency Flattener settings in a similar way to InDesign 2.0.

More information about Acrobat 6.0 is here: Acrobat 6.0 Professional: Graphics, Print, Prepress Overview

Option 3:

RIP the PDF on a device that truly supports PDF 1.4. Devices such as Creo Prinergy 2.1 support PDF 1.4 files and flatten the InDesign created transparency as a part of the normalisation process. Other RIPs such as the Fujifilm Celebrant Extreme RIP support PDF 1.4 in a similar way.

Some device manufacturers claim they support Acrobat 5.0/PDF 1.4 files. This may be the case, but the key question is “does it flatten transparency in the RIP?” Some devices will discard the transparency, resulting in a print job that is vastly different to the InDesign file.

What Not to Do:

Many customers email me asking about placing InDesign EPS files into QuarkXpress 4.x or 5 , and using tools like Imposition (an Xpress plugin) to impose InDesign jobs. There are many reasons why this is an outdated workflow. Ignoring InDesign for a moment, what happens when you receive a QuarkXpress 6.0 file? Life is going to get difficult very quickly. Time to move to true PDF based imposition environment. Squeezing all the PDFs you receive back through QuarkXpress is just asking for trouble.

• Do Not Open in Acrobat 5.0, save as EPS and place into QuarkXpress, any version. This is largely the same as Printing as Postscript from Acrobat 5.0.
• Do Not Place the PDF into QuarkXpress as a PDF. QuarkXpress’s engine for opening and placing PDFs only works with Acrobat 3.0/PDF 1.2 files, nothing later. For instance, spot colours that may be in the PDF are not recognised.
• Do Not Open the PDF in Illustrator, any version, and attempt to print. Illustrator is not a universal PDF editing application.
• Do Not Place the PDF in InDesign 1.5. This version of InDesign does not have a transparency flattener, and therefore a quality result cannot be obtained.
• Do Not Open and convert the PDF in Photoshop, any version. Photoshop will rasterise the PDF into pixels. Even at 300dpi, this will result in a very large file and any vector graphics and text will “go fuzzy” at print time as it is passed through a halftone screen at the RIP-end.

If you are sending directly exported Acrobat 5.0 PDFs from InDesign 2.0, send this web page to your printer.

Thanks to Marcus for ensuring I got off my bum and wrote this article.

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

aka: True PDF Workflow with InDesign 2.0.x

aka: Its time to stop using legacy EPS!

A common workflow in the Print and Prepress industry is exporting a page from applications such as PageMaker and QuarkXpress and placing these back into the application and printing.

For example 1: You have created a page for publication, and you also need to create a larger, poster size version.

Generically, you have a page in InDesign document “A” that needs to be incorporated into InDesign document “B”

---

Traditionally, the process has been: Export the page as an EPS (Encapsulated Postscript) and place this back into a picture box in the layout application.

With the advent of InDesign 2.0.x, this workflow is outdated and potentially dangerous from a quality perspective.

The New, Modern workflow:

Step 1:

Export the Page from InDesign 2.0.x document “A” as a PDF, specifically a Acrobat 5.0 compatible PDF.

Step 2:

Import the PDF into InDesign 2.0.x document “B”. In this example, the PDF has been placed into another document. Here, the Links palette shows the source PDF exported from step 1 above.

Why use PDF?

1. PDFs are smaller than EPS files, and take less processing overhead to place in InDesign (no need to parse the Postscript)
2. Exporting Acrobat 5.0 PDFs from InDesign 2.0.x is extremely quick as there is no need to flatten the transparency contained in the InDesign document.
3. Any transparency in the InDesign 2.0.x document “A” remains unflattened in the exported Acrobat 5.0 PDFs. Once placed back into InDesign 2.0.x, it will recognise this and flatten the transparency inside the PDF before exporting as Acrobat 4.0 PDF or printing to Postscript
4. If expanding the size of the placed PDF, if flattened, there is the chance that the flattened elements lose quality – especially at the places where elements stitch together.

Making Compact PDFs Using Acrobat Distiller

Portable Document Format (PDF) files are a defacto standard for publishing complex documents on web sites. Forms, visually rich brochures and long documents are presented in PDF form. Due to the ease of “printing” to a PDF generating Printer such as Distiller or PDFWriter rather than totally regenerating the page in HTML or SWF — and also not to mention the cost of the Reader (free!) — placing PDFs

The web, however, is not a fast, high bandwidth unlimited download environment many of us experience at work. Sometimes even at work our access to the internet is throttled by slow modems, slow proxy servers – or god forbid, both.

Whilst making PDF is relatively easy, making a small, compact and efficient PDF takes a little setup and thought.

This article is not going to describe how to make extreme PDFs. This would require a long, and indepth study of PDF as a file format and squeezing every last byte, redundant space and element from the file.

PDFWriter or Distiller?

Acrobat 3.0 and 4.0 installed a printer called ‘PDFWriter’. The PDFWriter takes GDI (Windows) or Quickdraw (MacOS 8/9) printer calls and converted these into a PDF file. Quickdraw and GDI is not as accurate as Postscript with positioning.

Using a baseline file containing one graph (vector graphics), one image and text using Times New Roman (Truetype) – we can compare the file size of Distiller vs. PDFWriter. In both instances, I have used JPEG-medium settings for image compression.

Source Microsoft Word 2000 Document: 221184 bytes
PDFWriter: 92656 bytes
Distiller: 20676 bytes

Why is the PDFWriter’s PDFs larger? Isn’t it the same file? Well, yes – but the path taken is different. You can travel from Perth to Sydney the quickest way – or – via Darwin. Using either path you will arrive at the destination. However, the distance you travel is very different. PDFWriter is no different: we are asking the application to generate a representation of the page using the operating system’s internal graphics; perfectly tuned for screen output – but no so for printed output.

Looking “inside” the PDF using the Acrobat 5.0 “PDF Consultant>Audit Space Usage” reveals the differences when comparing the two files:

Distiller: Fonts 9.5% [1966 bytes], Images 68% [14053 bytes]

Above: Tools>PDF Consultant>Audit Space Usage Report for Distiller created PDF

PDFWriter: Fonts 4.5% [4162 bytes], Images 92.3% [85491 bytes]

Above: Tools>PDF Consultant>Audit Space Usage Report for PDFWriter created PDF

Why does the Distiller produce a smaller PDF? Postscript is targetted at printed output from low end laser and inkjet printers to very high quality printed output.

The Distiller, at its core, has a Postscript engine. It takes the Postscript generated by the application and interprets it; leaving the resulting images, text and graphics in an internal object-based state. This state is then easily represented in PDF.

I recommend we use the Distiller as our preferred process/path for creating a compact PDF.

The Creating Application

– Print in RGB. Firstly, images in the CMYK colour space takes extra disk space, contain less colour information (technically: it has a smaller colour gamut) compared to RGB and the the Distiller will convert from CMYK to RGB anyway. So try to keep the source images and colour definitions in RGB.

– Reduce Image Use. As images take the most space in a PDF, use vector graphic elements whereever possible. This is especially the case with elements such as corporate logos: they are usually available in vector format, and are used many times in a PDF.

– Do not convert text to outlines. Applications such as Illustrator and Freehand have the ability to convert type to outlines. Usually, this is done to reduce the need for a font to be sent along with a document. When creating PDFs each time a character is used in the document there is a vector shape describing that glyph (character). If it is used again, the whole shape is described again – this may take many bytes to do. Just reusing the character and referencing a typeface will take a byte.

– Reduce font variation. Try to minimise the number of typefaces used in a document. For every font or typeface used, Acrobat may (depending on the settings) embed more font information. This increases the resulting file size.

– Use Zapf Dingbats as much as Possible. Widget style fonts such as Wingdings et al are ok, but each time they are used, the Distiller has to embed the characters into the PDF. If Zapf Dingbats has a similar character or mark, use that glyph instead. As Zapf Dingbats in installed along with Acrobat Reader, the local version of the font will be used.

Tuning The Distiller

Now we are using the Distiller as the tool to create compact PDFs, its time to understand the process it goes through to create the resulting PDF.

The creating application firstly creates Postscript. Applications that do not produce Postscript directly use the AdobePS printer driver to generate the Postscript.

The heart of the Distiller is a Postscript engine, but the instructions it uses are contained in a file called the ‘joboptions’ file. It instructs the Distiller as to what fonts to embed and how; as well as what compression to apply to images.

Included with Acrobat 4.0 and 5.0 is a joboptions file called “ScreenOptimized” or “Screen” (respectively). This applies a certain, standardised joboption settings the source Postscript and directly effect the resulting PDF. These settings are a good first start when creating compact PDFs.

We can however further tweak these settings to make even smaller, more compact PDFs.

Launch Acrobat Distiller, and go to Settings>Job Options menu.

We are going to look at each of these 5 tabs and choose the most appropriate options to reduce the size of our PDF.

The first panel that is displayed is the “General Settings”

Compatibility

Acrobat 3, 4 or 5? What we are really comparing is not the version of Acrobat, but the underlying version of the PDF file format. Each time Adobe releases a new version of Acrobat, there are underlying changes to the file format as it adds extra features to the application. For instance,

Acrobat 4.0 added the ability to embed Times, Helvetica, Courier and Symbol to a PDF. Acrobat 5.0 added the ability to have expressed transparency.

Acrobat 3 = PDF 1.2
Acrobat 4 = PDF 1.3
Acrobat 5 = PDF 1.4

The decision to use a particular version of Acrobat, and therefore new features added to the file format must be based on what you believe your readers have installed on their PCs. The Reader, whilst a free download from the Adobe web site, is not something that end users will install the first day it is release.

Personally, I’ve found it usually takes 6-12 months from a new release of Acrobat to the PDF format for it to become the ‘standard’ and therefore target for your compatibility when creating PDFs.

The good news is that Acrobat 5.0 Distiller can make Acrobat 3, 4 or 5 PDFs.

When using Acrobat 4 or 5 compatibility, the file gets marginally larger than Acrobat 3. Why is this so? The reason is that the Distiller is adding an colour management profile (ICC) to each image.

Unless you are being very particular about colour, there is no need to attach a colour profile to each image. This will further reduce the file size of our compact PDF.

There are other limitations with Acrobat 3.0 compatibility. There is a limitation to the number of pages (32768) and dimensions (45×45 inches).

Optimise for Fast Web View: The Distiller restructures the file to prepare for page-at-a-time downloading. It also compresses text and line art, overriding the setting in the Compression panel. This makes for faster access and viewing when downloading.

Thumbnails: Thumbnails are a small, miniature versions of the PDF’s pages included inside the document for the Thumbnails palette in Acrobat. Prior to Acrobat 5.0, the Acrobat Distiller had to create these Thumbnails or they had to be manually created inside Acrobat itself. In Acrobat 5.0, Thumbnails are dynamically created.

Therefore to keep our PDF small, its best to turn this off.

Compression

As images constitute a large ratio of the space taken in PDFs, its best we optimise the images as much as possible.

The first important setting is to set the ‘target’ dpi to 72. This is the dpi that most computer screens will experience the compact PDF. New in Distiller 5.0 is the ‘for images above’ — this forces all images to be downsampled to 72dpi.

The quality setting you choose will be dependant on the end quality you are willing to ‘live with’ in the compact PDF. Remember, we are looking at images and how the average end user will experience them.

In our baseline document, the three difference compressions result in the following file sizes:

Medium: 20876 bytes
Low: 16615 bytes
Minimum: 12677 bytes

See the difference between the three generated PDFs using Medium, Low and Minimum Compression: Link To Compression Comparison

Fonts

Before discussing fonts, we need to first look at how the Acrobat Reader renders type. The optimal way of displaying type if for the Acrobat Reader to use the inbuilt operating system’s type display mechanism. Apple and Microsoft spend many engineer-years of time tuning their operating systems to make the display of text as quick as possible.

In an environment where a PDF can go anywhere, you cannot rely on all the fonts (and versions of fonts) used in a document being everywhere in the world. Even relying on a fairly common font, Arial, is not recommended. What about Linux users – do they have the exact same version. What about different versions of Windows? What about different language versions of the same operating system?

This is where PDF first made its name: a universal format where the font issues are handled by the Reader. Fonts used on the creating computer are embedded into the PDF file as a part of the Distiller process — if the font permits embedding. (this is a ‘switch’ on the font, turned on or off by the maker/designer of the typeface)

There is no need to send your fonts along with the PDF – the Acrobat Reader is supplied with a collection of typefaces that represent the fonts supplied as standard with Postscript Level 1: Times, Helvetica, Courier, Symbol and Zapf Dingbats. If you use these fonts in your design, and as they are supplied with the Acrobat Reader, there is no need to embed them in a compact PDF. This is not recommended for PDFs for High Quality output as the versions of these fonts will be slightly different depending on the version of the Acrobat Reader the end user has; as well as differences related to platform.

Another smart feature of Acrobat since the first version is creating faux fonts. Faux means false in French (I think!) Acrobat Reader has a mechanism where it looks at typeface information embedded in the PDF by the Distiller that describes the features of the font (serif or sans serif, weight, descender details etc) and creates a false version of this typeface when displaying at the end user’s display. It will not look exactly the same, but it may suffice for the purposes of your compact PDF.

In our baseline Microsoft Word document, there are three fonts used. Times New Roman (TrueType), Arial (TrueType) in the graph and Imago ExtraBold (Headline). If you view the File>Document Info>Fonts you notice that these fonts are listed:

The important column to notice in the above screen dump is the ‘Actual Font’. It lists the actual font used in place of the original font. You will see that Imago ExtraBold lists “AdobeSansMM” — this indicates that the actual font being used to display is a Multiple Master font (MM) Sans Serif (without little bits on the end of the character shapes). What this means is that Acrobat has created a false (faux) version of this font to reduce file size. Not all fonts can be fauxed.

In contrast, Times New Roman and Arial list other typefaces: TimesNewRomanPSMT and ArialMT. This indicates that the original font is not embedded, and Acrobat is substituting the font to display it as accurately as possible.

The most appropriate option is to Subset embedded fonts, and never embedding the base-14 fonts. The result is that the Distiller:

– does not embed the base -14 fonts that are included with Acrobat Reader
– does not embed fonts that can be substitued with AdobeSerif/AdobeSansMM faux fonts
– does embed fonts that cannot be created

Color

Distiller, to create the smallest files, converts all colour to sRGB (Acrobat 4 or 5) or Calibrated RGB (Acrobat 3). To reduce the file size further, its best to set the ‘Working Spaces’ RGB and CMYK to ‘None’ This discards any ICC colour profiles from the images prior to embedding them into the PDF.

Advanced

The only option relevant to file size is the option called “ASCII format”. This option is a legacy of the bad old days of dial-in bulletin board systems and modems. Sometimes, the most appropriate way to transfer files was not via 8-bit communications but rather 7-bit ASCII style communications. This increases the file size by up to 12% when this option is turned on.

Individual Graphic Tuning

In some circumstances, you may need to individually tune the images in a PDF to reduce file size or increase the quality of certain elements.

Tools such as Quite A Box of Tricks or Enfocus Pitstop will permit individual optimisation of PDFs.

Beware and Be-aware Of

– Combining PDFs (taking the pages from one PDF and inserting them into another using Acrobat) will result in larger PDFs than combing the pages in the source document.

– Adding Form fields, buttons, Tags and Bookmarks to PDFs add to the size. Whilst the size of these added elements are much smaller than images that may be in a PDF, they do add to the size. Consider the audience of the PDF. If you expect visually impared people to view the PDF, tagging is a requirement for screen readers to correctly interpret and read the document. Long, text filled documents scream out for bookmarks for viewing. Visually rich documents for onscreen may require links created by Acrobat.

– Applications such as Adobe Illustrator 9, 10 and InDesign 2.0 can create bitmap images where you don’t first expect. In the process of creating a Postscript file where transparency exists in the source document, these applications may flatten the transparency into a bitmap, even where the source elements are vector graphics or text.

“Flattening” is a function of printing, and is executed to ensure the printed result matches the designer’s intent. As we know, bitmaps take more space than vector and text elements. Therefore, the file size may blow out.

– Some typefaces are large. Complex fonts containing more intricate paths will take more space than simple typefaces. If you use bold, semibold, italic variations to a typeface, this will force Distiller to further embed more font data.

– In Microsoft Word, and other applications, you can “scale up” images. The applications hold the image data at 100%, and at print time, scale the pixels up. With Acrobat and PDF, the image is always held at 100% after printing from the application. This will cause the PDF to potentially become larger than the original application document – as the PDF have more pixels.

How Low Can it Go?

Out of interest, how small can you make a PDF?

The simplest piece of Postscript that will render a PDF is the command ‘showpage’ Normally, the Postscript interpretter treats this as “take all that has been rendered to the page and generate output” In the instance of nothing on the page, you get, well, nothing.

Acrobat Distiller 5.0.5 generates a PDF of 2116 bytes in length from this simple Postscript command.

– half of this is populated with XMP (eXtensible Metadata Platform) packet containing information about the file.

Handcoding a PDF would result in a smaller file, but I’ll leave this as an excerise for the reader.

Obviously, a PDF with no contents is as useful as a non-alcoholic beverage at an australian barbeque.

If you are interested in a tool that will automatically enhance the size (that is, make smaller) – I suggest you look into PDFEnhancer

Nick Hodge is an Enthusiast Evangelist for Microsoft, Australia. A self-proclaimed Professional Geek, Nick is known for the online series “The Geek Stories” (http://thegeekstories.com/) and his personal blog (http://nickhodge.com) covering Australia, technology, learning, the web and other geek-related topics.

Prior to Microsoft, Nick spent 8 years at Adobe Systems; and 3 years at Apple Computer. He holds an MBA from LaTrobe Univeristy and a Post Graduate Diploma of Technology Management from Monash University.

When not working, Nick enjoys cutting code for his web site, http://www.nickhodge.com/ and driving one of his two MINI Cooper S’s (Many Pictures of Nicks MINI Cooper S).

---

Click Here: Nick Hodge Press Photo

Adobe Acrobat, Forms and Email; no custom CGI

Adobe Acrobat 5.0 permits the submission of form data via an email connection. However, the free Adobe Acrobat Reader does not permit the submission of form data to via an email message, directly out of the application. Or does it?

As I get this question about once a week, I decided to research and write a small article about how to implement this using Adobe Acrobat here.

There are two solutions: either code your own CGI script in a variety of languages: PERL, PHP, ASP, JSP etc (see: PDF Forms and Javascript), or alternatively use one of the most common pre-installed CGIs: Formmail.

Programming CGI is not a simple “Learn programming CGI in 24-hours” topic. There is a smarter way: by using a CGI that has been “previously cooked in the oven”. Most ISPs and web hosting organisations know that expecting every customer to be a code-jockey is a little steep, therefore they install some default CGIs. “Formmail.cgi” is a simple CGI that takes form data, and emails the result to a nominated email account.

What is Formmail?

From Matt Wright’s Formmail: FormMail is a generic HTML form to e-mail gateway that parses the results of any HTML form and sends them to the specified users. This script has many formatting and operational options, most of which can be specified within each form, meaning you don't need programming knowledge or multiple scripts for multiple forms.

As this is an extremely common CGI, you should be able to get your ISP or web server administrator to install it; or at least a CGI that replicates its functionality. Hostforweb is my third ISP/host since early 2002 and each of the previous two also had a Formmail CGI facility.

So how does CGI and HTML forms relate to Adobe PDF?

Normally, this CGI relies on HTML forms. But aren’t we talking about PDF Forms? With Acrobat, one of the submission formats for forms data is HTML encoded data being received by a CGI. All we need to do is ensure that we submit the PDF form data in a HTML format; and submit fields that the Formmail CGI recognises.

This also does not require any special configuration of email application on the client end. All they need is Acrobat Reader/Adobe Reader installed.

How to Create the Form in Acrobat

Let’s take the example form: a simple web site feedback form for this site. I’ll set up this the forms submission occur via CGI-Formmail.

Download the form here: Feedback Form. Open in Adobe/Acrobat Reader and complete the form.

This form is a simple feedback form. If you click on the Submit button, it will send me an email that looks something like this:

The creation of form fields in Acrobat is simple: just use the Form tool, creating text fields in the places you would like them to appear.

Behind the scenes on this form live 3 hidden fields. It is extremely important to name the fields with the required names for the CGI. These fields are sent to the CGI and are used to inform the CGI where to send the data, the subject on the email, who the form is sent from. In Formmail’s instance, its needs at least one field: recipient. In my example, I have added two extra fields: subject (this will be the subject of the email sent to me) and redirect (this is where the CGI will return the user once it has sent the email)

Below, I have clicked on the field “recipient” There are two important things I have configured on this field: firstly, the fact that it is “hidden” – you cannot see it when you are entering data into the form. The second change I have made is to set a default value: the recipient of the form data.

The Submit button has special options. When you create the button, in the Actions I have created a “mouse up, Submit Form” action as below:

When you edit the Action, you can nominate the URL. In my instance, the URL is http://thedigitalcottage.com/cgi-sys/FormMail.cgi. This URL WILL NOT WORK in your form – you will need to determine from your hosting provider the URL to their Formmail installation. There are some special anti-SPAM features in Formmail that does not permit the use of my CGI with your site. To have the PDF form submit the data in a format that Formmail can understand, its important that the Export Format is HTML.

One of the Anti-SPAM features of Formmail is restricting who can be the recipient of a particular form. You will need to consult with your ISP to determine what security settings they have applied to their Formmail installation to restrict who the data from the form can be sent to.

Once completed, you load the PDF form or email the form in the normal manner.

Now the final user of the form will be able to Submit data back, without the need of a custom CGI script.

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

Generating Composite, Trapped PDF from InDesign

Well created, Composite PDFs are the most commonly requested format for printers in this part of the world. So, how exactly do you generate a composite, trapped PDF?

InDesign 1.5.x and 2.0.x have support for Application Built-in trapping. The trapping engine inside InDesign matches many of the features of Adobe’s InRIP trapping engine; usually found in higher end RIPs from Creo, Heidelberg, Agfa etc.

Trapping is the process of changing the shapes of certain objects in a printed output to reflect the way the ink will run when printed. Each of the inks are laid down on the paper by different cylinders of the press. As the paper (substrate) are impressed by these cylinders, there might be a misregistration – where the inks don’t quite line up. In this instance, the plates contain extra overprinted areas of dots to cover up any unsightly gaps that may be left in the final output.

Highend PDF workflows today are bedevilled by the “Who is trapping this?” question. In controlled workflows, this is easy to coordinate. Prepress know which press a job is running on, and apply some visual checks on the job and may make trapping changes to the source file prior to output. However, when there is a hand-off to an unknown press and printing environment, the creator will not know the press conditions. Today’s digital workflows largely assume that the final RIP (imagesetter/platesetter) will trap the document.

In some workflows, this trust may not be enough. We need to make composite PDFs for onscreen viewing, simplicity and size; yet trap them so there are no ugly white patches at print time.

With InDesign 1.5 and 2.0, the Adobe trapping engine takes InDesign created object (text and vector objects) and traps them to placed images, text and vector shapes. InDesign does not trap placed PDF or EPS objects. These are assumed to be trapped. InRIP trapping solutions will trap the entire page stream. To use Adobe In-RIP Trapping, you must use In-RIP separations. Built-in trapping limits trap widths to 4 points, regardless of the value you enter for the trap widths. For larger trap widths, you will need to use Adobe In-RIP Trapping.

From QuarkXpress, the trapping information as setup by the Trap palette only comes into play when printing separations. With QuarkXpress 4.0 and above, overprinting and knockouts as set by the palette are retained in composite Postscript output. Saving Pages as EPS from QuarkXpress 3.32 and higher does result in overprints/knockouts being retained. (see QuarkXpress, PDF, Trapping and Overprint)

InDesign 1.5.x and 2.0 preserve knockout/overprint attributes in Composite Postscript output. As you would expect, Adobe’s Applicaton Built-In trapping works when printing separations.

Back to the topic at hand: A significant workflow difference with InDesign is that it can also apply Application Built-In traps when printing InRIP separations.

What are InRIP separations? Essentially, programs like InDesign send composite Postscript to the RIP with some extra commands telling the RIP to take a completed page and produce n-plates based on the colourants on the page. For simple process jobs, this would result in a page each for C, M, Y and K.

The implementation of the Postscript showpage and copypage operators have the capability of producing pages for each colourant. If the output device doesn’t have this colourant, then the alternativeSpace colourant is used; failing this, DeviceCMYK is used. There is a Postscript engine inside the Distiller, and it yields DeviceCMYK colourant output.

Acrobat Distiller has a Postscript engine inside: what does it do with InRIP separation-marked Postscript? Distiller 4.0x and 5.0.x ignore the operators for producing pages for each colourant, and produce a composite PDF. The colours stay as CMYK (or spot colours if used) if the Distiller options are left to “Leave Colour Unchanged”

Now InDesign 1.5.x and 2.0.x allow you to apply Application Built-in trapping to the composite Postscript when printing as InRIP separations. Yes, I know its a little mind bending! If you then send the resulting Postscript to Acrobat Distiller, the Distiller discards the commands to separate, but it does retain the extra “trapping” Postscript commands. The final PDF holds these trap commands, and they can be viewed in Acrobat 5.0 by turning on Overprint Preview.

The final outcome is a Composite, Trapped PDF.

Steps

1. Ensure that your Trapping settings are correct. Please consult the InDesign CS Printing Guide, the manual and your Prepress/Printer before assuming the defaults are correct.

2. Assign a Trap Style to the page. In this instance, I am using InDesign’s [Default] Trap style as viewed above.

3. Here is the File>Print dialog box. In Output, Color is set to In-RIP Separations, and Trapping is set to Application Built-In

4. To make it easier to use the same settings on the next document, you can save a Printer Style for later use.

5. This is the source document in InDesign 2.0

6. The final PDF inside of Acrobat 5.0, (available here: Example CompositeTrapped PDF)with Overprint Preview turned on. The darker areas around the type etc. display where overprinted strokes have been applied based on the inking requirements.

7. Using Quite Revealing from Quite Software, I can also reveal the overprinted strokes separately.

For more information about Trapping, and specifically the Adobe trapping engine:

Adobe Trapping Technology (white paper)

Adobe In-RIP Trapping Workflow

How to Trap Using Adobe Trapping Technologies

Thanks to Steve Amerige, Matt Phillips for their assistance with some of the finer details.

The database

There are three basic tables in the MySQL database. One contains content, and referenced by mcid. The second contains any binary content (images, PDFs) and is referenced by an miid The final contains sections referenced by msid, which are a mechanism for organising and classifying content and images.

Around these three tables live some utility tables that contain basic logging information (a simple page count per fragment) and caching. Each page, when first rendered, is cached to a separate table. This enables faster response on the second and subsequent hits. To ensure that only current pages are cached, there is a simple cachetree implemented such that when a change or deletion to a record is made, the cache is pruned. (the page cache is presently disabled)

These sections provide the underlying hierachical structure of the site. This structure is tightly bound to the navigation through the site. As this is in the database, it is very easy to change the underlying navigation structure without changing how individual pieces of content are referenced. Usually, the structure is defined as a folder hierachy on the server. As soon as you start moving the folders around, the references from one piece of content to another breaks – forcing a mass update of the site. External search engines that may have the links stored in their database get 404 style errors. In the mungenetengine, each content, image and section is referenced individually without relationship to its place in the navigation structure.

Each fragment of content or image belongs to a section. These sections are the hierarchy behind the site – so each section can belong to another section, and a particular section can have multiple child-sections. A section can only have one parent. As you move to another section or page within another section, the mungenetengine (renamed to mne.php) can generate a navigation fragment; which is a template driven hierarchy containing links to the parent section, sibling content pages or sections, and any children.

In fact, each content entry is just a fragment of html. A whole page may actually comprise of other referenced contents elements, each inserted into the final html that you seen in the browser. The process of building one page from another is a process called templating. Its not as simple as one page referencing another; this process is recursive – one page refers to another until the final page is built. The concept here is to only enter data only once into the database. Each content can insert other fragments (or images) into the html output.

There are different types of content fragments. For instance, one is an external link – these contents are resolved, and wrapped into an HREF style link.

Images are in fact binary objects. They are wrapped into the final html – except they refer back to the mungenetengine – which grabs the images out of the database and serves them up. html fragments that are generated automatically add the width/height, alts and hrefs normally associated with images – if they are in the database. PDFs and other non-image content can be linked to so the user can download them, or embedded into the HTML fragment.

SWF and SVG can also be stored in the database. When rendering out these elements, the engine generates object embed tags. The width and height are not read from either format as yet. Whilst it is possible to read the ‘twips’ from a SWF, it seems overly complex to do – just yet

The engine can also generate what are known as breadcrumbs (next and previous links) — this is easy as the structure and relationship between objects are known and map to the navigation

PHP, the server side scripting language, munges the data together from the MySQL database and serves it up. As at early February 2003 this code is about 1870 lines long. I am testing this on WindowsXP and transferring it here to a Linux box for production. I really only have a choice of three server side scripting systems (PHP, Perl or Python). Learning PHP has been a great experience.

The next stage is to transfer the experience here into using a better language that has more object orientation. I have chosen Python as the langauge to accomplish this task. How long this takes, only time will tell!