Category: mungenet

The database

There are three basic tables in the MySQL database. One contains content, and referenced by mcid. The second contains any binary content (images, PDFs) and is referenced by an miid The final contains sections referenced by msid, which are a mechanism for organising and classifying content and images.

Around these three tables live some utility tables that contain basic logging information (a simple page count per fragment) and caching. Each page, when first rendered, is cached to a separate table. This enables faster response on the second and subsequent hits. To ensure that only current pages are cached, there is a simple cachetree implemented such that when a change or deletion to a record is made, the cache is pruned. (the page cache is presently disabled)

These sections provide the underlying hierachical structure of the site. This structure is tightly bound to the navigation through the site. As this is in the database, it is very easy to change the underlying navigation structure without changing how individual pieces of content are referenced. Usually, the structure is defined as a folder hierachy on the server. As soon as you start moving the folders around, the references from one piece of content to another breaks – forcing a mass update of the site. External search engines that may have the links stored in their database get 404 style errors. In the mungenetengine, each content, image and section is referenced individually without relationship to its place in the navigation structure.

Each fragment of content or image belongs to a section. These sections are the hierarchy behind the site – so each section can belong to another section, and a particular section can have multiple child-sections. A section can only have one parent. As you move to another section or page within another section, the mungenetengine (renamed to mne.php) can generate a navigation fragment; which is a template driven hierarchy containing links to the parent section, sibling content pages or sections, and any children.

In fact, each content entry is just a fragment of html. A whole page may actually comprise of other referenced contents elements, each inserted into the final html that you seen in the browser. The process of building one page from another is a process called templating. Its not as simple as one page referencing another; this process is recursive – one page refers to another until the final page is built. The concept here is to only enter data only once into the database. Each content can insert other fragments (or images) into the html output.

There are different types of content fragments. For instance, one is an external link – these contents are resolved, and wrapped into an HREF style link.

Images are in fact binary objects. They are wrapped into the final html – except they refer back to the mungenetengine – which grabs the images out of the database and serves them up. html fragments that are generated automatically add the width/height, alts and hrefs normally associated with images – if they are in the database. PDFs and other non-image content can be linked to so the user can download them, or embedded into the HTML fragment.

SWF and SVG can also be stored in the database. When rendering out these elements, the engine generates object embed tags. The width and height are not read from either format as yet. Whilst it is possible to read the ‘twips’ from a SWF, it seems overly complex to do – just yet

The engine can also generate what are known as breadcrumbs (next and previous links) — this is easy as the structure and relationship between objects are known and map to the navigation

PHP, the server side scripting language, munges the data together from the MySQL database and serves it up. As at early February 2003 this code is about 1870 lines long. I am testing this on WindowsXP and transferring it here to a Linux box for production. I really only have a choice of three server side scripting systems (PHP, Perl or Python). Learning PHP has been a great experience.

The next stage is to transfer the experience here into using a better language that has more object orientation. I have chosen Python as the langauge to accomplish this task. How long this takes, only time will tell!

There is an RSS feed implemented from www.nickhodge.com

RSS 0.92 Feed
www.nickhodge.com RSS 2.0 Feed

Stephen Canning from Creative Consultants asked if there was a Windows version of the Add Tab to Cell Contents Script (MacOS .sit from adobeevangelists.com).

I found an hour spare, quickly relearnt Visual Basic, and managed to pop this out:

Add Tab to Beginning of Cell (zip downloadable)

Just download, unzip and place the .exe into your InDesign 2.0 Scripts folder.

Select Table Cells you would like to insert a Tab Character into, then run this by double clicking on the script in the Window>Scripts palette.

Tim Cole has been hassling me to do this for 12 months. Today he found me in a receptive, VB mode, so here it is.

Paste as Text Only (Windows zip downloadable)

Just download, unzip and place the .exe into your InDesign 2.0 Scripts folder.

Instead of using Edit>Paste, select the insertion point or textframe you wish to paste into, and using the Windows>Scripts palette, double click on this script. It will then paste as text only — no formatting, into relevant item.

For MacOS users, Tim has a MacOS version here: Paste as Text Only (MacOS .sit from adobeevangelists.com).

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

There are a variety of methods for taking a Photoshop file and placing to into InDesign. After over a year of InDesign 2.0 in production use, I thought it opportune to expose another workflow choice taken by these customers. It directly relates to the placement of files from Photoshop into InDesign 2.0 layouts.

Apart from supporting the traditional TIFF (.tif) file format, InDesign also permits the placement of JPEG (.jpeg, .jpe, .jpg), Macintosh PICT, GIF, Portable Network Graphics (PNG) and other bitmap-style formats.

The most commonly touted method is to save an Adobe Photoshop .psd file, and place this into InDesign 2.0. Transparency, otherwise known as alpha channel support, is fully supported in InDesign 2.0. What does not work, however, are vector layers. When placed into InDesign 2.0, vector layers including text layers are rasterised at the resolution of the Photoshop file (or transparency flattener, depending on the mechanics of the file).

Example 1: Photoshop image, containing Transparency as shown by the checkerboard pattern.

Example 2: The same Photoshop image, containing Transparency, placed into InDesign. The transparency is respected.

With Photoshop 6.0, and included in Photoshop 7.0, Adobe added the ability to save a Photoshop PDF file. Photoshop PDF has many benefits, and also shares many of the features of the Photoshop PSD format.

This format, like the Photoshop PSD format, has the following features

• Transparency support.
• Photoshop layers. including text layers, adjustment layers, layer masks etc are saved in the same file

Additionally, Photoshop PDF has the following features that Photoshop PSD does not have:

• Its PDF. can be displayed in the free Adobe Acrobat Reader
• Vector layers. keeping text/vector shapes kept as vector shapes for crisp printing and screen viewing)
• Security. using the security features of Adobe PDF, your images can be locked from viewing/editing by other people.
• Compression. images can be compressed using ZIP(lossless: no pixels are hurt) or JPEG (lossy: image data is changed)

With InDesign, Adobe has added support for the placement of PDF files without having to resort to “making and EPS” to ensure quality output. InDesign 2.0 also permits the placement of PDF files that contain transparency information – and holds the transparency information.

To save a Photoshop PDF for placement into InDesign, the go to File>Save As.

Example 4: File>Save As. Choosing Photoshop PDF, here I am saving the layers (permitting late stage editing of the text layer if I wish)

Example 5: File>Save As. Choosing Photoshop PDF: a subsequent dialog box permits the saving of Transparency, applying security and in this particular file: retain the vector data as vector.

This Photoshop PDF can be placed into InDesign – the the same as any PDF. When placed into InDesign 2.0, the file looks similar to the following. Notice I am only seeing the Proxy View of the image:

In the above example, I have turned on “High Quality Display”. The vector quality of the text, and any vector shape layers (including vector clipping groups) are respected; just as they are respected when printing or exporting as PDF or EPS.

Additionally, it is worth noting that the file extensions supported by Photoshop PDF are .pdf and .pdp

What is the difference? Nothing, apart from the file extension. The contents are exactly the same. The reason why there are two options is for the Windows platform. On Windows, the extension is used to associate a file with an application. .pdf is associated with Adobe Acrobat. When you double click on a file, this dot extension is used to assist Windows in determining which application to launch to view the file. With InDesign 2.0, when using the contextual menu “Graphics>Edit Original“, this association is also used. As this is a Photoshop PDF, and normally when editing/viewing the file – you would like to show it in Photoshop, an additional extension is permitted: .pdp, or Photoshop PDF.

I would recommend using the .pdp extension when saving a file from Photoshop as a Photoshop PDF. There were changes in InDesign 2.0 and 2.0.1 to support .pdp as equivilent to .pdf.

For MacOS users, the extension for InDesign 2.0 files is .indd

In fact, I would always use Photoshop PDF as the primary format when taking files from Adobe Photoshop to InDesign. There are no downsides: the file size is smaller, vector text and layers are supported and kept as vector.

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

InDesign: Use the Distiller or Export PDFs?

This was originally posted to the InDesign-Blueworld mailing list on 14-October-2002

Firstly, there is nothing technically wrong with Exported PDFs from InDesign. At all. I (personally) have had great success with exported PDFs from InDesign RIPping to Prinergy and various other imagesetters/platesetters in production in the field.

When you send, or you have received an Acrobat 5.0/PDF 1.4 – directly exported from InDesign 2.0, the workflow choices are a little different: Printing Acrobat 5.0/PDF1.4 Generated by Adobe InDesign 2.0

In either workflow, you will get a high quality PDF that will generate great output.

However, I do recommend using a Print to Postscript-Distill workflow in the following situations:

1. When you are sending a PDF “blind”.

   In other words, where you are not sure of the provenance/age/version/vendor of your printer’s RIP — they will more than likely have determined an internal workflow for Distiller-made PDFs. They will have .joboptions available for your use, and have tested Distiller made PDFs from QuarkXpress, InDesign and other sources. If they use tools like Pitstop, they probably have created preflight checks based on Distiller-made PDFs.

   This is especially the case if you are sending advertisements, sending files to remote countries or doing work for a client where your client nominates a printer and it is not your choice. In these style workflows, there is a blind handoff.

   Therefore, Creating Postscript and Distilling is the safest path.

2. Your Printer’s RIPs are Harlequin < 5.3

   This is the CID font encoding issue. As you probably know by now, InDesign to accurately represent glyphs like ligatures, InDesign encodes the text in its PDFs in a form known as “CID”.

   THERE IS NOTHING WRONG, TRICKY, HIDDEN OR EVIL about CID font encoding. It’s a valid part of the PDF specification that certain vendors had not implemented in their software. By Print to Postscript-Distill, there is no CID font encoding, whereas exported PDFs do. Well build (that is: to
   specification) RIPs/Imagesetters work successfully with CID font encoding.

   A large InDesign customer here in Australia have a *very* old Harlequin RIP which is integral in their workflow. This forces the Distiller-route PDF generation: which works flawlessly, day in and day out.

   Again, if you do not worry, understand or even care what your printer is
   using: the Distiller is a common standard method.

3. Your printer/publisher is conservative, and provides a Distiller-workflow option.

   OK, so your printer accepts PDFs and provides a series of steps and a Distiller 4/5 .joboptions file. In this case, I sometimes recommend people export a PDF from InDesign to see if it works successfully (prepare to be surprised!) — however, to make life easier and have less Prepress technical people getting hot under the collar, use the Print to Postscript-Distill route.

All of this said, Exporting PDFs is a better option. Why?

1. Its quicker. Much quicker.
2. There are less translations (InDesign->Postscript->PDF, vs.
   InDesign->PDF)
3. Once there are more RIPs with InRIP flattening (next revision of Prinergy, Fujifilm etc) are out there, we get even faster output to Acrobat 5 (PDF 1.4). A sight to behold, people!

Therefore, if you have the chance to test Exported PDFs with your
workflow, please do.

Please note that Australia is far, far along the High Quality PDF path. PDF is the industry standard here in Australia (independent study) with a majority of printers getting a majority of their work in as PDF. This involves a plethora of RIPs, workflow software, imposition tools etc. Therefore in Australia, Distiller is a consistent known entity, and why we pragmatically recommend Print to Distiller PDF generation for our InDesign customers here.

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

Comments on Creo Prinergy and InDesign 2.0

In Australia, the most common RIP I see in highend platemaking work is the Creo Prinergy series

This particular system takes in Postscript and PDF, and foes through a process of Normalising the input into PDF. As a part of this process, it can preflight and check the incoming stream to ensure a quality printed result.

Prinergy takes natively exported InDesign 2.0 PDFs sucessfully, and the 2.1 version of Prinergy permits in-RIP transparency flattening (not discarding like some competitors!).

However, there are two options in the Refining stage that can create issues for InDesign created jobs. These issues are to do with Prinergy correcting common issues with QuarkXpress generated Composite Postscript (and therefore PDF)

They can be found in the Refine Process Plan section of Prinergy, and specifically inside Color Convert, Overprint Conversion

• Set Colours to Knockout
  Setting process colours to knockout globally is dangerous. It overrides the overprint setting in the input file and forces the colurs to knockout. As a result, colour printing order affects which separations knocks out.

  You would use this feature for disabling overprint and enabling content colour matching for all objects in a PDF page. When you select this feature, you also let the printed PDF page match the onscreen PDF page only when Overprint Preview is turned off.

  Therefore, you are creating a printed result which only matches the onscreen view in Acrobat 4.0 or the Reader. Not InDesign 2.0 or Acrobat 5.0 with Overprint Preview turned on, or more importantly, as the RIP that you created proofs from probably proofed the job!

  As I understand it, this is turned on in Prinergy to “clean up bad Postscript” eminating from QuarkXpress in Composite style workflows. I am willing to hear other issues it solves.
  

  One of the very significant impacts of this is in CMYK+Spot colour PDF workflows with PDF 1.3 files. (ref: InDesign 2.0: Spot Colors, Transparency) InDesign’s flattener creates white process-only-knockout boxes where a spot colour needs to print. You may notice these boxes when viewing the PDF in the Reader or Acrobat without Overprint Preview. When this option is turned on, the spot plates will be knocked out by these boxes, and the resulting plates are not printable.

  Another impact is with Composite, Trapped PDFs. ((ref: InDesign 2.0: Generating Composite, Trapped PDFs)
  The trapping created by the referred technique are discarded when this option is turned on.

• Set Black to Overprint
  All Black elements (where the ink settings are 100%K, 0 CMY) are converted into overprinted black.

My recommendation is to be very, very careful with these options when working with InDesign 2.0 created PDFs with Prinergy

Visit the new InDesign Prepress Section: Adobe InDesign: Prepress Techniques

The holy grail with InDesign 2.0 and its support for spot/special colours is the complete replacement of DCS as an input file format.

DCS was invented to permit applications such as Photoshop to create a pre-separated file and place into QuarkXpress. Quark then didn’t need much intelligence to output the separated file: it just passed each of the plates as an EPS in the page stream when generating film/plates.

InDesign 2.0 does support DCS as a placeable file format. In fact, InDesign’s support for DCS is more comprehensive than that of Illustrator. However:

• Placed DCS elements cannot interact with transparency.
• Output is restricted to printing as separations; not Composite CMYK or InRIP separations
• Composite PDF, the most common exchange format today, cannot be exported or distilled.

In a world where composite PDF is the norm, this DCS workflow no longer fits. With simple CMYK work from Photoshop, placing elements and generating separations is simple. The graphic is still held as separate plates, but presents to the user as a composite image.

The problem arises, however, when attempting to take a CMYK+spot(s) file from Photoshop and attempt place into any application – including InDesign 2.0. Ultimately, it would be great to be able to place a single file that contains bitmap and vector information, as well as holding transparency and spot colours. Presently, the only file format that supports this fully is DCS. Whilst you can create Illustrator 10 .ai or .pdf files containing CMYK bitmap and spot colour vector objects, Photoshop is still largely the tool of choice for creation of special effects.

Photoshop allows spot colour channels to be created, but the only supported output formats are EPS, DCS and Photoshop PDF. In all of these formats, any transparency is not retained.

Therefore, this technique (which has been used in production) may assist you in combining the power of Photoshop and extending its spot channel support, and the power of transparency in InDesign to assist us in creating a printable job.

The process:

The overview is: create two Photoshop PDF files, place them on top of each other in InDesign 2.0. The bottom-most file is a CMYK only Photoshop PDF, and the topper-most file is a Spot-only Photoshop PDF. This technique relies on the Transparency Flattener in InDesign 2.0 to weave its magic to generate a composite output.

Step 1: create the source graphic element in Photoshop. In this example, we have a masthead that contains a Bevel&Emboss Photoshop layer style. Our goal is to make the colour of the masthead a special colour. (in the realworld example, this was printed as a metallic silver colour). Don’t create the spot channel yet. Save it as a Photoshop PDF (retaining transparency, vector information and resulting in a compact file).

In this file, the text has a white (or knockout) colour. Its goal in life is to act like a “cookie cutter” and remove the ink from underlying elements at print time. The Bevel&Emboss effect is created using a Black ink, so it will still appear on the black plate at print time.

Step 2: Take the same Photoshop PDF file:

• Turn off any effects.
• Change the white/knockout colour to Black. Choose a % of black that equals the % of spot colour ink you would like. In this example, as the element is a vector text element, it is just filled with 100% Black.
• double check and ensure where you have black, you want the spot ink to appear
• Mode>Convert>Grayscale. The will convert the black elements to 100% Black.
• Mode>Convert>Duotone. This will then permit you to change the Black to a named spot channel. Thankfully in Photoshop 6 or 7, this will be clipped inside the vector text element.
• For Advanced Users: you can also use other Layer Styles in Photoshop to ‘feather off’ the spot ink to create a highlight effect.

Do not make any position changes to the file. Set the channel in the Duotone to the same spot ink you are going to use in InDesign. This can be “Ink Aliased” at output time.

Step 3: As you will notice, saving as an EPS or PDF – transparency is not retained in this process. Don’t panic, we’re going to fix this in InDesign.

So we have two files: one being the Composite CMYK object saved as a Photoshop PDF, and another saved as a Photoshop PDF containing a Spot colour. In both files, there is vector data ensuring high quality output. In this example, I am using the .PDP file extension: the data inside the Photoshop PDF file is exactly the same; all I have done is adjust the extension. This will enable InDesign to Edit Original into Photoshop automatically.

Step 4: Go to InDesign 2.0. Place the file saved in step 1.

Step 5: Place the file saved in step 3, positioning it exactly over the top of the file placed in step 4. Use the transform palette to get 100% placement accuracy. Don’t worry about the “white” (knockout) colour from the placed file.

Step 6: set the top object to 100% Multiply using Window>Transparency. Almost magically, the white colour is removed, yet the spot colour remains. Leaving this at 100% Multiply, at output time the spot colour is retained (not converted to process)

To make the printing process a little more difficult, here the layer which the two elements are placed onto have been moved behind the image.

Step 7: Export as PDF, or Print to the Acrobat Distiller. In this example, I printed using Composite CMYK. Its a little difficult to fathom; but spot colours are held in this process (unless you use InDesign 2.0’s Ink Manager to convert them back to process at print time)

In the above screen dump, I am using Quite Revealing to show the background colour from the Composite PDF. As you can see, the first placed CMYK PDF ‘cuts out’ the colour in the background: in this instance, the PANTONE 264C Spot Colour.

This above screen dump shows the PANTONE 340 C as created in the second Photoshop file

Here is the black plate. The Bevel&Emboss added in the first Photoshop PDF is retained, and overprints the spot colour correctly.

How does this work?

The first file you place (CMYK) element acts like a ‘knockout’ element, removing any items underneath. The second file placed (EPS Spot) then overprints the underlying CMYK object. As InDesign’s flattener is smart, it does not knockout underlying elements. It also does not change their colour in the flattening as the top object is a spot colour. Other blend modes such as Lighten or Darken do attempt to change the colours – so the final document may be forced into CMYK.

What’s the Benefit?

What does this provide that DCS does not? The ability to generate a composite PDF. As soon as you place a DCS file into QuarkXpress, InDesign or PageMaker – you are forcing the output to be separations. In modern Postscript 3 or Extreme workflows, recombining preseparated output is difficult, and not the default workflow.

If you are in Australia, you may have seen the June 2002 issue of the Foxtel magazine. It used this technique. Supplied as a composite PDF with Spot colour, it correctly separated, was trapped and printed as a metallic. The Bevel&Emboss in the K plate overprinted the silver-metallic special colour. This plate is still a vector element.

Thanks to: Matt Phillips, Ben Hewitt (who tested this out on a live job!) and Alan Rosenfeld (for listening to my ranting about this in Brisbane). A big thankyou to Aaron Cliff from Foxtel magazine for sending a better quality image – and more importantly, being brave enough to pioneer this technique.