Technical Article

Custom XMP Properties and the PDF/A Part They Land In

An ERP invoice number added to the XMP of a PDF/A-2b file draws an undeclared-property finding, yet the same packet in a PDF/A-4 file conforms. Parts 1 to 3 demand a pdfaExtension:schemas description for every custom property and part 4 does not, so the PDF/A validator in PDFium Component for Delphi reads pdfaid:part before choosing a rule set

Decision flow of PDFium Component for Delphi reading pdfaid:part to choose extension-schema checks under PDF/A parts 1 to 3 and packet-level rules under PDF/A-4
the validator reads pdfaid:part first, so a packet claiming part 4 skips the four extension-schema findings that parts 1 to 3 raise in PDFium Component for Delphi

What do PDF/A parts 1 to 3 require of a custom property?

A description of its namespace, plus a line in that description naming the property. ISO 19005-1 §6.7.2 allows XMP properties only from the predefined schemas or from extension schemas that comply with §6.7.8, which wants each description in the same metadata stream; ISO 19005-2 §6.6.2.3.1 carries the rule into parts 2 and 3. A description is no blanket permit. It names properties one by one through pdfaProperty:name, so a packet that describes inv:InvoiceNumber and also writes inv:DueDate fails on the second. PDFium Component reports these cases under four TPdfAValidationIssue values, one finding per fault: a renamed prefix, for instance, yields the prefix finding alone

Issue valueReported whenClause
pvaiMissingExtensionSchemaA non-predefined namespace is in use and the packet has no pdfaExtension:schemas container19005-1 §6.7.8, 19005-2 §6.6.2.3.1
pvaiXmpPropertyUndeclaredThe container exists but no description in it names the property19005-1 §6.7.2, 19005-2 §6.6.2.3.1
pvaiExtensionSchemaPrefixA description namespace is bound to a prefix other than the required one19005-1 §6.7.8, 19005-2 §6.6.2.3.2
pvaiExtensionSchemaEntryMissingA description leaves out a field its table lists19005-1 §6.7.8, 19005-2 §6.6.2.3.3

Which of the first two you get depends on the producer. Many PDF/A writers already describe the pdfaid identification schema in a container, SaveAsPdfA in PDFium Component included, so a field added later lands beside a container that has no entry for it. The routine below prints only the extension-schema findings. Expected: a PDF/A-2b file written by SaveAsPdfA prints pac2b and pvaiXmpPropertyUndeclared, a part 1 or 2 packet without a container prints pvaiMissingExtensionSchema, and a PDF/A-4 file prints its conformance line alone

uses
  System.SysUtils, System.Classes, System.TypInfo,
  PDFium, FPdfPdfa, FPdfXmp;

const
  InvoiceNs = 'http://ns.example.com/invoice/1.0/';
  ExtensionIssues: TPdfAValidationIssues = [pvaiMissingExtensionSchema,
    pvaiXmpPropertyUndeclared, pvaiExtensionSchemaPrefix,
    pvaiExtensionSchemaEntryMissing];

procedure PrintExtensionIssues(Source: TStream);
var
  Res: TPdfAValidationResult;
  Issue: TPdfAValidationIssue;
begin
  Res := ValidatePdfAComplianceEx(Source, TPdfAValidationOptions.Default);
  Writeln(GetEnumName(TypeInfo(TPdfAConformance), Ord(Res.Conformance)));
  for Issue in Res.Issues * ExtensionIssues do
    Writeln('  ', GetEnumName(TypeInfo(TPdfAValidationIssue), Ord(Issue)));
end;

procedure ProbeCustomProperty(const FileName: string);
var
  Pdf: TPdf;
  Packet: TPdfXmpPacket;
  Output: TMemoryStream;
begin
  Output := TMemoryStream.Create;
  try
    Pdf := TPdf.Create(nil);
    try
      Pdf.FileName := FileName;
      Pdf.Active := True;
      Packet := Pdf.ReadXmpPacket;
      if Packet = nil then
        raise Exception.CreateFmt('%s has no XMP packet', [FileName]);
      try
        Packet.SetSimpleValue(InvoiceNs, 'inv', 'InvoiceNumber', 'INV-2026-0412');
        if not Pdf.SaveAsWithXmpPacket(Output, Packet) then
          raise Exception.Create('Saving the edited packet failed');
      finally
        Packet.Free;
      end;
    finally
      Pdf.Free;
    end;
    Output.Position := 0;
    PrintExtensionIssues(Output);
  finally
    Output.Free;
  end;
end;

The predefined set moved between XMP 2004 and ISO 16684-1

Part 1 and part 2 disagree on what predefined means. ISO 19005-1 §6.7.2 points at the XMP Specification of January 2004, ISO 19005-2 §6.6.2.3.1 at ISO 16684-1, whose list is longer. The gap shows in everyday files through crs (Camera Raw settings), aux (additional EXIF data) and xmpDM (dynamic media), which the PDF/A Competence Center's TechNote 0008 footnotes as new in the XMP 2005 specification. A crs:AutoBrightness or xmpDM:videoColorSpace property needs a description under part 1 and is predefined from part 2 on. The public veraPDF corpus has PDF/A-1b failure files that add one such property and nothing else, while veraPDF's own test-file builder writes a camera-raw or dynamic-media hint into its PDF/A-2b files, so a checker blind to the part number gets one side wrong. Declarations shifted too: photoshop:SupplementalCategories is Text in 2004 and a bag of Text in ISO 16684-1, so the bag form fails PDF/A-1 and is correct in PDF/A-2

The opposite shortcut fails too. Calling everything outside the ten schema tables of TechNote 0008 an extension would flag the sType structures and the Identifier qualifier namespace that ordinary xmpMM:History entries use, and TechNote 0008 allows those URIs in PDF/A-1. PDFium Component reads the part from pdfaid:part: a part 1 packet without a description block that uses crs, aux or xmpDM gets pvaiMissingExtensionSchema, a part 2 packet gets no extension-schema finding. For PDF/A-1 output, strip camera-raw and dynamic-media properties nobody downstream reads. A full crs description runs to dozens of entries, each a chance for an entry-missing finding

Required prefixes and the trailing # in the schema URIs

XML namespaces normally let a writer pick any prefix. §6.7.8 takes that freedom away for the description vocabulary, giving each structure a URI and a required prefix: pdfaSchema, pdfaProperty, pdfaType for a value type and pdfaField for a field of one; ISO 19005-2 §6.6.2.3.2 restates all four. Binding http://www.aiim.org/pdfa/ns/property# to nonpdfaProperty is equivalent XML and still fails, and the veraPDF corpus files for this clause keep every URI correct and rename one prefix. Your own schema is another matter: pdfaSchema:prefix records a preferred prefix, so inv is a recommendation. Keep it equal to the prefix the packet binds anyway

Anatomy of the four PDF/A extension-schema vocabularies showing required prefixes, schema URIs and mandatory fields, with the trailing hash normalization PDFium Component applies in Delphi
each description vocabulary carries a URI and a required prefix, and the component drops a trailing # on both sides before comparing namespace URIs

The URIs have a quirk. The prose of ISO 19005-1 §6.7.8 prints http://www.aiim.org/pdfa/ns/schema with no trailing #, while its tables and conforming files end the URI with one, so a strict comparison against the prose matches nothing. PDFium Component drops a trailing # on both sides before comparing. Write the # form, and copy your own namespace into pdfaSchema:namespaceURI character for character. The closing sentence of ISO 19005-2 §6.6.2.3.2 then wants every field the tables of §6.6.2.3.3 list, and a gap is reported as pvaiExtensionSchemaEntryMissing:

  • Schema (pdfaSchema): schema, namespaceURI and prefix, plus the property and valueType lists, which a schema with nothing to declare may leave out
  • Property (pdfaProperty): name, valueType, category and description, where category is either internal or external
  • Value type (pdfaType): type, namespaceURI, prefix, description and field, required because a custom value type is a structure
  • Field (pdfaField): name, valueType and description

How should a structured entry be spelled in RDF/XML?

Any of three ways, as long as the item stays a structure. ISO 32000-2 §14.3.2 hands the grammar of a metadata stream to ISO 16684-1, which writes XMP as a subset of RDF/XML. The short form puts rdf:parseType="Resource" on the rdf:li and lists the fields under it; the long form wraps them in a nested rdf:Description, and both mean the same anonymous structure. A third spelling moves plain text fields into attributes on that rdf:Description. Only text fits there: pdfaSchema:property holds a sequence and stays an element. Strip rdf:parseType from a short-form item and the result is not RDF/XML. A parser then expects text or one node element inside the rdf:li, takes pdfaSchema:schema for a typed node, and a node element may not hold non-whitespace text, so the first field already breaks the grammar. An XSLT that rebuilds list items, or a template that regenerates the block from a dictionary, can drop the attribute while keeping every element. Read such output back with a different RDF parser from the one that wrote it to see the damage

Three legal RDF/XML spellings of one PDF/A extension-schema entry, and the grammar break when rdf:parseType is stripped, as PDFium Component preserves them in Delphi
short form, long form and attribute shorthand all describe the same structure, and stripping rdf:parseType from a short-form item breaks the RDF grammar at the first field
<!-- Short form: the invoice schema described in full -->
<rdf:li rdf:parseType="Resource">
  <pdfaSchema:schema>Invoice data from the ERP</pdfaSchema:schema>
  <pdfaSchema:namespaceURI>http://ns.example.com/invoice/1.0/</pdfaSchema:namespaceURI>
  <pdfaSchema:prefix>inv</pdfaSchema:prefix>
  <pdfaSchema:property>
    <rdf:Seq>
      <rdf:li rdf:parseType="Resource">
        <pdfaProperty:name>InvoiceNumber</pdfaProperty:name>
        <pdfaProperty:valueType>Text</pdfaProperty:valueType>
        <pdfaProperty:category>external</pdfaProperty:category>
        <pdfaProperty:description>Invoice number assigned by the ERP</pdfaProperty:description>
      </rdf:li>
    </rdf:Seq>
  </pdfaSchema:property>
</rdf:li>

<!-- Long form: the same structure as a nested node -->
<rdf:li>
  <rdf:Description>
    <pdfaSchema:schema>Invoice data from the ERP</pdfaSchema:schema>
    ...
  </rdf:Description>
</rdf:li>

<!-- Attribute shorthand: text fields as attributes, the list stays an element -->
<rdf:li>
  <rdf:Description pdfaSchema:schema="Invoice data from the ERP"
      pdfaSchema:namespaceURI="http://ns.example.com/invoice/1.0/"
      pdfaSchema:prefix="inv">
    <pdfaSchema:property>...</pdfaSchema:property>
  </rdf:Description>
</rdf:li>

PDFium Component keeps whichever spelling arrived. Each value in the TPdfXmpPacket that TPdf.ReadXmpPacket returns records the rdf:parseType of its source element, Resource, Literal or Collection, and ToXml writes it back; a structure with no record leaves with rdf:parseType="Resource". That writer serves SaveAsWithXmpPacket and NormalizePdfAMetadata, the call that brings Info entries and their XMP equivalents into line, and an extension block in any of the three spellings validates the same before and after a read-and-save cycle. Write new entries in the short form. Every field stays an element, which keeps edits uniform, and it matches the pdfaid entry SaveAsPdfA writes

Adding your description to the container already there

Merge the new entry into the existing rdf:Bag. One container per packet. veraPDF's XMP parser refuses a packet with a second pdfaExtension:schemas element (Duplicate property or field node 'pdfaExtension:schemas'), and the failures land on general metadata rules. Each property needs a name, value type, category and description, the entries a Factur-X invoice fills in to declare its fx properties in an extension schema. The listing splices a short-form item before </rdf:Bag>, which ToXml writes on the container's own line. The item declares pdfaSchema and pdfaProperty itself, so it parses the same wherever it lands

const
  ExtensionNs = 'http://www.aiim.org/pdfa/ns/extension/';
  ContainerEnd = '</rdf:Bag></pdfaExtension:schemas>';
  TopDescriptionEnd = '</rdf:Description>'#10' </rdf:RDF>';

function InvoiceSchemaItem: string;
begin
  Result := '<rdf:li rdf:parseType="Resource"' +
    ' xmlns:pdfaSchema="http://www.aiim.org/pdfa/ns/schema#"' +
    ' xmlns:pdfaProperty="http://www.aiim.org/pdfa/ns/property#">' +
    '<pdfaSchema:schema>Invoice data from the ERP</pdfaSchema:schema>' +
    '<pdfaSchema:namespaceURI>' + InvoiceNs + '</pdfaSchema:namespaceURI>' +
    '<pdfaSchema:prefix>inv</pdfaSchema:prefix>' +
    '<pdfaSchema:property><rdf:Seq><rdf:li rdf:parseType="Resource">' +
    '<pdfaProperty:name>InvoiceNumber</pdfaProperty:name>' +
    '<pdfaProperty:valueType>Text</pdfaProperty:valueType>' +
    '<pdfaProperty:category>external</pdfaProperty:category>' +
    '<pdfaProperty:description>Invoice number assigned by the ERP' +
    '</pdfaProperty:description>' +
    '</rdf:li></rdf:Seq></pdfaSchema:property></rdf:li>';
end;

// True when a schema entry describes Uri, whichever spelling it uses
function DescribesNamespace(const V: TPdfXmpValue; const Uri: string): Boolean;
var
  Q: Integer;
begin
  for Q := 0 to High(V.Qualifiers) do      // fields as attributes on rdf:li
    if (V.Qualifiers[Q].Name = 'namespaceURI') and
       (string(V.Qualifiers[Q].Value) = Uri) then
      Exit(True);
  Result := (Pos('>' + Uri + '</', string(V.RawXml)) > 0) or    // elements
    (Pos('namespaceURI="' + Uri + '"', string(V.RawXml)) > 0);   // attributes
end;

// Returns a new packet; the caller frees it
function WithInvoiceSchema(Packet: TPdfXmpPacket): TPdfXmpPacket;
var
  Xml: string;
  Index, J, P: Integer;
  HasContainer: Boolean;
begin
  HasContainer := Packet.TryGetProperty(ExtensionNs, 'schemas', Index);
  if HasContainer then
    for J := 0 to High(Packet.Properties[Index].Values) do
      if DescribesNamespace(Packet.Properties[Index].Values[J], InvoiceNs) then
        Exit(Packet.Clone);                // already described
  Xml := Packet.ToXml;
  P := Pos(ContainerEnd, Xml);
  if P > 0 then
    Insert(InvoiceSchemaItem, Xml, P)      // join the existing bag
  else if HasContainer then
    raise EPdfXmpError.Create('Extension container uses an unexpected prefix')
  else
  begin
    P := Pos(TopDescriptionEnd, Xml);
    if P = 0 then
      raise EPdfXmpError.Create('Unexpected XMP packet layout');
    Insert('<pdfaExtension:schemas xmlns:pdfaExtension="' + ExtensionNs +
      '"><rdf:Bag>' + InvoiceSchemaItem + ContainerEnd + #10'  ', Xml, P);
  end;
  Result := TPdfXmpPacket.Parse(Xml);
end;

procedure TagInvoice(const SourceFile, TargetFile, InvoiceNo: string);
var
  Pdf: TPdf;
  Packet, Described: TPdfXmpPacket;
  Written: TFileStream;
begin
  Pdf := TPdf.Create(nil);
  try
    Pdf.FileName := SourceFile;
    Pdf.Active := True;
    Packet := Pdf.ReadXmpPacket;
    if Packet = nil then
      raise Exception.CreateFmt('%s has no XMP packet', [SourceFile]);
    Described := nil;
    try
      Packet.SetSimpleValue(InvoiceNs, 'inv', 'InvoiceNumber', InvoiceNo);
      Described := WithInvoiceSchema(Packet);
      if not Pdf.SaveAsWithXmpPacket(TargetFile, Described) then
        raise Exception.CreateFmt('Could not write %s', [TargetFile]);
    finally
      Described.Free;
      Packet.Free;
    end;
  finally
    Pdf.Free;
  end;
  Written := TFileStream.Create(TargetFile, fmOpenRead or fmShareDenyWrite);
  try
    PrintExtensionIssues(Written);         // from the first listing
  finally
    Written.Free;
  end;
end;

Expected: for a PDF/A-1, -2 or -3 source written by SaveAsPdfA, TagInvoice prints the conformance line and none of the four extension-schema values, with the invoice entry in the short form beside the pdfaid one. A second run leaves the bag alone whatever spelling the existing entry uses: DescribesNamespace searches RawXml and, for attributes on an empty rdf:li, the qualifiers. A container under another prefix raises an exception, since a fresh one would be the duplicate veraPDF refuses. When the ERP adds a due date, extend the existing pdfaSchema:property sequence

Does PDF/A-4 check extension schemas at all?

No. ISO 19005-4 §6.7.2.1 keeps only packet-level rules: the catalog Metadata key, the deprecated bytes and encoding header attributes, well-formed XML and UTF-8. An undescribed property, a description under nonpdfaSchema and one missing pdfaSchema:schema all leave a file conforming under the veraPDF PDF/A-4 profile, and PDFium Component runs its four extension-schema checks only when pdfaid:part is 1, 2 or 3. The identification schema keeps its rules everywhere. Under §6.7.3 of part 4 its namespace must still be bound to pdfaid, and its property set grows by part: part, conformance and, since Amendment 1 of 2007, amd in part 1, corr from ISO 19005-2 §6.6.4 on, rev in part 4. A wrong prefix is reported as pvaiXmpIdPrefixWrong and an extra name as pvaiXmpIdPropertyUnknown in every part

To compare parts on one document, change the packet and nothing else: insert the namespace binding inside the XMP and delete as many bytes of whitespace between tags, so /Length, the cross-reference offsets and the %PDF-2.0 header stay put. A pypdf rewrite can hand a PDF/A-4 file back with a %PDF-1.3 header whose failures bury the metadata result. For production output, write the description into PDF/A-4 packets too: it costs a few hundred bytes, the validator in PDFium Component leaves it alone under part 4, and the packet can later move into a PDF/A-3 deliverable unchanged