messaging validation rules

[matthew.d.rawlings@xxxxxxxxxxxx]

At today's MTF meeting the Chair (Brian),
asked me to reply to these comments on the mail list.

Validation of Matthew's contract message validation
rules proposal
==================================================================

We discussed this at some length (in the absence of Matthew), and came
to a
number of areas of consensus or where we felt that additional clarity was
required.

MR: A long time ago I sent in a request to add some
additional rules to the Contracts messaging. This was based on work in
2006/7 on getting SWIFT support for FpML - the Contracts Messages, and
the missing rules were sent through in 2007. The rules covered gaps in
the current definitions. The rules were considered by the VWG during 2007
and approved for addition, subject to the VWG producing examples. 

1) We liked the use of the validation rule _expression_ syntax for documenting
these types of rules for processing messages.  We felt that these
types of
rules would be useful documentation for implementers.  There may be
some
advantage to this type of rule as opposed to other documentation mechanisms.
For example, they do not require the specification of a standardized state
model, which could be difficult to get agreement on.

MR: Getting clear definitions, such as "you can't
cancel a cancel message" is invaluable for implementers. It is much
better than ambiguous situations. We already agreed to add this as text
to the Standard (AWG 3rd April 2008), but as a testable rule with a concrete
implementation it is less ambiguous to interpretation.

MR: These rules are defined as operating on the set
of all messages known to a Messaging Endpoint (Sender/Receiver). Members
of the VWG offered two other alternatives. The first alternative was modal
logic or temporal logic, which would work fine but would likely be alien
to many modellers and implementers, and we would still need to define the
modes. The second alternative was to define some sort of standardized processing
model based on FSMs that had a strong sense of message order and 'current'
state. Standardized processing is difficult to describe and we can't verify
whether it is followed or not. I went for the simple validation rules because
they were simple to understand and implement and only standardized what
was in public messages not how systems were implemented. I agree with all
the statements here.

2) We did not feel that these rules were like traditional validation rules,
because these rules require access to the current state of the processing
system (i.e. they are "dynamic" or "state-dependent").
 Other validation
rules express constraints about the content of a single message, and can
be
applied without consulting other messages.  This makes them amenable
to
being applied in a stand-alone validation application.  The rules
in
Matthew's proposal can only be applied with access to a database containing
the current processing state.

MR: When people say "current state" it is
normally meant in the OO sense as in "current state of the world".
These rules do not require current state of the world. However when I asked
Brian by email he explained in this case it meant in effect "current
knowledge of the messages". This is highly desirable, because if the
rules required you to know about messages not received or not know about
messages you had received then it couldn't be made to work.

MR: All these rules operate over the entire set of
messages a message endpoint knows about. This means you never need to wait
for more messages or to 'forget' old messages. The rules do work cross
message, so you can't evaluate one message at a time in isolation. I had
implemented these messages in Saxon by running the rules over a large set
of messages on a filesystem. They also worked well when using a leading
XML DBMS and a leading RDBMS with XML support. You can implement them in
anything that can see the messages.

For this reason, we felt that these types of rules should be clearly
distinguished from validation rules.  They should be associated with
the
business process definitions and be termed something like "message
processing" or "request processing" rules, rather than "validation"
rules,
for clarity for implementers.

MR: This is what I said, and what the VWG said, and
now what the MTF says. We all agree. I suggest "Messaging Constraints"
or "Messaging Rules".

We think it would be beneficial to develop a suite of these rules, and
felt
that a quite number of the rules in Matthew's document could be useful
as a
starting point for this set.

MR: This is what I said, and what the VWG said, and
now what the MTF says. We all agree.

3) We had questions about what the expected handling of different severities
would be.  For example, would a "Mandatory" rule violation
automatically
result in an immediate failure report to the sender, or could it be handled
by a repair function?  Different people on the call had different
interpretations of the likely handling of these types of rules.

MR: This is the same issue as invalid messages today.
Jon Postel said: "Be liberal in what you accept, and conservative
in what you send.". Most client facing interfaces seem to be very
liberal and rather than reject instead they hospitalize messages and repair
them, whereas most internal facing interfaces systems seem to be conservative
in accepting messages. What you do in the face of failure is the same as
for receiving bad XML. This is a different issues to what the rules are.

MR: It was the VWG that invented the term "Mandatory",
I never cared for it. With dynamic constraints on messaging there are two
kinds of inconsistency - those you can never recover from, and those that
you can recover from with extra information. 

4) We felt that it was essential that the context of the "collection"
of
business objects be defined very clearly for the rules to be clearly
defined.  For example, is uniqueness for business object identifiers
(e.g.
contract IDs, as listed here) required across all generators of documents,
or only within a single generator?  Should a recipient of messages
from
multiple sources validate the IDs against IDs submitted by other sources
as
well, or must they be kept distinct?  If a sender attaches multiple
IDs each
with a different scheme, shouldn't they be checked for uniqueness
including/within the scheme?

MR: It is very clear. I included the XQuery collection
function in the context. The context applies to all messages a Messaging
Endpoint knows about.

5) Can an identifier (e.g. trade identifier, message identifier) ever be
recycled, for example in case of an explicit instruction that the identifier
was incorrectly assigned?  Or, for example, could a message ID be
recycled
after, say, a month?

MR: The rule can be changed to match whatever the
definition says. If the definition in the schema says it can be recycled
after 10 weeks, then a rule can cope. The definition of MTF identifiers
is still under discussion. When complete it can be expressed as a rule.

6) It appears that the uniqueness rules apply to all identifiers included
by
the document creator, whether or not they were actually assigned by the
document creator (the contract/header/identifier element can occur multiple
times, and the rule appears to apply to all of the occurrences).  This
means
that if a document creator attaches identifiers allocated by another party
to a contract/trade, it will be difficult or impossible to change that
association if in the future a mistake is discovered, since these
identifiers cannot be reused in another contract/trade (without causing
a
uniqueness violation).  Is it possible to distinguish to which identifier(s)
the uniqueness constraint should be applied?  (Brian suggested this
may be
more of a modeling than a validation issue).

MR: The rule can be changed to match whatever the
definition says. The definition of MTF identifiers is still under discussion.
When complete it can be expressed as a rule.

7) Since the "identifier" element in contract has some sub-structure
(e.g.
versioned and non-versioned variants), it would be useful to have a more
precise definition of the ID matching rules. 

MR: Yes. If the definition is refined the rule can
be refined to match. Whatever the definition is, the rule can be changed
to match.

Matthew Rawlings 
+44 7917 596 827

---

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities.