On-boarding a New Auth Interface in jPOS, Part 6

Part of a continuing series…

We’ve now on-boarded a wide range of authorization endpoints onto our jPOS payment transaction frameworks.  In my On-Boarding Guide, I give relatively short shrift to the thinking you ought to go though in terms of the implications of supporting a wide range of endpoints via your Transaction Manager(s) (hereafter, I’ll refer to this as a ‘TM’).  In the section of the guide entitled “Add new participants to TransactionManager,” I go through the ‘vanilla’ exercise of assuming that the new endpoint gets added into your main TM.

Now, let’s go beyond vanilla.  In a situation like we have at one of our acquirer-side customers, it could prove problematic to have all endpoints piled into one TM.  First there’s just the sheer maintenance of the thing:  even with some separation (see discussion belong), our main TM there has extended to 42kb.  There enters at that level the simple baseline concern of over-complexity and the resulting potential for error during new service on-boarding. 

Moreover, there’s my previously stated concern that smaller, less proven endpoints could prove unstable.  Excessive timeouts or simply elongated response times could easily suck up all your tasks leaving incoming transaction requests piling up in the server queue.  Not to cast aspersions at stored value card providers, but that’s where I had most of my concern because that class of authorizer is subject to massive run-ups in holiday volume (20x vs. a ‘typical’ day is not unusual).  Your good performance becomes totally predicated on some external party doing their peak planning correctly.  Which, frankly, ought to scare the crap out of you.

This situation is especially acute in situations like the one I’m describing here where we have transactions coming from 4,900+ locations across the four US continental all funneling into one listening point.  Here, we have a server that look like this:

<xx-server class="org.jpos.xx.Server" logger="Q2">
<property name="port"    value="30000" />
<property name="space"   value="tspace:default" />
<property name="queue"   value="NNNNN.TXN" />
<property name="timeout" value="60000" />
</xx-server>

Now, we forward to our main TM.  That entity starts off looking like this:

<xx-txnmgr class="org.jpos.transaction.TransactionManager" logger="Q2">
<property name="sessions" value="120" />
<property name="space" value="tspace:default" />
<property name="queue" value="NNNNN.TXN" />
<property name="debug" value="FALSE" />

<participant class="org.jpos.xx.PrepareContext">
   <!-- need to know the Space and Queue in order to get the
        number of outstanding transactions at the start of
        this transaction -->
   <property name="space" value="tspace:default" />
   <property name="queue" value="NNNNN.TXN" />
</participant>

<participant class="org.jpos.transaction.Open" logger="Q2" realm="open">
  <property name="checkpoint" value="open" />
</participant>

<participant class="org.jpos.xx.ParseRequest"
    logger="Q2" realm="parse-request">
  <property name="checkpoint" value="parse-request" />
</participant>

Now, the all important switch step – you can see that the thing here is that we process Debit, EBT and Credit in the main TM while using jPOS’ powerful Continuations facility to push the stored value stuff out to other TMs:

<participant class="org.jpos.xx.Switch" logger="Q2" realm="switch"> 
<property name="X.41" value="EmployeeVerification LogAndReply" />
<property name="X.42" value="RewardsProgram LogAndReply" />
<property name="X.70" value="CheckPurchase LogAndReply" />
<property name="X.93" value="CreateDebit Decrypt DebitSale" />
<property name="X.96" value="CreateDebit FindOrigPurchaseSoft CopyFromOriginal DebitSaleReversal LogAndReply" />
<property name="X.60" value="CreateDebit Decrypt EBT_FS_Sale" />
<property name="X.61" value="CreateDebit FindOrigPurchaseSoft CopyFromOriginal EBT_FS_Void LogAndReply" />
<property name="X.62" value="CreateDebit Decrypt EBT_FS_Return" />
<property name="X.63" value="CreateDebit Decrypt EBT_FS_BalInq LogAndReply" />
<property name="X.68" value="CreateDebit FindOrigReturnSoft CopyFromOriginal EBT_FS_Return_Rev LogAndReply" />
<property name="X.69" value="CreateDebit FindOrigPurchaseSoft CopyFromOriginal EBT_FS_Sale_Rev LogAndReply" />
<property name="X.65" value="CreateDebit Decrypt EBT_Cash_Sale" />
<property name="X.66" value="CreateDebit FindOrigPurchaseSoft CopyFromOriginal EBT_Cash_Void LogAndReply" />
<property name="X.67" value="CreateDebit FindOrigPurchaseSoft CopyFromOriginal EBT_Cash_Sale_Rev LogAndReply" />
<property name="X.54" value="CreateCredit Decrypt CreditSale" />
<property name="X.55" value="CreateCredit FindOrigPurchaseSoft CopyFromOriginal CreditSaleReversal LogAndReply" />
<property name="X.08" value="CreateCredit FindOrigPurchaseSoft CopyFromOriginal CreditSaleReversal LogAndReply" />
<property name="X.06" value="CreateCredit Decrypt CreditReturn" />
<property name="X.07" value="CreateCredit FindOrigReturnSoft CopyFromOriginal CreditReturnReversal LogAndReply" />

<!—svc provider 1 Transactions -->
<property name="S.43" value="ForwardToProvider1TxnMgr" />
<property name="S.44" value="ForwardToProvider1TxnMgr" />
<property name="S.45" value="ForwardToProvider1TxnMgr" />

<!—svc provider 2 Transactions -->
<property name="T.43" value="ForwardToProvider2TxnMgr" />
<property name="T.44" value="ForwardToProvider2TxnMgr" />
<property name="T.45" value="ForwardToProvider2TxnMgr" />   

<!—svc provider 3 Transactions -->
<property name="V.43" value="ForwardToProvider3TxnMgr" />
<property name="V.44" value="ForwardToProvider3TxnMgr" />
<property name="V.45" value="ForwardToProvider3TxnMgr" /> 

<!-- Unhandled transactions -->
<property name="Unhandled" value="Unhandled" /> 
</participant>

The problem here:  we’ve inoculated ourselves against unexpectedly poor performance by the SVC class endpoints.  But – if the Debit/Credit/EBT endpoints were to suffer – then SVC-bound transactions would sit in the server queue and not be able to traverse the first steps of the main TM.  [Side note:  Your friend here is the threshold timer mechanism.  Don’t leave home without it.]  In short, the continuations don’t help us at all in that situation. 

That’s exactly what happened to us in May of this year.  Despite some vaunted geographical separation in data centers by the gateway provider, Debit/EBT transactions were found to have a single point of failure (since resolved) somewhere in their solution framework.  The lesson learned for us is that you’ve got to provide endpoint separation where ever and whenever possible.  We had assumed “hey, let’s protect the strong from the weak.”  In practice, SVC providers like Stored Value Systems have been superstars, while other presumed stalwarts have been a source of some unwanted surprises.

For that reason, we’re in the process of re-architecting the main TM – it’ll do those preliminary steps, handle those first transactions (EmployeeVerification, RewardsProgram, CheckPurchase) internally, then take the Continuations road on all transactions that are externally authorized.  We end up like so (we don’t separate Debit, Credit and EBT in this instance because the majority of the time they end up getting routed through the same gateway provider):

<participant class="org.jpos.xx.Switch" logger="Q2" realm="switch">

<!-- These first three we do right inside the main TM here –>
<property name="X.41" value="EmployeeVerification LogAndReply" />
<property name="X.42" value="RewardsProgram LogAndReply" />
<property name="X.70" value="CheckPurchase LogAndReply" />

<!-- Everything else, we forward in order to protect one set of
     transaction classes from bad performance from another  -->

<!-- Debit Transactions -->
<property name="X.93" value="ForwardToDebitCreditTxnMgr" />
<property name="X.96" value="ForwardToDebitCreditTxnMgr" />

<!-- EBT Transactions -->
<property name="X.60" value="ForwardToDebitCreditTxnMgr" />
<property name="X.61" value="ForwardToDebitCreditTxnMgr" />
<property name="X.62" value="ForwardToDebitCreditTxnMgr" />
<property name="X.63" value="ForwardToDebitCreditTxnMgr" />
<property name="X.68" value="ForwardToDebitCreditTxnMgr" />
<property name="X.69" value="ForwardToDebitCreditTxnMgr" />
<property name="X.65" value="ForwardToDebitCreditTxnMgr" />
<property name="X.66" value="ForwardToDebitCreditTxnMgr" />
<property name="X.67" value="ForwardToDebitCreditTxnMgr" />

<!-- Credit Transactions -->
<property name="X.54" value="ForwardToDebitCreditTxnMgr" />
<property name="X.55" value="ForwardToDebitCreditTxnMgr" />
<property name="X.08" value="ForwardToDebitCreditTxnMgr" />
<property name="X.06" value="ForwardToDebitCreditTxnMgr" />
<property name="X.07" value="ForwardToDebitCreditTxnMgr" />

<!—svc provider 1 Transactions -->
<property name="S.43" value="ForwardToProvider1TxnMgr" />
<property name="S.44" value="ForwardToProvider1TxnMgr" />
<property name="S.45" value="ForwardToProvider1TxnMgr" />

<!—svc provider 2 Transactions -->
<property name="T.43" value="ForwardToProvider2TxnMgr" />
<property name="T.44" value="ForwardToProvider2TxnMgr" />
<property name="T.45" value="ForwardToProvider2TxnMgr" />   

<!—svc provider 3 Transactions -->
<property name="V.43" value="ForwardToProvider3TxnMgr" />
<property name="V.44" value="ForwardToProvider3TxnMgr" />
<property name="V.45" value="ForwardToProvider3TxnMgr" /> 

<!-- Unhandled transactions -->
<property name="Unhandled" value="Unhandled" /> 
</participant>