Transaction Playback in jPOS

A question Alejandro and I often hear is: "Hey, I need the ability to capture transactions and play them back.  Can I do that in a jPOS implementation?"

Yes.

The jPOS Logger facility supports full transaction capture and was designed with playback in mind. The jPOS framework has a ‘LogChannel’ component that basically reads a log out of the XML stream – ignoring everything but <isomsg>...</isomsg> - and is capable of replaying them.

To help frame this discussion, it would help to post here an instance of a transaction captured in a q2 log.  This example (see below) is a Debit request acquired at a locally-controlled point-of-sale.  We parse the request, translate the PIN block, switch the transaction out to FDR as an ISO 0200 request, get the 0210 back, parse it, log it and send a response back to the device.  The quietly beautiful thing about the q2 logger is the way in which it collects and presents all four legs of the transaction request into one nice little package.  For troubleshooting, research, certification, ad-hoc testing - you name it - this layer of simple elegance is a godsend (take this from a guy who has wasted countless hours of his life piecing together transaction legs from purely time sequential logs in legacy payment systems frameworks).

So, here is a complete transaction as it is captured in the jPOS/OLS.Switch log file (note masking or wiping of all PIN- and PAN-related information per CISP/PCI standards):

<log realm="Debug" at="Fri Jan 05 15:29:53 EST 2007.649">
  <commit>
    <id>4</id>
    <context>
     <transient>
      <entry   
      key="EV_SESSION">org.jpos.ev.Session:java.nio.channels.SocketChannel
     [connected local=/10.46.1.30:33000 remote=/10.254.6.102:52908]</entry>
      <entry key="PROFILER">
       <profiler>
         open [0/0]
         parse-request [15/15]
         create-debit-tranlog [32/47]
         populate-debit-tranlog [15/62]
         validate-terminal [32/94]
         find-duplicate [31/125]
         translate-pin [78/203]
         create-fdr-request [16/219]
         query-host-or-reverse [1843/2062]
         prepare-ebt-response [0/2062]
         close [16/2078]
         send-response [0/2078]
         end [2078/2078]
       </profiler>
      </entry>
      <entry key="DB">org.jpos.ee.DB@f7db8c</entry>
      <entry key="REQUEST">
       <fsdmsg schema='file:cfg/v2-base'>
         header: '20202020'
         record-format: 'E'
         application-type: '0'
         field-sep-1: '.'
         acquirer-bin: '628702'
         message-version: '02'
         shift-number: '10249'
         merchant-number: '0010'
         store-number: '00660'
         terminal-number: '0001'
         category-code: '4444'
         country-code: '987'
         city-code: '97208'
         timezone-differential: '010'
         transaction-code: '60'
         terminal-serial-number: '12345678'
         encryption-indicator: '3'
         transaction-sequence-number: '5651'
         card-id-source: 'A'
         account-entry-mode: 'D'
         magnetic-strip-info: '507680______1999=__________________'
         cid-device-attached: 'X'
         pin-action-code: 'F'
         pin-function-code: '0'
         pin-length: '00'
         pin-block: '00'
         encrypted-pin-block: '22DC9C______C62D'
         pin-dukpt-ksn: '011401______018E'
         amount: '1300'
         additional-amount: '0'
         ebt-cashier: '111111111'
         ebt-supervisor: '000000000'
         register-number: '12'
         tran-id: '1658744'
         tender-attempt-indicator: '1'
         tender-number: '8200'
         tender-attempt: '1947'
         magnetic-strip-info-encrypted: 'SSYyNK________________82GX'
       </fsdmsg>
      </entry>
      <entry key="switch-key">E.60</entry>
      <entry
       key="TRANLOG">org.jpos.ee.DebitTranLog@1e76a9[id=1971086]</entry>
      <entry key="RA_CRYPTO_RC">0</entry>
      <entry key="AUDIT">true</entry>
      <entry key="MERCHANT">org.jpos.ee.Merchant@1e41b05[id=0010]</entry>
      <entry key="TERMINAL">org.jpos.ee.Terminal@1067fe5[id=2]</entry>
      <entry key="STORE">org.jpos.ee.Store@189406a[id=00660]</entry>
      <entry key="BALANCE_MAP">{98.18.0840=000000000000,
                  98.02.0840=000000000000, 98.01.0840=000000000000}</entry>
      <entry key="debug-f63">
       <ltvmsg>
         <tag id="22">
0000  41 50 50 52 4F 56 41 4C  20 20 20 20 20 20 20 20  APPROVAL       
         </tag>
         <tag id="62">
0000  03 3F 01 08 40 43 00 00  00 00 00 00 3F 02 08 40  .?..@C......?..@
0010  43 00 00 00 00 00 00 3F  18 08 40 43 00 00 00 00  C......?..@C....
0020  00 00                                             ..
         </tag>
       </ltvmsg>
      </entry>
      <entry key="RESPONSE">
       <fsdmsg schema='file:cfg/v2-ebt-resp-base'>
         header: '20202020'
         record-format: '0'
         terminal-serial-number: '12345678'
         encryption-indicator: '3'
         transaction-sequence-number: '5651'
         magnetic-strip-info: '507680______1999=__________________'
         register-number: '12'
         tran-id: '1658744'
         tender-attempt-indicator: '1'
         tender-number: '8200'
         tender-attempt: '1947'
         ebt-cash-balance: '0000000'
         ebt-fs-balance: '0000000'
         response-code: 'AA'
         display-message: 'APPROVAL'
         approval-number: '510730'
         ebt-auth-code: '510730'
         date: '010507'
         capture-date: '010507'
       </fsdmsg>
      </entry>
     </transient>
     <persistent>
      <entry key="TIMESTAMP">Fri Jan 05 15:29:51 EST 2007</entry>
      <entry key="OUTSTANDING">0</entry>
      <entry key="TRANLOG_ID">1971086</entry>
      <entry key="CAPTURE_DATE">Fri Jan 05 00:00:00 EST 2007</entry>
      <entry key="ISO_REQUEST">
       <isomsg direction="outgoing">
         <field id="0" value="0200"/>
         <field id="3" value="009800"/>
         <field id="4" value="1300"/>
         <field id="7" value="0105202951"/>
         <field id="11" value="002358"/>
         <field id="12" value="152951"/>
         <field id="13" value="0105"/>
         <field id="18" value="5912"/>
         <field id="22" value="901"/>
         <field id="24" value="001"/>
         <field id="25" value="00"/>
         <field id="35" value="507680______1999=__________________"/>
         <field id="37" value="700500660651"/>
         <field id="41" value="00519977"/>
         <field id="42" value="000992001999999"/>
         <field id="49" value="840"/>
         <field id="63"
          value="0024363220202020202020202020202020202020202020202020"
          type="binary"/>
       </isomsg>
      </entry>
      <entry key="ISO_RESPONSE">
       <isomsg direction="incoming">
         <field id="0" value="0210"/>
         <field id="3" value="009800"/>
         <field id="4" value="000000001300"/>
         <field id="7" value="0105153016"/>
         <field id="11" value="002358"/>
         <field id="12" value="152951"/>
         <field id="13" value="0105"/>
         <field id="24" value="001"/>
         <field id="25" value="00"/>
         <field id="37" value="700500660651"/>
         <field id="38" value="510730"/>
         <field id="39" value="00"/>
         <field id="41" value="00519977"/>
         <field id="63"
value="00183232415050524F56414C20202020202020200036363203980108404300
000000000098020840430000000000009818084043000000000000" type="binary"/>
       </isomsg>
      </entry>
      <entry key="RC">0000</entry>
     </persistent>
    </context>
  </commit>
</log>

[For an in-depth discussion of the jPOS Logger, please refer to Chapter 4 of the jPOS Programmer’s Guide. See also references to LogChannel in Tables 3.2 and 8.1 of the same document.  If you're serious about giving jPOS a shot, please do the needful and buy the Guide.  It's a great investment.  It will be instrumental in your ability to be successful with jPOS.]

The complicating factor nowadays is that most implementations need to be PCI-compliant, so a playback solution has to be more sophisticated. The reader here can see that in the log example provided above, all PAN- and PIN-related fields are either wiped or masked, per the governing standards. This level of obfuscation renders the transactions unplayable.

Therefore, the bar is raised to where – if playback is desired – you must store the replayable information in encrypted form. We would suggest the use of a Public Key Infrastructure (‘PKI’) to encrypt the message flow using the public key of a key pair.

Since users would not know the private key, they can’t replay the message, unless, under special conditions, probably using a secure, offline, decrypting station, you manually enter a passphrase to unlock the securely-held private key that in turn gives you the ability to decrypt the log with the full content. Then, we could make use of LogChannel to replay the transactions (or something similar if we are not talking about ISO8583).

[NOTE: PCI standards do not allow – in any circumstances – the storing of full card track or PIN block information from production systems even in encrypted form. We assume that playback requirements refer to test systems only, or, alternatively, refer to a facility in which test and/or randomly-generated card and PIN data is substituted for production data. Optimally, your audit staff ought to review any proposed playback facility.]

As an alternative approach for consideration, jPOS’ Space + TransactionManager architecture gives the jPOS framework implementer the ability to simultaneously queue a transaction into more than one Transaction Manager running in different nodes. For testing purposes, it is feasible to temporarily queue a copy of all production traffic into a test cluster (with simulated endpoints) without affecting the speed and integrity of online, production operations. This approach also mitigates the PCI compliance implications of playback as described above.

Now, let's talk about a manner to playback transactions in a more ad-hoc, one-by-one fashion.  We've designed our jPOS application with 'audit' capabilities.  So, for every terminal defined within the jPOS application framework, we can elect to turn on the 'audit' flag.  Then, we implemented our transaction log ('tranlog') with 'request' and 'response' columns.  If the 'audit' flag is turned on for a specific device, we throw copies of the raw request and response into those columns.  [NOTE:  Per PCI/CISP requirements, visibility and access to those columns in production is limited to appropriate admin-level personnel.] 

With the that data in hand, we can playback at will.  The one twist is that the data gets logged in raw (i.e., Hex) representation.  So, we can have a 'request' column that looks like this (I've broken this up into multiple chunks so it fits...in reality, this is one long string of Hex):

47302E3632383730323030303030303030303130303337373130303031343434343938373937
3230383031303936313233343536373833313339394D44725A6D734E795463703878716C7A1
C58461C333530341C301C303330353632383231313039393531323930

You can make use of online tools like Nick Ciske's neat little 'Sweet Sixteen' facility to translate that Hex into ASCII.  The result is something you can put into a transaction injector.  [You ought to have a nice tool like that at your disposal...you don't want to have to rely on someone to do a 'real' POS transaction to test your stuff.]

Comments

[J. Patrick Bedell]

Thank you for your great work! I am hoping to use your excellent software in the next version of the information currency issuance software at http://infoeng.sf.net . I'm hopeful that I will release a service (and software at http://sf.net/projects/sficm ) that enables the information currency ("IC" - neologism for digital financial instruments with information as the underlying asset) from jPOS commits (see icsvn) to be bought and sold to compensate and incentivize developers, and create a price structure for the individual contributions to jPOS. Thanks again for your work!