Adding timeout, keep-alive properties to your channel

I’ve written in the past about some of the operational improvements we’ve made in our QMUX configurations over time.  As a prime example, we’ve had two situations where an authorizer’s tight disconnect model has forced us to go with a MUX Pool approach.  Now here recently, we’ve been presented with another ‘opportunity’ for improvement.  Specifically, we’ve wanted to address a situation seen in production at a client site where our connections to Stored Value Systems (‘SVS’) channels disconnected, then got into a hung state after reconnect.  The status lights stayed green and transactions repeatedly timed-out over the impacted channel.  [NOTE:  SVS is a really good service provider; the issue here was on our end, so it makes for a good, real-life example.]

Alejandro advised us to implement a property value that will set a socket-level timeout.  The ‘receive’ function in the multiplexer’s ‘Channel Adapter’ will fail (with log event ‘<io_timeout>’) if nothing is received within the specified timeout period.  The channel will disconnect and then attempt a reconnection. 

While you can apply these property values to any QMUX channel, we had to take some special care for this one because of the MUX Pool considerations.  The specified value of the <timeout> property should be greater than the related <echo-interval> specified in the related logon manager.  SVS features a tight (3 minutes 30 seconds) disconnect model, leading us to have to implement the ‘MUX pool’ approach with aggressive echo intervals (180000 ms, or three minutes).  The timeout property value specified needs to provide breathing space to allow the echoes to operate in their intended fashion.  An appropriate value is 300000 (five minutes).  Since the MUX pool forces an echo on each channel in the SVS MUX every three minutes, not getting ‘receive’ activity in five minutes raises the possibility of a ‘hung’ line.  Accordingly, we put the channel through a disconnect/reconnect cycle as a proactive measure.

The final result:  our channel configuration now looks like this – see new lines in bold, red (NOTE: I’ve obfuscated the ‘host’ value here):

<channel-adaptor name='svs'
    class="org.jpos.q2.iso.ChannelAdaptor" logger="Q2">
<channel class="org.jpos.iso.channel.NACChannel" logger="Q2"
       realm="svs-channel"
       packager="org.jpos.iso.packager.GenericPackager">
  <property name="packager-config" value="cfg/svs.xml" />
  <property name="host" value="299.999.999.999" />
  <property name="port" value="24110" />
  <property name="timeout" value="300000" />
  <property name="keep-alive" value="true" /> 
</channel>
<in>svs-send</in>
<out>svs-receive</out>
<reconnect-delay>10000</reconnect-delay>
</channel-adaptor>

For the record, the corresponding Logon Manager configuration – which remains unchanged in this exercise – looks like this:

<svs-logon-mgr class="org.jpos.svs.LogonManager" logger="Q2">
<property name="persistent-space" value="jdbm:svslogon:log/svslogon" />
<property name="mux"            value="svs-mux-0" />
<property name="channel-ready"  value="svs.ready" />
<property name="timeout"        value="900000" />
<property name="echo-interval"  value="180000" />
<property name="logon-interval" value="43200000" />
</svs-logon-mgr>