[QFJ-544] Header with BeginString=FIXT.1.1 incorrectly parsed by FIXMessageDecoder (manifests as missing messages) Created: 20/Jul/10  Updated: 04/Sep/12  Resolved: 02/Aug/12

Status: Resolved
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: 1.5.3

Type: Bug Priority: Default
Reporter: Stelios Papadopoulos Assignee: Christoph John
Resolution: Fixed Votes: 3
Labels: None

Attachments: Text File quickfixj-1.4.patch     Text File quickfixj-1.5.patch    
Issue Links:
Duplicate
is duplicated by QFJ-505 FixMessageDecoder.startsWith() wrongl... Closed
Relates
is related to QFJ-687 Lost message on package fragmentation... Closed

 Description   

When BeginString=FIXT.1.1 messages can be skipped if the buffer ends before the last '=' of "8=FIXT.1.1\0019=". That will happen if the buffer contains a correct message before that.

Example:

in messages:

8=FIXT.1.1\0019=12\00135=X\001108=30\00110=036\0018=FIXT.1.1\0019=12\00135=X\001108=30\00110=036\001

the second message will fail to be decoded if the buffer contents presented to the FixMessageDecoder in two subsequent decode() invocations are:

"8=FIXT.1.1\0019=12\00135=X\001108=30\00110=036\0018=FIXT.1.1\0019"

and

"=12\00135=X\001108=30\00110=036\001"

I have created tests to replicate this issue (and also issue QFJ-505) and included fixes (for this and the fix proposed in QFJ-505).
Here is the diff:

Index: core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java
===================================================================
— core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java (revision 46)
+++ core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java (revision 49)
@@ -348,7 +348,22 @@

@Test
public void testMinaDemux() throws Exception

{ - DemuxingProtocolCodecFactory codecFactory = new DemuxingProtocolCodecFactory(); + String message = "8=FIX.4.2\0019=12\00135=X\001108=30\00110=036\001"; + + doTestMinaDemux(message); + + }

+
+ @Test
+ public void testMinaDemuxFixt() throws Exception

{ + String message = "8=FIXT.1.1\0019=12\00135=X\001108=30\00110=036\001"; + + doTestMinaDemux(message); + + }

+
+ private void doTestMinaDemux(String message) throws Exception, UnsupportedEncodingException {
+ DemuxingProtocolCodecFactory codecFactory = new DemuxingProtocolCodecFactory();
codecFactory.register(FIXMessageDecoder.class);

ProtocolDecoder decoder = codecFactory.getDecoder();
@@ -359,7 +374,7 @@
int count = 5;
String data = "";
for (int i = 0; i < count; i++)

{ - data += "8=FIX.4.2\0019=12\00135=X\001108=30\00110=036\001"; + data += message; }

for (int i = 1; i < data.length(); i++)

{ @@ -379,9 +394,8 @@ output.reset(); buffer.clear(); }

-

  • }
    -
    + }
    +
    private void assertMessageFound(String data) throws ProtocolCodecException { assertMessageFound(data, 1); }

    Index: core/src/main/java/quickfix/mina/message/FIXMessageDecoder.java
    ===================================================================

      • core/src/main/java/quickfix/mina/message/FIXMessageDecoder.java (revision 46)
        +++ core/src/main/java/quickfix/mina/message/FIXMessageDecoder.java (revision 49)
        @@ -123,7 +123,7 @@
        }
        if (messageCount > 0) {
        // Mina will compact the buffer because we can't detect a header
  • if (in.remaining() < minMaskLength(HEADER_PATTERN))
    Unknown macro: {+ if (state == SEEKING_HEADER) { position = 0; } return MessageDecoderResult.OK;@@ -297,7 +297,8 @@ }

private static BufPos indexOf(ByteBuffer buffer, int position, byte[] data) {

  • for (int offset = position, limit = buffer.limit() - minMaskLength(data) + 1; offset < limit; offset++) {
    + int limit = buffer.limit() - minMaskLength(data) + 1;
    + for (int offset = position ; offset < limit; offset++)
    Unknown macro: { int length; if (buffer.get(offset) == data[0] && (length = startsWith(buffer, offset, data)) > 0) { return new BufPos(offset, length); @@ -337,6 +338,10 @@ return -1; } }

    + if(dataOffset != data.length)

    { + // when minMaskLength(data) != data.length we might run out of buffer before we run out of data + return -1; + }

    return bufferOffset - initOffset;
    }



 Comments   
Comment by Stelios Papadopoulos [ 20/Jul/10 ]

the svn references are unrelated to the QuickFixJ svn repository.

Comment by Stelios Papadopoulos [ 20/Jul/10 ]

Found the same issue in 1.5.0 here's my diffs for the tests and the fix:

Index: C:/dev/projects/QUICKFIXJ-IG/core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java
===================================================================
— C:/dev/projects/QUICKFIXJ-IG/core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java (revision 52)
+++ C:/dev/projects/QUICKFIXJ-IG/core/src/test/java/quickfix/mina/message/FIXMessageDecoderTest.java (working copy)
@@ -327,7 +327,22 @@

@Test
public void testMinaDemux() throws Exception

{ - DemuxingProtocolCodecFactory codecFactory = new DemuxingProtocolCodecFactory(); + String message = "8=FIX.4.2\0019=12\00135=X\001108=30\00110=036\001"; + + doTestMinaDemux(message); + + }

+
+ @Test
+ public void testMinaDemuxFixt() throws Exception

{ + String message = "8=FIXT.1.1\0019=12\00135=X\001108=30\00110=036\001"; + + doTestMinaDemux(message); + + }

+
+ private void doTestMinaDemux(String message) throws Exception, UnsupportedEncodingException {
+ DemuxingProtocolCodecFactory codecFactory = new DemuxingProtocolCodecFactory();
codecFactory.register(FIXMessageDecoder.class);

ProtocolDecoder decoder = codecFactory.getDecoder();
@@ -338,7 +353,7 @@
int count = 5;
String data = "";
for (int i = 0; i < count; i++)

{ - data += "8=FIX.4.2\0019=12\00135=X\001108=30\00110=036\001"; + data += message; }

for (int i = 1; i < data.length(); i++)

{ @@ -358,9 +373,8 @@ output.reset(); buffer.clear(); }

+ }

  • }
    -
    private void assertMessageFound(String data) throws ProtocolCodecException { assertMessageFound(data, 1); }

Index: C:/dev/projects/QUICKFIXJ-IG/core/src/main/java/quickfix/mina/message/FIXMessageDecoder.java
===================================================================
— C:/dev/projects/QUICKFIXJ-IG/core/src/main/java/quickfix/mina/message/FIXMessageDecoder.java (revision 52)
+++ C:/dev/projects/QUICKFIXJ-IG/core/src/main/java/quickfix/mina/message/FIXMessageDecoder.java (working copy)
@@ -123,7 +123,7 @@
}
if (messageCount > 0) {
// Mina will compact the buffer because we can't detect a header

  • if (in.remaining() < minMaskLength(HEADER_PATTERN))
    Unknown macro: {+ if (state == SEEKING_HEADER) { position = 0; } return MessageDecoderResult.OK;@@ -337,6 +337,10 @@ return -1; }

    }
    + if(dataOffset != data.length)

    { + // when minMaskLength(data) != data.length we might run out of buffer before we run out of data + return -1; + }

    return bufferOffset - initOffset;
    }

Comment by Stelios Papadopoulos [ 01/Aug/12 ]

revision numbers are unrelated to the official repository.

fix for the 1.4.x branch

Comment by Stelios Papadopoulos [ 01/Aug/12 ]

revision numbers are unrelated to the official repository.

fix for the 1.5.x branch

Comment by Christoph John [ 02/Aug/12 ]

Stelios, thanks for the patch!

Committed as r1075 on trunk.

Generated at Sat Nov 23 09:05:10 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.