[QFJ-971] Intermittently we receive the "Required Tag Missing" error for existing fields Created: 08/Mar/19  Updated: 17/Jun/20  Resolved: 23/Dec/19

Status: Closed
Project: QuickFIX/J
Component/s: Message Generation
Affects Version/s: 1.5.3
Fix Version/s: 2.2.0

Type: Bug Priority: Default
Reporter: Michael Efis Assignee: Christoph John
Resolution: Fixed Votes: 0
Labels: Message

Issue Links:
Duplicate
is duplicated by QFJ-835 Required tag missing, field=447 Closed

 Description   

Hello QuickFix support,

Could you advise on the following issue, please?

We receive intermittent "Required Tag Missing" error while parsing incoming fix messages, even though the missing field is present in the message. For example in the following message the tag 447 is present on the message, and yet sometimes we receive “Required Tag Missing, field=447” while parsing it.

8=FIX.4.4|9=284|35=F|49=TEST_49|56=TEST_56|34=420|52=20190302-07:31:57.079|115=TEST3|116=TEST_116|11=TEST_11|41=TEST_41|55=TEST_55|48=TEST_48|22=4|54=2|60=20190302-07:31:56.933|38=TEST_38|207=TEST_207|454=3|455=TEST_455_1|456=TEST_456_1|455=TEST_455_2|456=TEST_456_2|455=TEST_455_3|456=TEST_456_3|453=1|448=TEST_448|447=D|452=3|10=XXX|

This issue has been observed with cancel messages and it normally happens after application startup. We do use a data dictionary, and 99.8% of the time such cancellation messages work fine. We use quick fix version 1.5.3. (quickfixj-core-1.5.3.jar and quickfixj-msg-fix44-1.5.3.jar)

Could you take a look at it?

Many thanks
Michael



 Comments   
Comment by Christopher Hurst [ 08/Mar/19 ]

Hi,

It might be a different problem and we use a different version of FIX but ..

I think we saw the same thing a year back (my memory is slightly hazy) but like you say its intermittent and would stop happening after a while and not be reproducible . It only occurred for us in E2E tests on VMs so we just so it was not important as it would go away (We only use QFJ as part of a test tool)

The message logs always saw the correct tags indicated as missing, it may well have been the same tag you quote (I can't remember the message) and we validated the offending FIX with an external tool (all fine).

I managed to hack quickfix to serialise on failure the message then reloaded it in a test bed but could find nothing wrong with the object.

Eventually the problem just stopped happening and wasn't reproducible , though it does seem to come back periodically but not often enough to debug it properly,

Comment by Michael Efis [ 18/Mar/19 ]

The same issue has been observed with other message types

Comment by Christoph John [ 08/Jul/19 ]

I am afraid we cannot analyse this without a reproducible test case.

Comment by Michael Efis [ 08/Jul/19 ]

Christopher,

I appreciate that without knowing someone else's code, it's difficult to say where the problem is.

In our case we send strings with messages directly to QuickFix, where we get the intermittent errors. There is basically nothing in between. If we could reliably reproduce this error, then we would have known the root cause of the issue and then probably we could create a reliable, testable workaround (for example some sort re-try mechanism) without involving quick fix support.

Unfortunately this error is too unpredictable and we can't reliably test any of our solutions.

The symptoms that we see (intermittent errors on the application startup) look similar to a concurrency issue around QuickFix initialization. However, without full understanding of the QuickFix code we cannot be sure if this is the case. This is where we would appreciate help from QuickFix support.

Thanks
Michael

Comment by Christoph John [ 08/Jul/19 ]

Hi,
I also answered this on the mailing list:

Hi,

unfortunately I do not have a solution but also did see such problems with a client of ours and it was also the exact same tag 447 that was the problem (although another message type). The application was already running for several hours.
We also could not reproduce it. It just appeared once every few months... :-/

Cheers,
Chris.

At least for our client it was no startup issue.

Comment by Christoph John [ 05/Sep/19 ]

Hi Michael Efis,

would it be possible to see the code snippet how you send the message strings to QFJ as you mentioned above? Are you using MessageUtils.parse() or another method? How does the party group in your data dictionary look like?

Thanks,
Chris.

Comment by Christoph John [ 20/Dec/19 ]

See issue and pull request by AndreyNudko:
https://github.com/quickfix-j/quickfixj/issues/240
https://github.com/quickfix-j/quickfixj/pull/241

Generated at Fri Nov 22 23:59:53 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.