Originally observed in quickfix/j 1.0.0, jdk 1.4.2, mina 0.9.3.
Also Reproduced with quickfix/j trunk, jdk 1.5, mina 0.9.5 trunk.

I actually think this is a bug in MINA. The PooledThreadModel which is used by the default configuration creates a new ThreadPoolFilter which is what is creating these wayward threads. I don't think the destroy method of this filter ever gets called. If the PooledThreadModel used a ReferenceCountingIoFilter to wrap the ThreadPoolFilter before adding it to the chain then destroy would be called

I'm also not sure if quickfix should even be using the PooledThreadModel - I think the event handling strategies do this part instead. So you might be able to do something like this in IoSessionInitiator to use no threading:

public synchronized void connect() {
lastReconnectAttemptTime = SystemTime.currentTimeMillis();
try

catch (Throwable e)

}

It seems to work ok for Banzai stops all the extra thread creation, but I haven't tried it beyond that.

Slightly off topic:

I think the PooledThreadModel/ThreadPoolFilter would take care of quickfix/j's thread pool needs as described in http://sourceforge.net/mailarchive/forum.php?thread_id=10923994&forum_id=103.

ThreadPoolFilter uses an internal queue (ThreadPoolFilter$SessionBuffer) per MINA session. Each queue can be processed by at most one worker thread from the thread pool at once, so each message would be processed in order. The EventHandlingStrategies would no longer be used as onMessage would be called by one of the worker threads.

The worker thread will process events on the queue until it is empty, and more events can be added while it is running. I think this means a busy Session could starve other sessions if the thread pool is too small so it may not be directly suitable as a SingleThreadedEventHandlingStrategy replacement - an alternate ThreadPoolFilter that only processes one event from the queue may be required. ThreadPerSession could be implemented by supplying an unbounded thread pool.

This all assumes that the ThreadPoolFilter gets cleaned up properly when the session is closed, of course.

Brad, thanks for investigating this issue. I'm traveling internationally right now but I'll review all your comments when I return. I definitely appreciate the help.

I added the code for your original suggestion. The other suggestion about using MINA thread pooling is interesting, but I'd like to wait until the 0.9.x API stabilizes (MINA 1.0 release) before I tie to the QFJ code into it very deeply. The changes are on the branch: QFJ_RELEASE_1_0_x (and merged to the trunk).

Thanks Steve. Note that my code was a bit naughty in that it used the base class instead of the interface for IoServiceConfig.

I noticed some additional discussion about MINA thread models. It appears that Trustin and Peter want to remove the interface from MINA and recommend using the "manual" thread pool management (which we are doing). Brad (or anyone else), have you been following the discussion and are you familiar with the latest thoughts on th issue?

http://www.nabble.com/Is-ThreadModel-really-useful--t2778892.html

Thanks for the link, no I hadn't been following it. Sounds like you'll just have to remove the setThreadModel calls and come back a full circle

To implement thread handling I think they're still recommending using a ThreadPoolFilter but adding it manually to the filter chain, as opposed to using the ThreadModel abstraction. So most of the "slightly off topic" stuff about ThreadPoolFilter above should still apply if anyone is after a thread pool in quickfix.

Cheers,
Brad.