I'm working in a multi-module project which packaged as EAR files and uses WebLogic V12.2.1.4 as the Application Server. Speaking about the architecture, Communication between the critical modules were carried out through queues internally which was a loosely coupled way and the rest were achieved by EJB lookups.
Recently I observed that some of the messages passed were not submitted to the Async Listener setup in the Specific module which was placed to Audit the user. The sole purpose of this Audit module is to persists States related to user who is performing the action.
The listener is a MDB and the implementation is as follows
@MessageDriven(
name = "UserAudit",
messageListenerInterface = MessageListener.class,
mappedName = "jms/UserAudit",
activationConfig = {
@ActivationConfigProperty(propertyName = "ConnectionFactoryJndiName", propertyValue = "Jms/ConnectionFactory"),
@ActivationConfigProperty(propertyName = "DestinationType", propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "DestinationName", propertyValue = "jms/UserAudit"),
@ActivationConfigProperty(propertyName = "ReceiverThreads", propertyValue = "1")
}
)
public class UserAudit extends GenericMessageDrivenBean implements MessageListener {
@PersistenceContext(unitName = "audit-datasource")
private EntityManager entityManager;
@Override
@TransactionAttribute(TransactionAttributeType.REQUIRED)
public void onMessage(Message message) {
try {
//perform action
entityManager.flush();
} catch(Exception ex) {
Logger.error("Unexpected behavior in recording user audit: " + ex.getMessage());
}
}
}
Here is the class I'm using to push messages to queue
import javax.jms.*;
import javax.naming.InitialContext;
import javax.naming.NamingException;
public class PublishToQueue {
private String defaultConnectionFactory = "Jms/ConnectionFactory";
private PublishToQueue() {}
public static PublishToQueue getQueueHandler() {
return new PublishToQueue();
}
public void publish( Event event, String destination) {
try {
InitialContext ctx = new InitialContext();
QueueConnectionFactory factory = (QueueConnectionFactory) ctx.lookup(this.defaultConnectionFactory);
QueueConnection queueConn = factory.createQueueConnection();
QueueSession queueSession = queueConn.createQueueSession(false, Session.DUPS_OK_ACKNOWLEDGE);
Queue queue = (Queue) ctx.lookup(destination);
QueueSender queueSender = queueSession.createSender(queue);
queueSender.setDeliveryMode(DeliveryMode.PERSISTENT);
ObjectMessage objectMessage = queueSession.createObjectMessage(event);
queueSender.send(objectMessage);
queueSender.close();
} catch (JMSException ex) {
Logger.error(ex, "Unable to load connection factory context");
} catch(NamingException ex) {
Logger.error(ex, "Unable to load connection factory context");
}
}
}
I found out below Debug statements from WebLogic Console for the DEBUG level
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | AsyncSendCallbackRunner created for Session@1727035488, WorkManager@1370915913[null@null@JmsAsyncQueue, weblogic.work.ServerWorkManagerImpl] | JMSMessagePath | DOMAIN |
|---|---|---|---|---|---|
| BEA-000000 | Debug | FESession.checkPartition: connPartiton=DOMAIN, destPartition=null | JMSFrontEnd | DOMAIN | |
| BEA-000000 | Debug | FEProducer.() | JMSFrontEnd | DOMAIN | |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | CLIENT/JMSProducer (id: <6887940678975293522.242>) : Sending message deliveryMode = 2 priority = 4 timeToLive = 0 timeToDeliver = 0 redeliveryLimit = -1 | JMSMessagePath | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | CLIENT/JMSProducer (id: <6887940678975293522.242>) : Dispatching message to FRONTEND/FEProducer for destination: JmsManager!UserAudit, appListener=null, dispatch mode=0, isJMSSessionPooledInWrapper=false | JMSMessagePath | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | FRONTEND/FEProducer: Assigning message ID <244143.1708267182584.0> | JMSMessagePath | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | FRONTEND/FEProducer: Dispatching message to BACKEND/BEDestination <244143.1708267182584.0> request weblogic.jms.frontend.FEProducerSendRequest@468b1421[parentRequest.jmsAsyncSend=false] dispatcher Local_Admin dest JmsManager!UserAudit | JMSMessagePath | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | Checking Sequence Mode: needReorcer = false forwarded = false | JMSBackEnd | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | Putting new message <244143.1708267182584.0> on JmsManager!UserAudit | JMSBackEnd | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | Message <244143.1708267182584.0> successfully enqueued | JMSBackEnd | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | CLIENT/JMSProducer (id: <6887940678975293522.242>) : Successfully sent message ID:<244143.1708267182584.0> | JMSMessagePath | DOMAIN |
| BEA1-03912F3A40D660319C07 | BEA-000000 | Debug | CLIENT/JMSProducer.waitTillAllJMSAsyncSendProcessed(): jmsAsyncSendCount=0, this=connection240.session241.producer242 | JMSMessagePath | DOMAIN |
After the waitTillAllJMSAsyncSendProcessed() method call I can't see the message progress through the JMS system.
I observed this after pausing the UserAudit queue for pause consumption which was placed for a requirement to test the queue.
The project deployed on Weblogic V12.2.1.4 and the JMS dependency is com.oracle:jms:10.1.3
My initial though would be that some behavior causes Thread Deadlock and the Producer can't push the message to JMSBackend. What could cause this behavior and what are the probable ways to overcome this behavior?