PLEASE HELP--2 instances of the application reading the messages using IMAP

CodeSeeker

I have an application, which uses pop3 to read the messages from the mailbox, and it has been working fine for so many year. We recently have started changing this application to use java mail IMAP 4 instead of Pop3.

Primary reason for that :- Is we want to better meet out SLA, so we have decided to go with the 2 instances of this application, and hence decided to use IMAP4 to concurrently accessing the mailboxes via IMAP4.

My issues:- Now when, these 2 differet instances is trying to read same mailbox exactly at same time, It is reading the same message twice, as 1 instance is not aware of state changes in the 2nd instance.

Here is code that checks the mailboxes. The primary method, which checks a particular mailbox is checkMailbox

Please help with your suggestions, in terms of how to avoid reading the same message multiple times by these 2 different instances will be highly appreciated.
================================================== ========

Expand|Select|Wrap|Line Numbers

  import java.io.ByteArrayOutputStream;

import java.io.File;

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.OutputStream;
 
import javax.mail.Address;

import javax.mail.Flags;

import javax.mail.Folder;

import javax.mail.Message;

import javax.mail.MessagingException;

import javax.mail.Session;

import javax.mail.Store;

import javax.mail.URLName;

import javax.mail.internet.MailDateFormat;

import javax.mail.internet.MimeMessage;

import javax.mail.Flags;

import javax.mail.event.*;
 
import mypackage.core.application.AppManager;

import mypackage.core.base.ClydeException;

import mypackage.core.utility.ClydeUtil;

import mypackage.core.utility.MsgTrace;
 
import com.northwesternmutual.mail.util.LoggingWrapper;

import com.northwesternmutual.mailreader.application.MailReaderAppInit;

import com.northwesternmutual.mailreader.handler.access.INewMailHandler;

import com.northwesternmutual.mailreader.handler.access.NewMailHandlerHome;
 
/**

* Timer class that polls a mailbox and processes any messages

*/

class MailboxWatcher implements Runnable 

{

private MailBox mailbox = null;

MailDateFormat mailDateFormat = new MailDateFormat();

File problemCheckpointFile = null;

String problemMessageId = null;
 
private boolean checkForCorruptedFrom = ClydeUtil.getZimbuEnvVar("REJECT_DUAL_FROM", "N").equalsIgnoreCase("Y");

private boolean quarantineAvailable = ClydeUtil.getZimbuEnvVar("QUARANTINE_CHECKPOINT_DIR", false) != null;

private boolean deleteAfterQuarantine = quarantineAvailable && ClydeUtil.getZimbuEnvVar("DELETE_AFTER_QUARANTINE", "N").equalsIgnoreCase("Y");
 
/**

* MailboxWatcher constructor comment.

* 

* @param mailBox

* MailBox mailbox to poll

*/

public MailboxWatcher(MailBox mailBox) {

super();

mailbox = mailBox;

}

/**

* Check for new messages. 

* 

* @param aAlarm

* the alarm ringing me

*/

protected synchronized void checkMailbox() 

{

Store store = null;

Folder f = null;

try

{

Session s = Session.getDefaultInstance(System.getProperties());

//s.setDebug(true);

// open the mailbox

store = s.getStore(mailbox.getIMAP4Url());
 
store.connect();
 
// And get the folder we want.

f = store.getFolder("INBOX");
 
if(f.isOpen()){

//f.close(false);

return;

}

f.open(Folder.READ_WRITE);
 
Message[] messages = f.getMessages();
 
LoggingWrapper.trace(messages.length+ " messages @ " +mailbox, 

MailReaderAppInit.GENERAL_CATEGORY, MsgTrace.MINIMAL);
 
if (messages.length > 0)

{

// Loop while there are messages to process, and the application

// isn't on

// its way down. Since this thread will stick around we check

// isInitialized().

for (int loop = 0; 

loop < messages.length && AppManager.isInitialized();

loop++)

{

boolean flag = false;

//Before the message processing continue we will mark the message system flag as SEEN
 
LoggingWrapper.trace(messages[loop].getMessageNumber()+ "with messages messaage number got changed " , 

MailReaderAppInit.GENERAL_CATEGORY, MsgTrace.MINIMAL);

// If we don't process the next messag,e then don't continue
 
if (!processNextMessage(messages[loop]))

{
 
loop = messages.length;

f.expunge();

}
 
}

}
 
f.close(true);

}

catch (MessagingException me)

{

ClydeException.raise(ClydeException.INTERNAL_ERROR,

"EMCR001",

"Error polling mailbox " + mailbox,

me.getMessage(),

true,

false,

me);

}catch(Exception ex){
 
}

finally

{

//We close the folder and delete the messages that succeeded

try

{

if (f !=null && f.isOpen())

{

f.close(true);

}

}

catch (MessagingException me) {}
 
try

{

if (store !=null && store.isConnected())

{

store.close();

}

}

catch (MessagingException me) {}

}

}
 
/**

* Get a unique value from the mail messages

* 

* @return java.lang.String

* @param msg

* javax.mail.Message

*/

private String getUniqueMessageId(Message msg) 

throws MessagingException

{

// This works 99% of the time

String msgIds[] = msg.getHeader("Message-Id");

if (msgIds == null)

{

// this is needed when a message is returned to sender from the OpenMail

// server

// .... I think OM is broke in this regard.

msgIds = msg.getHeader("In-Reply-To");

if (msgIds == null)

{

ClydeException.raise(ClydeException.INTERNAL_ERROR, 

"EMCR014",

"Couldn't get UNIQUE message id",

"No messge Id, or reply-to");

}

}
 
String id = msgIds[0];

// Strip off the < and the >

id = id.substring(1, id.length()-1);

// And we'll also remove the @ since it's kind of ugly

id = id.replace('@', '_'); 

return id;

}
 
/**

* returns false to stop processing this mailbox or true to continue

*/ 

private boolean processNextMessage(Message msg)

throws MessagingException

{

INewMailHandler handler = NewMailHandlerHome.instance().getHandler();
 
String messageId = getUniqueMessageId(msg);
 
Message copyOfMessage = copyMessage(msg);
 
// Then we send this message out.

try

{

isCorruptedFrom(copyOfMessage);

if (handler.message(mailbox.getMessageQueue(),

mailbox.getAttachmentQueue(),

copyOfMessage))

{

msg.setFlag(Flags.Flag.DELETED, true);
 
}

else 

{

// This stops sending all messages if the handler failed

return false;

}

}

catch (MessagingException me)

{

quarantineAndDelete(msg, messageId, copyOfMessage, me);

return false;

}

catch (ClydeException ce)

{

if (ce.techMsg().indexOf("2030") > -1 ||

ce.techMsg().indexOf("2031") > -1)

{

quarantineAndDelete(msg, messageId, copyOfMessage, ce);

return false;

}

else

{

throw ce;

}

}
 
return true;

}
 
/**

* Returns a read/write version of the message. POP3 messages are read only, and

* we want to be able to update it a bit.

* 

* @param msg

* @return Message

* @throws MessagingException

*/

private Message copyMessage(Message msg) throws MessagingException {

// ..First mark which mailbox we read it from.

Message copyOfMessage = new MimeMessage((MimeMessage)msg);

fixExciteDotComBoundaryProblem(msg, copyOfMessage);

copyOfMessage.addHeader("X-MailReader-MailBox", mailbox.getUserId());

flushOutCopyOfMessage(copyOfMessage);

return copyOfMessage;

}
 
/**

* Some WebMail providers don't appear to follow the RFC spec for mail messages.

* Specifically, they add an extra semicolon to the Content-Type. See RFC 2387

* section 3.4

* 

* @param msg

* @param copyOfMessage

* @throws MessagingException

*/

private void fixExciteDotComBoundaryProblem(Message msg, Message copyOfMessage)

throws MessagingException 

{

// Check for a mishandled content-type header... Darn WebMail providers...

String contentTypes[] = msg.getHeader("Content-Type");

if (contentTypes != null && contentTypes.length > 0)

{

String contentType = contentTypes[0].trim();

if (contentType.endsWith(";"))

{

contentType = contentType.substring(0,contentType.length()-1);

}

copyOfMessage.setHeader("Content-Type", contentType);

}

}
 
/**

* Certain fields aren't flushed out unless the message writeTo() is called.

* This prevents problems in the send.

* 

* @param copyOfMessage

* @throws MessagingException

*/

private void flushOutCopyOfMessage(Message copyOfMessage)

throws MessagingException 

{

// Flush out extra fields by dumping it out...

OutputStream os = new ByteArrayOutputStream(10240);

try { copyOfMessage.writeTo(os); } catch (IOException io) {}

try { os.close(); } catch (IOException e) {}

}
 
/**

* Log a message, then if quarantine is available, quarantine it. If

* deleteAfterQuarantine is set, then the message is removed from the mailbox

* 

* @param f

* @param msg

* @param messageId

* @param copyOfMessage

* @param originalException

* @throws MessagingException

*/

private void quarantineAndDelete(Message msg,

String messageId,

Message copyOfMessage,

Throwable originalThrowable)

throws MessagingException 

{

try 

{

ClydeException.raise(ClydeException.INTERNAL_ERROR,

"QUAR001", 

"Attempting Quarantining message",

"Quarantining message: "+getUniqueMessageId(msg) +" from INBOX",

true, false, originalThrowable);

}

catch (ClydeException justForLogging) {}
 
if (quarantineAvailable)

{

quarantineMessage(copyOfMessage);

if (deleteAfterQuarantine)

{

msg.setFlag(Flags.Flag.DELETED, true);

}

}

}
 
/**

* Method isAutoReplyMessage.

* 

* @param msg

* @return boolean

*/

private boolean isAutoReplyMessage(Message msg) throws MessagingException {

// X-OpenMail-Autoreplied: TRUE

String headers[] = msg.getHeader("X-OpenMail-Autoreplied");

// If the headers aren't there, it definetly isn't an autoreply

if (headers == null || headers.length == 0)

return false;

if (headers[0].equalsIgnoreCase("true"))

return true;

return false;

}
 
/**

* Method isCorruptedFrom

* 

* @param msg

* @return boolean

*/

private boolean isCorruptedFrom(Message msg) 

throws MessagingException 

{

if (checkForCorruptedFrom == true)

{

Address froms[] = msg.getFrom();

System.out.println("From address:-"+froms[0]);
 
// If the headers aren't there, it definetly isn't an autoreply

if (froms != null && froms.length > 1)

{

throw new MessagingException("Message contains corrupted From: " + msg.getHeader("From")[0]);

}

}

return false;

}
 
private void quarantineMessage(Message msg)

throws MessagingException

{

try

{

File quarantine = new File(ClydeUtil.getZimbuEnvVar("QUARANTINE_CHECKPOINT_DIR"), "quarantine." getUniqueMessageId(msg)".msg");

FileOutputStream fos = new FileOutputStream(quarantine);

msg.writeTo(fos);

fos.close();

LoggingWrapper.trace("Quarantining message: "+getUniqueMessageId(msg) " into " quarantine + " from INBOX",

MailReaderAppInit.GENERAL_CATEGORY, MsgTrace.VERBOSE);
 
try 

{

ClydeException.raise(ClydeException.INTERNAL_ERROR,

"QUAR001", 

"Quarantining message",

"Quarantining message: "+getUniqueMessageId(msg) " into " quarantine + " from INBOX");

}

catch (ClydeException justForLogging) {}

}

catch (IOException io)

{

throw new MessagingException("Problem quarantining message", io);

}

}
 
/**

* @see java.lang.Runnable

*/

public void run() 

{

try {
 
checkMailbox();
 
} catch (Throwable th)

{

ClydeUtil.dump(th);

LoggingWrapper.trace("checkMailbox threw an uncaught throwable", MailReaderAppInit.GENERAL_CATEGORY, 

MsgTrace.MINIMAL);

}

finally

{

mailbox.setNextPollTime();

}

}

}

Plater

It was my impression of both pop/imap standards that the server for these should not be allowing concurrent access to the same mailbox. (For just such reasons I would imagine)
A valid mailbox connection access is supposed to mark the mailbox as locked until such a time as the connection is terminated.
It would seem that your server is not doing that?

Is it possible for you to set up some sort of 'mutex' type system, i.e. implement your own methods for locking a mailbox while it's in use (however brief). If the idle disconnect is short for the connections, you could just have your application block until the mailbox becomes unlocked. As long as the individual access times to your server remain short, there wouldn't really be a slow down.

PLEASE HELP--2 instances of the application reading the messages using IMAP

re: PLEASE HELP--2 instances of the application reading the messages using IMAP

/bytes/development

/bytes/it

/bytes/about