Hi,
I am futzing around with Andrew Stuarts "Catchmail" program
that stores emails into a postgresql database.
I want to avoid inserting the same email more than once...
(pieces of the email actually get emplaced into several
tables).
Is the "Message-ID" header field a globally unique identifer?
I eventually want to have a cron job process my inbox and don't
want successive cron tasks to keep re-entering the same email :)
Jerry
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postg resql.org so that your
message can get through to the mailing list cleanly 10 4262
No, it's not a global unique identifier. In fact you cannot even be sure
it will always be there in all mails, depending on your mail processing
chain. Most of the email clients will add one, and most of the mail
transfer agents will also add one if missing, but there's no general
rule of how to create the mail id. Usually it will be unique for the
specific instance of the user/transfer agent which generates it, as it
serves for exact this purpose, to identify the mail instance on the
agent instance, but nothing more.
So: don't use it as a unique identifier.
Cheers,
Csaba.
On Tue, 2004-10-12 at 17:01, Jerry LeVan wrote: Hi, I am futzing around with Andrew Stuarts "Catchmail" program that stores emails into a postgresql database.
I want to avoid inserting the same email more than once... (pieces of the email actually get emplaced into several tables).
Is the "Message-ID" header field a globally unique identifer?
I eventually want to have a cron job process my inbox and don't want successive cron tasks to keep re-entering the same email :)
Jerry
---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to ma*******@postg resql.org so that your message can get through to the mailing list cleanly
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings
On Tue, 2004-10-12 at 17:01, Jerry LeVan wrote: Hi, I am futzing around with Andrew Stuarts "Catchmail" program that stores emails into a postgresql database.
I want to avoid inserting the same email more than once... (pieces of the email actually get emplaced into several tables).
Is the "Message-ID" header field a globally unique identifer?
Its supposed to be that, yes. Unless you meet a broken client
or MTA.
I eventually want to have a cron job process my inbox and don't want successive cron tasks to keep re-entering the same email :)
Why not a script which receives the mail in behalf of
the user? Time to run some spam checkers too.
Regards
Tino
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Jerry LeVan <je*********@ek u.edu> writes: Hi, I am futzing around with Andrew Stuarts "Catchmail" program that stores emails into a postgresql database.
I want to avoid inserting the same email more than once... (pieces of the email actually get emplaced into several tables).
Is the "Message-ID" header field a globally unique identifer?
It is intended to be, but since it is generated by either the MUA or
the first MTA that sees the message, there is a lot of scope for
broken software to screw things up.
-Doug
--
Let us cross over the river, and rest under the shade of the trees.
--T. J. Jackson, 1863
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
On Tue, 12 Oct 2004 11:01:08 -0400, Jerry LeVan <je*********@ek u.edu> wrote: Hi, I am futzing around with Andrew Stuarts "Catchmail" program that stores emails into a postgresql database.
I want to avoid inserting the same email more than once... (pieces of the email actually get emplaced into several tables).
Is the "Message-ID" header field a globally unique identifer?
I eventually want to have a cron job process my inbox and don't want successive cron tasks to keep re-entering the same email :)
In terms of Internet mail? Answer is... almost.
The idea is that each mail has an unique Message-ID, but there are
cases when few "different" mails get same Message-ID. Such can be
the case with mailing lists, like the one you are reading right now.
Suppose you are "crosssendi ng" a message, telling:
To: pg***********@p ostgresql.org
Cc: li**********@ke rnel.org
the message will arrive here and a copy will be sent to each mailing list.
Then these twin messages will be processed by mailing list software,
subjects will have [something] prepended in case of pgsql-general,
the linux-kernel will have custom "signature" at the end of a message,
pgsql-general will have "TIPS" as a signature.
Then suppose you are subscribed to both lists. You will receive both
messages (which look slightly different) but with same Message-ID.
Oh, and if you store a "Sent-mail" in same/similar fold^H^H^H^Htab le,
be warned that when this message comes back from pgsql-general or
most any other mailing list it will have the same Message-ID.
So... I think you might want to discard messages with duplicate
Message-IDs (loosing one of lkml- or pgsql-general- list, whichever
comes later), but you should do it silently. Mail should not be rejected
or you're risking getting bounced of the mailing list.
HTH, HAND,
Dawid
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postg resql.org so that your
message can get through to the mailing list cleanly
Well, this is for my personal mail...I think I will
probably give it a try.
The program can log errors so in the few cases that
might occur I think I can "manufactur e" my own message
id.
I guess that the bottom line is that if it exists it
is unique...
Jerry
On Oct 12, 2004, at 11:25 AM, Doug McNaught wrote: Jerry LeVan <je*********@ek u.edu> writes:
Hi, I am futzing around with Andrew Stuarts "Catchmail" program that stores emails into a postgresql database.
I want to avoid inserting the same email more than once... (pieces of the email actually get emplaced into several tables).
Is the "Message-ID" header field a globally unique identifer?
It is intended to be, but since it is generated by either the MUA or the first MTA that sees the message, there is a lot of scope for broken software to screw things up.
-Doug -- Let us cross over the river, and rest under the shade of the trees. --T. J. Jackson, 1863
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
On Tue, Oct 12, 2004 at 11:01:08AM -0400, Jerry LeVan wrote: Hi, I am futzing around with Andrew Stuarts "Catchmail" program that stores emails into a postgresql database.
I want to avoid inserting the same email more than once... (pieces of the email actually get emplaced into several tables).
Is the "Message-ID" header field a globally unique identifer?
Not a postgresql related issue, but, yes Message-ID: is, by
definition, a globally unique identifier. If there are two
messages with the same Message-ID then the sender is asserting
that those two messages are identical. See RFC 2822 section 3.6.4.
You will sometimes see a message generated without a Message-ID at
all, but that will usually have had a Message-ID added by some MTA
along the delivery route. If your MX doesn't add Message-IDs when
missing then you may well see incoming email without Message-IDs
(mostly spam).
In practice there are varying levels of competence in implementation
of Message-ID generation, so you'll very rarely see syntactically
incorrect Message-IDs that may, in theory, clash.
I eventually want to have a cron job process my inbox and don't want successive cron tasks to keep re-entering the same email :)
I wouldn't try and use Message-ID as a primary key, though. Give
yourself a serial field.
I don't use Message-ID at all in my postgresql-based
mailstore. Instead I use a maildir style spool directory for incoming
mail and the processes that import those spooled messages into the
mailstore use standard maildir techniques for locking the message on
disk, writing it to the DB, moving it atomically from the new/ to the
cur/ directory, then commiting the database write. I've pumped
millions of emails through this in production with no problems.
Cheers,
Steve
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Jerry LeVan writes I eventually want to have a cron job process my inbox and don't want successive cron tasks to keep re-entering the same email :)
That's the hard way to do it, it's easier to route messages to individual files.
BTW I'm doing just that in a GPL'ed project, see the URL in my sig if you're
interested. It also comes with a GUI to access the mail in the database.
--
Daniel
PostgreSQL-powered mail user agent and storage: http://www.manitou-mail.org
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
On Tue, 2004-10-12 at 17:11, Csaba Nagy wrote: No, it's not a global unique identifier.
[snip]
Hey, you know what ? Good that I read this thread :D
I'm in the process of writing a Java SMTP server, and had no clue about
this rule... although I have read a few times the relevant RFC. It's
true that currently I don't touch the mail, I just use internal IDs
which I don't care if they are not globally unique, but if those IDs
would have made it to the final product in some mails, there would have
been some chance for collisions... as some other poster said, there is a
good chance of non-conforming MTAs to be out there.
Cheers,
Csaba.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
On Tue, Oct 12, 2004 at 08:50:30AM -0700, Steve Atkins wrote: Is the "Message-ID" header field a globally unique identifer? Not a postgresql related issue, but, yes Message-ID: is, by definition, a globally unique identifier. If there are two messages with the same Message-ID then the sender is asserting that those two messages are identical. See RFC 2822 section 3.6.4.
Except that they usually recommend adding the domain as part of the
message-id. This is where your problem comes in. A mail server hiding
behind a NAT firewall that's a relay and doesn't receive incoming mail
directly, is not going to have a domain. So you'll get things like
localhost.local domain or something completely fake. Indeed, two
messages in this thread have this.
Secondly, clients creating their own message-ids are not always that
great. For example, Outlook, uses the computer name as the domain.
They're not likely to be unique worldwide either. You'd still have to
be pretty unlucky to get two the same though. It's possible Microsoft
mashes the ethernet MAC ID in there too.
The answer is, it very very close to unique, but not really guarenteed.
All mail systems I'm aware of generate their own ID's anyway, look
through the received headers...
Hope this helps,
--
Martijn van Oosterhout <kl*****@svana. org> http://svana.org/kleptog/ Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a tool for doing 5% of the work and then sitting around waiting for someone else to do the other 95% so you can sue them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQFBbAccY5T wig3Ge+YRAnbPAK DQqvl2q1YEuRsgi 3RFY+5i3QR6dwCg pSDL
XogzUbnIARx82/f4kYITmWQ=
=qMPb
-----END PGP SIGNATURE----- This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Hugh Welford |
last post by:
Hi Using WIN XP and FP2000
Using CDO.message to out put a message from my site, I get the following
error message 8004020f from using the code:-
strsql = "SELECT * FROM details WHERE patid = " & request("to") & ";"
objrec.open strsql, objconn, adopenforwardonly, adlockoptimistic, adcmdtext
thetext = "Hello, " & request("nickto") & " you have a new message on online
Connect from " & session("username") & "<br> <a target='_top'
|
by: MLH |
last post by:
I've followed the instructions found on Microsoft's site...
http://support.microsoft.com/default.aspx?scid=kb;en-us;149119&Product=acc97
Here are those instructions...
To play and record sounds, follow these steps:
Create the following new table in Design view: Table: Messages
-----------------------
Field Name: Message
Data Type: OLE Object
|
by: Serious_Practitioner |
last post by:
Hello, all -
A few days ago, I posted the following message -
Hello all, and thank you for your help in the recent past.
Got a new error message in the last few days.
While working with different sized tables, in datasheet view, I've
gotten this message -
|
by: Brian Keating EI9FXB |
last post by:
Would I be correct in saying that the only way to get a user message into a
Windows form would be to use P/Invoke with Message?
Of is there some part of the .NET API that I am totally un aware of?
Thanks for any help
Brian
|
by: ronscottlangham |
last post by:
I am working on a custom WCF EndpoingBehavior that will modify the
messages coming in and out of my Service. I am having an error
related to the modification of the message in the
IEndpointBehavior.AfterReceiveRequest(ref Message request,...) method.
In the method that creates a new message, I have something similar
to ...
/// xmlReader defined with new modified content....
/// 'message' is the original message...
| |
by: =?Utf-8?B?QWRl?= |
last post by:
HI All,
I am encountering the following error when I try to send an email through a
SMTP server. I believe the problem lies with the authentication part when the
network crednetials are used, error is thrown at the .send point.
Error is:
The following error occured Sending an email: System.ApplicationException:
An error occurred sending an email To helen@HerMail.com
From ade@MyMail.com Cc Subject A test with Finlistener #1. The body...
|
by: Steven Allport |
last post by:
I am working on processing eml email message using the email module (python
2.5), on files exported from an Outlook PST file, to extract the composite
parts of the email. In most instances this works fine, the message is read
in using message_from_file, is_multipart returns True and I can process each
component and extract message attachments.
I am however running into problem with email messages that contain emails
forwarded as...
|
by: Philluminati |
last post by:
I have a Perl SOAP Server which returns this SOAP Message when
invoked:
<?xml version="1.0" encoding="UTF-8"?><soap:Envelope xmlns:xsi="http://
www.w3.org/2001/XMLSchema-instance" xmlns:soapenc="http://
schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/
XMLSchema" soap:encodingStyle="http://schemas.xmlsoap.org/soap/
encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/...
|
by: bernd |
last post by:
Hi netties,
posted this in a different group already, which seems to be
inappropriate.
I create a message queue with msgget(), sent a 5-byte message to it
with msgsnd() and try to receive the queue's content with msgrcv().
Everything seems to work fine, apart from the fact that I get back
only the last byte back from the original message (after sending the
message ipcs -qA shows clearly that there are 5 Bytes waiting in the
|
by: forums_mp |
last post by:
Facing a design problem here and I'm not sure how to solve this. The
system consists of bi-directional communication between a subject
communicating with two (at least for now ) listeners. Message
translation needs to occur during communication. Each message
between subject and listener contains 30 words where each word is
16bits in size. The first word in each message is a header.
Communication between Subject and Listeners can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |