Wednesday, April 25, 2012

Google Calendar Bug-Fixes

I have discovered some bugs in my previous entry regarding Google Calendar synchronization. Firstly there is a bug in the locking mechanism causing it to sometimes hang while running fetchmail. The following is the corrected .procmailrc entries:

# === Google Calendar (from Microsoft Exchange; iCalendar) ===
:0 c:
* H ?? ^Content-Type: multipart/(alternative|mixed)
* B ?? ^Content-Type: text/calendar
 :0 Wac: Mail/multipart.lock
 | munpack -q -t -C ~/Mail/.munpack 2> /dev/null
 :0 Wac: Mail/multipart.lock
 | /home/oliver/bin/
 :0 Wac: Mail/multipart.lock
 | rm -f ~/Mail/.munpack/*

I discovered the second bug when synchronizing a particular iCalender format file containing several BEGIN:VEVENT blocks. Events are expected to have a UID field. I do not know whether this should be unique within the scope of your entire calendar, but it must be unique within the scope BEGIN:VCALENDAR block (i.e. the entire iCalender file.)

Exchange, in all its infinite stupidity decides to assign the same UID to every VEVENT, and as soon as you attempt to upload it to Google Calender via WebDav you will receive a cryptic error message:
500 Server error code: ........ (Pseudo-random characters)

You'll receive an even less helpful error message when trying the "Import Calendar" feature of the web interface:
Failed to import events: Could not upload your events because you do not have sufficient access on the target calendar.

What is the solution? Well, funnily enough a Unique Identifier (UID) should actually be unique! Changing those UIDs to some unique values and uploading the file again works perfectly! (Anything works, I used echo $RANDOM | md5sum | cut -d ' ' -f 1 i.e. MD5 hash of a pseudo-random number.)

I am only aware of one such formatted calendar entry in my dataset, so I adjusted it manually. However, I'd feel safer using sed or awk to replace each "UID" with a real unique identifier. I'll post such a modified in the second part, whenever I feel like playing with awk again! You could always beat me to it... Happy hacking!


  1. Don't you have a race condition now? The lockfile will be locked and unlocked tree timesmwhile handling a single email, which means those parts might get interleaved, no?

    1. I do not believe so, because of the 'W' flag:

      w Wait for the filter or program to finish and check its exitcode (normally ignored); if the filter is unsuccessful, then the text will not have been filtered.

      W Has the same meaning as the `w' flag, but will suppress any `Program failure' message.

      You're right though, the lock file is recycled for three different (but related) rules. I don't claim to be a procmail expert, though.

  2. I had the same error message as you:
    Failed to import events: Could not upload your events
    because you do not have sufficient access on the target calendar.
    when trying to import events in my primary calendar. The reason for doing so was that I wanted to make another calendar my primary one. Since there's no procedure to do this, I exported the other calendar as ICS file, deleted my primary one (or, more accurately, deleted all the events in my primary calendar) and tryed to import the ICS file into my primary one.

    Anyway, I tried to follow the procedure mentioned above. A quick Linux command:
    egrep -i '\' myfile.ics | sort | uniq -c | grep -v ' 1'
    showed that I had indeed a few duplicated UID. Fixing that did not solve my problem as I had hoped. However, completely removing all the UID fields as suggested here:!msg/calendar/5vLrmBPXyxc/Sd76gunaaaEJ
    in my file (with a small vi command) did it. Si if anybody's have the same issue, you might try that too, worked great!
    Thanks anyway for pointing in the right direction (of the UID).

  3. Some characters were stripped in the Linux command. It should be:
    egrep -i ' \ < UID \ > ' myfile.ics | ...
    (without any space between the single quote characters)
    Sorry about that...

  4. Thank you Oliver!

    I was getting the "Failed to import events: Could not upload your events because you do not have sufficient access on the target calendar" error as well. The difference in my case was that I had only one event listed in my .ics file (and one UID), so uniqueness within the .ics file was not an issue. However, I changed the last character in the UID anyway and it worked! I am guessing that the UID was already in use in another item in my Google calendar...

    Thanks again!