Google Calendar Bug-Fixes

I have discovered some bugs in my previous entry regarding Google Calendar synchronization. Firstly there is a bug in the locking mechanism causing it to sometimes hang while running fetchmail. The following is the corrected .procmailrc entries:

#
# === Google Calendar (from Microsoft Exchange; iCalendar) ===
#
:0 c:
* H ?? ^Content-Type: multipart/(alternative|mixed)
* B ?? ^Content-Type: text/calendar
{
 :0 Wac: Mail/multipart.lock
 | munpack -q -t -C ~/Mail/.munpack 2> /dev/null
 :0 Wac: Mail/multipart.lock
 | /home/oliver/bin/calendar.sh
 :0 Wac: Mail/multipart.lock
 | rm -f ~/Mail/.munpack/*
}

I discovered the second bug when synchronizing a particular iCalender format file containing several BEGIN:VEVENT blocks. Events are expected to have a UID field. I do not know whether this should be unique within the scope of your entire calendar, but it must be unique within the scope BEGIN:VCALENDAR block (i.e. the entire iCalender file.)

Exchange, in all its infinite stupidity decides to assign the same UID to every VEVENT, and as soon as you attempt to upload it to Google Calender via WebDav you will receive a cryptic error message:
500 Server error code: ........ (Pseudo-random characters)

You'll receive an even less helpful error message when trying the "Import Calendar" feature of the web interface:
Failed to import events: Could not upload your events because you do not have sufficient access on the target calendar.

What is the solution? Well, funnily enough a Unique Identifier (UID) should actually be unique! Changing those UIDs to some unique values and uploading the file again works perfectly! (Anything works, I used echo $RANDOM | md5sum | cut -d ' ' -f 1 i.e. MD5 hash of a pseudo-random number.)

I am only aware of one such formatted calendar entry in my dataset, so I adjusted it manually. However, I'd feel safer using sed or awk to replace each "UID" with a real unique identifier. I'll post such a modified calendar.sh in the second part, whenever I feel like playing with awk again! You could always beat me to it... Happy hacking!

From fetchmail/procmail to Google Calendar

I spent many hours scouring Google before finally figuring out how to implement this easily. Calendar synchronization is very important to me; I want to be able to look at my phone, my tablet, or my browser and see my entire schedule (otherwise I'd never show up anywhere.)

.procmailrc:

:0 Wc: Mail/multipart.lock
* H ?? ^Content-Type: multipart/(alternative|mixed)
* B ?? ^Content-Type: text/calendar
{
        :0 Wac:
        | munpack -q -t -C ~/Mail/.munpack 2> /dev/null
        :0 Wac:
        | /home/oliver/bin/calendar.sh
        :0 Wc:
        | rm -f ~/Mail/.munpack/*
}

This beautiful undocumented recipe is actually quite simple, but perhaps needs some explanation. From man procmailrc:

A line starting with ':' marks the beginning of a recipe.
It has the following format:

       :0 [flags] [ : [locallockfile] ]

       zero or more conditions (one per line)
       exactly one action line

W means wait until the program finishes and ignore any failure messages. a means the preceding recipe must have successfully completed before procmail will execute this recipe. Finally, c means carbon-copy (this recipe does not handle delivery of the mail, so it must proceed further through the chain of recipes.)

It may be more appropriation to use f (this pipe is a filter) instead. I am new to procmail configuration; c works for me.

First match messages containing a Content-Type header of either multipart/alternative or multipart/mixed, second check the message body for a Content-Type of text/calendar (which is the data we want.) If these conditions are true unpack the MIME multipart message into separate files with munpack into a temporary directory. Proceed and run calendar.sh which does the magic, and finally clean up after ourselves.

calendar.sh:

#!/bin/bash

http_proxy=http://proxy.example.com:8080

CALENDAR_ID='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX@group.calendar.google.com'
for i in ~/Mail/.munpack/*; do
        if [ -n "$(grep 'BEGIN:VCALENDAR' ${i} 2> /dev/null)" ]; then
                expect -dc \
                "spawn /usr/bin/cadaver -p ${http_proxy/http:\/\/} \
                https://www.google.com/calendar/dav/${CALENDAR_ID}/events ; \
                expect \"dav:\" ; \
                send \"put ${i}\\r\" ; \
                expect \"dav:\" ;
                send \"bye\\r\""
        fi
done

Newlines have been inserted into the script so that it doesn't break layouts on other pages; you will need to fix those if copying this script.

I use expect and cadaver to upload the iCalender file extracted by munpack to Google Calendar via their WebDav interface. Finding the CALENDAR_ID is a little confusing. You can find it from the "Settings" option when you login to the Google Calender via a browser. You must use the public Calendar Address, not the Private Address, but you do not need to share the calender publicly; Google will request your login details via WebDav. This is easiest to configure in .netrc. chmod 600 the file for some additional safety.

machine www.google.com
login john.doe
password example123

The login should not include the @gmail.com otherwise it will fail. Happy hacking!