Chat logs in Nepomuk

Prototyping is fun. You don't need to care about proper libraries. Your
code can be absolutely horrible, cause "Hey! It's just a prototype!"

Yesterday, I started the process of importing my entire gTalk chat
history into Nepomuk. It turned out to be a lot simpler that I thought
it would be.

Step 1: Get the chat logs

GMail fortunately allows you to export your chat logs via SMTP. They
don't implement the traditional XMPP-0136 for fetching offline
messages. But at least, unlike Facebook, they provide a mechanism.

I landed up using getmail for importing all chat logs.

getmailrc :

type = SimpleIMAPSSLRetriever
server =
mailboxes = ("[Gmail]/Chats",)

username = *****
password = ********

type = Maildir
path = ~/Chats/

I originally wanted to use offlineimap but they seem to have a
problem fetching the Chats in GMail.

There were a lot of logs!

Step 2: Write a parser

The chat logs are presented in a custom xml format encapsulated in the
email. The content was in the traditional quoted-printable format,
as most emails are. Writing a parser didn't take too long. Plus, with
the new Nepomuk Datamanagement APIs, pushing them into Nepomuk was even

The API is fun to use!

Ideally, this should be implemented as a strigi analyzer, so that it
becomes a part of Nepomuk's Indexing framwork. But hey! It's a

What's the point of having your chat logs in Nepomuk

Well, for one, the Telepathians can use this to show chat logs. We'll
obviously need a better way of importing the chat logs. Manually calling
nepomuk-chat-feeder obviously isn't an option. So we'll need to find a proper way of fetching chat logs.

The second, more personal, use is that I finally have a usable dataset
to determine important people in my life - based on the chat frequency
and timings. AFAIK Facebook internally uses a combination of likes,
comments, chat history and stalking to determine how important a person
is to you, and accordingly place them higher in the auto-completion list
and chat sidebar.

This obviously has many other applications like altering the chat list
based on the people you converse with when you're doing one activity.

Source Code: kde:scratch/vhanda/nepomuk-gtalk-chatlogs