Chat logs in Nepomuk

Prototyping is fun. You don’t need to care about proper libraries. Your code can be absolutely horrible, cause “Hey! It’s just a prototype!”

Yesterday, I started the process of importing my entire gTalk chat history into Nepomuk. It turned out to be a lot simpler that I thought it would be.

Step 1: Get the chat logs

GMail fortunately allows you to export your chat logs via SMTP. They don’t implement the traditional XMPP-0136 for fetching offline messages. But at least, unlike Facebook, they provide a mechanism.

I landed up using getmail for importing all chat logs.

getmailrc :

[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmail.com
mailboxes = ("[Gmail]/Chats",)

username = *****@gmail.com
password = ********

[destination]
type = Maildir
path = ~/Chats/

I originally wanted to use offlineimap but they seem to have a problem fetching the Chats in GMail.

Step 2: Write a parser

The chat logs are presented in a custom xml format encapsulated in the email. The content was in the traditional quoted-printable format, as most emails are. Writing a parser didn’t take too long. Plus, with the new Nepomuk Datamanagement APIs, pushing them into Nepomuk was even simpler.

Ideally, this should be implemented as a strigi analyzer, so that it becomes a part of Nepomuk’s Indexing framwork. But hey! It’s a prototype!

What’s the point of having your chat logs in Nepomuk

Well, for one, the Telepathians can use this to show chat logs. We’ll obviously need a better way of importing the chat logs. Manually calling nepomuk-chat-feeder obviously isn’t an option. So we’ll need to find a proper way of fetching chat logs.

The second, more personal, use is that I finally have a usable dataset to determine important people in my life - based on the chat frequency and timings. AFAIK Facebook internally uses a combination of likes, comments, chat history and stalking to determine how important a person is to you, and accordingly place them higher in the auto-completion list and chat sidebar.

This obviously has many other applications like altering the chat list based on the people you converse with when you’re doing one activity.

Source Code: kde:scratch/vhanda/nepomuk-gtalk-chatlogs