Chat logs in Nepomuk
Prototyping is fun. You don't need to care about proper libraries. Your
code can be absolutely horrible, cause "Hey! It's just a prototype!"
Yesterday, I started the process of importing my entire gTalk chat
history into Nepomuk. It turned out to be a lot simpler that I thought
it would be.
Step 1: Get the chat logs
GMail fortunately allows you to export your chat logs via SMTP. They
don't implement the traditional XMPP-0136 for fetching offline
messages. But at least, unlike Facebook, they provide a mechanism.
I landed up using getmail for importing all chat logs.
[retriever] type = SimpleIMAPSSLRetriever server = imap.gmail.com mailboxes = ("[Gmail]/Chats",) username = *****@gmail.com password = ******** [destination] type = Maildir path = ~/Chats/
I originally wanted to use offlineimap but they seem to have a
problem fetching the Chats in GMail.
Step 2: Write a parser
The chat logs are presented in a custom xml format encapsulated in the
email. The content was in the traditional quoted-printable format,
as most emails are. Writing a parser didn't take too long. Plus, with
the new Nepomuk Datamanagement APIs, pushing them into Nepomuk was even
Ideally, this should be implemented as a strigi analyzer, so that it
becomes a part of Nepomuk's Indexing framwork. But hey! It's a
What's the point of having your chat logs in Nepomuk
Well, for one, the Telepathians can use this to show chat logs. We'll
obviously need a better way of importing the chat logs. Manually calling
nepomuk-chat-feeder obviously isn't an option. So we'll need to find a
proper way of fetching chat logs.
The second, more personal, use is that I finally have a usable dataset
to determine important people in my life - based on the chat frequency
and timings. AFAIK Facebook internally uses a combination of likes,
comments, chat history and stalking to determine how important a person
is to you, and accordingly place them higher in the auto-completion list
and chat sidebar.
This obviously has many other applications like altering the chat list
based on the people you converse with when you're doing one activity.
Source Code: kde:scratch/vhanda/nepomuk-gtalk-chatlogs