Documentation Generator for Google Protocol Buffers

As part of my summer job these past two summers I’ve had to work quite a lot with Protocol Buffers, Google’s data interchange format. The Protocol Buffers compiler (protoc) takes message definitions like

message Person {
  required int32 id = 1;
  required string name = 2;
  optional string email = 3;
}

and generates C++, Java or Python code for message manipulation and (de)serialization. Protocol Buffers has been around for a while, and nowadays there are cooler kids on the block like Cap’n Proto and MessagePack. But Protocol Buffers remains a quite popular serialization format, and it is widely used.

One thing that always bothered me is that Protocol Buffers has no built-in support for documentation comments, something like Doxygen or JavaDoc. I found one third party tool, but it could only generate DocBook and was quite Windows-centric.

Long story short, I wrote my own little tool. It’s quite simple, but supports /// or /** */ documentation comments for messages, fields, enums and enum values within Protocol Buffer message definitions. Like this:

/**
 * A Person is something with a personality.
 * Most of the time...
 */
message Person {
  required int32 id = 1;    /// How impersonal...
  required string name = 2; /// Yay!

  /** Optional? In this day and age?! */
  optional string email = 3;
}

The tool works as a plugin for the protoc compiler. It runs on Linux, Windows and Mac OS X and can output HTML, Markdown or DocBook documentation natively, with support for custom output formats through Mustache templates. See the GitHub repo for more examples and sample output.

I hope someone else finds it useful :)

Opinions on Lenovo X220?

(Sorry in advance for this possible misuse of the planet, but I feel this might be geeky enough).

My old Fujitsu-Siemens Amilo Si1520 laptop, which has served me well all in all, has started to fall apart on me hardware-wise. Since I recently got hold of a bit of extra cash, I’m looking for a replacement. I’m considering the Lenovo X220. It’s basically the same form factor as my current one, which is nice since it’s what I’m looking for. And it seems it has gotten some good reviews. I’ve never owned a Thinkpad, or an IBM/Lenovo machine for that matter.

Anyone have an X220 who can give me the inside scoop on what annoys you most about the machine? I see in the spec sheet that it’s certfied for Redhat/Novell/Ubuntu, but has anyone had any problems getting things to work?

If I get this model, I’d probably be getting the Core i7 one, but without an SSD since I already have a 180 GB SSD to put in it.

Thanks for any answers/recommendations!

Brief GSoC Update — Interactive Table Resizing and Cell Selections

Hi everybody,

In the coming days, I’ll try to write a longer post with more technical details about my recent work on the project. Until then, here’s a short screencast showing some of the new features; interactive table resizing and cell selections. This time it includes me speaking a bit (!). And the format is now WebM/VP8 instead of Ogg/Theora. White areas in the capture got some weird tinting, but that’s a screencasting problem and not my code :)

Download Video (~12 MB WebM)

Bye ’til next time!

Simple Backups with rsnapshot — 3 Step Guide

Introduction

Since I recently set up a simple backup scheme for my laptop, workstation and server that I’m quite happy with, I thought I should share. The thing about backups is that everyone talks about how one should have them, but who really has an adequate backup scheme?

My laptop and workstation runs Arch Linux. The server is an old FreeBSD 7.0 install. I know the FreeBSD version is ancient, but the installed ports are kept up to date, and the machine has been running fine for years. It hosts this blog, along with the blogs and websites of some friends, a DNS server and a private SILC server me and some friends use for chatting.

So in short, here’s what I did for backups on the server. For the laptop and workstation, the instructions are pretty much identical, except you can leave out the parts about MySQL backups.

Step 1 — Install rsnapshot

portinstall rsnapshot (pacman -S rsnapshot on Arch)

Step 2 — Configure rsnapshot

I use the following configuration to tell rsnapshot to keep seven days of daily backups along with one montly backup in /usr/.rsnapshot.

config_version  1.2
snapshot_root   /usr/.snapshots/
cmd_rm          /bin/rm
cmd_rsync       /usr/local/bin/rsync
cmd_logger      /usr/bin/logger

cmd_postexec    /usr/local/bin/backup-strongspace.sh

interval        daily   7
interval        monthly 1

verbose         2
loglevel        3
logfile /var/log/rsnapshot
lockfile        /var/run/rsnapshot.pid

rsync_long_args --delete --numeric-ids --relative --delete-excluded --filter="dir-merge,n- .backup-exclude"
link_dest       1

backup  /usr/home/              localhost/
backup  /etc/           localhost/
backup  /var/named/     localhost/
backup  /var/www/       localhost/
backup  /usr/local/etc/ localhost/
backup_script   /usr/local/bin/backup-mysql.sh  localhost/mysql/

The interval directives tell rsnapshot how many backups to keep. E.g. with the above configuration, if I execute rsnapshot daily ten times, the last seven of the backups will be kept. rsnapshot uses hard linking to save space, so the disk usage won’t be horrible.

By using the cmd_postexec directive, I give the path to a script to execute after each backup run. In my backup-strongspace.sh script I have:

#!/bin/sh

/usr/local/bin/rsync -az --delete --delete-excluded /usr/.snapshots/daily.0 estan@estan.strongspace.com:/strongspace/estan/dose

This will sync the latest backup to my Strongspace account (40 GB Starter account, $4.99/month).

The --filter="dir-merge,n- .backup-exclude" is obscure rsync syntax and means that I can put stuff to be excluded from backup in directory specific .backup-exclude files.

Next comes the backup directives, these simply specify the directories that should be backed up.

Using the backup-script directive, I specify the path to script to be run in an empty temporary directory before that directory is backed up. The backup-mysql.sh script I’ve specified contains the following:

#!/bin/sh

/usr/local/bin/mysqldump -u root -pmypassword --all-databases | gzip > all-databases.sql.gz

This will simply make a gzipped dump of all MySQL databases on the machine, which will then be backed up by rsnapshot.

An important note for Linux users is that you probably want to specify cmd_cp /bin/cp, as rsnapshot can take advantage of some features of GNU cp.

Step 3 — Configure cron job

I created two scripts that are run from cron as part of FreeBSD’s regular periodic maintenance scripts:

/etc/periodic/daily/001.backup:

#!/bin/sh

/usr/local/bin/rsnapshot daily > /tmp/rsnapshot.out 2>&1 || cat /tmp/rsnapshot.out | mail -s "daily backups failed on `hostname`" my@email.com

and

/etc/periodic/montly/001.backup:

#!/bin/sh

/usr/local/bin/rsnapshot monthly > /tmp/rsnapshot.out 2>&1 || cat /tmp/rsnapshot.out | mail -s "monthly backups failed on `hostname`" my@email.com

This means I’ll get an e-mail if the backups fail for some reason, and can then inspect the rsnapshot log in /var/log/rsnapshot. On the FreeBSD server I already had an e-mail server configured, but on my laptop and workstation I set up msmtp instead, which is a simple SMTP mailer, and the configuration of the cron jobs is a bit different from FreeBSD.

Result

If I ever mess something up, I’ll have backups from the past seven days to restore from, or from the last monthly backup. And if the HDD crashes, I’ll always have a copy of the latest stuff on my Strongspace account. I’m very happy and it feels good to finally have backups.

Feel free to share your own backup strategy in the comments.

Cheers,
Elvis