Recent Posts

Puppet and JRuby, a love and hate story

3 minute read

Since I heard about JRuby about a year ago, I wanted to try to run my favorite ruby program on it. I’m working with Java almost all day long, so I know for sure that the Sun JVM is a precious tool for running long-lived server. It is pretty fast, and has a very good (and tunable) garbage collector.

In a word: the perfect system to run a long-lived puppetmaster!

The first time I tried, back in February 2009, I unfortunately encountered the bug JRUBY-3349 which prevented Puppet to run quite early, because the Fcntl constants weren’t defined. Since my understanding of JRuby internal is near zero, I left there.

But thanks to Luke Kanies (Puppet creator), one of the JRuby main developers Charles Oliver Nutter fixed the issue a couple of weeks ago (thanks to him, and they even fixed another issue at about the same time about fcntl which didn’t support SET_FD).

That was just in time for another test…

But what I forgot was that Puppet is not every ruby app on the block. It uses lots of cryptography behind the scene. Remember that Puppet manages its own PKI, including:

  • a full Certification Authority.
  • a CRL.
  • authenticated clients connections, through SSL.

That just means Puppet exercise a lot the Ruby OpenSSL extension.

The main issue is that MRI uses OpenSSL for all the cryptographic stuff, and JRuby uses a specific Java version of this extension. Of course this later is still young (presently at v 0.5) and doesn’t contain yet everything needed to be able to run Puppet.

In another life I wrote a proprietary cryptographic Java library, so I’m not a complete cryptography newcomer (OK, I forgot almost everything, but I still have some good books to refer to). So I decided to implement what is missing in JRuby-openssl to allow a webrick Puppetmaster to run.

You can find my contributions in the various JRUBY-3689, JRUBY-3690, JRUBY-3691, JRUBY-3692, JRUBY-3693 bugs.

I still have another a minor patch to submit (OpenSSL::X509::Certificate#to_text implementation).

So the question is: with all that patches applied, did I get a puppetmaster running?

And the answer is unfortunately no.

I can get the puppetmaster to start on a fresh configuration (ie it creates everything SSL related and such), but it fails as soon a client connects (hey that’s way better than before I started :-)).

All comes from SSL. The issue is that with the C OpenSSL implementation it is possible to get the peer certificate anytime, but the java SSL implementation (which is provided by the Sun virtual machine) requires the client to be authenticated before anyone get access to the peer certificate.

That’s unfortunate because to be able to authenticate a not-yet-registered client, we must have access to its certificate. I couldn’t find any easy code fix, so I stopped my investigations there.

There is still some possible workarounds, like running in mongrel mode (provided JRuby supports mongrel which I didn’t check) and let Nginx (or Apache) handle the SSL stuff, but still it would be great to be able to run a full-fledged puppetmaster on JRuby.

I tried with a known client and get the same issue, so maybe that’s a whole different issue, I guess I’ll have to dig deeper in the Java SSL code, which unfortunately is not available :-)

Stay tuned for more info about this. I hope to be able to have a full puppetmaster running on JRuby soon!

EDIT: I could run a full puppetmaster on webrick from scratch under JRuby with a normal ruby client. I’ll post the recipe in a subsequent article soon.

In love with zsh, part 1

10 minute read

Note: when I started writing this post, I didn’t know it would be this long. I decided then to split it in several posts, each one covering one or more interesting aspect of zsh. You’re now reading part 1.

I first used a Unix computer in 1992 (it was running SunOS 4.1 if I remember correctly). I’m using Linux since 1999 (after using VMS throughout the 90s in school, but I left the Unix world while I was working with RAYflect doing 3D stuff on Mac and Windows).

During the time I worked with those various unices (including Irix on a Crimson), I think I’ve used almost every possible shell with various level of pleasure and expertise, including but not limited to:

When my own road crossed ZSH (about 6 years ago), I felt in love with this powerful shell, and it’s now my default shell on my servers, workstations and of course my macbook laptop.

The point of this blog post is to give you an incentive to switch from insert random shell here to zsh and never turn back.

The whole issue with zsh is that the usual random Linux distribution ships with Bash by default (that’s not really, true as GRML ships with zsh, and a good configuration). And Bash does its job well enough and is wide-spread, that people have usually only low incentive to switch to something different. I’ll try to let you see why zsh is worth the little investment.

Which version should I run?

Right now, zsh exists in 2 versions a stable one (4.2.7) and a development one (4.3.9). I’m of course running the development version (I usually prefer seeing new bugs than old bugs :-))

I recommend using the development version.

UTF-8 support, anyone?

Some people don’t want to switch to zsh because they think zsh doesn’t support UTF-8. That’s plain wrong, if you follow my previous advice which is to run a version greater than 4.3.6, UTF-8 support is there and works really fine.


One of the best thing in zsh is the TAB completion. It’s certainly the best TAB completion I could use in every shell I tried. It can completes almost anything, from files (of course), to users, including but not limited to hosts, command options, package names, git revisions/branches etc.

Zsh ships with completions for almost every shipped apps on earth. And the beauty is that completion is so much configurable that you can twist it to your own specific taste.

To activate completion on your setup:

% zmodload zsh/complist
% autoload -U compinit && compinit

The completion system is completely configurable. To configure it we use the zstyle command:

  zstyle <context> <styles>

</styles></context> How does it work?

The context defines where the style will apply. The context is a string of ‘:’ separated strings: ‘:completion:function:completer:command:argument:tag’ Some part can be replaced by , so that ‘:completion:’ is the least specific context. More specific context wins over less specific ones of course.

The various styles selects the options to activate (see below).

If you want to learn more about zsh completion, please read the zsh section completion manual.

Zsh completion is also:

Formatting completion

When zsh needs to display completion matches or errors, it uses the format style for doing so.

  zstyle ':completion:*' format 'Ouch: %d :-)'

%d will be replaced by the actual text zsh would have been printed if no format style were applied. You can use the same escape sequences as in zsh prompts.

Since there are many different types of messages, it is possible to restrict to warnings or messages by changing the tags part of the context:

  zstyle ':completion:*:warnings' format 'Too bad there is nothing'

And since it is possible to use all the prompt escapes, you can add style to the formats:

# format all messages not formatted in bold prefixed with ----
zstyle ':completion:*' format '%B---- %d%b'
# format descriptions (notice the vt100 escapes)
zstyle ':completion:*:descriptions'    format $'%{\e[0;31m%}completing %B%d%b%{\e[0m%}'
# bold and underline normal messages
zstyle ':completion:*:messages' format '%B%U---- %d%u%b'
# format in bold red error messages
zstyle ':completion:*:warnings' format "%B$fg[red]%}---- no match for: $fg[white]%d%b"

And the result:

Grouping completion

By default matches comes in no specific order (or in the order they’ve been found). It is possible to separate the matches in distinct related groups:

  # let's use the tag name as group name
  zstyle ':completion:*' group-name ''

An example of groups: Grouping matches

Menu completion is when you press TAB several times and the completion changes to cycle through the available matches. By default in zsh, menu completion activates the second time you press the TAB key (the first one triggered the first completion).

Menu selection is when zsh displays below your prompt the list of possible selections arranged by categories.

A short drawing is always better than thousands words, so hop an example:

Menu Selection - here I'm cycling through gzip options

In this example I typed gzip -<TAB> then navigated with the arrows to –stdout.

To activate menu selection:

  # activate menu selection
  zstyle ':completion:*' menu select

There’s also approximate completion

With this, zsh corrects what you already have typed. Approximate completion is controlled by the


completer. Approximate completion looks first for matches that differs by one error (configurable) to what you typed. An error can be either a transposed character, a missing character or an additional character. If some corrected entries are found they are added as matches, if none are found, the system continues with 2 errors and so on. Of course, you want it to stop at some level (use the max-errors completion style).

  # activate approximate completion, but only after regular completion (_complete)
  zstyle ':completion:::::' completer _complete _approximate
  # limit to 2 errors
  zstyle ':completion:*:approximate:*' max-errors 2
  # or to have a better heuristic, by allowing one error per 3 character typed
  # zstyle ':completion:*:approximate:*' max-errors 'reply=( $(( ($#PREFIX+$#SUFFIX)/3 )) numeric )'

Completion of about everything

From X windows, to hosts from users, almost everything including shell variables can be completed or menu-selected.

Here I typed “echo $PA<TAB>” and navigated to PATH:

Variable Completion

Now, one thing that is extremely useful is completion of hosts:

# let's complete known hosts and hosts from ssh's known_hosts file
( [ -r .ssh/known_hosts ] && awk '{print $1}' .ssh/known_hosts | tr , '\n');\
echo $basehost; ) | sort -u) )

zstyle ':completion:*' hosts $hosts


Yeah, I see, you’re wondering, aliases, pffuuuh, every shell on earth has aliases.

Yes, but does your average shell has global or suffix aliases?

Suffix Aliases

Suffix aliases are aliases that matches the end of the command-line.


% alias -s php=nano

Now, I just have to write:

% index.php

And zsh executes nano index.php. Clever isn’t it?

Global Aliases

Global aliases are aliases that match anywhere in the command line.

Typical uses are:

  % alias -g G='| grep'
  % alias -g WC='| wc -l'
  % alias -g TF='| tail -f'
  % alias -g DN='/dev/null'

Now, you just have to issue:

% ps auxgww G firefox

to find all firefox processes. Still not convinced?

Too risky?

Some might argue that global aliases are risky because zsh can change your command line behind your back if you need to have let’s say a capital G in there.

Because of this I’m using the GRML way: I use a special key combination (see in an upcoming post about key binding) that auto-completes my aliases directly on the command line, without defining a global alias.


One of the best feature, albeit one of the more difficult to master is zsh extended globing.

Globbing is the process of matching several files or paths with an expression. The most usually known forms are * or ?, like: *.c to match every file ending with .c.

Zsh pushes the envelop far away, supporting the following:

Let’s say our current directory contains:


Wildcard: *

This is the well known wildcard. It matches any amount of characters. As in:

% echo *.c

Wildcard: ?

This matches only one character. As in:

% echo test.?
test.c test.h

Character classes: […]

This is a character class. It matches any character listed between the braces. The content can be either single characters:

[abc0123] will match either a,b,c,0,1,2,3

or range of characters:

[a-e] will match from a to e inclusive

or POSIX character classes

[[:space:]] will match only spaces (refer to zshexpn(1) for more information)

The character classes can be negated by a leading ^:

[^abcd] matches only character outside of a,b,c,d

If you need to list - or ], it should be the first character of the class. If you need both list ] first.


% echo test.[ch]
test.c test.h

Number ranges <x-y>

x and/or y can be omitted to have an open-ended range. <-> match all numbers.

% echo test.<0-10>
test.1 test.2 test.3

% echo test.<2->
test.2 test.3

Recursive matching: **

You know find(1), but did you know you can do almost everything you need with only zsh?

% echo **/*.txt
a/a.txt b/1/b.txt b/2/d.txt

Alternatives: (a|b)

Matches either a or b. a and b can be any globbing expressions of course.

% echo test.(1|2)
test.1 test.2
% echo test.(c|<1-2>)
test.1 test.2 test.c

Negated matches ^ (only with extended globbing)

There are two possibilities:

leading ^: as in ^*.o which selects every file except those ending with .o
pattern1^pattern2: pattern1 will be matched as a prefix, then anything not matching pattern2 will be selected

% ls te*
test.c test.h team.txt term.txt
% echo te^st.*
team.txt term.txt

If you use the negation in the middle of a path section, the negation only applies to this path part:

% ls /usr/^bin/B*
/usr/lib/BuildFilter  /usr/sbin/BootCacheControl

Pattern exceptions (~)

Pattern exceptions are a way to express: “match this pattern, but not this one”.

# let's match all files except .svn dirs
% print -l **/*~*/.svn/* | grep ".svn"
# an nothing prints out, so that worked

It is to be noted that * after the ~ matches a path, not a single directory like the regular wildcard.

Globbing qualifiers

zsh allows to further restrict matches on file meta-data and not only file name, with the globbing qualifiers.

The globbing qualifier is placed in () after the expression:

# match a regular file with (.)
% print -l *(.)

We can restrict by:

  • (.): regular files
  • (/): directories
  • (*): executables
  • (@): symbolic links
  • (R),(W),(X),(U): file permissions
  • (LX),(L+X),(L-X),(LmX): file size, with X in bytes, + for larger than files, - for smaller than files, m can be modifier for size (k for KiB, m for MiB)
  • (mX),(m+X),(m-X): matches file modified “more than X days ago”. A modifier can be used to express X in hours (h), months (M), wweks (W)…
  • (u{owner}): a specific file owner
  • (f{permission string ala chmod}): a specific file permissions

% ls -al
total 0
drwxr-xr-x  8 brice wheel 272 2009-04-14 18:59 .
drwxrwxrwt 11 root  wheel 374 2009-04-14 20:04 ..
-rw-r--r--  1 root  wheel   0 2009-04-14 18:59 test.c
-rw-r--r--  1 brice wheel  10 2009-04-14 18:59 test.h
-rw-r--r--  1 brice wheel  20 2009-04-12 16:30 old
# match only files we own
% print -l *(U)
# match only file whose size less than 2 bytes
% print -l *(L-2)
# match only files older than 2 days
% print -l *(m-2)

It is possible to combine and/or negate several qualifiers in the same expressions

# print executable I can read but not write
% echo *(*r^w)

And there’s more, you can change the sort order, add a trailing distinctive character (ala ls -F). Refer to zshexpn(1) for more information.

What’s next

In the next post, I’ll talk about some other interesting things:

  • History
  • Prompts
  • Configuration, options and startup files
  • ZLE: the line editor
  • Redirections
  • VCS info, and git in your prompt

newcomers, use GRML

But that’s all for the moment. Newcomer, new switchers, if you want to get bootstrapped in a glimpse, I recommend using the GRML configuration:

# IMPORTANT: please note that you might override an existing
# configuration file in the current working directory!
wget -O ~/.zshrc

Introducing mysql-snmp!

1 minute read

Thanks to Days of Wonder the company I work for, I’m proud to release in Free Software (GPL):

mysql-snmp - monitor a MySQL server with SNMP


At Days of Wonder, we’re using MySQL for almost everything since the beginning of the company. We were initially monitoring all our infrastructure with mon and Cricket, including our MySQL servers.

Nine months ago I migrated the monitoring infrastructure to OpenNMS, and at the same we lost the Cricket MySQL monitoring (which was done with direct SQL SHOW STATUS LIKE commands).

I had to find another way, and since OpenNMS excels at SNMP, it was natural to monitor MySQL through SNMP. My browsing crossed this blog post. At about the same time I noticed that Baron Schwartz had released some very good MySQL Cacti Templates, so I decided I should cross both project and started working on mysql-snmp on my free time.

Hopefully, Days of Wonder has an IANA SNMP enterprises sub-number (20267, we use this for monitoring our game servers), so the MIB I wrote for this project is hosted in a natural place in the MIB hierarchy.

What’s this?

It’s a Net-SNMP perl subagent that connects to your MySQL server, and reports various statistics (from show status or show innodb status, or even replication) through SNMP.## But wait, there’s more, there’s OpenNMS support!

If you followed this blog from the very start, you know we’re using OpenNMS to monitor Days of Wonder infrastructure. So I included the various OpenNMS configuration bit to display nice and usable graphs, inspired by the excellent MySQL Cacti Templates.

Here are some examples:

InnoDB transactions

InnoDB Buffer Pool

So, I want it! Where should I look?

The code is hosted in my github repository, and everything you should know is in the mysql-snmp page on my site.

If you use this software, please do not hesitate to contribute, and/or fix bugs :-)