Recent Posts

Handwired Keyboard Build Log - Part 2

17 minute read

In the handwired build log part 1 we saw a technique to build a nice keyboard matrix without using a PCB.

In this part we’ll discover how to hook the teensy controller to the matrix.

The needed tools & parts

For this part, we’ll use:

  • the soldering station and solder
  • a pair of tweezers
  • a sharp knife
  • a wrench
  • a philips screwdriver

We’ll also need those parts:

Preamble

We plan to put the controller on a ribbon cable DIP connector:

the ribbon cable DIP connector

This is a special connector normally used to solder a ribbon cable to a PCB. This connector has the same footprint as a teensy, so instead of soldering it to a PCB, we’ll solder the teensy on the connector.

The ribbon cable will go from this connector to the matrix. To secure the ribbon cable, we need to use some wrench to crimp the connector on the ribbon cable. Each conductor from the ribbon cable ends up on a pin of the DIP connector:

the ribbon cable and the teensy

For the controller to read the matrix we need to wire each row and each column to a given port on the MCU. The good thing is that any port will do it, we don’t need to wire specifically a column or a row to a specific port (this would have been different if we had backlight leds which work better with a PWM capable pin).

An upcoming case?

I didn’t plan any case, but I want in the end to put a transparent PMMA bottom on which I could glue the controller. Since the plate has 5 M2 screw holes, I plan to secure the bottom plate through these holes by using the screws and five PCB brass spacers.

Wiring the matrix to the controller

We have 12 columns and 4 rows in the matrix, all those needs to be connected to the MCU ports. That means we’re going to use 16 conductors out of 24 on our ribbon cable.

For aesthetic reasons, and since 12 is a multiple of 4, I’ve splitted the 16 conductors ribbon cable in 4 pieces of 4 conductors.

The idea is to route the 4 conductors ribbon up to where the individual connector will be soldered to the matrix.

The big difficulty is to plan the length of the 4 conductors ribbons and when to split them in individual conductors. Again for aesthetic reasons, I decided to keep the conductors bound together in the ribbon as much as physically possible.

The other good news is that a small ribbon of 4 conductors is about the same size as the distnce between 2 switches. So I can route those ribbons easily under the matrix wires and between the switches up to their respective destinations.

For aesthetic reason again, I decided to route all the ribbons starting from the controller at the same place in the back of the board, and make some 90º turns when needed. So I need to route them sorted by length (ie start by routing the longest ribbon and finish by the smallest one).

routing 2 first 4 conductors ribbons

As you can see in the picture, the ribbons turn around the brass spacers. I started routing from the middle lane which was free of spacers until the middle of the plate, then move up or down to access the needed columns.

One thing I didn’t plan very well was that I wanted the controller and its USB port to be on the left of the keyboard. But I did the routing from the left when the keyboard was on the front, so in the end the controller happens to be on the right side. Unfortunately it was to late to change it when I noticed it.

To connect the conductors to the matrix columns, I splitted the ribbon in individual connectors and routed them each to one switch:

end of a ribbon

Then it is a matter of cutting the conductor to the right length and remove the insulator. Usually it is not possible to use the wire stripper because it requires a large distance between the cut position and the end of the cable which is not possible when the conductor comes from the plate. I had to remove the insulator using a sharp knife and my nails.

To solder the conductors on the columns, I did a loop with the copper conductors around an existing solder junction, then used the solder iron to heat that existing solder. The loop was incorporated into the existing junction solder easily:

soldering individual conductors

Since we have 12 columns, I decided to route the first 4 conductors ribbon to the column 1 to 4 (the right ones on the back), the second ribbon to the middle ones (5 to 8), and the last column ribbons on the 8 to 12 columns. To balance a bit the routing, the first ribbons connect to the bottom row, the second one to the top row:

wiring all columns

The very next step is to route the last 4 conductors ribbon to the rows. The simples solution was to split the 4 conductors ribbon into 2 parts, one going up and one going down. Then solder the wires to the rows on the same column.

routing to the rows

And the final routing result:

routing final results

Soldering the controller

The first step is to crimple the DIP support on the ribbon cable. To help aligning the 16 conductors ribbon, I kept the unused 8 conductors part (this way the ribbon can’t move while closing the support).

the DIP support

To prevent any electric short between the controller and the switches, the controller is placed upside down, so the support pins are facing down in usual conditions (so the pins are facing up when the keyboard is reversed as in the picture).

Before soldering the controller to the support, I used the multimeter to make sure all the support pins are correctly connected to the matrix. To do that place the black electrode of the multimeter to one of the pin and check it is connected to the correct column or row.

At the same time note which pin is connected to which column or row, as we’ll have to use this information for the firmware:

the Teensy pins/port

If you follow exactly this tutorial, you’ll end up with this table for the rows:

row pin port
1 23 C6
2 11 D7
3 24 D7
4 12 D6

And this one for the columns:

Column pin port
1 19 D0
2 7 F7
3 20 D1
4 8 B6
5 22 D3
6 9 B5
7 21 D2
8 10 B4
9 9 F6
10 18 B7
11 17 B3
12 6 F5

Then solder the Teensy controller. It’s not easy as the Teensy PCB pads are very small, so better use a magnifying glass and a very small solder diameter (0.6mm for instance).

Teensy soldered

Programming the controller

To program the controller we’ll use QMK. This is an open source keyboard firmware forked and enhanced from TMK. It supports a miriad of custom keyboards and MCU (including various ATmega and ARM micro-controllers).

First, let’s clone the project:

git clone git@github.com:qmk/qmk_firmware.git
cd qmk_firmware

Then install the ATmega toolchain to be able to build the firmware for our keyboard. It’s very easy with the install process:

./util/qmk_install.sh

On macos it requires Homebrew.

Once done, check that you can compile a firmware, for instance the default GH60 keymap (a very well known 60% PCB):

% make gh60:default
QMK Firmware 0.6.193
Making gh60 with keymap default

avr-gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiling: keyboards/gh60/gh60.c                                                                    [OK]
Compiling: keyboards/gh60/keymaps/default/keymap.c                                                  [OK]
Compiling: quantum/quantum.c                                                                        [OK]
Compiling: quantum/keymap_common.c                                                                  [OK]
Compiling: quantum/keycode_config.c                                                                 [OK]
Compiling: quantum/matrix.c                                                                         [OK]
...
Compiling: lib/lufa/LUFA/Drivers/USB/Core/USBTask.c                                                 [OK]
Linking: .build/gh60_default.elf                                                                    [OK]
Creating load file for flashing: .build/gh60_default.hex                                            [OK]
Copying gh60_default.hex to qmk_firmware folder                                                     [OK]
Checking file size of gh60_default.hex                                                              [OK]
 * The firmware size is fine - 16926/28672 (11746 bytes free)

You should obtain the gh60_default.hex file. You can remove it, we won’t use it.

QMK supports many keyboards and many layouts (called keymaps in QMK) for each keyboard. A keyboard is defined by a directory in the keyboards/ folder, and each keymap is also a directory in the keymaps/ folder of a keyboard. To build such keymap, one need to use the make <keyboard>:<keymap> command.

The make command produces a hex file that can be flashed on the controller with QMK Toolbox, which is the recommended method. We can flash from the command line if we know the controller bootloader type, but QMK Toolbox is able to autodetect the correct bootloader, check the file size and so on. QMK Toolbox also acts as a console for the controller allowing to see debug statements.

For the Teensy, we’ll use the “halfkay” bootloader. One advantage of the Teensy compared to the Pro Micro controller (which we could have used), is that the bootloader is very forgiving: for instance a Pro Micro can be bricked if we flash a firmware that is too large for it.

Let’s implement our own Planck layout. The very first step is to create a new kind of keyboard in the handwired/ keyboard folder. Since it is a Planck keyboard, let’s create a planck folder in which we need to add the following files:

  • a keymaps/ folder (in which we’ll create our own default keymap)
  • a rules.mk makefile which contains our keyboard definition and QMK features enabled
  • a config.h which defines how our matrix is connected to the controller ports
  • a planck.c and planck.h which only defines the keymap macro in our case

You can find all the files in my QMK Handwired Planck branch.

Here’s a condensed version of my config.h:

/* key matrix size */
#define MATRIX_ROWS 4
#define MATRIX_COLS 12

/* Our handwired pin-out */
#define MATRIX_ROW_PINS { C6, D7, C7, D6 }
#define MATRIX_COL_PINS { D0, F7, D1, B6, D3, B5, D2, B4, F6, B7, B3, F5 }
#define UNUSED_PINS { B0, B1, B2, F0, F1, F4, D4, D5, E6 }

/* COL2ROW or ROW2COL */
#define DIODE_DIRECTION COL2ROW

We defined here that the matrix is 4x12, and the ports of the rows and columns (in increasing order). Also, we tell QMK that we hooked the diodes between the columns and the rows.

In rules.mk, we tell QMK everything about the used controller:

# This is a teensy 2.0
BOOTLOADER = halfkay
# running this MCU
MCU = atmega32u4
# Processor frequency.
F_CPU = 16000000
# Target architecture (see library "Board Types" documentation).
ARCH = AVR8
# Input clock frequency.
F_USB = $(F_CPU)
# Interrupt driven control endpoint task(+60)
OPT_DEFS += -DINTERRUPT_CONTROL_ENDPOINT
# Boot Section Size in *bytes*
#   Teensy halfKay   512
OPT_DEFS += -DBOOTLOADER_SIZE=512
# Build Options
BOOTMAGIC_ENABLE = no       # Virtual DIP switch configuration(+1000)
MOUSEKEY_ENABLE = no        # Mouse keys(+4700)
EXTRAKEY_ENABLE = yes       # Audio control and System control(+450)
CONSOLE_ENABLE = yes        # Console for debug(+400)
COMMAND_ENABLE = yes        # Commands for debug and configuration
NKRO_ENABLE = no            # Nkey Rollover
BACKLIGHT_ENABLE = no       # There are no leds
MIDI_ENABLE = no            # No MIDI controls
AUDIO_ENABLE = no           # We don't have audio
UNICODE_ENABLE = no         # Unicode
BLUETOOTH_ENABLE = no       # We don't have BT
RGBLIGHT_ENABLE = no        # We don't have underglow

I then created the default keymap. Since this is a Planck replica, I copied over the default Planck keymap of the MIT (2u space) layout. A keymap is a folder in the keymaps/ folder.

Usually the layout is described in the keymap.c file.

This keymap is a 3 layers keymap (base, raise, lower). The base layer can be either qwerty (the default), colemak or dvorak.

A layer is a 2D array representing the keycode associated with a matrix switch. A keymap is an array of layouts (see the keymaps symbol in the keymap.c), one per layer.

The keyboard can be in only one layer at a time, and can be programmed to switch to a given layer with a key combination as explained below.

Here’s for example the keymap of the base qwerty layer of my Planck handwired keyboard:

const uint16_t PROGMEM keymaps[][MATRIX_ROWS][MATRIX_COLS] = {
/* Qwerty
 * ,-----------------------------------------------------------------------------------.
 * | Tab  |   Q  |   W  |   E  |   R  |   T  |   Y  |   U  |   I  |   O  |   P  | Bksp |
 * |------+------+------+------+------+-------------+------+------+------+------+------|
 * | Esc  |   A  |   S  |   D  |   F  |   G  |   H  |   J  |   K  |   L  |   ;  |  '   |
 * |------+------+------+------+------+------|------+------+------+------+------+------|
 * | Shift|   Z  |   X  |   C  |   V  |   B  |   N  |   M  |   ,  |   .  |   /  |Enter |
 * |------+------+------+------+------+------+------+------+------+------+------+------|
 * | Brite| Ctrl | Alt  | GUI  |Lower |    Space    |Raise | Left | Down |  Up  |Right |
 * `-----------------------------------------------------------------------------------'
 */
[_QWERTY] = LAYOUT_planck_grid(
    KC_TAB,  KC_Q,    KC_W,    KC_E,    KC_R,    KC_T,    KC_Y,    KC_U,    KC_I,    KC_O,    KC_P,    KC_BSPC,
    KC_ESC,  KC_A,    KC_S,    KC_D,    KC_F,    KC_G,    KC_H,    KC_J,    KC_K,    KC_L,    KC_SCLN, KC_QUOT,
    KC_LSFT, KC_Z,    KC_X,    KC_C,    KC_V,    KC_B,    KC_N,    KC_M,    KC_COMM, KC_DOT,  KC_SLSH, KC_ENT ,
    BACKLIT, KC_LCTL, KC_LALT, KC_LGUI, LOWER,   KC_SPC,  KC_SPC,  RAISE,   KC_LEFT, KC_DOWN, KC_UP,   KC_RGHT
),
...
}

This corresponds to this layout:

Planck qwerty layout

All the people I showed the keyboard asked my why the Esc key is placed below the Tab key. I’m assuming that the original Planck layout has been built this way to enhance the VIM experience. This way the Esc key is on the homerow and can be reached without moving the left hand.

The LAYOUT_planck_grid macro has been defined in our planck.h file. It just maps a keycode to a spot in the layer array representing the matrix.

Notice the two RAISE and LOWER special keycodes. They are layer keycodes defined like this:

#define LOWER MO(_LOWER)
#define RAISE MO(_RAISE)

The MO(layer) macro allows to temporarily activate the given layer when the key is pressed.

The _LOWER and _RAISE layers are defined like this:

...
/* Lower
 * ,-----------------------------------------------------------------------------------.
 * |   ~  |   !  |   @  |   #  |   $  |   %  |   ^  |   &  |   *  |   (  |   )  | Bksp |
 * |------+------+------+------+------+-------------+------+------+------+------+------|
 * | Del  |  F1  |  F2  |  F3  |  F4  |  F5  |  F6  |   _  |   +  |   {  |   }  |  |   |
 * |------+------+------+------+------+------|------+------+------+------+------+------|
 * |      |  F7  |  F8  |  F9  |  F10 |  F11 |  F12 |ISO ~ |ISO | | Home | End  |      |
 * |------+------+------+------+------+------+------+------+------+------+------+------|
 * |      |      |      |      |      |             |      | Next | Vol- | Vol+ | Play |
 * `-----------------------------------------------------------------------------------'
 */
[_LOWER] = LAYOUT_planck_grid(
    KC_TILD, KC_EXLM, KC_AT,   KC_HASH, KC_DLR,  KC_PERC, KC_CIRC, KC_AMPR,    KC_ASTR,    KC_LPRN, KC_RPRN, KC_BSPC,
    KC_DEL,  KC_F1,   KC_F2,   KC_F3,   KC_F4,   KC_F5,   KC_F6,   KC_UNDS,    KC_PLUS,    KC_LCBR, KC_RCBR, KC_PIPE,
    _______, KC_F7,   KC_F8,   KC_F9,   KC_F10,  KC_F11,  KC_F12,  S(KC_NUHS), S(KC_NUBS), KC_HOME, KC_END,  _______,
    _______, _______, _______, _______, _______, _______, _______, _______,    KC_MNXT,    KC_VOLD, KC_VOLU, KC_MPLY
),

/* Raise
 * ,-----------------------------------------------------------------------------------.
 * |   `  |   1  |   2  |   3  |   4  |   5  |   6  |   7  |   8  |   9  |   0  | Bksp |
 * |------+------+------+------+------+-------------+------+------+------+------+------|
 * | Del  |  F1  |  F2  |  F3  |  F4  |  F5  |  F6  |   -  |   =  |   [  |   ]  |  \   |
 * |------+------+------+------+------+------|------+------+------+------+------+------|
 * |      |  F7  |  F8  |  F9  |  F10 |  F11 |  F12 |ISO # |ISO / |Pg Up |Pg Dn |      |
 * |------+------+------+------+------+------+------+------+------+------+------+------|
 * |      |      |      |      |      |             |      | Next | Vol- | Vol+ | Play |
 * `-----------------------------------------------------------------------------------'
 */
[_RAISE] = LAYOUT_planck_grid(
    KC_GRV,  KC_1,    KC_2,    KC_3,    KC_4,    KC_5,    KC_6,    KC_7,    KC_8,    KC_9,    KC_0,    KC_BSPC,
    KC_DEL,  KC_F1,   KC_F2,   KC_F3,   KC_F4,   KC_F5,   KC_F6,   KC_MINS, KC_EQL,  KC_LBRC, KC_RBRC, KC_BSLS,
    _______, KC_F7,   KC_F8,   KC_F9,   KC_F10,  KC_F11,  KC_F12,  KC_NUHS, KC_NUBS, KC_PGUP, KC_PGDN, _______,
    _______, _______, _______, _______, _______, _______, _______, _______, KC_MNXT, KC_VOLD, KC_VOLU, KC_MPLY
),
...

Since on a 40% keyboard we can’t have access to the numbers, function keys, and most of the symbols, those are placed on a different layer than the regular direct access keys. The two raise/lower keys can be actionned by the left and right thumb while at the same time pressing another key to obtain the number or symbol. This is very efficient.

The _______ is an alias for KC_TRANS which means that this key isn’t defined in this layer. When pressing this key while being in this layer, the keycode that will be emited is the first one to not be KC_TRANS in the layer stack. That means that Enter for instance is still Enter in any of the RAISE or LOWER layer.

The rest of the keymap.c file contain special code that overrides the default QMK behavior.

In QMK, a keyboard can override some functionalities, and a keymap can override the keyboard override.

For instance we overrode the process_record function by defining the process_record_user function in our keymap. This is a function which is called each time a key event happens (a key pressed or released). In our case, this is used to switch to a different base layer when going to the ADJUST layer and pressing a base layer key (for instance it is K to switch to colemak). The ADJUST layer is obtained by pressing at the same time the LOWER and RAISE keys.

We also overrode layer_state_set_user to make the LOWER + RAISE = ADJUST layer switching work. The layer_state_set_user function is called whenever QMK is switching to another layer, giving a chance to modify the target layer. We used update_tri_layer_state to return ADJUST when we switched to both LOWER and RAISE.

Now let’s build our firmware:

% make handwired/planck:default
QMK Firmware 0.6.193
Making handwired/planck with keymap default

avr-gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiling: keyboards/handwired/planck/planck.c                                                      [OK]
Compiling: keyboards/handwired/planck/keymaps/default/keymap.c                                      [OK]
Compiling: quantum/quantum.c                                                                        [OK]
Compiling: quantum/keymap_common.c                                                                  [OK]
Compiling: quantum/keycode_config.c                                                                 [OK]
Compiling: quantum/matrix.c                                                                         [OK]
Compiling: tmk_core/common/host.c                                                                   [OK]
Compiling: tmk_core/common/keyboard.c                                                               [OK]
Compiling: tmk_core/common/action.c                                                                 [OK]
Compiling: tmk_core/common/action_tapping.c                                                         [OK]
Compiling: tmk_core/common/action_macro.c                                                           [OK]
Compiling: tmk_core/common/action_layer.c                                                           [OK]
Compiling: tmk_core/common/action_util.c                                                            [OK]
Compiling: tmk_core/common/print.c                                                                  [OK]
Compiling: tmk_core/common/debug.c                                                                  [OK]
Compiling: tmk_core/common/util.c                                                                   [OK]
Compiling: tmk_core/common/eeconfig.c                                                               [OK]
Compiling: tmk_core/common/report.c                                                                 [OK]
Compiling: tmk_core/common/avr/suspend.c                                                            [OK]
Compiling: tmk_core/common/avr/timer.c                                                              [OK]
Compiling: tmk_core/common/avr/bootloader.c                                                         [OK]
Assembling: tmk_core/common/avr/xprintf.S                                                           [OK]
Compiling: tmk_core/common/magic.c                                                                  [OK]
Compiling: tmk_core/common/command.c                                                                [OK]
Compiling: tmk_core/protocol/lufa/lufa.c                                                            [OK]
Compiling: tmk_core/protocol/usb_descriptor.c                                                       [OK]
Compiling: tmk_core/protocol/lufa/outputselect.c                                                    [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Class/Common/HIDParser.c                                       [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/Device_AVR8.c                                        [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/EndpointStream_AVR8.c                                [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/Endpoint_AVR8.c                                      [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/Host_AVR8.c                                          [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/PipeStream_AVR8.c                                    [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/Pipe_AVR8.c                                          [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/USBController_AVR8.c                                 [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/AVR8/USBInterrupt_AVR8.c                                  [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/ConfigDescriptors.c                                       [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/DeviceStandardReq.c                                       [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/Events.c                                                  [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/HostStandardReq.c                                         [OK]
Compiling: lib/lufa/LUFA/Drivers/USB/Core/USBTask.c                                                 [OK]
Linking: .build/handwired_planck_default.elf                                                        [OK]
Creating load file for flashing: .build/handwired_planck_default.hex                                [OK]
Copying handwired_planck_default.hex to qmk_firmware folder                                         [OK]
Checking file size of handwired_planck_default.hex                                                  [OK]
 * The firmware size is fine - 17618/32256 (14638 bytes free)

Our firmware is in the handwired_planck_default.hex file.

To flash it:

  1. Connect the Teensy to the computer
  2. Open QMK Toolbox
  3. Press the Teensy reset button
  4. QMK Toolbox will notice a Teensy is connected by displaying *** Halfkay device connected
  5. Load the firmware
  6. Choose the ATMega32U4 microcontroller
  7. Press the flash button

You should see something like this:

*** Halfkay device connected
*** Attempting to flash, please don't remove device
>>> teensy_loader_cli -mmcu=atmega32u4 /Users/brice/devl/qmk_firmware/handwired_planck_default.hex -v
    Teensy Loader, Command Line, Version 2.1
    Read "handwired_planck_default.hex": 17618 bytes, 54.6% usage
    Found HalfKay Bootloader
    Programming..........................................................................................................................................
    Booting
*** Halfkay device disconnected
*** masterzen - Planck connected -- 0xFEED:0x6060

At this point your computer should recognize that a new keyboard has been connected. If you press any switches it should produce a letter.

You can now test the keyboard and the keymap with the Keyboard Tester.

Adding keycaps

It can be hard to find nice 40% ortho keycaps. I used the MDA Big Bang set. It’s a nice, smooth (and not that expensive) thick PBT keyset with dye-sub legends that covers a wide range of ortholinear keyboards, including the Planck.

The MDA (also called MIX or EDRUG) profile is a newer key profile that we could call the little brother of the SA profile. It’s less sculpted than SA, but still more than the other profiles.

Here’s how it looks on this handwired Planck:

My Handwired Planck with MDA Big Bang

What’s coming next

I haven’t had the time to work again on the keyboard, but I want to make it a bit slimmer (it has currently a height of 2cm not including keycaps and switches), and add a transparent bottom plate with small rubber feets.

I plan the bottom plate to be a transparent plexiglass plate (so that we can see the matrix), cut at the size of the metal switch plate. The complex part will be to correctly align the holes for the brass spacer screws along with making sure the hole head fits inside the plate.

To reduce the keyboard height, I will have to carve a small part of the bottom plate so that a small part of the teensy height can fit inside.

If possible, I’d like to build a full case made of plexiglass. I need to design it properly and find the correct tools to do that.

This will probably be the part 3 of this series!

Handwired Keyboard Build Log - Part 1

10 minute read

Update 1: I’ve finished the second part of the serie

Update 2: The plate I’m using has a height of 1.5mm, not 2.5mm as some astute readers have pointed out.

For the last 7 months, I’ve been discovering a new hobby: DIY mechanical keyboards. I’ve been using mechanical keyboards for ages (I was mostly typing on a Code keyboards lately), but lately moved to type only on my macbook keyboard (hopefully this is an early 2015, so the keyboard ie bearable).

So 7 months ago, I was browsing the Internet when I discovered a new world: there were passionate people that are building their keyboards and even programming them with QMK.

I soon was embarked in a no return journey to find the perfect keyboard, made of custom keyboards purchased from Korean Group Buys, properly prepared keyboard switches, custom color keycaps, split backspace layouts…

Forward to a month ago, I discovered that some people were building their own keyboards without any PCB, I then decided to try following the best handwiring guide with one of the smallest existing keyboard.

This serie of posts is the story behind this keyboard:

the end result

What’s a keyboard anyway

A keyboard if the combination of the following elements:

  • a matrix of switches soldered to a PCB
  • a metal or aluminium plate on which the switches are clipsed to
  • a controller that “reads” the matrix and sends keycodes to the USB port
  • keycaps

So in a very fast loop, the controller’s firmware will power on one column of the matrix and “read” the tension back on the matrix rows. If there’s some tension on one row given current on a column, then the controller can deduce which switch has been pressed. The firmware will then send the corresponding keycode based on the layout to the USB port, and continue with the next column and so on infinitely.

But there’s a problem: if one presses more than one key at a time, it is possible for the controller to register ghost keypresses. See the following schema:

Ghosting Matrix

When the controller will power the Col0, and if K00, K01 and K11 are depressed simultaneously, the controller will read current on both Line0 and Line1, because the current will flow from Col0 to K00 pin 1, then pin 2 because the switch is depressed, then to switch K01, then to switch K11, then to Line1. For the controller it is as if all the switches have been pressed, instead of the 3 that were indeed depressed.

To prevent this we add diodes between the switch and the row it is connected to:

Anti Ghosting Matrix

In the same hypothetical scenario as before, the current will be prevented to flow back from K00 to K01 and from Line0 by the D01 diode. Thus when powering Col0, the controller will only see a tension on Line0. And when powering Col1 it will see a tension from Line0 and Line1, thus registering 3 key presses.

Handwiring

Handwiring as its name implies is an electronic technique of building electronic circuits without using a PCB, and instead wiring all components one by one manually with small electric wires. It is often used to perform prototype of electronic boards.

The aim of this build log is to show how to wire a fully working (but small) keyboard.

The BOM

We’re going to build a Planck like keyboard in MIT layout, that is a matrix of 4x12, with a 2u spacebar, accounting for 47 keys. I chose this keyboard and layout because it’s one of the smaller keyboard (a 40%) and it is ortholinear (all switches are aligned) making it easy to wire.

So what will we need to build our keyboard:

  • 47 switches. I had a left-over of Aliaz silent 70g switches
  • a planck plate (which I purchased pre-made by Laserboost). You can get one either from them or Lasergist if you send them the CAD files. You can easily get the CAD files from the KLE layout tool and swillkb. I choose a 1.5mm metal plate to make sure it is sturdy enough to type on it.
  • electric wire of 0.2mm2 (24 AWG) of different colors
  • 47 diodes 1N4148
  • a controller: teensy 2.0. It can be a Pro Micro or even the newer QMK Proton C.
  • around 30cm of 24 way 1.27mm pitch ribbon cable
  • a ribbon DIP connector to attach to the ribbon cable and solder the teensy

The MIT layout Planck plate looks like this:

Planck Plate

Note that this plate has holes for a PCB mount stabilizer for the 2u space bar. I should have taken a version for a plate mount stabilizer, because with a PCB we won’t be able to put a stabilizer under the space bar.

We’ll also need the following tools:

  • a wire stripper. Being in Europe I got myself a nice Stanley one.
  • a set of tweezers
  • a precision wire cutter
  • a multimeter with continuity mode
  • a soldering station (preferrably temperature controlled) and solder
  • a sharp knife or razor (to remove insulators on very small cables)
  • an usb A to usb mini cable for programming the controller

The most important part is the insulator stripper:

Insulator Stripper

You can get a Vise Grip or any other tool like this one. You might need to tune it’s strength (there’s usually a small knob on the tool) so that it doesn’t cut the small wires.

In this part of the story, we’ll only need the wire stripper, some colored wires, the plate, 47 switches, tweezers, a multimeter, the wire cutter, solder and the soldering station.

Placing the switches

The very first step of our handwiring work is to firmly clips the switches on the plate:

switches on plate

switches on plate

I put the switches facing north (the led hole is at the top) so that the higher pins when seen from the back of the plate will be the pin connected to the rows, and the other one will be connected to the columns.

With a 2.5mm plate, the switches should clipse correctly on it. Make sure the plate is using MX switch holes without “top-opening” punches (straight square holes).

Preparing the diodes

We’ll have to solder the diodes on the switch pin connected to a row. Since there’s no circuit copper pads to put the solder on like on a PCB, the best way to solder something on a pin is to form a small wire loop and put solder on it. The solder will “flow” between the pin and the loop and stick firmly.

So our first task is to form a small loop with one of the leg of the diodes. Make sure to do it on the correct leg: the one opposite to the diode black mark:

diode loop

To ease the process and especially if you got your diodes in bands, you can bend them all in one shot on your desk table like this:

bending diodes

Next, remove the diodes from the strip, and using a tweezer block the wire in it and form a loop by turning around the diode leg. With the tweezer you can make sure the loop is flat. Make sure the loop is big enough to be placed on a switch pin, if not open it a little bit with the tweezers.

Repeat this for the other 46 diodes.

After this you can cut the extraneous diode leg just after the loop:

all diodes

Soldering the diodes

The very next step is to place the diode loops on each of the switch row pins:

placing diodes

And then soldering them:

soldering diodes

Make sure to orient correctly the diodes and leave the other leg correctly aligned.

Notice that I started placing and soldering the diodes from the top row (as seen from the back) so that the other rows diodes long legs doesn’t hinder soldering the next row.

two rows of diodes

We’ll then keep adding diodes and soldering them until we’ve covered all switches:

all diodes soldered

It is important to not cut the remaining leg yet. We’ll use it to connect to the row wire as you’ll find in the next step.

Building the rows

To build the rows, we’ll take a long piece of wire (I used black wire). The first thing to do is to remove the insulator at the loose end on something like 5cm with the wire stripper.

We know that we have 12 switches on a row (except the bottom one which has only 11 switches). There is 19mm between two switches.

Instead of cutting 11 pieces of wire which will be hard to solder in place correctly, we’ll use only one long straight piece of wire on which we’ll split the insulator with the wire stripper into 11 pieces of around 16mm each (without cutting the cable). Since it is hard to correctly measure the insulator length while using the wire stripper, I used a visual clue on the wire stripper to approximate the correct length and aligned the insulator pieces with it before cutting.

To solder the wire, we’re going to bend the diode leg around the cable to form a half loop and solder between the insulator pieces. At the same time secure the row wire with the switch central bumps.

We’ll start by soldering the loose end on the first diode, then proceed to the next diode: push the insulator piece toward the first junction, solder and move back the insulator at equal distance:

soldering first diode in a row

For the first diode, I formed a complete loop with the leg around the wire. For the subsequent diode since it is not practical to do that, I’ve done only half loops.

splitting the insulator

Another option which I used at first is to cut the insulator with the wire stripper on each step instead of cutting the 11 pieces at once. So solder one diode leg, cut the insulator at the right lenght, push it toward the last soldered diode, solder the next one, and so on. This is more effective if the distance between switches is variable, otherwise use the first method.

The last diode in the row should also be soldered by forming a full loop with the diode leg around the wire.

It is important to solder the diode leg on the wire before moving to the next diode leg along the row, otherwise the cable could move during the process and nothing would be correctly aligned.

Cut the extraneous wire at both ends and all the remaining legs with the wire cutter and you should obtain something like this:

1st row done

Apply the same technique to the 3 remaining rows. It takes around 10 minutes to solder a 12 switches row:

rows are almost finished

At this stage, you can check with a multimeter that each switch is correctly connected to the wire. Use the multimeter in continuity mode (the multimeter will beep if there is continuity between two junctions), and put the black electrode on one row end and place the red one on every diode junction, there should be a beep. You can also test the continuity of the switches and diodes combination: still with the black electrode on the row, place the red one on the other switch pin and press the switch: the multimeter should beep.

Once you’ve made sure everything works electrically, it is time to move to the columns wiring.

Wiring the columns

For a better visual effect, I’ve decided to wire each column with a different wire color. Unfortunately I couldn’t find 12 different wire colors, so I’ve used only 6 that I repeated twice. I arranged the colors in an approximation of a rainbow.

We’ll use the exact same technique as for the rows, except that we need to split the insulator into only 3 pieces of equal length (there are only 4 rows on this keyboard). To make sure we have enough wire, I didn’t cut it before soldering the last switch in a column.

Since we don’t have the diodes leg to form a loop around the wire, we’ll build loops with the wire around the switch pins:

wire loop around switch leg

Once soldered we can move to the next switch, push back the insulator toward the previous switch, solder the current one and so on:

wire loop around switch leg

Keep doing this until you’ve done all columns. It takes only a few minute per columns once we get the habit.

Since there are only 11 switches on the bottom row, one of the column will span only 3 switches.

The result should look like this:

the matrix is finished

You can then use the multimeter to check the colums are correctly wired, and that no rows is electrically connected to a column.

What’s coming next

In the handwired build log part 2, I’ll explain how to connect the controller to the matrix and how to program the controller to become a working keyboard. We’ll also put some keycaps on.

Another part will explain how I’m going to build a bottom plate for the keyboard.

Bootstrapping Windows servers with Puppet

16 minute read

All started a handful of months ago, when it appeared that we’d need to build some of our native software on Windows. Before that time, all our desktop software at Days of Wonder was mostly cross-platform java code that could be cross-compiled on Linux. Unfortunately, we badly needed a Windows build machine.

In this blog post, I’ll tell you the whole story from my zero knowledge of Windows administration to an almost fully automatized Windows build machine image construction.

Jenkins

But, first let’s digress a bit to explain in which context we operate our builds.

Our CI system is built around Jenkins, with a specific twist. We run the Jenkins master on our own infrastructure and our build slaves on AWS EC2. The reason behind this choice is out of the scope of this article (but you can still ask me, I’ll happily answer).

So, we’re using the Jenkins EC2 plugin, and a revamped by your servitor Jenkins S3 Plugin. We produce somewhat large binary artifacts when building our client software, and the bandwidth between EC2 and our master is not that great (and expensive), so using the aforementioned patch I contributed, we host all our artifacts into S3, fully managed by our out-of-aws Jenkins master.

The problem I faced when starting to explore the intricate world of Windows in relation with Jenkins slave, is that we wanted to keep the Linux model we had: on-demand slave spawned by the master when scheduling a build. Unfortunately the current state of the Jenkins EC2 plugin only supports Linux slave.

Enter WinRM and WinRS

The EC2 plugin for Linux slave works like this:

  1. it starts the slave
  2. using an internal scp implementation it copies ‘slave.jar’ which implements the client Jenkins remoting protocol
  3. using an internal ssh implementation, it executes java -jar slave.jar. The stdin and stdout of the slave.jar process is then connected to the jenkins master through an ssh tunnel.
  4. now, Jenkins does its job (basically sending more jars, classes)
  5. at this stage the slave is considered up

I needed to replicate this behavior. In the Windows world, ssh is inexistent. You can find some native implementation (like FreeSSHd or some other commercial ones), but all that options weren’t easy to implement, or simply non-working.

In the Windows world, remote process execution is achieved through the Windows Remote Management which is called WinRM for short. WinRM is an implementation of the WSMAN specifications. It allows to access the Windows Management Instrumentation to get access to hardware counters (ala SNMP or IPMI for the unix world).

One component of WinRM is WinRS: Windows Remote Shell. This is the part that allows to run remote commands. Recent Windows version (at least since Server 2003) are shipped with WinRM installed (but not started by default).

WinRM is an HTTP/SOAP based protocol. By default, the payload is encrypted if the protocol is used in a Domain Controller environment (in this case, it uses Kerberos), which will not be our case on EC2.

Digging, further, I found two client implementations:

I started integrating Overthere into the ec2-plugin but encountered several incompatibilities, most notably Overthere was using a more recent dependency on some libraries than jenkins itself.

I finally decided to create my own WinRM client implementation and released Windows support for the EC2 plugin. This hasn’t been merged upstream, and should still be considered experimental.

We’re using this version of the plugin for about a couple of month and it works, but to be honest WinRM doesn’t seem to be as stable as ssh would be. There are times the slave is unable to start correctly because WinRM abruptly stops working (especially shortly after the machine boots).

WinRM, the bootstrap

So all is great, we know how to execute commands remotely from Jenkins. But that’s not enough for our sysadmin needs. Especially we need to be able to create a Windows AMI that contains all our software to build our own applications.

Since I’m a long time Puppet user (which you certainly noticed if you read this blog in the past), using Puppet to configure our Windows build slave was the only possiblity. So we need to run Puppet on a Windows base AMI, then create an AMI from there that will be used for our build slaves. And if we can make this process repeatable and automatic that’d be wonderful.

In the Linux world, this task is usually devoted to tools like Packer or Veewee (which BTW supports provisioning Windows machines). Unfortunately Packer which is written in Go doesn’t yet support Windows, and Veewee doesn’t support EC2.

That’s the reason I ported the small implementation I wrote for the Jenkins EC2 plugin to a WinRM Go library. This was the perfect pet project to learn a new language :)

Windows Base AMI

So, starting with all those tools, we’re ready to start our project. But there’s a caveat: WinRM is not enabled by default on Windows. So before automating anything we need to create a Windows base AMI that would have the necessary tools to further allow automating installation of our build tools.

Windows boot on EC2

There’s a service running on the AWS Windows AMI called EC2config that does the following at the first boot:

  1. set a random password for the ‘Administrator’ account
  2. generate and install the host certificate used for Remote Desktop Connection.
  3. execute the specified user data (and cloud-init if installed)

On first and subsequent boot, it also does:

  1. it might set the computer host name to match the private DNS name
  2. it configures the key management server (KMS), check for Windows activation status, and activate Windows as necessary.
  3. format and mount any Amazon EBS volumes and instance store volumes, and map volume names to drive letters.
  4. some other administrative tasks

One thing that is problematic with Windows on EC2 is that the Administrator password is unfortunately defined randomly at the first boot. That means to further do things on the machine (usually using remote desktop to administer it) you need to first know it by asking AWS (with the command-line you can do: aws ec2 get-password-data).

Next, we might also want to set a custom password instead of this dynamic one. We might also want to enable WinRM and install several utilities that will help us later.

To do that we can inject specific AMI user-data at the first boot of the Windows base AMI. Those user-data can contain one or more cmd.exe or Powershell scripts that will get executed at boot.

I created this Windows bootstrap Gist (actually I forked and edited the part I needed) to prepare the slave.

First bootstrap

First, we’ll create a Windows security group allowing incoming WinRM, SMB and RDP:

aws ec2 create-security-group --group-name "Windows" --description "Remote access to Windows instances"
# WinRM
aws ec2 authorize-security-group-ingress --group-name "Windows" --protocol tcp --port 5985 --cidr <YOURIP>/32
# Incoming SMB/TCP 
aws ec2 authorize-security-group-ingress --group-name "Windows" --protocol tcp --port 445 --cidr <YOURIP>/32
# RDP
aws ec2 authorize-security-group-ingress --group-name "Windows" --protocol tcp --port 3389 --cidr <YOURIP>/32

Now, let’s start our base image with the following user-data (let’s put it into userdata.txt):

<powershell>
Set-ExecutionPolicy Unrestricted
icm $executioncontext.InvokeCommand.NewScriptBlock((New-Object Net.WebClient).DownloadString('https://gist.github.com/masterzen/6714787/raw')) -ArgumentList "VerySecret"
</powershell>

This powershell script will download the Windows bootstrap Gist and execute it, passing the desired administrator password.

Next we launch the instance:

aws ec2 run-instances --image-id ami-4524002c --instance-type m1.small --security-groups Windows --key-name <YOURKEY> --user-data "$(cat userdata.txt)"

Unlike what is written in the ec2config documentation, the user-data must not be encoded in Base64.

Note, the first boot can be quite long :)

After that we can connect through WinRM with the “VerySecret” password. To check we’ll use the WinRM Go tool I wrote and talked about above:

./winrm -hostname <publicip> -username Administrator -password VerySecret "ipconfig /all"

We should see the output of the ipconfig command.

Note: in the next winrm command, I’ve omitted the various credentials to increase legibility (a future version of the tool will allow to read a config file, meanwhile we can create an alias).

A few caveats:

  • BITS doesn’t work in the user-data powershell, because it requires a user to be logged-in which is not possible during boot, that’s the reason downloading is done through the System.Net.WebClient
  • WinRM enforces some resource limits, you might have to increase the allowed shell resources for running some hungry commands: winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"} Unfortunately, this is completely broken in Windows Server 2008 unless you install this Microsoft hotfix The linked bootstrap code doesn’t install this hotfix, because I’m not sure I can redistribute the file, that’s an exercise left to the reader :)
  • the winrm traffic is not encrypted nor protected (if you use my tool). Use at your own risk. It’s possible to setup WinRM over HTTPS, but it’s a bit more involved. Current version of my WinRM tool doesn’t support HTTPS yet (but it’s very easy to add).

Baking our base image

Now that we have our base system with WinRM and Puppet installed by the bootstrap code, we need to create a derived AMI that will become our base image later when we’ll create our different windows machines.

aws ec2 create-image --instance-id <ourid> --name 'windows-2008-base'

For a real world example we might have defragmented and blanked the free space of the root volume before creating the image (on Windows you can use sdelete for this task).

Note that we don’t run the Ec2config sysprep prior to creating the image, which means the first boot of any instances created from this image won’t run the whole boot sequence and our Administrator password will not be reset to a random password.

Where does Puppet fit?

Now that we have this base image, we can start deriving it to create other images, but this time using Puppet instead of a powershell script. Puppet has been installed on the base image, by virtue of the powershell bootstrap we used as user-data.

First, let’s get rid of the current instance and run a fresh one coming from the new AMI we just created:

aws ec2 run-instances --image-id <newami> --instance-type m1.small --security-groups Windows --key-name <YOURKEY>

Anatomy of running Puppet

We’re going to run Puppet in masterless mode for this project. So we need to upload our set of manifests and modules to the target host.

One way to do this is to connect to the host with SMB over TCP (which our base image supports):

sudo mkdir -p /mnt/win
sudo mount -t cifs -o user="Administrator%VerySecret",uid="$USER",forceuid "//<instance-ip>/C\$/Users/Administrator/AppData/Local/Temp" /mnt/win

Note how we’re using an Administrative Share (the C$ above). On Windows the Administrator user has access to the local drives through Administrative Shares without having to share them as for normal users.

The user-data script we ran in the base image opens the windows firewall to allow inbound SMB over TCP (port 445).

We can then just zip our manifests/modules, send the file over there, and unzip remotely:

zip -q -r /mnt/win/puppet-windows.zip manifests/jenkins-steam.pp modules -x .git
./winrm "7z x -y -oC:\\Users\\Administrator\\AppData\\Local\\Temp\\ C:\\Users\\Administrator\\AppData\\Local\\Temp\\puppet-windows.zip | FIND /V \"ing  \""

And finally, let’s run Puppet there:

./winrm "\"C:\\Program Files (x86)\\Puppet Labs\\Puppet\\bin\\puppet.bat\" apply --debug --modulepath C:\\Users\\Administrator\\AppData\\Local\\Temp\\modules C:\\Users\\Administrator\\AppData\\Local\\Temp\\manifests\\site.pp"

And voila, shortly we’ll have a running instance configured. Now we can create a new image from it and use it as our Windows build slave in the ec2 plugin configuration.

Puppet on Windows

Puppet on Windows is not like your regular Puppet on Unix. Let’s focus on what works or not when running Puppet on Windows.

Core resources known to work

The obvious ones known to work:

  • File: beside symbolic links that are supported only on Puppet >3.4 and Windows 2008+, there are a few things to take care when using files:
    • NTFS is case-insensitive (but not the file resource namevar)
    • Managing permissions: octal unix permissions are mapped to Windows permissions, but the translation is imperfect. Puppet doesn’t manage Windows ACL (for more information check Managing File Permissions on Windows)
  • User: Puppet can create/delete/modify local users. The Security Identifier (SID) can’t be set. User names are case-insensitive on Windows. To my knowledge you can’t manage domain users.

  • Group: Puppet can create/delete/modify local groups. Puppet can’t manage domain groups.

  • Package: Puppet can install MSI or exe installers present on a local path (you need to specify the source). For a more comprehensive package system, check below the paragraph about Chocolatey.

  • Service: Puppet can start/stop/enable/disable services. You need to specify the short service name, not the human-reading service name.

  • Exec: Puppet can run executable (any .exe, .com or .bat). But unlike on Unix, there is no shell so you might need to wrap the commands with cmd /c. Check the Powershell exec provider module for a more comprehensive Exec system on Windows.

  • Host: works the same as for Unix systems.

  • Cron: there’s no cron system on Windows. Instead you must use the Scheduled_task type.

Do not expect your average unix module to work out-of-the-box

Of course that’s expected, mostly because of the used packages. Most of the Forge module for instance are targeting unix systems. Some Forge modules are Windows only, but they tend to cover specific Windows aspects (like registry, Powershell, etc…), still make sure to check those, as they are invaluable in your module Portfolio.

My Path is not your Path!

You certainly know that Windows paths are not like Unix paths. They use \ where Unix uses /.

The problem is that in most languages (including the Puppet DSL) ‘' is considered as an escape character when used in double quoted strings literals, so must be doubled \\.

Puppet single-quoted strings don’t understand all of the escape sequences double-quoted strings know (it only parses \' and \\), so it is safe to use a lone \ as long as it is not the last character of the string.

Why is that?

Let’s take this path C:\Users\Administrator\, when enclosed in a single-quoted string 'C:\Users\Administrator\' you will notice that the last 2 characters are \' which forms an escape sequence and thus for Puppet the string is not terminated correctly by a single-quote. The safe way to write a single-quoted path like above is to double the final slash: 'C:\Users\Administrator\\', which looks a bit strange. My suggestion is to double all \ in all kind of strings for simplicity.

Finally when writing an UNC Path in a string literal you need to use four backslashes: \\\\host\\path.

Back to the slash/anti-slash problem there’s a simple rule: if the path is directly interpreted by Puppet, then you can safely use /. If the path if destined to a Windows command (like in an Exec), use a \.

Here’s a list of possible type of paths for Puppet resources:

  • Puppet URL: this is an url, so /
  • template paths: this is a path for the master, so /
  • File path: it is preferred to use / for coherence
  • Exec command: it is preferred to use /, but beware that most Windows executable requires \ paths (especially cmd.exe)
  • Package source: it is preferred to use /
  • Scheduled task command: use \ as this will be used directly by Windows.

Windows facts to help detection of windows

To identify a Windows client in a Puppet manifests you can use the kernel, operatingsystem and osfamily facts that all resolves to windows.

Other facts, like hostname, fqdn, domain or memory*, processorcount, architecture, hardwaremodel and so on are working like their Unix counterpart.

Networking facts also works, but with the Windows Interface name (ie Local_Area_Connection), so for instance the local ip address of a server will be in ipaddress_local_area_connection. The ipaddress fact also works, but on my Windows EC2 server it is returning a link-local IPv6 address instead of the IPv4 Local Area Connection address (but that might because it’s running on EC2).

Do yourself a favor and use Chocolatey

We’ve seen that Puppet Package type has a Windows provider that knows how to install MSI and/or exe installers when provided with a local source. Unfortunately this model is very far from what Apt or Yum is able to do on Linux servers, allowing access to multiple repositories of software and on-demand download and installation (on the same subject, we’re still missing something like that for OSX).

Hopefully in the Windows world, there’s Chocolatey. Chocolatey is a package manager (based on NuGet) and a public repository of software (there’s no easy way to have a private repository yet). If you read the bootstrap code I used earlier, you’ve seen that it installs Chocolatey.

Chocolatey is quite straightforward to install (beware that it doesn’t work for Windows Server Core, because it is missing the shell Zip extension, which is the reason the bootstrap code installs Chocolatey manually).

Once installed, the chocolatey command allows to install/remove software that might come in several flavors: either command-line packages or install packages. The first one only allows access through the command line, whereas the second does a full installation of the software.

So for instance to install Git on a Windows machine, it’s as simple as:

chocolatey install git.install

To make things much more enjoyable for the Puppet users, there’s a Chocolatey Package Provider Module on the Forge allowing to do the following

package {
  "cmake":
    ensure => installed,
    provider => "chocolatey"
}

Unfortunately at this stage it’s not possible to host easily your own chocolatey repository. But it is possible to host your own chocolatey packages, and use the source metaparameter. In the following example we assume that I packaged cmake version 2.8.12 (which I did by the way), and hosted this package on my own webserver:

# download_file uses powershell to emulate wget
# check here: http://forge.puppetlabs.com/opentable/download_file
download_file { "cmake":
  url                   => "http://chocolatey.domain.com/packages/cmake.2.8.12.nupkg",
  destination_directory => "C:\\Users\\Administrator\\AppData\\Local\\Temp\\",
}
->
package {
  "cmake":
    ensure => install,
    source => "C:\\Users\\Administrator\\AppData\\Local\\Temp\\"
}

You can also decide that chocolatey will be the default provider by adding this to your site.pp:

Package {
  provider => "chocolatey"
}

Finally read how to create chocolatey packages if you wish to create your own chocolatey packages.

Line endings and character encodings

There’s one final things that the Windows Puppet user must take care about. It’s line endings and character encodings. If you use Puppet File resources to install files on a Windows node, you must be aware that file content is transferred verbatim from the master (either by using content or source).

That means if the file uses the Unix LF line-endings the file content on your Windows machine will use the same. If you need to have a Windows line ending, make sure your file on the master (or the content in the manifest) is using Windows \r\n line ending.

That also means that your text files might not use a windows character set. It’s less problematic nowadays than it could have been in the past because of the ubiquitous UTF-8 encoding. But be aware that the default character set on western Windows systems is CP-1252 and not UTF-8 or ISO-8859-15. It’s possible that cmd.exe scripts not encoded in CP-1252 might not work as intended if they use characters out of the ASCII range.

Conclusion

I hope this article will help you tackle the hard task of provisioning Windows VM and running Puppet on Windows. It is the result of several hours of hard work to find the tools and learn Windows knowledge.

During this journey, I started learning a new language (Go), remembered how I dislike Windows (and its administration), contributed to several open-source projects, discovered a whole lot on Puppet on Windows, and finally learnt a lot on WinRM/WinRS.

Stay tuned on this channel for more article (when I have the time) about Puppet, programming and/or system administration :)