Ecukes - Cucumber for Emacs

There are plenty of unit/regression testing tools for Emacs, and even some for functional testing. What Emacs is missing though is a really good testing framework for integration testing. This is where Ecukes comes in.

Cucumber is a great integration testing tool, used mostly for testing web applications. Ecukes is Cucumber for Emacs. No, it's not a major mode to edit feature files. It is a package that makes it possible to write Cucumber like tests for your Emacs packages.

Ecukes is not a complete clone of Cucumber and is not intended to be. If however Ecukes is missing some feature that you feel really should be included, please make a bug report.

If you don't know anything about Cucumber I suggest you read up a bit about it before continuing with Ecukes.

To get an idea of how Ecukes can be used, here is an example from a feature in drag-stuff.

~/Code/drag-stuff $ carton exec ecukes features/line.feature
Feature: Drag line
  • In order to move a line up and down
  • As an Emacs user
  • I want to drag it
Background:
  • Given I am in the buffer "*drag-stuff*"
  • And the buffer is empty
  • And I insert
    • """
    • line 1
    • line 2
    • """
Scenario: Drag line up
  • When I go to line "2"
  • And I press "<M-up>"
  • Then I should see:
    • """
    • line 2
    • line 1
    • """
Scenario: Drag line down
  • When I go to line "1"
  • And I press "<M-down>"
  • Then I should see:
    • """
    • line 2
    • line 1
    • """
  • 2 scenarios (0 failed, 2 passed)
  • 11 steps (0 failed, 11 passed)

Carton

It is highly recommended to install Ecukes is by using Carton. Install Carton then add a Carton file looking something like this.

(source "melpa" "http://melpa.milkbox.net/packages/")

(package "super-duper" "0.0.1" "Super Duper.")

(development
  (depends-on "ecukes")
  (depends-on "espuds"))

Next, run the carton command to install all dependencies.

A simple way to run your features is to create a Makefile.

ECUKES = $(shell find elpa/ecukes-*/ecukes | tail -1)

all:
	carton exec ${ECUKES} features

This is the Ecukes usage information:

USAGE: ecukes [--script|--win] [file|dir] [options]

OPTIONS:
 -h, --help             Display this help message
 --script               Run Ecukes as a script/batch job
 --win                  Run Ecukes with full GUI window
 --new                  Create new Ecukes setup for project
 --list-steps           Print all available steps defined for this project.
 --with-doc             Include docstring when printing steps with --list-steps.
 --with-file            Include file name when printing steps with --list-steps.
 --verbose              Show `message' output
 --dbg                  Run in debug mode (enable as much debug options in Emacs as possible)
 --tags TAG_EXPRESSION  Only execute the scenarios with tags matching TAG_EXPRESSION.
                        TAG_EXPRESSION Examples: --tags @dev, --tags @dev,~@local
                        A tag starting with ~ excluded from the scenarios.

Let's say you have a project called super-duper that you want to test with Ecukes.

First, create all boilerplate files:

$ carton exec ecukes --new
create features
create   step-definition
create     super-duper-steps.el
create   support
create     env.el
create   super-duper.feature

The project will now look like this:

$ ls super-duper

  super-duper/
  |-- README
  |-- features
  |   |-- step-definitions
  |   |   `-- super-duper-steps.el
  |   |-- super-duper.feature
  |   `-- support
  |       `-- env.el
  `-- super-duper.el

Files in the support directory will be loaded (env.el first, then the rest) each time Ecukes run. All files that ends with -steps.el in the step-definitions directory will be loaded as step definitions.

Modes

Ecukes can run in a few different modes.

No win (default)

Running with no win is the default way to run features. In this mode, Ecukes will run Emacs with the -nw option.

Example:

$ carton exec ecukes features

Win

This will start up an Emacs window and run the features. To use this mode, specify --win as first option to ecukes.

Example:

$ carton exec ecukes --win features

Script

This will run the features as a batch job. This mode has a few quirks and is only recommended as a way to run the features fast.

Example:

$ carton exec ecukes --script features

Inline

You can also run features directly in your current Emacs session. To to that, make sure Ecukes and its dependencies are available in your load-path. If you are using Carton for your Emacs setup, simply add ecukes as a dependency and you are set. Then require ecukes.

(require 'ecukes)

To run features, visit a file in your project and run the ecukes function.

M-x ecukes

Examples

Here are a few ways to run Ecukes:

Create boilerplate files for current project.

$ carton exec ecukes --new

Show usage information.

$ carton exec ecukes --help

Run all features in no win mode.

$ carton exec ecukes

Run all features in win mode.

$ carton exec ecukes --win features

Run all features in script mode.

$ carton exec ecukes --script

Run one feature in no win mode.

$ carton exec ecukes features/super-duper.feature

Run one feature in win mode with verbose output.

$ carton exec ecukes --win features/super-duper.feature --verbose

Run all scenarios tagged with @super in script mode with debug.

$ carton exec ecukes --script features/super-duper.feature --dbg --tags @super

Run all scenarios not tagged with @duper.

$ carton exec ecukes features/super-duper.feature --tags ~@duper

Run all features in no win mode. Use Emacs binary installed via Homebrew.

ECUKES_EMACS="/usr/local/Cellar/emacs/24.2/Emacs.app/Contents/MacOS/Emacs" carton exec ~/Code/ecukes/ecukes

Ecukes supports most Cucumbers features, such as: backgrounds, scenarios, scenario outlines and tags. Take a look at Cucumbers manual for more information about each.

There are three different kind of steps: Regular, Table and Py string.

Regular Step

A regular step always only consumes one line. The most basic step looks like this:

Given a known state

Corresponding step definition:

(Given "a known state"
       (lambda ()
         ;; Do something
         ))

The second argument could also have been the symbol of a function name:

(Given "a known state" 'do-something)

Steps can take arbitrary many arguments. The arguments sent to the step definition function are all regular expression match groupings from the matching of the step name and the match string.

Single argument:

Given I am in buffer "buffer-name"

Corresponding step definition:

(Given "I am in buffer \"\\(.+\\)\""
       (lambda (buffer-nane)
         ;; Do something with buffer
         ))

Multiple arguments:

Given I am in buffer "buffer-name" with text "Foo"

Corresponding step definition:

(Given "I am in buffer \"\\(.+\\)\" with text \"\\(.+\\)\""
       (lambda (buffer text)
         ;; Do something with buffer and text
         ))

Table Step

A table step looks like this:

Given these meals:
  | meal      | price |
  | Hamburger | $4.50 |
  | Pizza     | $5.30 |

Corresponding step definition:

(Given "these meals:"
       (lambda (meals)
         ;; Do something with text
         ))

The argument meals is a simple list where the car of the list is the header and the cdr or the list is the rows.

The header would in the above case be:

("meal" "price")
And the rows would be:
(("Hamburger" "$4.50")
 ("Pizza" "$5.30"))

To pick out the header and rows attribute from a table you do:

(let* ((table ...)
       (header (car table))
       (rows   (cdr table)))
  ;; Do something with header and rows
  )

It is also possible to send arguments to a table step. The table list will be the last argument.

Given these meals at "fast food":
  | meal      | price |
  | Hamburger | $4.50 |
  | Pizza     | $5.30 |

Corresponding step definition:

(Given "these meals at \"\\(.+\\)\":"
       (lambda (restaurant meals)
         ;; Do something with restaurant and meals
         ))

Py-string Step

A Py-String (or Python string) step looks like this:

Given the following text:
  """
  some text
  """

Corresponding step definition:

(Given "the following text:"
       (lambda (text)
         ;; Do something with text
         ))

It is also possible to send arguments to a Py-string step. The Py-string will be the last argument.

Given the following text in buffer "buffer-name":
  """
  some text
  """

Corresponding step definition:

(Given "the following text in buffer \"\\(.+\\)\":"
       (lambda (buffer text)
         ;; Do something with buffer and text
         ))

Calling steps from steps

In order to keep your steps DRY, you can call steps from other steps like this:

(Given "I go to a"
       (lambda ()
         ;; Go to a
         ))

(Given "I go to b"
       (lambda ()
         ;; Go to b
         ))

(Given "I go from a to b"
       (lambda ()
         (Given "I go to a")
         (Given "I go to b")))

You can also pass arguments when calling other steps, like this:

(Given "I am in buffer \"%s\"" buffer-name-variable)

Or like this:

(Given "I am in buffer \"buffer-name\"")

Asynchronous steps

If you pass one extra parameter to a step definition, that is used as a callback. Once called, the step is considered done.

(When "^run command \\(.+\\)$"
       (lambda (command callback)
         (run-command command callback)
         ))

Ecukes provides four different hooks

Setup

Runs once before anything runs.

(Setup
 ;; Run code
 )

Before

Runs once before each step runs.

(Before
 ;; Run code
 )

After

Runs once after each step runs.

(After
 ;; Run code
 )

Teardown

Runs once when everything is done.

(Teardown
 ;; Run code
 )

Hooks are useful if you test your program and change the state in some feature. Since all scenarios execute in the same environment, the state change will affect all scenarios after. You can solve that by resetting the state in a before or after hook.

Hooks should be placed in any file in the project features/support directory.

All contributions are much welcome and appreciated!

Before submitting a patch, make sure to write a test for it (if possible). Ecukes is tested with a testing framework called Ert (Emacs Lisp Regression Testing).

To run the tests, you have to install Carton if you haven't already. Once installed, run the carton command to install all dependencies.

To run all test, simply run the make command.

You can always report issues on Github if you're not up to fixing it yourself.

Fork me on GitHub