There are plenty of unit/regression testing tools for Emacs, and even some for functional testing. What Emacs is missing though is a really good testing framework for integration testing. This is where Ecukes comes in.
Cucumber is a great integration testing tool, used mostly for testing web applications. Ecukes is Cucumber for Emacs. No, it's not a major mode to edit feature files. It is a package that makes it possible to write Cucumber like tests for your Emacs packages.
Ecukes is not a complete clone of Cucumber and is not intended to be. If however Ecukes is missing some feature that you feel really should be included, please make a bug report.
If you don't know anything about Cucumber I suggest you read up a bit about it before continuing with Ecukes.
To get an idea of how Ecukes can be used, here is an example from a feature in drag-stuff.
And more...
It is highly recommended to install Ecukes is by using Carton. Install Carton then add a Carton file looking something like this.
(source "melpa" "http://melpa.milkbox.net/packages/")
(package "super-duper" "0.0.1" "Super Duper.")
(development
(depends-on "ecukes")
(depends-on "espuds"))
Next, run the carton command to install all dependencies.
A simple way to run your features is to create a Makefile.
ECUKES = $(shell find elpa/ecukes-*/ecukes | tail -1)
all:
carton exec ${ECUKES} features
This is the Ecukes usage information:
USAGE: ecukes [--script|--win] [file|dir] [options]
OPTIONS:
-h, --help Display this help message
--script Run Ecukes as a script/batch job
--win Run Ecukes with full GUI window
--new Create new Ecukes setup for project
--list-steps Print all available steps defined for this project.
--with-doc Include docstring when printing steps with --list-steps.
--with-file Include file name when printing steps with --list-steps.
--verbose Show `message' output
--dbg Run in debug mode (enable as much debug options in Emacs as possible)
--tags TAG_EXPRESSION Only execute the scenarios with tags matching TAG_EXPRESSION.
TAG_EXPRESSION Examples: --tags @dev, --tags @dev,~@local
A tag starting with ~ excluded from the scenarios.
Let's say you have a project called super-duper that you want to test with Ecukes.
First, create all boilerplate files:
$ carton exec ecukes --new
create features
create step-definition
create super-duper-steps.el
create support
create env.el
create super-duper.feature
The project will now look like this:
$ ls super-duper
super-duper/
|-- README
|-- features
| |-- step-definitions
| | `-- super-duper-steps.el
| |-- super-duper.feature
| `-- support
| `-- env.el
`-- super-duper.el
Files in the support directory will be loaded (env.el first, then the rest) each time Ecukes run. All files that ends with -steps.el in the step-definitions directory will be loaded as step definitions.
Ecukes can run in a few different modes.
Running with no win is the default way to run features. In this mode, Ecukes will run Emacs with the -nw option.
Example:
$ carton exec ecukes features
This will start up an Emacs window and run the features. To use this mode, specify --win as first option to ecukes.
Example:
$ carton exec ecukes --win features
This will run the features as a batch job. This mode has a few quirks and is only recommended as a way to run the features fast.
Example:
$ carton exec ecukes --script features
You can also run features directly in your current Emacs session. To to that, make sure Ecukes and its dependencies are available in your load-path. If you are using Carton for your Emacs setup, simply add ecukes as a dependency and you are set. Then require ecukes.
(require 'ecukes)
To run features, visit a file in your project and run the ecukes function.
M-x ecukes
Here are a few ways to run Ecukes:
Create boilerplate files for current project.
$ carton exec ecukes --new
Show usage information.
$ carton exec ecukes --help
Run all features in no win mode.
$ carton exec ecukes
Run all features in win mode.
$ carton exec ecukes --win features
Run all features in script mode.
$ carton exec ecukes --script
Run one feature in no win mode.
$ carton exec ecukes features/super-duper.feature
Run one feature in win mode with verbose output.
$ carton exec ecukes --win features/super-duper.feature --verbose
Run all scenarios tagged with @super in script mode with debug.
$ carton exec ecukes --script features/super-duper.feature --dbg --tags @super
Run all scenarios not tagged with @duper.
$ carton exec ecukes features/super-duper.feature --tags ~@duper
Run all features in no win mode. Use Emacs binary installed via Homebrew.
ECUKES_EMACS="/usr/local/Cellar/emacs/24.2/Emacs.app/Contents/MacOS/Emacs" carton exec ~/Code/ecukes/ecukes
Ecukes supports most Cucumbers features, such as: backgrounds, scenarios, scenario outlines and tags. Take a look at Cucumbers manual for more information about each.
There are three different kind of steps: Regular, Table and Py string.
A regular step always only consumes one line. The most basic step looks like this:
Given a known state
Corresponding step definition:
(Given "a known state"
(lambda ()
;; Do something
))
The second argument could also have been the symbol of a function name:
(Given "a known state" 'do-something)
Steps can take arbitrary many arguments. The arguments sent to the step definition function are all regular expression match groupings from the matching of the step name and the match string.
Single argument:
Given I am in buffer "buffer-name"
Corresponding step definition:
(Given "I am in buffer \"\\(.+\\)\""
(lambda (buffer-nane)
;; Do something with buffer
))
Multiple arguments:
Given I am in buffer "buffer-name" with text "Foo"
Corresponding step definition:
(Given "I am in buffer \"\\(.+\\)\" with text \"\\(.+\\)\""
(lambda (buffer text)
;; Do something with buffer and text
))
A table step looks like this:
Given these meals:
| meal | price |
| Hamburger | $4.50 |
| Pizza | $5.30 |
Corresponding step definition:
(Given "these meals:"
(lambda (meals)
;; Do something with text
))
The argument meals is a simple list where the car of the list is the header and the cdr or the list is the rows.
The header would in the above case be:
("meal" "price")
And the rows would be:
(("Hamburger" "$4.50")
("Pizza" "$5.30"))
To pick out the header and rows attribute from a table you do:
(let* ((table ...)
(header (car table))
(rows (cdr table)))
;; Do something with header and rows
)
It is also possible to send arguments to a table step. The table list will be the last argument.
Given these meals at "fast food":
| meal | price |
| Hamburger | $4.50 |
| Pizza | $5.30 |
Corresponding step definition:
(Given "these meals at \"\\(.+\\)\":"
(lambda (restaurant meals)
;; Do something with restaurant and meals
))
A Py-String (or Python string) step looks like this:
Given the following text:
"""
some text
"""
Corresponding step definition:
(Given "the following text:"
(lambda (text)
;; Do something with text
))
It is also possible to send arguments to a Py-string step. The Py-string will be the last argument.
Given the following text in buffer "buffer-name":
"""
some text
"""
Corresponding step definition:
(Given "the following text in buffer \"\\(.+\\)\":"
(lambda (buffer text)
;; Do something with buffer and text
))
In order to keep your steps DRY, you can call steps from other steps like this:
(Given "I go to a"
(lambda ()
;; Go to a
))
(Given "I go to b"
(lambda ()
;; Go to b
))
(Given "I go from a to b"
(lambda ()
(Given "I go to a")
(Given "I go to b")))
You can also pass arguments when calling other steps, like this:
(Given "I am in buffer \"%s\"" buffer-name-variable)
Or like this:
(Given "I am in buffer \"buffer-name\"")
If you pass one extra parameter to a step definition, that is used as a callback. Once called, the step is considered done.
(When "^run command \\(.+\\)$"
(lambda (command callback)
(run-command command callback)
))
Ecukes provides four different hooks
Runs once before anything runs.
(Setup
;; Run code
)
Runs once before each step runs.
(Before
;; Run code
)
Runs once after each step runs.
(After
;; Run code
)
Runs once when everything is done.
(Teardown
;; Run code
)
Hooks are useful if you test your program and change the state in some feature. Since all scenarios execute in the same environment, the state change will affect all scenarios after. You can solve that by resetting the state in a before or after hook.
Hooks should be placed in any file in the project features/support directory.
All contributions are much welcome and appreciated!
Before submitting a patch, make sure to write a test for it (if possible). Ecukes is tested with a testing framework called Ert (Emacs Lisp Regression Testing).
To run the tests, you have to install Carton if you haven't already. Once installed, run the carton command to install all dependencies.
To run all test, simply run the make command.
You can always report issues on Github if you're not up to fixing it yourself.