Writing Tests for Plugins

Artur Hefczyc <artur.hefczyc@tigase.net> v2.0, June 2014: Reformatted for AsciiDoc. :toc: :numbered: :website: http://tigase.net/ :Date: 2010-04-06 21:22

You can write tests in form of simple text file which is loaded during test suite runtime.

You simply specify what should be send to the server and what response should be expected from the server. No need to write Java code and recompile whole test suite for new tests. It means new test cases can be now written easily and quickly which hopefully means more detailed tests for the server.

How it works:

Let’s take XEP-0049 Private XML Storage. Looking into the spec we can see the first example:

Example 1. Client Stores Private Data

CLIENT:

<iq type="set" id="1001">
  <query xmlns="jabber:iq:private">
    <exodus xmlns="exodus:prefs">
      <defaultnick>Hamlet</defaultnick>
    </exodus>
  </query>
</iq>

SERVER:

<iq type="result" id="1001"/>

This is enough for the first simple test. I have to create text file JabberIqPrivate.test looking like this:

send: {

<iq type="set" id="1001">
  <query xmlns="jabber:iq:private">
    <exodus xmlns="exodus:prefs">
      <defaultnick>Hamlet</defaultnick>
    </exodus>
  </query>
</iq>
}

expect: {
<iq type="result" id="1001"/>
}

And now I can execute the test:

testsuite $ ./scripts/all-tests-runner.sh --single JabberIqPrivate.test

Tigase server home directory: ../server
Version: 2.8.5-b422
Database:         xmldb
Server IP:        127.0.0.1
Extra parameters: JabberIqPrivate.test
Starting Tigase:
Tigase running pid=6751

Running: 2.8.5-b422-xmldb test, IP 127.0.0.1...
Script name: scripts/single-xmpp-test.xmpt
Common test:  Common test  ...        failure!
FAILURE,  (Received result doesnt match expected result.,
Expected one of: [<iq id="1001" type="result"/>],
received:
[<iq id="1001" type="error">
  <query xmlns="jabber:iq:private">
    <exodus xmlns="exodus:prefs">
      <defaultnick>Hamlet</defaultnick>
    </exodus>
  </query>
  <error type="cancel">
    <feature-not-implemented xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/>
    <text xml:lang="en" xmlns="urn:ietf:params:xml:ns:xmpp-stanzas">
      Feature not supported yet.</text>
  </error>
</iq>]),

Total: 100ms
Test time: 00:00:02
Shutting down Tigase: 6751

If I just started working on this XEP and there is no code on the server side the result is perfectly expected although maybe this is not what we want. After a while of working on the server code I can execute the test once again:

testsuite $ ./scripts/all-tests-runner.sh --single JabberIqPrivate.test

Tigase server home directory: ../server

Version: 2.8.5-b422

Database:         xmldb

Server IP:        127.0.0.1

Extra parameters: JabberIqPrivate.test

Starting Tigase:

Tigase running pid=6984

Running: 2.8.5-b422-xmldb test, IP 127.0.0.1...

Script name: scripts/single-xmpp-test.xmpt

Common test:  Common test  ... success,  Total: 40ms

Test time: 00:00:01

Shutting down Tigase: 6984

This is it. The result we want. In simple and efficient way. We can repeat it as many times we want which is especially important in longer term. Every time we change the server code we can re-run tests to make sure we get correct responses from the server.

You can have a look in current, with more complete test cases, file for JabberIqPrivate.

Now my server tests are no longer outdated. Of course not all cases are so simple. Some XEPs require calculations to be done before stanza is sent or to compare received results. A good example for this case is user authentication like SASL and even NON-SASL. But still, there are many cases which can be covered by simple tests: roster management, privacy lists management, vCard, private data storage and so on…​.