Save time, Test your Code Part 2

In the last blog post I explained how our baseline test works and it’s pros and cons, if you missed it you can find it here. In this article I hope to explain another type of test that we use that could prove helpful or inspirational to others so let’s get started.

Functional Testing Against Test Data

Like the previous blog post on our baseline test our functional tests are also a form of black-box testing; we are only really interested in the end result and whether things work as they should or do not.

In order to explain exactly what we do it’s probably worth first explaining the problem these tests try to address:

The PDF file format is a very robust format that enables content creators to integrate many different actions into a PDF file, some of these make use of JavaScript as scripting language while others are actions defined by the PDF specification itself. These advanced features should be supported by good PDF Viewers and mimicked when converting from PDF to another file format (e.g. our PDF to HTML5 converter). Doing this is no simple task though as a scripting engine needs to be used when creating a viewer (e.g. Rhino, Nashorn or V8) and when converting to another format you need to ensure that the actions are converted correctly using either JavaScript (as is the case with HTML5) or a scripting language supported by the file type.

With the agile approach we have to development it is also important that we make changes incrementally and have an easy way of testing that we have not broken or negated previous advances.

What our functional tests do is similar to our baseline tests except they test the actual functions of our output in the case of the HTML5 Converter or in the case of our viewer we test against how form fields and the document change after simulating an action.

The viewer test make use of a Java library called Jemmy that lets us simulate mouse clicks, typing and any other user input. The HTML test makes use of another Java library called HTMLUnit that does almost the exact same thing except it uses a headless browser.

The basic steps of the viewer test are:

  • First start off with a set of files with JavaScript embedded in them
  • We then open each file with our viewer using Jemmy (this is automated so we don’t need to manually do this for each file)
  • We next check to see if we have a script file for this particular page and document (the script file is something we use to determine the actions to perform on the page)
    • If we don’t have a script file we generate one based upon all the form elements on the page and any simple data we gleam from their current contents (e.g. if a field contains a number we make sure the script file tries to add a number to that field to test its formatting)
    • We then fail the test here to ensure the person running it is aware of the fact there wasn’t a test file and that they will potentially have to edit the generated one.
  • With the script file we run through it performing the actions it requires and checking against the wanted results. If at any point there is a differences the test fails and we get a description of what should of happened versus what actually happened.
  • We then repeat for each page of each file we are testing against.
Diagram showing how the functional test works

Diagram showing how the functional test works

As you can see we make use of a file which I called a script to test the various features of pages against. This file is just a plain text file that normally looks something like this:

TEST
# This is a comment line, below is a line that prints out to System.out
ECHO Autogenerated test file:  sampleForm2_javascript_page1.txt    Page: 1
# Some commands may need removing or modifying
SET_TEXT fullName=testString
CHECK_TEXT fullName=testString
SET_TEXT email=example@example-website.com
CHECK_TEXT email=example@example-website.com
SET_TEXT phone=testString
CHECK_TEXT phone=testString
SET_TEXT address=testString
CHECK_TEXT address=testString
SET_TEXT hobbies=testString
CHECK_TEXT hobbies=testString

In our code we have a simple interpreter that reads each line and executes an action based upon the command given. In order to make these script files a lot easier to create the test tries to guess at correct data to use, in this case it gets the email format correct but could not figure out that the phone field should probably test using a number instead of a string.

This excerpt also doesn’t test against bad data input e.g. typing in a normal string instead of an email address into the email field. The ability to do this is a major advantage of using these kinds of tests as when testing you should always ensure that you test for both successes and failures.

The HTML5 test is incredibly similar so does not really need as in-depth an explanation as the viewer test; it simply does the same but with HTML files and also runs a lot faster due to it’s headless nature (Jemmy will still open up your program as a window and can be timing dependent).

The advantages of using script file over hard coding the values into our tests is flexibility. Using a framework built to interpret these files let’s us add files to the tests at our leisure and tweak the script files to continually improve our test data, it also let’s us add new features to the test framework letting us improve it over time. For a small company these tests can save time spent hunting down bugs after a release by alerting us to their presence earlier on.

Next Time

That concludes this part of this testing series. So far we have only looked at 2 forms of black-box testing we use here at IDR Solutions; however we also make use of unit tests which is a form of white-box testing (or structural testing) which may be covered in the future.

This post is part of our “Testing Articles Index” in these articles we provide a guide to Testing.

If you’re a first-time reader, or simply want to be notified when we post new articles and updates, you can keep up to date by social media (TwitterFacebook and Google+) or the Blog RSS.

The following two tabs change content below.
Lyndon is a Developer at IDR Solutions. He currently focuses mostly on the JavaScript in the Viewer and PDF to HTML5 Converter and also the Android PDF Viewer. He gave a short talk at the GlassFish UnConference before JavaOne 2012. Outside of IDR Solutions he has a keen interest in AI and Games Programming and runs a blog that he periodically updates.

Related Posts:

lyndon

About Lyndon Armitage

Lyndon is a Developer at IDR Solutions. He currently focuses mostly on the JavaScript in the Viewer and PDF to HTML5 Converter and also the Android PDF Viewer. He gave a short talk at the GlassFish UnConference before JavaOne 2012. Outside of IDR Solutions he has a keen interest in AI and Games Programming and runs a blog that he periodically updates.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>