Thursday, April 9, 2009

Readable Test Code and Readable Test Output - Get Both Using Matchers

Who doesn't like readable code? The benefit is obvious: bugs are harder to hide in it, and you can change it more easily when the requirement changes.

Test code is code too. Thus it also benefits from being readable. In fact, I'd argue that readability is even more important for test code, as you have to be able to trust it (remember that most tests don't have their own tests).

If you have used a mocking framework before, you are probably familiar with the concept of a matcher: you give it a value, and it tells you whether the value has a certain property -- if yes, we say that it matches the value.

For example, in Google Mock, ContainsRegex("Ahcho+!") is a matcher that matches any string that has the regular expression "Ahcho+!" in it. For instance, it matches "Ahchoo!" and "Ahchoooo! Sorry.", but not "Aha!".

What's this to do with test readability, anyway? It turns out that matchers lend themselves easily to a style of writing assertions that resembles natural languages (and thus is easy for human to understand). For example, Google Mock supports the following assertion syntax, borrowed from the Hamcrest project:
EXPECT_THAT(value, matcher);
The assertion succeeds if the given value matches the given matcher. Therefore,
EXPECT_THAT(my_string, ContainsRegex("ab*"));
verifies that my_string contains regular expression "ab*".

Now, pretend the punctuations aren't in the above C++ statement and read it:
"Expect that my string contains regex ab*".
Does that sound like English?

That's not all. When an EXPECT_THAT assertion fails, it prints an informative message that includes the expression being validated, its value, and the property we expect it to have -- thanks to a matcher's ability to describe itself in human-friendly language. Therefore, not only is the test code readable, the test output it generates is readable too. For example,
EXPECT_THAT(GetUserList(), Contains(admin_id));
might produce:
Value of: GetUserList()
Expected: contains "root"
Actual: { "peter", "paul", "mary" }

This message contains relevant information to help you diagnose the problem, often without having to use a debugger. To get the same effect without using a matcher, you'd have to write something like:
std::vector<std::string> users = GetUserList();
EXPECT_TRUE(VectorContains(users, admin_id))
<< " GetUserList() returns " << users
<< " and admin_id is " << admin_id;
where << is the syntax Google Test uses for appending custom messages to an assertion failure. Clearly this is more tedious to write and harder to read than the one-liner using EXPECT_THAT.

Google Mock provides dozens of matchers for you to use out-of-box. Here's a list of them and their usage.

So matchers are nice. But what if you cannot find a matcher for the property you want to test? You can either combine existing matchers, or define your own from scratch. Both are quite easy to do in Google Mock. I'll show you how in another post. Stay tuned.

8 comments:

  1. Nice! I strongly agree that "readability is even more important for test code, as you have to be able to trust it "
    I have heard about Google Mock only a few days ago, but it attracts me.

    ReplyDelete
  2. I wonder why EXPECT_TRUE is in gmock but not gtest?

    ReplyDelete
  3. Abu, EXPECT_TRUE *is* in gtest.

    Perhaps you mean why EXPECT_THAT is in gmock instead of gtest. The reason is historical. The matcher library was initially developed as part of gmock, as a reasonable mocking framework cannot do without matchers. EXPECT_THAT depends on matchers and thus ended up in gmock.

    ReplyDelete
  4. Hi Wan,
    Sorry, I meant to say about EXPECT_THAT as you guess.

    I think it will nice to gtest have EXPECT_THAT because it seems allow
    me to write tests more BDD like style.

    Maybe EXPECT_PRED in gtest provide same functionality, but
    I prefer to actual to come first and then expected result like EXPECT_THAT.

    Thanks,

    ReplyDelete
  5. Abu, you are welcome.

    I agree it would be nice to move EXPECT_THAT and matchers to gtest. It's just a lot of refactoring work, and we don't have time for that soon.

    I wonder what's preventing you from using gmock. It gives you EXPECT_THAT and more. Even if you only want to use EXPECT_THAT and nothing else, there's nothing wrong with using gmock. And it's easy to set up. Does gmock not compile in your environment?

    ReplyDelete
  6. Hi,

    gmock requires VC8.1+boost, and that is too higher requirement just for me as I maintain some old windows projects which written in VC6/7.

    (Maybe I should migrate to new compilers , but migration of test-less project is risky..)

    ReplyDelete
  7. Abu, thanks for clarifying. I agree it's desirable to move EXPECT_THAT to gtest, but it will take time. Meanwhile, perhaps you can extract EXPECT_THAT and the matcher library from gmock and use it in your project. This part doesn't depend on boost and may be compilable with minor tweaks in your environment. Just a thought.

    BTW, gmock requires VC 8.0 SP1, not VC 8.1. It only needs boost where tr1/tuple is not available. Recent versions of gcc have tr1/tuple, so gmock doesn't require boost if used with gcc 4.0+.

    ReplyDelete
  8. Thanks Wan for give me useful information.

    I'll try to extract and tweak.

    ReplyDelete

Note: Only a member of this blog may post a comment.