Make disposable web servers for testing

If you project depends on a interaction with a web server, especially a remote one, you have some challenges with testing that portion. Even if you can get it working for you, when you distribute your code, someone else might not be able to reach your server for testing. Instead of relying on an external server, you can use a local server that you write especially for your test suite.

This problem has a couple tricky parts. To run a test server from your own test suite, your server needs to bind to a port that’s not already in use. When it has a port, it needs to communicate that to your test script.

Another problem, which you won’t consider for this Item, involves the configurability of your program so you can change the hostname and port during the test. This Item assumes you’ve taken are of that bit.

The Test::Fake::HTTPD module can create a web server directly from your test script. This example creates a web server that returns the same JSON response for every request:

use Test::More;
use Test::Fake::HTTPD;

use Mojo::UserAgent;

my $httpd = run_http_server {
	my $request = shift;

	return [ 
		200, 
		[ 'Content-Type' => 'application/json' ], 
		[ '{ "cat": "Buster" }' ] 
		];
	};

ok( defined $httpd, 'Got a web server' );

diag( sprintf "You can connect to your server at %s.\n", $httpd->host_port );

my $response = Mojo::UserAgent->new->get(
	$httpd->endpoint
	)->res;

diag( $response->to_string );
is( $response->json->{cat}, 'Buster', 'Cat is Buster' );

done_testing();

The test output shows the a message telling you the host and port, as well as the response:

ok 1 - Got a web server
# You can connect to your server at 127.0.0.1:50602.
# HTTP/1.1 200 OK
# Content-Type: application/json
# Date: Wed, 07 Dec 2011 08:44:40 GMT
# Content-Length: 19
# Server: libwww-perl-daemon/6.00
# 
# { "cat": "Buster" }
ok 2 - Cat is Buster
1..2

When the test script ends (or the web server variable goes out of scope, so there’s nothing for you to cleanup. It’s a disposable web server.

In your web server, you can do anything that you like. It doesn’t have to implement everything, or even close to everything, that the production server does. It just has to return responses that you can use in your tests. That means that you get to control not only the success, but also the failures.

That Server line gives you a hint about what created the $request object—it’s LWP behind the scenes, so it’s HTTP::Request. You can get the requested path with the uri method and decide what to do:

use strict;
use warnings;

use Test::More;
use Test::Fake::HTTPD;

use Mojo::UserAgent;
use URI;

my $httpd = run_http_server {
	my $request = shift;

	my $uri = $request->uri;

	return do {
		if( $uri->path eq '/' ) {
			[ 
				200, 
				[ 'Content-Type' => 'text/plain' ], 
				[ "Ask about our cats!" ], 
			]
			}
		elsif( $uri->path eq '/cats' ) {
			[ 
				200, 
				[ 'Content-Type' => 'application/json' ], 
				[ '{ "cat": "Buster" }' ], 
			]
			}
		elsif( $uri->path eq '/dogs' ) {
			[ 
				408, 
				[ 'Content-Type' => 'text/plain' ], 
				[ "We don't walk dogs" ], 
			]
			}
		else {
			[ 
				404, 
				[ 'Content-Type' => 'text/plain' ], 
				[ "Not Found" ], 
			]
			}
		}		
	};

ok( defined $httpd, 'Got a web server' );
diag( sprintf "You can connect to your server at %s.\n", $httpd->host_port );

my $uri = URI->new( $httpd->endpoint );
isa_ok( $uri, 'URI' );


subtest '/' => sub {
	plan tests => 3;
	
	my $this = $uri->clone;
	isa_ok( $this, 'URI' );
	$this->path( '/' );
	
	my $response = Mojo::UserAgent->new->get( $this )->res;
	
	is( $response->headers->content_type, 
		'text/plain', 'Top level is plain text' );
	like( $response->body , qr/cats/, 'Top level has cats' );
	};
	
subtest '/cats' => sub {
	plan tests => 2;
	
	my $this = $uri->clone;
	isa_ok( $this, 'URI' );
	$this->path( '/cats' );
	
	my $response = Mojo::UserAgent->new->get( $this )->res;
	
	is( $response->headers->content_type, 
		'application/json', '/cats returns JSON' );
	};

subtest '/not_there' => sub {
	plan tests => 2;
	
	my $this = $uri->clone;
	isa_ok( $this, 'URI' );
	$this->path( '/not_there' );
	
	my $response = Mojo::UserAgent->new->get( $this )->res;

	is( $response->code, 
		'404', '/not_there returns 404' );
	};

done_testing();

With subtests organized around accesses to paths, the TAP isn’t so hard to read (although you probably won’t have to look at it yourself):

ok 1 - Got a web server
# You can connect to your server at 127.0.0.1:50498.
ok 2 - The object isa URI
    1..3
    ok 1 - The object isa URI
    ok 2 - Top level is plain text
    ok 3 - Top level has cats
ok 3 - /
    1..2
    ok 1 - The object isa URI
    ok 2 - /cats returns JSON
ok 4 - /cats
    1..2
    ok 1 - The object isa URI
    ok 2 - /not_there returns 404
ok 5 - /not_there
1..5

If you need to use the same test webserver in more than one test script, you move it into its own file.

use strict;
use warnings;

use Test::More;

use Mojo::UserAgent;
use URI;

require 'server.pl';
my $httpd = get_http_server();

ok( defined $httpd, 'Got a web server' );
diag( sprintf "You can connect to your server at %s.\n", $httpd->host_port );

my $uri = URI->new( $httpd->endpoint );
isa_ok( $uri, 'URI' );

...

The server.pl file wraps the call to run_http_server. But, if you are going to do that, you might as well skip the convenience method and set up your own object. You can change the timeout value, for instance:

use Test::Fake::HTTPD;

sub get_http_server {
	my $httpd = Test::Fake::HTTPD->new(
		timeout => 30,
		);

	$httpd->run( sub {
		my $request = shift;
		...;		
		} );
		
	$httpd;
	}

Different test scripts each get their own test web server. If you’re running several tests scripts in parallel, you’ll start several servers at the same time, which not be that kind to your system or to the other people using it. If you don’t like that, you could set up a single server at the start of your test run, share it with all tests, and shut down everything at the end, although you won’t see that in this Item.

Things to remember

  • Test web interactions locally
  • Use Test::Fake::HTTPD to create cheap, disposable web servers