Record In Browser Network Communication inside End to End tests (Selenium* + NodeJs)

Posted on | Edited on | Comments:

One little and fun challenge.
We need to end to end a script with intensive network communication.
We have a standard and framework agnostic setup. It is doesn’t matter which of the framework we use on NodeJs. Let’s focus on communication. You could be surprised that the majority of protocols in the selenium test are JSON/HTTP.
So be careful and aware when you intensively use a framework features.
BTW selenium server and web drives could be hosted even on different servers.

We do not care about a network round trip main goal is opposite suppress and minimize any network calls.

We want to be able:

  • Have a history log of all HTTP calls.
  • Use this info as a part of the selenium-based framework. We use Nightwatch and web driver IO
  • It should be an IE8 friendly and mobile device friendly.
  • We should have the ability to configure and control ajax responses.

So we need a history calls DB lets call it this way.

Before we go to the dark wood.

If you need only AJAX mocking pls consider a PollyJS
One more of the source of inspiration for folks that use Nightwatch is nightwatch-xhr.

If you need more - Join me on a journey …

The mock servers

Just use some backend.

home brewed nodejs.

The most naive way is to create an embedded express server and give control on an instance in your test and expose recorded history calls as a method. Optionally this mock server could be used as a static assets host.

It is so trivial that I even do not want to write code.

The building one more bicycle.

source
Build only when you see a benefit.
In some point, you realize that It is no place like localhost so you decide to host your self as a service.
So pls stop for a moment. Yep, first, you will split API for getting a history, and the big fat part that will mock and reset mocks. This homebrew monster will grow.

Mountebank

Awesome person Brandon Byars did a great job and created mountebank - over the wire test doubles
Mountenbank contains a lot more than you need ready to use patterns to mock on fly extremely complex communication scenarios.

mbbook
If you want to know more . read the code or the book

Pros

  • it is super simple
  • You dont care how request was send.

Cons

  • The global state orchestration. All your history hosted on the server and disconnected with a test context.
  • It is required to point all to you mock service. Some times it is necessary to have a call to well-known services like Google, etc.
  • The network and network could be slow.

Lets spy

To address an issue with a location that could not refer to our mock. Lets maybe a spy?
So anyway we record calls. So it looks like Proxy could be the answer.

Proxies

We put a proxy that spy or manipulate with a request that happens on env

The external tools & broswer extensions.

Pros

  • Could record a manipulate any of requests.
  • You don’t care how the request was sent.

Cons

  • Not all of them have a programmable interface that could be easily integrated into an e2e test.
  • Perfect if you control a browser or environment but what to do if we want to test mobile device.

BrowserMob Proxy

It is one step forward. We could integrate BrowserMobb via Nodejs bindings to our e2e code and get traffic information.
So ready to use example nightmob for Nightwatch.

Pros

  • NodeJs interface
  • Easy setup

Cons

  • Still, we have a problem of environment control. if you don’t have this issue welcome

Web driver perf logs

Selenium server uses a JSON API for communication. Selenium itself use a web driver to talk to browsers. So as you can see a pattern, WebDrivers itself is like a smart proxy.
Web drivers have a open specification. It is excellent news, but … Implementations of drivers for different browsers are different. As a reference, you could take a look at a Web Log. The project is not actively supported.
It uses a performance logs feature for chrome web driver. Deep internals.

Pros

  • it is 100% based on selenium web driver. Just add a config

Cons

  • It is two layers down from your framework so it could be a trick to integrate this to it.
  • It depends on drivers not always you have a control on it.
  • Selenium grids do not always give access to it.

Lets go to the dark side (client side)

“Make the best use of what is in your power, and take the rest as it happens. Some things are up to us [eph’ hêmin], and some things are not up to us. Our opinions are up to us, and our impulses, desires, aversions—in short, whatever is our own doing. Our bodies are not up to us, nor are our possessions, our reputations, or our public offices, or, that is, whatever is not our own doing.

Epictetus

Or as later in pray

God, give me the grace to accept with serenity
the things that cannot be changed,
Courage to change the things
which should be changed,
and the Wisdom to distinguish
the one from the other.

Reinhold_Niebuhr Serenity prayer

What do I mean?
Instead of externalizing a task lets mock all parts that could create traffic.

It is work even if you don’t have full control on page content. You could inject a script snippet to a page.
The idea is just stupid.
We take all API that creates network calls.
In my case, it is what I mention on top.

  • Images
  • beacons
  • AJAX
    • XHR
    • XDomainRequest

It is much much more. We will not cover.

  • Html tags that are on page
  • Injected him via inner Html
  • DOM parser
  • DOM create element

It is loooong looong road. I believe that the method that I ‘ll demonstrate could be extended and adapted to list below.

Lets mock them all !!!

Simple idea lets mock all js classes and functions that could create calls used in our code. Simple.
Let’s start from most wired one.

How to handle bulk html insert?

To be honest I decide not to cover this case as far as it is not used in my code.
It could give you a notice about adding a complex htm to dom with a object trees.
We could solve a cases when some body add big chanks of html to page but it is usless for unattached DOM.

One of the helpers could be MutationObserver

It was promised to be IE 9 compatible but works only on IE 11.
But it is cool stuff to play with

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

var observeDOM = (function(){
  var MutationObserver = window.MutationObserver || window.WebKitMutationObserver;

  return function( obj, callback ){
    if( !obj || !obj.nodeType === 1 ) return; // validation

    if( MutationObserver ){
      // define a new observer
      var obs = new MutationObserver(function(mutations, observer){
          callback(mutations);
      })
      // have the observer observe foo for changes in children
      obs.observe( obj, { childList:true, subtree:true });
    }
    
    else if( window.addEventListener ){
      obj.addEventListener('DOMNodeInserted', callback, false);
      obj.addEventListener('DOMNodeRemoved', callback, false);
    }
  }
})();

//------------< DEMO BELOW >----------------
// add item
var itemHTML = "<li><button>list item (click to delete)</button></li>",
    listElm = document.querySelector('ol');

document.querySelector('body > button').onclick = function(e){
  listElm.insertAdjacentHTML("beforeend", itemHTML);
}

// delete item
listElm.onclick = function(e){
  if( e.target.nodeName == "BUTTON" )
    e.target.parentNode.parentNode.removeChild(e.target.parentNode);
}
// Observe a specific DOM element:
observeDOM( listElm, function(m){ 
   var addedNodes = [], removedNodes = [];
   m.forEach(record => record.addedNodes.length & addedNodes.push(...record.addedNodes))
   m.forEach(record => record.removedNodes.length & removedNodes.push(...record.removedNodes))
  console.clear();
  console.log('Added:', addedNodes, 'Removed:', removedNodes);
});

As far as all DOM listeners are not cover our case.
Next naive idea is simple - Let’s mock an Image Object itself.
It is trivial and relatively simple except one small fact.

Image src mock

So If you Image is an attached to DOM pls lets not to test a browser core functions. We trust that the browser will try to fetch it. Just check an image element on the page and analyze an src attribute if needed.
I am more fun case where selenium is blind because the image is a newer part of the DOM.

1
2
var pixel = new Image(1,1);
pixel.src = 'cute_cat?id=42#fd';

It is a hack way to send a get request even if a lot of script functions and XHR is not available.
The challenge - a lot of selenium frameworks will not see it.

Note: pls avoid using iframes or scripts for the same trick. I could write a book about this topic.
So the challenge.
As far as we not attaching it to a DOM majority of tricks will not work.

The Src is not a function but property

So the core challenge is that src is a property.

1
2
var pixel = new Image(1,1);
pixel.src = 'cute_cat?id=42#fd';

it will be much easier to have.

1
2
var pixel = new Image(1,1);
pixel.src('cute_cat?id=42#fd');

but we don’t want to change this for sure.

Ecma Script 6+ Proxies

JS Proxies allow to override and intercept property behavior.
Cool video about proxies.

Compact article here

Base on it you could try an implementation described here
I’ll take some code for clarity.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
const NativeImage = Image;
   
   class FakeImage {
     constructor(w, h) {
       const nativeImage = new NativeImage(w, h);
       const handler = {
         set: function(obj, prop, value) {
           if (prop === 'src') {
               window.mocks.callHistory.push{url: src, t : new Date.getTime()};
           }
           return nativeImage[prop] = value;
         },
         get: function(target, prop) {
           let result = target[prop];
           if (typeof result === 'function') {
             result = result.bind(target);
           }
           return result;
         }
       };
       const prox = new Proxy(nativeImage, handler);
       try {
         prox[Symbol.toStringTag] = 'HTMLImageElement';
       } catch(e){}
       return prox;
     }  
   }

One sed news It will not work in IE and a bunch of browsers.
I have tried a combinations ob babble + google Proxy polifil.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
"use strict";

function _instanceof(left, right) { if (right != null && typeof Symbol !== "undefined" && right[Symbol.hasInstance]) { return right[Symbol.hasInstance](left); } else { return left instanceof right; } }

function _classCallCheck(instance, Constructor) { if (!_instanceof(instance, Constructor)) { throw new TypeError("Cannot call a class as a function"); } }

var NativeImage = Image;

var FakeImage = function FakeImage(w, h) {
  _classCallCheck(this, FakeImage);

  var nativeImage = new NativeImage(w, h);
  var handler = {
    set: function set(obj, prop, value) {
      if (prop === 'src') {
            if (!window.mocks.callHistory) {
                window.mocks.callHistory = [];
            }
            window.mocks.callHistory.push{url: src, t : new Date.getTime()};
      }

      return nativeImage[prop] = value;
    },
    get: function get(target, prop) {
      var result = target[prop];

      if (typeof result === 'function') {
        result = result.bind(target);
      }

      return result;
    }
  };
  // handled by google library
  var prox = new Proxy(nativeImage, handler);

  try {
    prox[toStringTag] = 'HTMLImageElement';
  } catch (e) {}

  return prox;
};

After regression, I didn’t get the results that I expect. I was still failing in IE.

Pros
  • You have for free all functions and properties working as it exept src that now do a logging.
    Cons
  • Not riching a browser coverage. Forget about IE.

Setters

Why not use a setter API?According to the documentation, it is supported by a tonne of browsers.

1
2
3
4
5
var Image = function(h, w) {
    set src (url) {
        mocks.callHistory.push({url: url, t : new Date.getTime()});
    }
}

Dooone !!!!!

Pros
  • It is working !!!
Cons
  • You lost all functionality of the image. So If you need to test some extra stuff, you should tune the mock all the time
  • The image is gone from a page.
  • It is working !!! but not in IE 10 (

Internet explorer 11, 10, 9 , 8 friendly

Well, what about IE?
In docs, you see IE 9+, but I get an error. After research, I came with a solution that you could add properties to DOM elements.
So let’s try.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
var Image  = function (h, w) {
    var fImage = document.createElement('div');
    fImage.setAttribute('id', 'mock-image' + (new Date()).getTime());
    var imgUrl ;
    Object.defineProperty(fImage, 'src', {
    get: function() { return imgUrl},
    set: function(url) {
        if (!window.mocks.callHistory) {
        window.mocks.callHistory = [];
        }
        imgUrl = url;
        window.mocks.callHistory.push{url: url, t : new Date.getTime()};
    }
    });
    return fImage;
};

One more cool side effect . as far as every call mow is dome element you could attach it to a page and use in your test. This part of code on your controle.
So now all you favorite selenium commands inside you cool framework are working.
I prefer to collect all data as in memory array and fetch it with one single selenium call that executes a script.
It is more efficient.
One more cool stuff with a DOM element you could make a visual log on page and use these pages for local testing etc.
All other parts could use the same pattern.

Pros
  • It is working !!!
  • It is visible by selenium now
  • All in your hands
Cons
  • You lost all functionality of the image. So If you need to test some extra stuff, you should tune the mock all the time
  • The image is gone from a page.
  • It is working !!! but not in IE 10 (

Ajax

It is a tonne of the libraries that do a VCR recording and replays.
I have already mention pollyJS, and it is a lot of frameworks that do it for you.
One important part - We should be able to configure response so that we could model different situations in tests.

XMLHttpRequest

Fist of all de facto standard is XMLHttpRequest
As you could check browser support is one of the best even IE7.
As far as we decide to mock all on a browser side.
You could find a cool and practically complete xhr mock here.
It allows you not only record but simulates a response. One big downside - xhr-mock is quite fat and super big.
We don’t need all of the features.

I have made a minimal mock based on a feature that used.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
window.XMLHttpRequest = function () {
    var options = window.mocks.options && window.mocks.options :
            {status: 200 , body:"{\"m\": \"hallo \"}"};
    var self = this;

    var fakeAjax = document.createElement('div');
    fakeAjax.setAttribute('id', 'mock-xhr' + (new Date()).getTime());
    });

    fakeAjax.open = function _fxhropen() {
    console.log('xrh open', (new Date()).getTime());
    self.args = [].slice.call(arguments);
    self.url =   self.args.length > 2 ? self.args[1] : "";
    if (!window.mocks.callHistory) {
        window.mocks.callHistory = [];
    }
    window.mocks.callHistory.push({type:'ajax', url: self.url,  args: self.args, time : (new Date()).getTime()});
    };
    fakeAjax.withCredentials = true;
    fakeAjax.send = function _fxhrsend () {
    console.log('xrh send', (new Date()).getTime());
    debugger;
    self.args.concat([].slice.call(arguments));

    if(typeof self.onChangeHandler === 'function') {
        self.onChangeHandler.call({
        "readyState": 4,
        "status": options.status,
        "responseURL": self.url,
        "responseText": options.status === 200 ? options.body : ""
        });
    }
    };

    Object.defineProperty(fakeAjax, 'onreadystatechange', {
    get: function() { return self.handler},
    set: function(handler) {

        self.onChangeHandler = handler;
    }
    });
    return fakeAjax;
};

The idea is the same. We mock

  • open method
  • send method

  • onchange handler property.

    For simplicity, we use a window.mocks.options as a source of mock data. In the future we could create our logic to match URLs to payloads and do a smarter config based on tests needs.

XDomain Request

Is archaic and limited see docs. You have no info about request status and no ability to set anything except payload and URL.
It was IE 8-9 feature only. If you need to use it - you could mock it in the same way.

Fetch API

More modern and sophisticated XMLHttpRequest successor.
I haven’t touch (no, ie old mobile browser support) it but the same pattern could be used as with beacons.

Beacons API

It is a cool feature of modern browsers. It was designed for guaranteed delivery of request (mainly analytics data) before page closes. It sends SMALL messages in an async manner. So you get back a boolean if you call scheduled in a qued successful.

A mocking code is straightforward as far as it just a function.

1
2
3
4
5
6
7
window.navigator.sendBeacon = function mockBeaconSend(url , data) {
    if (!window.mocks.callHistory) {
    window.mocks.callHistory = [];
    }
    window.mocks.callHistory.push({type:'beacon', url: url,  args: [].slice.call(arguments), time : (new Date()).getTime()});
    return true;
};

CAUTION Pls return true if you want to simulate that beacons are working or false if you’re going to test cases when the call fails to schedule.

Summary

A solution to my challenge for now:

  • Mock all calls as an in-memory object on the client side with a script injection.

    Pros

  • Full control

  • No network latencies
  • Control over result
  • small size of mocking code
  • test framework agnostic

  • selenium or any other drivers and black magic agnostic.

    Cons

  • You need to analyze code and do mocking. It could be subject to change.

  • Lose a lot of functionality.
    • DOM element create (could play with mutation event or simulate your own)
      • innerHtml that could attach images
      • DOM Parces
      • appends
    • Not mocked methods like setHeader on XMLHttpRequest.
  • We change a page behavior and size.

  • Js injection. Injection should happen befor target script of test functionality.

    My general recommendation does a minimal that required for your tests and more essential test targets.
    Some times people test too much our they examine a standard browser behavior that is.
    I came with this solution mainly because of free DOM images that send a get request.
    If you rely on third party libs, it could be tricky, but you still could check the code.

    It is always hard to unreal to have a universal solution.