PhantomJS for automatic responsive page rendering

PhantomJS is a headless WebKit scriptable with a JavaScript API

While you might've already used PhantomJS as a part of a test automation stack, it's also a powerful standalone tool you can use for fun and profit. My experiment today:

How can I automatically check my website on multiple screen sizes?

There are some great tools out there that you can try, but if you are running low-budget (like a personal blog) or feel an insatiable urge to overengineer a simple problem, feel free to follow along. We are going to end up with a report that will look like this.

1. Download PhanthomJS , I'll be running phantomjs-2.1.1-windows.
2. Though there is sadly no support for ES6, the API is really reasonable and well documented. Let's start with a sanity check and just render our webpage. Create a file render.js, which will load the webpage module and save it to png file:

var page = require('webpage').create();
page.open('https://mostlybugless.com/', function() {
    page.render('mb.png');
    phantom.exit();
});

Link to screen capture API docs

You can run it with:

phantomjs.exe render.js

Resulting image will contain full-lenght page with minimum-width render. Not quite what we wanted, but ~it's something~. 3. We can configure our script to change the viewport with:

page.viewportSize = {
    width: 1024,
    height: 768
};

Aaand that doesn't look like 1024x768. Viewport is the size of actual headless browser but rendering in default saves full page length. Let's fix that by letting PhantomJS know which part of the page we want to clip when taking screenshot.

page.clipRect = { top: 0, left: 0, width: 1024, height: 768 };

In some cases we may want to render full lenght of page, but as you can see on this very blog, we wouldn't get actual user experience for fixed position elements. Altogether we are now at:

var page = require('webpage').create();
page.open('https://mostlybugless.com/', function() {
    page.viewportSize = {
        width: 1024,
        height: 768
    };
    page.clipRect = {
        top: 0,
        left: 0,
        width: 1024,
        height: 768
    };
    page.render('mb.png');
    phantom.exit();
});

Tremendous success! We have covered the basic screen capture guide from PhantomJS documentation. We could easily for-loop it here and call it a day. Build package that you downloaded should even contains an example script by Salih Sagdilek - responsive-screenshot.js, which pretty much does that.

Let's go a little bit deeper

Rendering page with scrolling #

PhantomJS API allows us to:

• Evaluate any function in the context of opened webpage

var actualHeight = page.evaluate(function() {
    return document.body.scrollHeight;
});

• Scroll page to a desired position

page.scrollPosition = {
    top: 200,
    left: 0
};

By combining those two features with simple math we can automatically save screenshots of a virtual full-page scroll to bottom.

var page = require('webpage').create();
page.open('https://mostlybugless.com/', function() {
    var viewPort = {
        width: 1024,
        height: 768
    }
    page.viewportSize = viewPort
    page.clipRect = {
        top: 0,
        left: 0,
        width: viewPort.width,
        height: viewPort.height
    };
    var actualHeight = page.evaluate(function() {
        return document.body.scrollHeight
    });

    var step = 200;
    var stepCount = ((actualHeight - viewPort.height) / step) + 1;

    for (var i = 0; i < stepCount; i++) {
        page.scrollPosition = {
            top: Math.min(i * step, actualHeight - viewPort.height),
            left: 0
        };
        page.render('mb' + i + '.png');
    }

    phantom.exit();
});

Now that we have got a series of pictures, how about

Making a page interaction gif animation automatically #

If it was ever created, it will be rewritten in javascript at some point. It's true for git clients, web servers and video-processing frameworks - for better or worse. But in the latter we are interested. There is a small gif-creating library from yahoo we are going to use:

vgifshot

var page = require('webpage').create();
var gifshot = require('./gifshot.js');
var fs = require('fs');

var webserver = require('webserver');
var server = webserver.create();
var service = server.listen(8083, function(request, response) {
    response.statusCode = 200;
    response.setHeader('Access-Control-Allow-Origin', '*');
    response.setHeader('Content-Type', 'image/png');
    response.setEncoding('binary');
    var path = request.url.substring(1);
    console.log('request ' + path);
    var file = fs.open(path, 'rb');
    var data = file.read();
    file.close();
    response.write(data);
    response.close();
});

const GifReadyMessage = '[Injected] Gif Ready';

console.log('server started ' + service)

page.open('http://localhost:54359/', function(status) {
    console.log(status);

    page.onConsoleMessage = function(msg, lineNum, sourceId) {
        console.log('CONSOLE: ' + msg + ' (from line #' + lineNum + ' in \"' + sourceId + '\")');
        handlePageConsoleMessage(msg, page);
    };
    page.onError = logError;
    var width = 1024;
    var height = 768;
    var gifDownScale = 2;
    var gifFrameDuration = 5; // 10 = 1 second
    var scrollingPixelStep = 200;
    var fileNamePrefix = 'mb'

    var files = renderPage(page, width, height, scrollingPixelStep, fileNamePrefix);

    console.log('rendered')

    var injectsuccess = page.injectJs('./gifshot.js');
    console.log('injecting gifshot success' + injectsuccess)

    var imagesToUse = files.map(function(f) {
        return 'http://localhost:8083/' + f
    });
    console.log(imagesToUse)

    page.evaluate(function(width, height, frameDuration, downscale, images, readyMessage) {
        gifshot.createGIF({
            'images': images,
            'frameDuration': frameDuration,
            'gifWidth': width / downscale,
            'gifHeight': height / downscale,
        }, function(obj) {
            if (!obj.error) {
                _injected_tempGifStorage = obj.image;
                console.log(readyMessage)
            } else {
                console.log(JSON.stringify(obj));
            }
        });
    }, width, height, gifFrameDuration, gifDownScale, imagesToUse, GifReadyMessage);

});

function handlePageConsoleMessage(msg, page) {
    if (msg === GifReadyMessage) {
        var gif = page.evaluate(function() {
            return _injected_tempGifStorage;
        });
        var path = 'output.gif';
        console.log('Saving image')
        var gifContent = gif.replace(/^data:image\/gif;base64,/, '');;
        fs.write(path, atob(gifContent), 'wb');
        phantom.exit();
    }
}

function renderPage(page, width, height, scrollingStep, prefix) {
    page.viewportSize = {
        width: width,
        height: height
    }
    page.clipRect = {
        top: 0,
        left: 0,
        width: width,
        height: height
    };
    var actualHeight = page.evaluate(function() {
        return document.body.scrollHeight
    });
    var step = scrollingStep;

    var stepCount = ((actualHeight - height) / step) + 1;
    var files = [];
    for (var i = 0; i < stepCount; i++) {

        page.scrollPosition = {
            top: Math.min(i * step, actualHeight - height),
            left: 0
        };

        var fileName = prefix + '_' + width + 'x' + height + '_' + i + '.png';
        page.render(fileName);
        files.push(fileName);
        console.log('rendered ' + fileName);
    }
    return files;
}

function logError(msg, trace) {
    var msgStack = ['ERROR: ' + msg];

    if (trace && trace.length) {
        msgStack.push('TRACE:');
        trace.forEach(function(t) {
            msgStack.push(' -> ' + t.file + ': ' + t.line + (t.function ? ' (in function \"' + t.function+'\")' : ''));
        });
    }

    console.error(msgStack.join('\n'));
}

Oops, that escalated quickly. Combining quirks of a headless browser, gif-rendering javascript library that abuses DOM to achieve its goals, local files and, worst of all, javascript, was less then pleasurable. Taking it piece by piece, we:

1. Set up a webserver that serves static files from the script directory
2. Render page just like before
3. Inject gifshot into the context of a webpage
4. Request gifshot to create animation of locally saved renders and notify us through console.log-ing
5. Read the result and save locally in a hook to opened page console message

So, this ~maybe~ is a ~little bit~ hacky solution. I created this abomination for various reasons that came up, but mostly this is because gifshot uses DOM to do its magic and in this situation - access to locally saved pictures. Asynchronisity of gif creation and sandboxing between PhantomJS context and opened webpage did not help.

But it works! We are now automatically generating a gif of a full-page scrolling.

Now, we are pretty much one for-loop from doing a cross-spectrum responsiveness checkup. But first,

Which screen resolutions to test? #

According to statcounter 80% of market share is covered by 20 screen sizes. We will as such apply the Pareto principle.

20 is still a pretty large number, but I will reduce some of them since height usually doesn't matter that much for overflowing content and some of them are "close enough". If your styling will break it will be either at smallest screens, largest screens, or right next to your breakpoints. I'm assuming the typical bootstrapesque 768/992/1200, so a following set seems pretty reasonable:

var screenSizes = [{
        width: 1024,
        height: 768
    },
    {
        width: 1280,
        height: 720
    },
    {
        width: 1366,
        height: 768
    },
    {
        width: 1600,
        height: 900
    },
    {
        width: 1920,
        height: 1080
    },
    {
        width: 320,
        height: 534
    },
    {
        width: 375,
        height: 667
    },
    {
        width: 540,
        height: 960
    },
    {
        width: 720,
        height: 1280
    },
    {
        width: 768,
        height: 1024
    },
];

Final solution #

There is nothing interesting in looping over an array, nor doing minor parametrization or using again previous hack for passing data through a sandboxed page. However final version contains automatic report creation and folder handling which make it much more usable.

There is always a lot more that I could improve, performance and code quality, file cleanup. But I'm calling it Pareto-done for now. You can find the script here and sample report here, enjoy!

Notes and issues #

• PhantomJS is supposed to support ES6 from version 2.5, we are stuck now with the bad parts of JS.
• PhantomJS is dead, project is no longer maintained. It will still be used for long time as it lies on the foundation of many testing setups, but don't count on issues being fixed. Take a look at headless Chrome and Puppeteer.
• PhantomJS has bugs. Quite a lot of them actually. So before you cry wolf, make sure you are not seeing an issue with media query parsing.
• Transparent background will be transparent on renders, and thus on gif, leading to really weird results. I can't imagine right now a valid case where your page's body would have transparent background, so just set it explicitly if it's not and you encounter problems.