Performance data collection using BrowserMob Proxy and Selenium

BrowserMob Proxy with Selenium

BrowserMob Proxy is a simple utility that makes it easy to capture performance data from browsers, typically written using automation toolkits such as Selenium and Watir.

BrowserMob Proxy can capture performance data for web apps (via the HAR format). Used to collect the performance data from the client side.

What is a HAR format ?

HAR(HTTP Archive) is a online tool visualizing. HAR is produced by the HTTP tracking tools, these files captures the details of the client/server communication and this can be used for analysis like Page Load Performance.

How the HAR files will look ?

proxy is programmatically controlled via a REST interface or by being embedded directly inside Java-based programs and unit tests.It can control HTTP trafic for,

  • blacklisting and whitelisting certain URL patterns
  • simulating various bandwidth and latency
  • remapping DNS lookups
  • flushing DNS caching
  • controlling DNS and request timeouts
  • automatic BASIC authorization

How to Use BrowserMob Proxy?
Step 1: Download the BrowserMob Proxy

Step 2: Start the proxy by running `browsermob-proxy` on Mac/*Nix based machines and `browsermob-proxy.bat`for Windows in the bin directory
On Mac it should be like sh browsermob-proxy -port 9090

Step 3: You can create a New Java Project in Eclipse as documented here

Step 4: You use the below mentioned Java code as an example,

package com.selenium.performancetest;

import java.io.FileOutputStream;

import org.browsermob.core.har.Har;
import org.browsermob.proxy.ProxyServer;
import org.openqa.selenium.By;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

public class PerfTest {

	public static void main(String[] args) throws Exception {

		String strFilePath = "";

		// start the proxy
	    ProxyServer server = new ProxyServer(4444);
	    server.start();
	    //captures the moouse movements and navigations
	    server.setCaptureHeaders(true);
        server.setCaptureContent(true);

	    // get the Selenium proxy object
	    Proxy proxy = server.seleniumProxy();

	    // configure it as a desired capability
	    DesiredCapabilities capabilities = new DesiredCapabilities();
	    capabilities.setCapability(CapabilityType.PROXY, proxy);

	    // start the browser up
	    WebDriver driver = new FirefoxDriver(capabilities);

	    // create a new HAR with the label "apple.com"
	    server.newHar("assertselenium.com");

	    // open yahoo.com
	    driver.get("http://assertselenium.com");

	    driver.get("http://assertselenium.com/2012/10/30/transformation-from-manual-tester-to-a-selenium-webdriver-automation-specialist/");

	    // get the HAR data
        Har har = server.getHar();
        FileOutputStream fos = new FileOutputStream(strFilePath);
        har.writeTo(fos);
        server.stop();
		driver.quit();

	}

}

You will have to add all the jar files that are in lib folder in the downloaded package to this Eclipse Project Build Path

Step 5: Run the script as Right click and Run As Java Application

Step 6: You can view the Eclipse console for run time logs generated

Step 7: Refresh your project, now you can see a folder HAR files is generated inside the HARFiles folder

You can now view the Har file for performance data here

Now we can view the performance data for http://assertselenium.com and it looks like the below screenshot,

Hope this post helps in setting up the BrowserMob Proxy.

Please share and leave your comments if any.

 

The same implementation in .NET C#

using AutomatedTester.BrowserMob.HAR;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;

namespace BrowserMob.PerfTest
{
    public class ExampleClass
    {
        public void ExampleUse()
        {
            // Supply the path to the Browsermob Proxy batch file
            Server server = new Server(@"C:\BrowserMobProxy\bin\browsermob-proxy.bat");
            server.Start();

            Client client = server.CreateProxy();
            client.NewHar("assertselenium");

            var seleniumProxy = new Proxy { HttpProxy = client.SeleniumProxy };
            var profile = new FirefoxProfile();

            profile.SetProxyPreferences(seleniumProxy);
            // Navigate to the page to retrieve performance stats for
            IWebDriver driver = new FirefoxDriver(profile);
            driver.Navigate().GoToUrl("http://assertselenium.com");

            // Get the performance stats
            HarResult harData = client.GetHar();

            // Do whatever you want with the metrics here. Easy to persist
            // out to a data store for ongoing metrics over time.

            driver.Quit();
            client.Close();
            server.Stop();
        }
    }
}

The Python Implementation

from browsermobproxy import Server
                server = Server("path/to/browsermob-proxy")
                server.start()
                proxy = server.create_proxy()

                from selenium import webdriver
                profile  = webdriver.FirefoxProfile()
                profile.set_proxy(proxy.selenium_proxy())
                driver = webdriver.Firefox(firefox_profile=profile)

                proxy.new_har("google")
                driver.get("http://www.google.co.uk")
                proxy.har # returns a HAR JSON blob

                proxy.stop()
                driver.quit()
Comments
17 Responses to “Performance data collection using BrowserMob Proxy and Selenium”
  1. Manish says:

    Nice Post.

  2. Ramesh Jhajharia says:

    Can you please post for .NET C# also ?

    • Thanks Ramesh for your support. I C# I havent tried yet, but will look into it and post it here. Keep visiting

      • Ramesh Jhajharia says:

        I am using this with ASP .Net C#.
        But I am facing an issue that I am not able to remap host or you can say the browser mob proxy is not reading my host file .
        I found that calls to remap a host is available in java but not in Asp .net c#.
        Please help me if you have any idea on this.

        Thanks,
        Ramesh Jhajharia

  3. Hey Ramesh,
    I saw your post in the Google Groups as well. I have just started looking at the .NET bindings of BrowserMob Proxy. I will post the code here for the same If I get to work them properly.

    Havent fully explored all the API’s in Java itself. Let me try to help you out soon.

    Thanks
    Manoj

  4. xrisb says:

    great article. I was able to implement the browsermob proxy using C# however what i’m finding it difficult to understand is how do i go about implementing it for my whole project? I mean i have a login page, and when the login button is being clicked i’m being redirected to the same page instead of having the app proceeding to the next screen. does this mean that browsermob proxy has to be implemented each time i want to monitor traffic?

  5. vikas says:

    Hi monoj, when i am trying to run https://www.gmail.com using above code. i am getting some proxy issues. Firefox is opening and giving a message stating: Firefox is configured to use a proxy server that is refusing connections.
    Check the proxy settings to make sure that they are correct.
    Contact your network administrator to make sure the proxy server is
    working.
    Do i need to change any settings to make it work? please let me know

    • That is completely dependent on the network that you use. I think you are under a firewall protected environment, where you are using your AD credentials to access the webpages ? If yes, you can configure Firefox with proxy or add the webpage under exemption list

  6. vikas says:

    Forgot to mention my firefox version is 18.0.2 and i am using latest browsermob jars: browsermob-proxy-2.0-beta-6

  7. bianca pirte says:

    using python on windows:
    for
    from browsermobproxy.server import Server
    from selenium import webdriver

    server = Server(“\Lib\browsermobproxy\bin”)
    server.start()
    proxy = server.create_proxy()

    profile = webdriver.FirefoxProfile()
    profile.set_proxy(proxy.selenium_proxy())
    driver = webdriver.Firefox(firefox_profile=profile)

    proxy.new_har(“google”)
    driver.get(“http://www.google.co.uk”)
    proxy.har # returns a HAR JSON blob

    proxy.stop()
    driver.quit()

    i get

    from browsermobproxy.server import Server
    ImportError: No module named server

    even if the import is the one Eclipse suggested for Server

    i have:
    python 2.7
    selenium 2.27
    browsermob 0.20
    and I run the test from eclipse

    any suggestions?

    thanks,
    -Bianca

  8. Melvin Lobo says:

    Hi All,

    My main goal is to create the har files.

    I tried using the above code and works fine but if I test a site which requires login cedentials i,e session id, it fails past the login screen. Below is the code snippet
    driver.get(_sBaseURL + “/Login/Login.aspx”);
    driver.findElement(By.id(“User.UserName”)).clear();
    driver.findElement(By.id(“User.UserName”)).sendKeys(new String[]{“jmathe02”});
    driver.findElement(By.id(“User.Password”)).clear();
    driver.findElement(By.id(“User.Password”)).sendKeys(new String[]{“cat1dog”});
    driver.findElement(By.id(“btnLogin”)).click();

    driver.findElement(By.xpath(“//table[@id=’tblAgenciesList’]/tbody/tr[13]/td”)).click();
    ===============
    However if I change to proxy to use ProxyType.Direct it works but then does not create the har entires.

    Was wondering if some could help.

    Melvin

  9. Thanks for the TracBack’s

Trackbacks
Check out what others are saying...


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: