Browser automation, this technology seemed like black magic or forbidden for some, but it may be your last resort of integration (not really an integration) if there is no interface to integrate with the web application you need to interact with and exchange information. for many years it was my favorite resort for quick wins and workaround solutions for many issues, to list a few:
- You have a list of thousands of data entry rows you need to update to a web form
- You have an application that needs to automatically type data to a web page and submit it in a macro style function
- You need to retrieve data from thousands of pages and compile them in one tabular format
- You need to create your own web SSO application
The list is not limited to the above but can be endless in terms of quick win solutions. some companies offer you frameworks to purchase that does this black magic for you but I need to make a point that it is too simple and easy to perform that you don’t need to rely on a specific framework to do it for you.
I am going to concentrate on IE automation due to the reason that most enterprise environments rely (unfortunately) on internet explorer to run their applications, even some web applications only run on IE. I will be writing a different guide for using the Selenium framework in java that is more suited for Firefox automation although it also has IE drivers to run automations on.
Some say why do you have to do it in IE, why not use the webbrowser control from the .Net framework? a simple answer is that the webbrowser control is limited and does not offer the complete features of running web applications on it, say for example running an activex, or making some SSO (Single Sign On) applications that also does automation to work with the application you are going to write for your desired automation, additionally it will offer the user the same experience of a native web browser which is a huge advantage.
Follow these steps and you will have the key to begin automating your tasks in internet explorer using .Net framework:
In the below example we are going to automate google.com to search for a specific term that the user will enter in your application.
- Create a windows forms (or console if you desire) project in visual studio (windows forms is used for simplicity)
- add reference to Microsoft internet controls (shdocvw.dll)
- import shdocvw to the windows form class
using SHDocVw; - add a button and a textbox to your form
- Create a new object that contains the internet explorer instance at the class level and create a Boolean variable to trigger automation when desired and a string variable to pass to the document complete event to avoid cross thread exception in your debugging environment
//Create a new object that contains the internet explorer instance InternetExplorer iExplorer = new InternetExplorer(); //Create a boolean variable to trigger automation when desired bool doAutomation = false; //Create a string variable to pass to the document complete event string searchTerm = ""; - add the following code to the form load event to make the internet explorer instance visible and add a handler to the document complete event of the webbrowser, the document complete event is the event that fires when the page is completely loaded in the browser (except for ajax loads)
private void Form1_Load(object sender, EventArgs e) { //add a handler to the document complete event in internet explorer iExplorer.DocumentComplete += iExplorer_DocumentComplete; //Make the internet explorer instance visible iExplorer.Visible = true; } - add the following code to the button click event to navigate to a desired URL, set the search term variable from the input text box and trigger the automation by setting the Boolean variable assigned to true:
private void button1_Click(object sender, EventArgs e) { //Set a flag to do the automation doAutomation = true; //Set the search term from the input textbox in the form application searchTerm = textBox1.Text; //Navigate to a URL like google iExplorer.Navigate("https://www.google.ae/"); } - add reference to the Microsoft HTML object library
- import mshtml to the form class:
using mshtml; - go to internet explorer developer tools (press f12) and search by clicking on the elements you want to automate and get their IDs, in our case, the google search input id is “q” and the search button id is “gbqfba”
- add the following code the browser complete event, the code is commented and you can understand how it works from the comments
void iExplorer_DocumentComplete(object pDisp, ref object URL) { //check if the desired URL is loaded & automation is triggered if (URL.ToString() == "https://www.google.ae/" && doAutomation) { //reset the automation trigger to avoid automation when not required doAutomation = false; //cast pDisp to an internet explorer variable InternetExplorer iEplorerDOC = (InternetExplorer)pDisp; //get the document object from the internet explorer instance mshtml.HTMLDocument doc = iEplorerDOC.Document; //get the search input element from the document object by ID mshtml.IHTMLElement searchElem = doc.getElementById("q"); //set the search value to the desired term by setting the attribute of the input textbox of google to any value searchElem.setAttribute("value", searchTerm); //get the search button object from the document loaded mshtml.IHTMLElement searchBtn = doc.getElementById("gbqfba"); //invoke the click command to simulate the onclick method of the element in the document loaded in the browser searchBtn.click(); } } - Run the application, input your search term then press the search button to observe the black magic (see how simple was that!)
Below is the full code example:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using SHDocVw;
using mshtml;
namespace WindowsFormsApplication3
{
public partial class Form1 : Form
{
//Create a new object that contains the internet explorer instance
InternetExplorer iExplorer = new InternetExplorer();
//Create a boolean variable to trigger automation when desired
bool doAutomation = false;
//Create a string variable to pass to the document complete event
string searchTerm = "";
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
//add a handler to the document complete event in internet explorer
iExplorer.DocumentComplete += iExplorer_DocumentComplete;
//Make the internet explorer instance visible
iExplorer.Visible = true;
}
void iExplorer_DocumentComplete(object pDisp, ref object URL)
{
//check if the desired URL is loaded & automation is triggered
if (URL.ToString() == "https://www.google.ae/" && doAutomation)
{
//reset the automation trigger to avoid automation when not required
doAutomation = false;
//cast pDisp to an internet explorer variable
InternetExplorer iEplorerDOC = (InternetExplorer)pDisp;
//get the document object from the internet explorer instance
mshtml.HTMLDocument doc = iEplorerDOC.Document;
//get the search input element from the document object by ID
mshtml.IHTMLElement searchElem = doc.getElementById("q");
//set the search value to the desired term by setting the attribute of the input textbox of google to any value
searchElem.setAttribute("value", searchTerm);
//get the search button object from the document loaded
mshtml.IHTMLElement searchBtn = doc.getElementById("gbqfba");
//invoke the click command to simulate the onclick method of the element in the document loaded in the browser
searchBtn.click();
}
}
private void button1_Click(object sender, EventArgs e)
{
//Set a flag to do the automation
doAutomation = true;
//Set the search term from the input textbox in the form application
searchTerm = textBox1.Text;
//Navigate to a URL like google
iExplorer.Navigate("https://www.google.ae/");
}
}
}
Introduction to Internet Explorer Browser Automation http://t.co/ijq8oDqpdS
— Mohammed Al-Solh (@alsolh) May 29, 2014