Browser automation, this technology seemed like black magic or forbidden for some, but it may be your last resort of integration (not really an integration) if there is no interface to integrate with the web application you need to interact with and exchange information. for many years it was my favorite resort for quick wins and workaround solutions for many issues, to list a few:

• You have a list of thousands of data entry rows you need to update to a web form
• You have an application that needs to automatically type data to a web page and submit it in a macro style function
• You need to retrieve data from thousands of pages and compile them in one tabular format
• You need to create your own web SSO application

The list is not limited to the above but can be endless in terms of quick win solutions. some companies offer you frameworks to purchase that does this black magic for you but I need to make a point that it is too simple and easy to perform that you don’t need to rely on a specific framework to do it for you.

I am going to concentrate on IE automation due to the reason that most enterprise environments rely (unfortunately) on internet explorer to run their applications, even some web applications only run on IE. I will be writing a different guide for using the Selenium framework in java that is more suited for Firefox automation although it also has IE drivers to run automations on.

Some say why do you have to do it in IE, why not use the webbrowser control from the .Net framework? a simple answer is that the webbrowser control is limited and does not offer the complete features of running web applications on it, say for example running an activex, or making some SSO (Single Sign On) applications that also does automation to work with the application you are going to write for your desired automation, additionally it will offer the user the same experience of a native web browser which is a huge advantage.

Follow these steps and you will have the key to begin automating your tasks in internet explorer using .Net framework:

In the below example we are going to automate google.com to search for a specific term that the user will enter in your application.

1.  Create a windows forms (or console if you desire) project in visual studio (windows forms is used for simplicity)
2. add reference to Microsoft internet controls (shdocvw.dll) 
3. import shdocvw to the windows form class
   using SHDocVw;
4. add a button and a textbox to your form 
5. Create a new object that contains the internet explorer instance at the class level and create a Boolean variable to trigger automation when desired and a string variable to pass to the document complete event to avoid cross thread exception in your debugging environment
   

           //Create a new object that contains the internet explorer instance
           InternetExplorer iExplorer = new InternetExplorer();
           //Create a boolean variable to trigger automation when desired
           bool doAutomation = false;
           //Create a string variable to pass to the document complete event
           string searchTerm = "";
           

6. add the following code to the form load event to make the internet explorer instance visible and add a handler to the document complete event of the webbrowser, the document complete event is the event that fires when the page is completely loaded in the browser (except for ajax loads)
   

   private void Form1_Load(object sender, EventArgs e)
           {
               //add a handler to the document complete event in internet explorer
               iExplorer.DocumentComplete += iExplorer_DocumentComplete;
               //Make the internet explorer instance visible
               iExplorer.Visible = true;
           }
           

7. add the following code to the button click event to navigate to a desired URL, set the search term variable from the input text box and trigger the automation by setting the Boolean variable assigned to true:
   

   private void button1_Click(object sender, EventArgs e)
           {
               //Set a flag to do the automation
               doAutomation = true;
               //Set the search term from the input textbox in the form application
               searchTerm = textBox1.Text;
               //Navigate to a URL like google
               iExplorer.Navigate("https://www.google.ae/");
           }
           

8. add reference to the Microsoft HTML object library 
9. import mshtml to the form class:
   using mshtml;
10. go to internet explorer developer tools (press f12) and search by clicking on the elements you want to automate and get their IDs, in our case, the google search input id is “q” and the search button id is “gbqfba” 
11. add the following code the browser complete event, the code is commented and you can understand how it works from the comments
    

    void iExplorer_DocumentComplete(object pDisp, ref object URL)
            {
                //check if the desired URL is loaded & automation is triggered
                if (URL.ToString() == "https://www.google.ae/" && doAutomation)
                {
                    //reset the automation trigger to avoid automation when not required
                    doAutomation = false;
                    //cast pDisp to an internet explorer variable
                    InternetExplorer iEplorerDOC = (InternetExplorer)pDisp;
                    //get the document object from the internet explorer instance
                    mshtml.HTMLDocument doc = iEplorerDOC.Document;
                    //get the search input element from the document object by ID
                    mshtml.IHTMLElement searchElem = doc.getElementById("q");
                    //set the search value to the desired term by setting the attribute of the input textbox of google to any value
                    searchElem.setAttribute("value", searchTerm);
                    //get the search button object from the document loaded
                    mshtml.IHTMLElement searchBtn = doc.getElementById("gbqfba");
                    //invoke the click command to simulate the onclick method of the element in the document loaded in the browser
                    searchBtn.click();
                }
            }
            

12. Run the application, input your search term then press the search button to observe the black magic (see how simple was that!)
    

Below is the full code example:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using SHDocVw;
using mshtml;

namespace WindowsFormsApplication3
{
    public partial class Form1 : Form
    {
        //Create a new object that contains the internet explorer instance
        InternetExplorer iExplorer = new InternetExplorer();
        //Create a boolean variable to trigger automation when desired
        bool doAutomation = false;
        //Create a string variable to pass to the document complete event
        string searchTerm = "";
        public Form1()
        {
            InitializeComponent();
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            //add a handler to the document complete event in internet explorer
            iExplorer.DocumentComplete += iExplorer_DocumentComplete;
            //Make the internet explorer instance visible
            iExplorer.Visible = true;
        }

        void iExplorer_DocumentComplete(object pDisp, ref object URL)
        {
            //check if the desired URL is loaded & automation is triggered
            if (URL.ToString() == "https://www.google.ae/" && doAutomation)
            {
                //reset the automation trigger to avoid automation when not required
                doAutomation = false;
                //cast pDisp to an internet explorer variable
                InternetExplorer iEplorerDOC = (InternetExplorer)pDisp;
                //get the document object from the internet explorer instance
                mshtml.HTMLDocument doc = iEplorerDOC.Document;
                //get the search input element from the document object by ID
                mshtml.IHTMLElement searchElem = doc.getElementById("q");
                //set the search value to the desired term by setting the attribute of the input textbox of google to any value
                searchElem.setAttribute("value", searchTerm);
                //get the search button object from the document loaded
                mshtml.IHTMLElement searchBtn = doc.getElementById("gbqfba");
                //invoke the click command to simulate the onclick method of the element in the document loaded in the browser
                searchBtn.click();
            }
        }

        private void button1_Click(object sender, EventArgs e)
        {
            //Set a flag to do the automation
            doAutomation = true;
            //Set the search term from the input textbox in the form application
            searchTerm = textBox1.Text;
            //Navigate to a URL like google
            iExplorer.Navigate("https://www.google.ae/");
        }
    }
}

Introduction to Internet Explorer Browser Automation http://t.co/ijq8oDqpdS

— Mohammed Al-Solh (@alsolh) May 29, 2014