Learning About .NET Web Access Classes

on

I mentioned about wanting to start a business in my previous post. Well it looks like I might have my chance. While it isn’t exactly what I was thinking of in my previous post, I foresee many opportunities to flex my programming muscle in this endeavor. Plus, I will be starting up with a good friend, so there is a good chance our motivation will actually produce some results.

A piece of software I am starting to research/design/create will be an internal application we will use to automatically extract some government provided public data. The current issue is time. With the amount of data and the requirement of using their web interface, it is not worth the time needed to do it by hand. The only other option would be to buy the data from the government, but that’s just an unnecessary cost seeing as there is a free option available.

So I will be writing a program in C# to automatically access a few websites and download the needed data in chunks. I have never delt with .NET’s web access objects, so I started looking at what they have to offer today…. which looks like a lot.

I modified an example from here:
http://msdn.microsoft.com/en-us/library/456dfw4f.aspx

To do a simple test to see how reading a web page works in .NET.

My revised code below:
Take note of the two windows form controls (webBrowser1) and (textbox1).

// due to the finally statement, these variables need to be created outside the try block
WebRequest request = null;
WebResponse response = null;
Stream dataStream = null;
StreamReader reader = null;

try {
    // Create a request for the URL.
    request = WebRequest.Create("http://page_to_access");
    // If required by the server, set the credentials.
    request.Credentials = CredentialCache.DefaultCredentials;
    // Get the response.
    response = request.GetResponse();
    // Display the status.
    textBox1.Text = ((HttpWebResponse)response).StatusDescription;
    // Get the stream containing content returned by the server.
    dataStream = response.GetResponseStream();
    // Open the stream using a StreamReader for easy access.
    reader = new StreamReader(dataStream);
    // Read the content.
    string responseFromServer = reader.ReadToEnd();
    // Display the content.
    webBrowser1.DocumentText = responseFromServer;
} catch (Exception error) {
    Console.Write(error.ToString());
} finally {
    // Clean up the streams and the response.
    if (reader != null) {
        reader.Close();
    }
    if (response != null) {
        response.Close();
    }
}


All it does is just request a webpage (only with get data) and then stream the response to a string. After that it takes the string and inserts it into the standard WebBrowser control.

Next up I will setup/research:

  • Searching the results from a web request.
  • POSTing data to a page as apposed to using the get string
  • Downloading files that are available from a webpage

Should be interesting!