Skip to main content

Learning About .NET Web Access Classes

I mentioned about wanting to start a business in my previous post. Well it looks like I might have my chance. While it isn’t exactly what I was thinking of in my previous post, I foresee many opportunities to flex my programming muscle in this endeavor. Plus, I will be starting up with a good friend, so there is a good chance our motivation will actually produce some results.

A piece of software I am starting to research/design/create will be an internal application we will use to automatically extract some government provided public data. The current issue is time. With the amount of data and the requirement of using their web interface, it is not worth the time needed to do it by hand. The only other option would be to buy the data from the government, but that’s just an unnecessary cost seeing as there is a free option available.

So I will be writing a program in C# to automatically access a few websites and download the needed data in chunks. I have never delt with .NET’s web access objects, so I started looking at what they have to offer today…. which looks like a lot.

I modified an example from here:
http://msdn.microsoft.com/en-us/library/456dfw4f.aspx

To do a simple test to see how reading a web page works in .NET.

My revised code below:
Take note of the two windows form controls (webBrowser1) and (textbox1).

// due to the finally statement, these variables need to be created outside the try block
WebRequest request = null;
WebResponse response = null;
Stream dataStream = null;
StreamReader reader = null;

try {
    // Create a request for the URL.
    request = WebRequest.Create("http://page_to_access");
    // If required by the server, set the credentials.
    request.Credentials = CredentialCache.DefaultCredentials;
    // Get the response.
    response = request.GetResponse();
    // Display the status.
    textBox1.Text = ((HttpWebResponse)response).StatusDescription;
    // Get the stream containing content returned by the server.
    dataStream = response.GetResponseStream();
    // Open the stream using a StreamReader for easy access.
    reader = new StreamReader(dataStream);
    // Read the content.
    string responseFromServer = reader.ReadToEnd();
    // Display the content.
    webBrowser1.DocumentText = responseFromServer;
} catch (Exception error) {
    Console.Write(error.ToString());
} finally {
    // Clean up the streams and the response.
    if (reader != null) {
        reader.Close();
    }
    if (response != null) {
        response.Close();
    }
}


All it does is just request a webpage (only with get data) and then stream the response to a string. After that it takes the string and inserts it into the standard WebBrowser control.

Next up I will setup/research:

  • Searching the results from a web request.
  • POSTing data to a page as apposed to using the get string
  • Downloading files that are available from a webpage

Should be interesting!

Labels:

Popular posts from this blog

ChatGPT is a new, and faster, way to do programming!

Currently ChatGPT is in a free “initial research preview” . One of its well known use cases at this point is generating software code. I’ve also just used it to write most of this article… Well, actually a future article about cleaning up SRT subtitle files of their metadata faster than I have been by hand with Notepad++ and its replace functionality. Update: I recorded a screencast of writing the SRT subtitle cleaner application loading and processing portion. I relied heavily on ChatGPT for code. It was a fun process! https://youtu.be/TkEW39OloUA ChatGPT, developed by OpenAI, is a powerful language model that can assist developers in a variety of tasks, including natural language processing and text generation. One such task that ChatGPT can help with is creating an SRT cleaner program. SRT, or SubRip Subtitle, files are commonly used to add subtitles to video files. However, these files can become cluttered with unnecessary information, such as timing lines or blank spaces. To clean...
Read more

Theme error in 2010s Android App after AppCompat Migration

I plan on releasing a lot of my old work as GPL open source, but most of it has aged to the point that it no longer functions, or if it does work it’s running in compatibility mode. Basically it’s no longer best practices. Not a good way to start off any new public GPL projects, in my opinion. The current project I’m working on is an Android app that calculates star trails meant to help photographers get or avoid that in their night time photos. For now I’m going to skip some of the import process because I didn’t document it exactly. It’s been mostly trial and error as I poke around Android Studio post import. The Android Studio import process… Removing Admob Google Play code before the project would run at all. After removing dependencies, it kind of worked, but when running it in the emulator it shows a pop-up message saying that the app was developed for an old version of Android. Going through the process of updating code to match current best practices… I had the IDE convert the ...
Read more

Blogger Notable theme pop-up header issue fix (thanks to Gemini Pro)

I've made a few half hearted attempts over the years to to fix Blogger's Notable theme's rendering of the pop-up header that shows up when you scroll down the page a decent amount and then pull back to reveal that secondary header. On Chrome mobile I noticed a gray box that forms next to the magnifying glass icon. I never looked in detail on  Chrome desktop, but it had an issue as well which I'll detail below.  If you are looking for a solution and don't want all of the extra talk about how I was able to find it, here it is:  .centered-top-container .sticky .main_header_elements { overflow : hidden !important ; } I decided to try using Gemini Pro 2.5 to see if it was capable of finding the issue and giving me a fix. Turns out that it was able, but it took a bit of collaboration back and forth to find the actual problem.  Here is a modified article I asked it to give me based on our debugging chat (it was very colorful in the article which I scaled back a lot, ...
Read more