Skip to main content

Learning About .NET Web Access Classes

I mentioned about wanting to start a business in my previous post. Well it looks like I might have my chance. While it isn’t exactly what I was thinking of in my previous post, I foresee many opportunities to flex my programming muscle in this endeavor. Plus, I will be starting up with a good friend, so there is a good chance our motivation will actually produce some results.

A piece of software I am starting to research/design/create will be an internal application we will use to automatically extract some government provided public data. The current issue is time. With the amount of data and the requirement of using their web interface, it is not worth the time needed to do it by hand. The only other option would be to buy the data from the government, but that’s just an unnecessary cost seeing as there is a free option available.

So I will be writing a program in C# to automatically access a few websites and download the needed data in chunks. I have never delt with .NET’s web access objects, so I started looking at what they have to offer today…. which looks like a lot.

I modified an example from here:
http://msdn.microsoft.com/en-us/library/456dfw4f.aspx

To do a simple test to see how reading a web page works in .NET.

My revised code below:
Take note of the two windows form controls (webBrowser1) and (textbox1).

// due to the finally statement, these variables need to be created outside the try block
WebRequest request = null;
WebResponse response = null;
Stream dataStream = null;
StreamReader reader = null;

try {
    // Create a request for the URL.
    request = WebRequest.Create("http://page_to_access");
    // If required by the server, set the credentials.
    request.Credentials = CredentialCache.DefaultCredentials;
    // Get the response.
    response = request.GetResponse();
    // Display the status.
    textBox1.Text = ((HttpWebResponse)response).StatusDescription;
    // Get the stream containing content returned by the server.
    dataStream = response.GetResponseStream();
    // Open the stream using a StreamReader for easy access.
    reader = new StreamReader(dataStream);
    // Read the content.
    string responseFromServer = reader.ReadToEnd();
    // Display the content.
    webBrowser1.DocumentText = responseFromServer;
} catch (Exception error) {
    Console.Write(error.ToString());
} finally {
    // Clean up the streams and the response.
    if (reader != null) {
        reader.Close();
    }
    if (response != null) {
        response.Close();
    }
}


All it does is just request a webpage (only with get data) and then stream the response to a string. After that it takes the string and inserts it into the standard WebBrowser control.

Next up I will setup/research:

  • Searching the results from a web request.
  • POSTing data to a page as apposed to using the get string
  • Downloading files that are available from a webpage

Should be interesting!

Labels:

Popular posts from this blog

ChatGPT is a new, and faster, way to do programming!

Currently ChatGPT is in a free “initial research preview” . One of its well known use cases at this point is generating software code. I’ve also just used it to write most of this article… Well, actually a future article about cleaning up SRT subtitle files of their metadata faster than I have been by hand with Notepad++ and its replace functionality. Update: I recorded a screencast of writing the SRT subtitle cleaner application loading and processing portion. I relied heavily on ChatGPT for code. It was a fun process! https://youtu.be/TkEW39OloUA ChatGPT, developed by OpenAI, is a powerful language model that can assist developers in a variety of tasks, including natural language processing and text generation. One such task that ChatGPT can help with is creating an SRT cleaner program. SRT, or SubRip Subtitle, files are commonly used to add subtitles to video files. However, these files can become cluttered with unnecessary information, such as timing lines or blank spaces. To clean...
Read more

Theme error in 2010s Android App after AppCompat Migration

I plan on releasing a lot of my old work as GPL open source, but most of it has aged to the point that it no longer functions, or if it does work it’s running in compatibility mode. Basically it’s no longer best practices. Not a good way to start off any new public GPL projects, in my opinion. The current project I’m working on is an Android app that calculates star trails meant to help photographers get or avoid that in their night time photos. For now I’m going to skip some of the import process because I didn’t document it exactly. It’s been mostly trial and error as I poke around Android Studio post import. The Android Studio import process… Removing Admob Google Play code before the project would run at all. After removing dependencies, it kind of worked, but when running it in the emulator it shows a pop-up message saying that the app was developed for an old version of Android. Going through the process of updating code to match current best practices… I had the IDE convert the ...
Read more

Printing to file in Linux WINE

I noticed that this post has been sitting as a draft since 2011. At this point I have no idea if it’s useful or what I was even doing, but I might as well make it public in case someone can find it helpful! So I’ve been trying to get one of those PDF print drivers working in WINE without success. I then came upon a process that might work. When printing you need to select the checkbox “Print to file” that creates a .prn file. Just Linux things... I was using a program that only has printing facilities, but I want to export around 100 pages of text and images. Once you have the .prn (postscript) file, you can do any number of things to it. In my case I want the postscript file to be converted to HTML. I am also considering PDF format because that has more conversion options to eventually get me to HTML or plain text. sudo apt-get install cups-pdf Or it looks like that package might have changed to this… sudo apt-get install printer-driver-cups-pdf Where PDFs would be generated in /home/...
Read more