I mentioned about wanting to start a business in my previous post. Well it looks like I might have my chance. While it isn’t exactly what I was thinking of in my previous post, I foresee many opportunities to flex my programming muscle in this endeavor. Plus, I will be starting up with a good friend, so there is a good chance our motivation will actually produce some results.

A piece of software I am starting to research/design/create will be an internal application we will use to automatically extract some government provided public data. The current issue is time. With the amount of data and the requirement of using their web interface, it is not worth the time needed to do it by hand. The only other option would be to buy the data from the government, but that’s just an unnecessary cost seeing as there is a free option available.

So I will be writing a program in C# to automatically access a few websites and download the needed data in chunks. I have never delt with .NET’s web access objects, so I started looking at what they have to offer today…. which looks like a lot.

I modified an example from here:

http://msdn.microsoft.com/en-us/library/456dfw4f.aspx

To do a simple test to see how reading a web page works in .NET.

My revised code below:
Take note of the two windows form controls (webBrowser1) and (textbox1).

//due to the finally statement, these variables need to be created outside the try block
WebRequest request = null;
WebResponse response = null;
Stream dataStream = null;
StreamReader reader = null;
try
{
    // Create a request for the URL.
    request = WebRequest.Create("http://page_to_access");

    // If required by the server, set the credentials.
    request.Credentials = CredentialCache.DefaultCredentials;

    // Get the response.
    response = request.GetResponse();

    // Display the status.
    textBox1.Text = ((HttpWebResponse)response).StatusDescription;

    // Get the stream containing content returned by the server.
    dataStream = response.GetResponseStream();

    // Open the stream using a StreamReader for easy access.
    reader = new StreamReader(dataStream);

    // Read the content.
    string responseFromServer = reader.ReadToEnd();

    // Display the content.
    webBrowser1.DocumentText = responseFromServer;
}
catch (Exception error)
{
    Console.Write(error.ToString());
}
finally
{
    // Clean up the streams and the response.
    if (reader != null)
    {
        reader.Close();
    }
    if (response != null)
    {
        response.Close();
    }
}

All it does is just request a webpage (only with get data) and then stream the response to a string. After that it takes the string and inserts it into the standard WebBrowser control.

Next up I will setup/research:
- Searching the results from a web request.
- POSTing data to a page as apposed to using the get string
- Downloading files that are available from a webpage

Should be interesting!

Thanks to the new functionality I am able to use the free blogger.com blogs as a website. It’s all thinks to functionally they added after Google acquired them. For example I created a custom template for my sister’s graphic/fashion/photography design website. In this post I’ll go over the basics that allow this to work.

1. Google allows custom domain names. This means the whole website can have a professional look and feel because all links connected to the site will look like www.nameofsite.com/revrefvrv/dfvdfvd… etc.

2. The use of labels, conditional statements, and the database data that blogger allows to be accessed through the template system.

To make a standard website I started with the simplest of the pre-created templates modifying it to suit my needs. I removed a lot of the functionality like comments, backlinks, and whatever else.

For a basic website you first need some links to blocks of site content. I generally hard code them into the template. Now if you’ve used the new blogger template system on a lower level (eg. the code level and not the drag and drop stuff) you will notice a few quips. They process any hard coded html data depending on where you put it, so you have to be careful what/where you place directly code into the template. Otherwise you can add “html/JavaScript” widgets to add whatever code functionality you want. The only problem with this is that all of that code you add is stored in the database and not the template. That kind of irritates me because it’s a pain to backup.

So I will add something like this where I want the menu to be:

<div class='divmenu'>
   <span class='spanmenuitem'>
     <a href='http://www.site.com/search/label/Whats%20New'>What's New</a>
   </span>
   <span class='spanmenuitem'>
     <a href='http://www.site.com/search/label/Fashion%20Design'>Fashion Design</a>
   </span>
 </div>

As you can see, those are just hard coded links. It uses the label search functionality to find all posts that have that specific label attached to it.

So now you have links to site content on every page. Now we want to have a special page display only when the user goes to www.site.com… aka. the index page.
You will have to find the post section of the template:

<b:widget id='Blog1' locked='true' title='Blog Posts' type='Blog'>
<b:includable id='main' var='top'>
  <div id='divcontentsection'>
  <!-- display the index page, or something else -->
  <b:if cond='data:blog.url == data:blog.homepageUrl'>
    <div id='divindexpage'>
      This is the index page!
    </div>
  <b:else/>
    <b:loop values='data:posts' var='post'>
      <b:include data='post' name='post'/>
    </b:loop>
  </b:if>
  <b:include name='nextprev'/>
  </div>
</b:includable>

Key points to note is the conditional statement checking ‘data:blog.url == data:blog.homepageUrl’. Every time the website is accessed it checks to see if the current page is the index page if it is it will display some html code, otherwise it will process site site as it generally would (show posts).

Now we have to figure out how to use labels to process the site content like a website would. I will talk through what I wanted to accomplish. After that will be the block of code that accomplishes the task.

* When a user clicks on a hard coded link to “Graphic Design”, I wanted it to display a list of post titles only related to graphic design, no actual content. Each title could be clicked on to go to that specific post. That means you need to have “Enable Post Pages?” set to YES in Settings->Archiving. Every post made has a unique link associated with it. It allows us to to use conditional statements to figure out to display the post content or not depending on what page they are on.

* I wanted to create html breadcrumbs. This is a goofy term for links that show your level of deepness in the site. For example “Home > Graphic Design > Ninjas” show what path the user took to get where they are. Each one can be clicked on so they can jump back. This is easily possible with blogger as you will see in the code later.

* Another thing is that I still want regular blog functionality in the site. When the user clicks on the “Whats New” link they should be shown all posts (title, content, everything…). All this takes is a quick check of what label the post has before it is processed.

Here is the code of the post includable that’s linked to from the previous code shown:

<b:includable id='post' var='post'>
  <div class='post'>
    <!-- this post functionality only works if there is ONE label -->
    <!-- look for special cases like whats new, we want that to display like a regular blog -->
    <!-- if it is a special case, display the content as well as the title+link -->
    <b:loop values='data:post.labels' var='label'>
      <b:if cond='data:label.name == "Whats New"'>
        <div class='divpostbody'>
          <p>
            <b:if cond='data:post.dateHeader'>
              <h2><data:post.dateHeader/></h2>
            </b:if>
            <h3><a expr:href='data:post.url'><data:post.title/></a> - <data:post.timestamp/></h3>
            <p>
              <data:post.body/>
            </p>
          </p>
          <div style='clear: both;'/> <!-- clear for photos floats -->
        </div>
      <b:else/>
        <!-- if this is not a single item, display only the title and link of each item -->
        <b:if cond='data:post.url == data:blog.url'>
          <!-- START display breadcrumb links -->
          <h4>
          <a expr:href='data:blog.homepageUrl'>Home</a> |
          <b:loop values='data:post.labels' var='label'>
            <a expr:href='data:label.url'><data:label.name/></a>
          </b:loop> |
          <a expr:href='data:blog.url'><data:post.title/></a>
          </h4>
          <!-- END display breadcrumb links -->

          <h3><data:post.title/></h3>
          <div class='divpostbody'>
            <p><data:post.body/></p>
            <div style='clear: both;'/> <!-- clear for photos floats -->
          </div>
       <b:else/>
        <h3><a expr:href='data:post.url'><data:post.title/></a></h3>
       </b:if>
      </b:if>
    </b:loop>   

  </div>
</b:includable>

It took me about 2 days to research and code the necessary functionality. Blogger’s current documentation on their template system leaves something to be desired. Hopefully they will improve on that. Anyways, that’s about it for the template! Now you should have a fully functional informational website that still has blogging functionality.

The code is a pain to paste into here, so if you have some troubles getting it to work, let me know.