• The default Ubuntu 12.04 LTS AWS image boots on a 8GB  root partition created on EBS. This allows you to run a a normal Ubuntu server. If you create a larger EBS partition, Ubuntu will automatically expand the file system on first boot, but 8GB should be plenty for a typical DNS server.
• The Micro Instances includes 600MB of RAM
• I thought I read that the Micro Instance includes 2 CPU cores, but the image I booted only shows 1 CPU.

This should be fine for a backup DNS server.

The AWS documentation is large and intimidating. In the past, AWS encouraged a lot of specialized AWS development to create paravirtualized kernels that worked under Xen combined with VM instances that booted up and automatically configured  themselves to use S3 storage. This is a great model for web sites and other Internet based services that require scaling across many nodes, it seemed like overkill for something like DNS that typically only requires a couple of high availability servers. Fast forward several years later and now AWS lets you easily create VM backed by EBS storage that can be managed like a typical server. Here are the steps I followed to create my AWS based DNS server:

1. Opened an AWS account  - credit card is required (!)
2. Configure and start a new Ubuntu 12.04 LTS micro instance. The free instance is clearly identified and the defaults end up with an EBS backed server with a 8GB root partition.
3. Add a public IP address: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html
4. Login to the new instance with the AWS generated SSH key pair: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html
5. Configure your new instance normally

Now we just have to wait and see if it is really free!

• http://en.wikipedia.org/wiki/Domain_tasting
• http://en.wikipedia.org/wiki/Domain_name_front_running

A friend forwarded me an email yesterday with a scam I haven’t seen before. He received domain name spam that looked like this:

Take time to secure friendsdomainnamewithans.com today.

We understand the importance of capturing all similar Domain Names in alignment with your business. Make an offer today in order for us to go after the domain name for you.
To confirm interest in owning this domain, fill out form here:

http://dr-gs.com/24343840mavidu-MCI

[...]

When I checked yesterday, the domain name was registered to a company with a name something like  “China Domains” (too bad I didn’t save the page). The domain was set to expire in a year. When I checked the domain name today, it was unregistered. It looks like somebody is combining spam with domain tasting… so creative!

Recently, I upgraded my development system from OpenOffice 2.x to OpenOffice 3.1. I can’t remember now the main reason I upgraded, but I was looking forward to being able to add the ability to convert docx to pdf (Office Open XML support was added in OpenOffice 3.x). I figured the upgrade might require some minor changes to my document conversion code, but it turned out not to be so simple.

The first problem was that OpenOffice no longer showed up under “Add Reference” in Visual Studio. After some Google searches, I finally stumbled across a solution. OK, wow, digging though install cab files for DLLs, hacking the registry and manipulating the PATH is ugly, but it worked – at least once. Almost as soon as my test program ran successfully, Visual Studio locked up. The only way to get control back was to kill the Visual Studio process, then restart it. Every time I opened the solution file, after just a few seconds, Visual Studio would lock up again. Removing every trace of the OpenOffice DLL references would prevent the lockups, but without OpenOffice, I could not convert documents. I eventually figured out that the problem was that the library project that used OpenOffice was also included in my web site project. Because the web site uses the library, too, the OpenOffice reference is automatically copied to the web site project. Normally, this would be no big deal – just a couple of small DLL files in the web site bin folder that would not be used – but in this case, Visual Studio could not successfully build the web site project as long as the OpenOffice DLLs were referenced.

Now that using OpenOffice was no longer as simple as “Add Reference”, it was time to take a look at a way to decouple my application from the OpenOffice DLLs. The API my application needs is pretty simple: Given the bytes of a file in Doc (or .docx) format, return the bytes of the file in Pdf format. Also, there are security risks whenever you open a file in a complicated piece of software like OpenOffice. These two facts pointed me to turning the Pdf conversion into a web service. The client application and web service can be partitioned across computers, so, if I wanted to, I could run the Pdf conversion service on a virtual machine with its own install of OpenOffice and tight security and resource control. The web service API didn’t need fancy XML: Clients could post the Doc file via a standard HTTP MIME encoded form. The result would be a the PDF byte stream returned via HTTP. This would make the service easy to develop, test and use from clients on many platforms.

I typically develop my large web based applications in C# and Asp.Net. However, for small services like document conversion, I often use PHP. Simple PHP applications can be just a file or two and I can move a PHP based service over to any web server that supports PHP. So, my first attempt at a document conversion service was written in PHP and used the OpenOffice COM API. Using my earlier OpenOffice API code as a guide and some PHP sample code I found on the web, I was able to quickly create a command line PHP application that could perform Pdf conversions. With this proof of concept working, I coded up the PHP web service. But the web service wouldn’t work. I could see in Task Manager that the OpenOffice executables were launched, but any code running under the web server that tried to access the OpenOffice COM objects would freeze up.

I spent a lot of time trying different security settings for various files and DCOM. I even tried creating a dedicated user and configuring DCOM to run OpenOffice as the dedicated user when the OpenOffice COM objects were activated. I changed Windows event log settings to log all security violations. Nothing would work and I couldn’t see anything useful in Event Viewer. I tried coding up a simple Asp.Net application and got the same results under the .Net OpenOffice API. Under both PHP/COM and C#/.Net, OpenOffice could be automated by a command line program but would not run under IIS. Right now, I develop on Vista and deploy to Windows Server 2008, so one might think the problem was trying to use IIS 7, but OpenOffice 2 didn’t have similar problems.

Finally, I gave up trying to run OpenOffice directly under IIS. The problem was either some type of security issue or that OpenOffice was trying to pop up registration dialogs. In Windows, the simple way to run a program under a given user account is to run the program as a Windows Service. It turns out that it is pretty easy to create a Windows Service in C#. The basic idea I came up is was:

1. Install OpenOffice
2. Create a Windows user account dedicated to Pdf conversion.
3. Login as the Pdf conversion user and make sure that OpenOffice opens without prompting for registration, crash feedback, etc. (I suspect that the reason OpenOffice 3 won’t run properly under IIS is that it is prompting the user to register).
4. Create a Windows Service with a simple API to convert documents
5. Install the Windows Service and configure it to run as Pdf conversion user
6. Access the Windows Service from my document conversion Web Service

Here is how the API calls flow: Application -> Web Service -> Windows Service

As I mentioned earlier, the API between the application and the web service was standard a HTTP Post. Between the Web Service and the Windows Service, I originally planned to use a simple, proprietary TCP/IP protocol. As long as you aren’t try to get too fancy coding thread pools and asynchronous I/O, at least in C/C++, it’s pretty easy to open a TCP socket and communicate between client and server with a line oriented protocol (like HTTP, POP3, SMTP, etc.). But, since both the Web Service and the Windows Service were going to run on Windows (at least for now), I decided to take a look at .Net Remoting. The .Net remoting solution was easy to code and use so I ended up sticking with it.

Below is some code to show you how everything fits together:

The C# interface IConverter is compiled into a DLL. The DLL is referenced by the client (Web Service) and server (Windows Service).

    public class ConverterEndpoints
    {
        public static int    Port = 7047;
        public static string Uri  = "DocumentToPdf";
    }

    public interface IConverter
    {
        void Convert(string srcDocument, string destPdf);
    }

The class ConverterEndpoints holds a couple of constants that the client and server use to setup the TCP port and service name. The code below demonstrates how ConverterEndpoints is used in the Windows Service. I’ve included code for the entire class so you can see how easy it is to program a Windows Service and access the Windows Event Log in C# – no 3rd party libraries are required.

The original version of this interface passed the files as simple byte[] variables. Theoretically, the files never need to be written to disk (input and output is via HTTP). But, I found that I could easily pass small byte[] variables across .Net remoting but large byte[] variables failed. Since the Web Service and the Windows Service run on the same machine and the file has to be written to disk for OpenOffice to do the conversion, I settled for creating temporary files and passing the names of the files between the Web Service and Windows Service.

   class WindowsService : ServiceBase
    {
        protected EventLog _log;
        protected TcpChannel _channel;

        public WindowsService()
        {
            ServiceName = "Teztech Document Conversion";
            EventLog.Source = "Teztech Document Conversion";
            EventLog.Log = "Application";
            
            // These Flags set whether or not to handle that specific
            //  type of event. Set to true if you need it, false otherwise.
            CanHandlePowerEvent = false;
            CanHandleSessionChangeEvent = false;
            CanPauseAndContinue = true;
            CanShutdown = true;
            CanStop = true;

            if (!EventLog.SourceExists("Teztech Document Conversion"))
                EventLog.CreateEventSource("Teztech Document Conversion", "Application");

            _log = new EventLog();
            _log.Source = "Teztech Document Conversion";

            _channel = new TcpChannel(ConvertInterface.ConverterEndpoints.Port);
            ChannelServices.RegisterChannel(_channel, false);

            RemotingConfiguration.RegisterWellKnownServiceType(typeof(DocumentToPdf), ConvertInterface.ConverterEndpoints.Uri, WellKnownObjectMode.Singleton);
        }

        static void Main()
        {
            ServiceBase.Run(new WindowsService());
        }

        /// <summary>
        /// Dispose of objects that need it here.
        /// </summary>
        /// <param name="disposing">Whether or not disposing is going on.</param>
        protected override void Dispose(bool disposing)
        {
            base.Dispose(disposing);
        }

        protected override void OnStart(string[] args)
        {
            _log.WriteEntry("Service Starting");

            _channel.StartListening(null);

            base.OnStart(args);

            _log.WriteEntry("Service Running");
        }

        protected override void OnStop()
        {
            _log.WriteEntry("Service Stopping");

            _channel.StopListening(null);

            base.OnStop();
        }

        protected override void OnPause()
        {
            _channel.StopListening(null);
            base.OnPause();
        }

        protected override void OnContinue()
        {
            _channel.StartListening(null);
            base.OnContinue();
        }
    }

Below is the code behind file for the Web Service client. As you can see, .Net makes this type of Web Service easy to create.

public partial class Pdf : System.Web.UI.Page
{
    protected TcpChannel _channel;

    protected void Page_Load(object sender, EventArgs e)
    {
        try
        {
            if (Request["Username"] == null || Request["Password"] == null)
                throw new ApplicationException("Username and Password are required");

            HttpPostedFile postedFile = Request.Files["InputFile"];
            if (postedFile == null || postedFile.FileName == null || postedFile.FileName == "")
                throw new ApplicationException("InputFile was not supplied or file name is empty.");

            string inputFileExtension = Path.GetExtension(postedFile.FileName);
            if (inputFileExtension.Length < 2)
                throw new ApplicationException("InputFile does not have a valid file name extension.");
            inputFileExtension = inputFileExtension.Substring(1);

            string outputFileName = Request["OutputFileName"];
            if (outputFileName == null || outputFileName == "")
                outputFileName = Path.GetFileNameWithoutExtension(postedFile.FileName) + ".pdf";

            DbRequest.CheckAuthorization(Request["Username"], Request["Password"]);

            if (_channel == null)
            {
                Dictionary<string, string> channelProperties = new Dictionary<string,string>();
                channelProperties["name"] = "";
                _channel = new TcpChannel(channelProperties, null, null);
                ChannelServices.RegisterChannel(_channel, false);
            }

            string url = string.Format("tcp://localhost:{0}/{1}", ConvertInterface.ConverterEndpoints.Port, ConvertInterface.ConverterEndpoints.Uri);

            ConvertInterface.IConverter converter = (ConvertInterface.IConverter)Activator.GetObject(typeof(ConvertInterface.IConverter), url);

            string path = MapPath(".");
            path = Path.Combine(Path.GetDirectoryName(path), "Temp");

            using (TempFileCollection tempFiles = new TempFileCollection(path))
            {
                string inputFile  = tempFiles.AddExtension(inputFileExtension);
                string outputFile = tempFiles.AddExtension("pdf");

                postedFile.SaveAs(inputFile);
                converter.Convert(inputFile, outputFile);

                Response.Clear();
                Response.ContentType = "application/pdf";
                Response.AddHeader("content-disposition", string.Format("inline; filename={0}.pdf", "sample"));

                byte[] pdfBytes = File.ReadAllBytes(outputFile);

                Response.OutputStream.Write(pdfBytes, 0, pdfBytes.Length);
            }
        }        
        catch (Exception ex)
        {
            Response.Clear();
            Response.Write(string.Format("<HTML><HEAD></HEAD><BODY><H1>Conversion Error</H1><P>{0}</P></BODY></HTML>", ex.Message));
            Response.StatusCode = 500;
            Response.StatusDescription = ex.Message;
        }
    }

For completeness, I have included the code for the actual Web Service’s .aspx file below. As you can see, it’s just standard .aspx. boilerplate – all HTTP output is generated in the code behind file’s Page_Load event (which will be invoked when the Web Service client posts to the page). This is quick and dirty, but it works.

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="Pdf.aspx.cs" Inherits="Pdf" EnableViewState="false" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
    <title>Untitled Page</title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
    
    </div>
    </form>
</body>
</html>

Here And here is the code for a little PHP application I used to test my conversion Web Service:

<?php $ErrorMessage = ''; ?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>
File Conversion Server Demo
</title>
</head>
<body>

<h1>File Conversion Server Demo</h1>

<form name="Demo" method="POST" enctype="multipart/form-data" action="/demo.pdf">

    <?php if($ErrorMessage): ?>
    <p><font size="5" color="#008000"><?php echo $ErrorMessage; ?></font></p>
    <?php endif; ?>
    
    <table>
        <tr>
            <td><b>Username:</b></td>
            <td><input name="Username" value=""></input></td>
        </tr>
        <tr>
            <td><b>Password:</b></td>
            <td><input name="Password" value=""></input></td>
        </tr>
        <tr>
            <td><b>Input File:</b></td>
            <td><input type="file" name="InputFile"></input> </td>
        </tr>
    </table>
    
    <input type="submit" value="Submit"></input> 
    
</html>

If you look the PHP form’s action tag, you’ll see I used /demo.pdf as the action. Using the very nice IIS7 URL Rewrite Module, I map all *.pdf URLs to my conversion Web Service. OK, this might be a little overkill, but this way the URLs look nice and all end in .pdf (which is appropriate).

The final piece of the puzzle is a small wrapper class I created to access the conversion Web Service from within my C# application:

    public class ConvertClient
    {
        public ConvertClient(string convertHost, string username, string password)
        {
            _ConvertHost = convertHost;
            _Username    = username;
            _Password    = password;
        }

        protected string _ConvertHost;
        public string ConvertHost { get { return _ConvertHost; } set { _ConvertHost = value; } }

        protected string _Username;
        public string Username { get { return _Username; } set { _Username = value; } }

        protected string _Password;
        public string Password { get { return _Password; } set { _Password = value; } }

        protected string _Response;
        public string Response { get { return _Response; } set { _Response = value; } }

        protected int _MaxConvertAttempts = 2;
        public int MaxConvertAttempts { get { return _MaxConvertAttempts; } set { _MaxConvertAttempts = value; } }

        protected string _ResponseCode;
        public string ResponseCode { get { return _ResponseCode; } set { _ResponseCode = value; } }

        protected static string _szBoundary    = "SEPARATORSTRINGTEZTECHDOTCOM1";
        protected static string _szBoundary2   = "\r\n--SEPARATORSTRINGTEZTECHDOTCOM1\r\n";
        protected static string _szBoundary3   = "\r\n--SEPARATORSTRINGTEZTECHDOTCOM1--";
        protected static string _szFileSizeHdr = "Content-Disposition: form-data; name=\"MAX_FILE_SIZE\"\r\n\r\n";
        protected static string _szFileHdrFmt  = "Content-Disposition: form-data; name=\"InputFile\"; filename=\"{0}\"\r\nContent-Type: application/octet-stream\r\n\r\n";

        public byte[] ConvertToPdf(byte[] inputFileBytes, string inputFileName, string outputFileName)
        {
            string url = string.Format("http://{0}/{1}?Username={2}&Password={3}", ConvertHost, outputFileName, Username, Password);

            // Calculate upload data size

            string szFileSizeData = string.Format("{0}", inputFileBytes.Length + 50000);
            string szFileHdr      = string.Format(_szFileHdrFmt, inputFileName);

            ASCIIEncoding ascii = new ASCIIEncoding(); // At this time, file names must be ascii
            List<byte> header = new List<byte>(); 
            List<byte> footer = new List<byte>(); 

            header.AddRange(ascii.GetBytes(_szBoundary2));      // MAX_FILE_SIZE field
            header.AddRange(ascii.GetBytes(_szFileSizeHdr));
            header.AddRange(ascii.GetBytes(szFileSizeData));
            header.AddRange(ascii.GetBytes(_szBoundary2));      // userfile field
            header.AddRange(ascii.GetBytes(szFileHdr));

            footer.AddRange(ascii.GetBytes(_szBoundary3));
            
            int cbContent = header.Count + inputFileBytes.Length + footer.Count;

            int attempts = 0;

            while (true)
            {
                try
                {
                    attempts++;

                    HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
                    webRequest.Method        = "POST";
                    webRequest.ContentType   = string.Format("multipart/form-data; boundary={0}\r\n", _szBoundary);
                    webRequest.ContentLength = cbContent;

                    using (Stream request = webRequest.GetRequestStream())
                    {
                        request.Write(header.ToArray(), 0, header.Count);
                        request.Write(inputFileBytes, 0, inputFileBytes.Length);
                        request.Write(footer.ToArray(), 0, footer.Count);
                    }

                    using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
                    {
                        using (BinaryReader reader = new BinaryReader(webResponse.GetResponseStream()))
                        {
                            byte[] pdfBytes = reader.ReadBytes((int)webResponse.ContentLength);
                            return pdfBytes;
                        }
                    }
                }
                catch
                {
                    if (attempts >= MaxConvertAttempts)
                        throw;
                }
            }
        }
    }

I didn’t see an easy way to build a MIME document in .Net, so I’m building up the MIME document (used in the HTTP Post) from scratch.

To debug the Windows Service, as shown in the code above, I added code to log to the Windows Event Viewer. In this case, I started with working document conversion code, so I didn’t end up needing to run the service under a debugger. But, one of the nice things about services running as a dedicated process (unlike Apache modules, ISAPI modules and Control Panel applets) is that, with just a simple change (adding a Main function), you can run the service as a regular command line application and attach the Visual Studio debugger to the running process.

To troubleshoot the Web Service, my first step was to disable “friendly errors”. With IIS7, you have to do it on both the web browser client and in IIS7. Debugging is just a matter of attaching the IIS7 process in Visual Studio (as usual for any web application).

All our production Windows servers use the 64 bit edition of Windows 2008 Server. Right now, I happen to do most of my development on a 32 bit edition of Windows Vista. Both platforms run IIS7 and usually, the 32 vs. 64 bit doesn’t cause any problems. However, the Windows version of OpenOffice only comes in a 32 bit version. This version runs fine on 64 bit Windows, but when programming OpenOffice via .Net, you need a 32 bit application, so you have to set the Target Platform in the Project Build Properties to X86.

http://www.lotterygold.com/ This was an interesting and very large project that recently went live. The application we created for this site is general purpose and can support many types of e-commerce. The backend of this site is very comprehensive and highly automated. Due to its sensitive nature, the there are multiple layers of security and a dedicated audit trail facility.

http://www.mountainhighoutfitters.com/ This site is based on our very comprehensive and sophisticated online store engine. Our client’s goal is to compete head on with the likes of rei.com (who probably have spent hundreds of thousands if not millions of dollars on their online store). We have a very full featured marketing capabilities, integrated product catalog management (with lots of JavaScript to make it friendly and easy to use), integrated order fulfillment and automated integration with in-house store inventory. Since Google is the source of so much traffic (both paid and non-paid), we support every service that Google offers: Google Base (a.k.a. Google Products and Froogle), Google Site Maps, Google Adwords, Google Checkout and Google Analytics. We have some interesting payment solutions for vendors that want to sell to non-traditional markets (overseas). All our web applications support our standard skin and content management system (see below), so just about any look and feel can be created without any programming or database changes. There is a lot of power and flexibility under the hood that is probably best demonstrated in a meeting.

http://www.hudhomeschattanooga.com/ This is a standard ListingsTech site to market HUD homes in the metro Chattanooga, TN area. One interesting thing about this project is that we allow visitors to search across subsets of two different listings databases. We needed to do this because the databases contain all HUD listings for Georgia and Tennessee, but the site is only for Chattanooga. Another interesting idea here is the general approach of keeping the site as simple as possible – we are trying to get visitors to the information or service they want in as few clicks as possible… Want to see listings for Walker County? Click the search term on the left. Need to get prequalified for a loan? Click the link on the top of the page and fill out the form. Need more information about a listing? Click the “Contact Us for More Info” link and type your question.  

Getting a Deal 

FIrst, let’s look at how to get the great deal everybody wants: Retail stores know you want a $400 laptop, so they use this item as a loss leader. If you watch the newspapers around popular days for big sales (back to school tax holidays, July 4th, Black Friday, etc), you will often find impossible deals advertised. These really are great deals if you are willing and able to jump through the required hoops. Be aware that there are more people looking for the deal then there are systems available at the sweet price. In fact, the stores are counting on this… they sell 4 systems at the loss leader price and 100 systems at the “sale” price that they can make money at. 

If you want the best deal, I highly recommend that you watch the newspaper advertisements. You’ll find prices you can’t even get online.  Also, check out this link: http://www.notebookreview.com/deals/. There are probably other lists of laptop deals – I don’t buy them very often – this is just one I happen to know of.

Since it is easy, it is always worth checking out the deals available online. Two online stores I check a lot are NewEgg and Buy.com. Amazon.com also has good deals from time to time, but I hate their search interface.

Then, there are just the random deals you have to be on the lookout for. Here is an example: I wanted to buy my two older kids notebooks for Christmas to take to school. I did the usual initial searching around. I wasn’t willing to get in line for black Friday sales. One day, I just happened to pull up microcenter.com and do a search. I found an impossible deal – budget Lenovo 15″ for $400. I printed the price and went to the store. The price on the display was like $700, but I asked. They guy said the $400 deal was for real and sold me two at that price. I am sure I could have sold them on ebay for $600! Crazy.

There are ultra budget systems available that normally cost around $400. A typical example is the Asus Eee PC. Though they are interesting and competition is always good, I personally would not want one of these notebooks… the keyboards are bad and the screens are too small. Most people I know want a Windows OS. Windows XP was a solid notebook OS. Everybody knows how to use it. I am starting to wonder if Ubuntu (a popular Linux distribution) is not a better choice than Windows Vista, though.

Choosing a Model

All the the name brands (Dell, Sony, NEC, HP, etc.) are going to have about the same reliability. Sony and Lenovo’s (formerly IBM) ThinkPad  are the premium laptops. If you are going to use the built-in keyboard a lot, consider the feel of the keyboard. I think the Lenovo’s keyboards are the best. I have a 13″, 4 pound Sony. I like the small form factor, but if I were to buy again, I’d look for a similar form factor in Lenovo because I really don’t like the Sony keyboard and I’ve now ended up using mine enough to where it has become annoying. The Lenovos also have a keyboard light… this is very useful and I wish my Sony had one.

I have a customer that bought 2 state of the art 13″ 3 pond Sonys. They are similar to mine but have carbon fiber cases and hybrid hard drives (normal hard drive plus some flash). It turns out that the hybrid hard drives aren’t any faster or better for batteries than normal hard drives… also, these 2 notebooks have already been replaced twice each (one is in the shop for a 3rd time) for failed hard drives. The problem with the hard drives is probably just a coincidence, but, overall, I would not opt for the hybrid hard drive if I had a choice.

Your notebook will come with Vista. Technically, OEMs can’t sell Vista after July 1. Too bad because Vista is a poor match for the slower components (RAM, hard drives) used in notebooks. On some recent Vista notebooks I have setup for a client, I had to uninstall Norton Anti-Virus just to make them usable. With Vista, you’ll want at least 1GB RAM. A 7200 RPM hard drive would be good, too, but these are hard to find.

Backups

The new high capacity hard drives are the major weakness of all modern notebooks. Your notebook’s hard drive will fail! Unlike failures with desktop hard drives, the chances of a total hard drive failure (where only a backup recovery service can get the data off for a cost of $500 and a week wait) are very high. Don’t lose your family pictures! You definitely need a backup solution. I recommend something you can set and forget. The home version of http://mozy.com/ is pretty good and the 2GB account is free – OK for My Documents (say for a computer to take back and forth to school), but not big enough for a large Outlook archive or a big collection of pictures. I am still searching for a really good and free or inexpensive set and forget backup solution.

Antivirus

I put AVG on my kid’s notebooks. The free version is very popular and gets OK reviews, but there are a lot of complaints about the resources required by latest version (8.0). Also, I noticed that at least on one of the notebooks, it had not been running automatic updates… Really, this kind of software must be set and forget.

Office Suite

You can save some money with the free Open Office. It really is a good program. If you configure it to save as .doc and .xls by default, most people will barely notice they aren’t using Microsoft Office. The only problem comes when somebody sends an Office 2007 docx or xlsx XML format file (the default “open” file format for Office 2007). Now that the competitors have mastered .doc and .xls, Microsoft had to change things up to keep customers coming back!

Eventually, my credit card processor teamed up with one of the first companies to offer an Internet credit card processing gateway, Cybercash. I hopped on board as one of the first customers. As with a lot of APIs created by hurried teams of inexperienced programmers, Cybercash was much more complex than was necessary to accomplish the simple task at hand. Credit card processing can be really simple: Pass the credit card processor a list of required fields and get back an Boolean success code and an error message.  HTTPS is a fine protocol for this type of stateless transaction that must occur over a secure channel. Input fields can be passed as a URL encoded query string. The error code and error message could be returned an HTTP response code or via a URL encoded fields in the response body. The modern, fancy name for this type of transaction protocol is REST. Under the hood of its overly complex API, Cybercash used the simple REST protocol I described earlier. Liked a lot of other Cybercash programmers, I eventually figured this out on my own and bypassed the complex and cumbersome API to use the simple REST protocol.

Time passed on and Verisign bought up all kinds of important technology related to e-commerce. They purchased Cybercash and started promoting their own gateway API, PayflowPro. Several times, we researched the fees and APIs offered by the various vendors. PayflowPro was the best and the one we used and recommended. The PayflowPro API was a tad bit easier to use than the official Cybercash API – easy enough that instead of bypassing the API for the REST protocol, I used the actual PayflowPro API function calls.

When the time came to automate the billing for T3city, I started out with PayflowPro. However when it came time to get a new merchant account, I took some time to review my options. On paper, PayPal Website Payments Pro looked pretty good – they have a nice and simple fee structure (unlike PayflowPro with all sorts of fees billed out at different times) and the fees are competitive if you can get your volume over $3,000/month (and not too bad if you can’t). The sign-up process was all online. I figured it couldn’t be too hard to change gateway APIs – I’d already used several others – so I made the switch.

Well, I was wrong about the difficulty level associated with switching from PayflowPro to PayPal. The PayPal API is much more complex. First, there is the fact that the API is SOAP based. In addition to the more complex SOAP protocol, the PayPal programming examples use another layer on top of the native SOAP API. Why they add this extra layer, I really don’t know – maybe it’s just a fact that every junior programmer feels like an extra layer or two will make things simpler. It doesn’t make the API any easier to use. In fact, I found the differences between the new layer and the documentation for the base SOAP protocol very confusing. In the end, I elected to ditch the PayPal examples and use PayPalSvc.wsdl and .Net’s SOAP support tools to generate my own SOAP client for the PayPal API. The resulting API is a little awkward is spots (mostly due to PalPals not so hot API design), but at least it matches the detailed and fairly comprehensive PayPal SOAP protocol documentation.

Once you get over the base complexity of the PayPal SOAP based API, then you have to deal with the stateful nature of PayPal. Credit card processing is normally a simple transaction, but the PayPal Website Payments Pro program requires that you offer your customers the option to pay via PayPal. And, what the heck, allowing payments by PayPal is not such a bad idea.  However, unlike a simple credit card transaction, payments with PayPal naturally flow over a series of pages, some on your site and some on PayPal’s site. Information such as billing and shipping addresses and invoice line items have to be exchanged in a delicate dance. Assuming you are willing to tackle the complexity of proper error handling, avoiding browser complaints about HTTPS redirects and giving the user the ability to cancel cleanly, getting PayPal transactions to flow smoothly turns out to be a fairly complex.

Is the extra complexity worth it? Consider these points:

• Both gateways are very reliable. 
• PayPal fee structure is simpler, clearly specified on their web site and may be less expensive – YMMV. Traditional store fronts may be able to get better rates from their current processors with a different gateway API.
• PayPal charges the same fees for all types of credit cards. AMEX fees are usually higher, so this may save you some money, but, again, YMMV.
• PayPal does not require any type of term or volume commitment
• Accepting PayPal is nice – giving customers more ways to pay you is a good thing!
• Implementing PayPal Website Payments Pro is a good deal harder than using PayflowPro. Depending on your web site/application, adding stateful credit card / PayPal transaction processing may require adding new database fields, new screens and other large scale programming changes.
• Under .Net, PayPal Website Payments Pro does not require any native (non .Net) code. To use PayflowPro in .Net, I load and make native function calls into the native PayflowPro DLL.
• PayPal Website Payments Pro requires full names for states (not the typical 2 letter codes) and proper ISO country codes for every transaction. If you don’t already have validation in place and tables to translate state codes into state names (users will use 2 letter state codes if you let them), this support will require additional programming work.
• With PayPal Website Payments Pro, you have to perform an additional step to transfer money from your PayPal account into your bank account.  Funds from credit card purchases show up in your PayPal account immediately, but transfers to your bank account take multiple business days (similar to the delay you’ll have with a traditional merchant service provider and PayflowPro). In theory, you can use funds in your PayPal account immediately for purchases via PayPal, but you would need a careful strategy to ensure proper accounting.
• To credit a customer via PayPal, you must create a new credit transaction that references a special code from the original sale transaction. Unlike PayflowPro, you cannot void a transaction and you cannot change a sale transaction to a credit transaction.
• PayPal has different and likely much more strict fraud protection measures. Overall, I like this capability, but it did cause some pain during our switch when credit cards on file from long term customers no longer worked because of mismatched a CVV number, a mistyped expiration date, etc..

I can understand why they wanted a bit of money for the work that was required for validation that first year, but overall, SSL certificates have long been overpriced for the value they provide. After that first validation, the next year’s renewal costs the CA practically nothing, but they used to give no renewal discounts at all and, even now, renewal discounts don’t exist and multi-year discounts are not as substantial as they could (should?) be.

Then there were (are) silly SSL cert upgrades that supposedly provided stronger encryption. Well, I suppose those upgrades actually could enhance encryption if you just happened to be running an old, obscure version of IE that was only available outside the US, only for a short time and has not been available since early 2000. Funny how even today, you’ll find that even the biggest CA charges extra for Server Gated Cryptography, even though no browser modern enough to be secure needs or supports it.

If you want to spend even more on your SSL cert, CAs will happily add on various types of hyper-specific insurance policies and all manner of “site seals” and  “trust logos”.

The entire verification and trust thing is just silly. The percentage of Internet users that would recognize Verisign, Thawte, Comodo or any other CA is vanishingly small. Even if we were to assume your average Internet buyer were a sophisticated, educated, rational consumer, why would they trust some company they’ve never heard of to tell them how trustworthy amazon.com is? On top of all this, given the year after year rape attempts committed by big CAs with their over pricing of renewals, fake SGC upgrades and other kinds of fake “strong cryptography” upgrades, the only thing I personally trust CAs to do is to make as much money as they possibly can with any means at their disposal.

With the useless state of verification and trust, it’s no wonder that some smaller CAs eventually started verifying less and charging a lot less. Now days, you can buy an SSL certificate that certifies nothing other that you are using SSL. Fair enough – even that tidbit is more than most consumers are interested in knowing. Practically speaking, SSL is only a technology for the vendor. Vendors should use SSL properly because they care about the consumer enough to not transmit their personal information in the clear over the Internet.  Frankly, if my wife, my Mom or one of my kids finds something they want to buy, as long as the browser don’t completely refuse to accept the connection, they will be happy to click through all manner of browser warnings to put in a credit card number. Haven’t we all been trained to ignore these peskey warning messages by now? A few shoppers might be consoled by a a happy, friendly padlock icon, but how many users are fully aware that the pad-lock icon is supposed to be in the browser’s status bar. “What’s a status bar”, you ask?

Given all this, I’ve been using the least expensive SSL certs I can find. Here are a couple of different examples – can you tell what kind of validation was used?

• https://www.searchenginecommando.com/order/  (cheap)
• https://www.t3city.com/ (cheaper)
• https://www.embracegroup.com/index2.html  (cheapest)

I suppose the SSL CAs all got together and decided something just had to be done before everybody started using self-signed SSL certificates. Enter now the “Green Address Bar” SSL Certificate. The real name is the “Extended Validation”  SSL Certificate, but my name would be better. At least with my name, there is a slight chance that consumers (of SSL certs) will notice.  If you have one of these super-duper certificates, IE’s address bar is supposed to turn green (as in the color of money, eh?). I think this is a Vista only feature, though.

The SSL vendors want $500 and more for these “EV” SSL certs. Personally, I think they are pricing themselves out of the market. Yes, a handful of sites like amazon.com will pay the extra $489 for the SSL cert that makes the browser’s address bar turn green (as in the color of envy?). I have to guess, though, that the vast majority of SSL certs are purchased for small e-commerce sites like searchenginecommando.com, t3city.com and embracegroup.com. These small operators will rightly assume that public will pay no attention to the fact that the address bar is not green. Personally, I doubt if many people really even look for the SSL lock icon anymore. If EV were just a $50 up-charge, then a lot of small shops might would go ahead and get it (at least for the first year until they find out it didn’t effect sales). Right now, since the new certs are so expensive, practically nobody will buy them and buyers will just forget all about the green address bar. If they do notice, they’ll probably just have a vague notion that something wrong with their computer (again). “Wasn’t Vista supposed to fix these kinds of problems?” they’ll wonder.

In modern times, Windows 2000 and Windows 2003 closed the gap for fast and reliable web hosting, but Linux still makes a better email server for ISP type email hosting. The Linux/UNIX security model is simpler than Windows while the Linux process model is slightly more featured. It turns out that these qualities make Linux a good fit for web and email hosting. You really don’t need complex ACL based permissions for web hosting. The simpler Linux security model works and because it’s simpler, it is easier to get right.  The Linux lightweight process model (including support for fork with copy on write) is a good match for virtual web hosting – think about all the perl CGI scripts out there. The lightweight process model is also handy for writing email servers. None of the nice email software I use for hosted email is even available for Windows. It’s not that it would be ultra-hard to port these packages to Windows. It’s just painful and really, why bother? DNS hosting is a wash. The only package I like is PowerDNS and it runs on both Linux and Windows.

PHP runs fine on Windows under ISS, but really, why bother? I run Asp.Net on Windows under IIS. It is possible to run .Net applications under Mono. But, from a practical standpoint, Mono still doesn’t fully support Asp.Net 2.0 (though it’s really close now) and Asp.Net 2.0 is so much better than .Net 1.1, I don’t bother with 1.1 for any of my own stuff. Overall, running Asp.Net on Linux seems like a lot of struggling against the machine.

When it comes to developer tools, nothing comes close to Microsoft. I can program in vi, emacs or even notepad if I have to, but Microsoft programming tools are better. The IDEs are faster and more integrated. The debuggers are better. The compilers and linkers are faster. My biggest complaint with Linux programming tools is lack of performance. I need a fast compile/link/debug cycle. For anything bigger than a single file project, I can’t seem to get it on Linux. Last time I had a serious project on Linux, I took the time to try out lots of different IDEs. As I would have guessed, Kdevelop was the best – it’s beautiful and has tons of features, but I still found it slow and awkward for day-to-day use. I figured out how to use “automake and friends”. Talk about weird (and slow). I do like Subversion. I’m already using it for all my cross-platform development projects and I’m thinking about converting everything else over. With Microsoft tools, debugging Asp.Net applications is just like debugging a GUI application. This alone justifies developing all my web application in Asp.Net and hosting them on Windows.

It’s been more than a couple of years since my last serious desktop GUI application. At that time, Windows and MFC were the obvious platform. If I had to run the application on Linux, I would have tried to get something going with Winelib, but MFC support under Winelib has always looked like a PITA. For new development, I would consider using Qt or wxWigits. My big complaint about both these libraries is the deliberate lack of support for exceptions – talk about stuck in the dark ages. I’ve done a little bit of WinForms programming. Overall, I think the library is pretty good (and with Mono, it could be cross-platform). My worry is the distribution requirements (though Windows 9X, ME and 2000 are finally starting to fade away) and the hefty resource requirements.

The web UI is a little clunky, but overall, it’s better than most other switch configuration UI’s I’ve used in the past. I still prefer a script based configuration, but you don’t get that in these lower cost “web managed” switches.

The switch’s rules for VLANs are a bit weird. For example, I don’t like the “default VLAN” concept they have implemented. Also, you can’t mix tagged and untagged traffic on a single port (as you can with most hosts and many other switches). Really, VLANs are simple. A packet can be VLAN tagged or untagged. A switch port can be granted access to 1 or more VLANS. For each VLAN for which a switch port has been granted access, the port can either tag or not tag outgoing traffic. That’s it. I can see why the default configuration of a switch should be to grant very port untagged access to VLAN 1, but that doesn’t imply to me that there needs to be some kind of “default” VLAN for each port.

The switch’s management IP does not have to be on VLAN 1. This is a nice touch that I appreciate.

The statistics in the web GUI are pretty sparse. You can view basic interface counters for each port. It would be nice to get some idea of the bandwidth in use on each port. The 5 minute input and output rates on Cisco’s “show interfaces” output is a good example of what is useful.

SNMP support is also basic, but I had no problem configuring Cacti to create traffic graphs for each port. That is all I need.

Jumbo frame support is either on or off. Most switches let you set this per port. Setting it per VLAN would make the most sense to me. But, unless you could program the switch to automatically fragment packets for hosts that don’t support jumbo frames (and what switch does this?), a global switch is fine. Ultimately, you have to deal with the individual hosts on each VLAN – support for jumbo frames on the switch is just a simple yes or no (and can probably always be set to yes as long as the switch itself does not generate any jumbo packets).

The switch supports trunking. That is fairly common for switches that support VLANs. I haven’t tried out trunking. A GigE pipe is big enough for me right now.

You can have up to two ports configured to monitor (sniff) traffic on other ports. I haven’t tried this out yet, but I’m sure it will come in handy when one of our beloved unmentionable security agencies hits me with a court order for eavesdropping.

A couple of weeks ago, Gary started having more problems with his guestbook. It was slow and, more importantly, it wasn’t taking new posts. After some digging around, I found out that it was slow because PYG stored all the entries in a simple text file. THis is OK (and maybe ideal) for a guestbook with a few posts. Gary’s guestbook is active and covers several years. Also, there were still a fair number of spam posts that were not visible (Gary’s guestbook is moderated), but still in the text file. Altogether, Gary has over 3000 posts in his guestbook. The combination of simple text file storage with some inefficient coding in PYG and 3000+ posts was resulting in slow page generation. Gary allows HTML in posts. This is mostly so that this guests can post links to pictures, other concerts and videos (on Youtube, etc.). Allowing HTML can cause problems, but, because his guestbook is moderated, the problems should be correctable. Unfortunately, posts with invalid HTML tripped up the PYG administration interface. So, I started looking for a new guestbook script. PYG doesn’t have a version that supports MySQL. I wanted something that:

• Used PHP 4.x
• Used MySQL
• Used templates to create the Guestbook pages
• Allowed the guestbook to be moderated

Oddly enough, I couldn’t find anything that met these objectives. I searched Google and SourceForge. I came across tons of Guestbook projects. Some required PHP 5. Others didn’t use templates. I found one that used PHP 4 and templates but didn’t support moderation. So, I ended up writing one from scratch. I’ve already got one PHP project on SourceForge, phpHitCounter. phpGuestbook is already taken, so I’m thinking phpMysqlGuestbook will be the name. Here are the highlights:

• Simple configuration (config.php) and installation (edit config.php and run .sql script to create database).
• Excellent error messages and error handing for end users ((JavaScript and in page error reporting).
• Extra feedback for end users via a special page that explains post moderation.
• Fast post display for both end users and administrators
• Emails sent to the moderator include a link to the post to be reviewed
• Simplified administration
• Simple template based customization of HTML output. Templates are valid HTML so they can be maintained in FrontPage, Dreamweaver, etc.
• Option to allow HTML content in end user posts. Careful handling of user supplied HTML content in administration interface.
• Comprehensive support for banned IPs and banned/moderated content

This script is a nice companion to phpHitCounter.