Recently, I upgraded my development system from OpenOffice 2.x to OpenOffice 3.1. I can’t remember now the main reason I upgraded, but I was looking forward to being able to add the ability to convert docx to pdf (Office Open XML support was added in OpenOffice 3.x). I figured the upgrade might require some minor changes to my document conversion code, but it turned out not to be so simple.

The first problem was that OpenOffice no longer showed up under “Add Reference” in Visual Studio. After some Google searches, I finally stumbled across a solution. OK, wow, digging though install cab files for DLLs, hacking the registry and manipulating the PATH is ugly, but it worked – at least once. Almost as soon as my test program ran successfully, Visual Studio locked up. The only way to get control back was to kill the Visual Studio process, then restart it. Every time I opened the solution file, after just a few seconds, Visual Studio would lock up again. Removing every trace of the OpenOffice DLL references would prevent the lockups, but without OpenOffice, I could not convert documents. I eventually figured out that the problem was that the library project that used OpenOffice was also included in my web site project. Because the web site uses the library, too, the OpenOffice reference is automatically copied to the web site project. Normally, this would be no big deal – just a couple of small DLL files in the web site bin folder that would not be used – but in this case, Visual Studio could not successfully build the web site project as long as the OpenOffice DLLs were referenced.

Now that using OpenOffice was no longer as simple as “Add Reference”, it was time to take a look at a way to decouple my application from the OpenOffice DLLs. The API my application needs is pretty simple: Given the bytes of a file in Doc (or .docx) format, return the bytes of the file in Pdf format. Also, there are security risks whenever you open a file in a complicated piece of software like OpenOffice. These two facts pointed me to turning the Pdf conversion into a web service. The client application and web service can be partitioned across computers, so, if I wanted to, I could run the Pdf conversion service on a virtual machine with its own install of OpenOffice and tight security and resource control. The web service API didn’t need fancy XML: Clients could post the Doc file via a standard HTTP MIME encoded form. The result would be a the PDF byte stream returned via HTTP. This would make the service easy to develop, test and use from clients on many platforms.

I typically develop my large web based applications in C# and Asp.Net. However, for small services like document conversion, I often use PHP. Simple PHP applications can be just a file or two and I can move a PHP based service over to any web server that supports PHP. So, my first attempt at a document conversion service was written in PHP and used the OpenOffice COM API. Using my earlier OpenOffice API code as a guide and some PHP sample code I found on the web, I was able to quickly create a command line PHP application that could perform Pdf conversions. With this proof of concept working, I coded up the PHP web service. But the web service wouldn’t work. I could see in Task Manager that the OpenOffice executables were launched, but any code running under the web server that tried to access the OpenOffice COM objects would freeze up.

I spent a lot of time trying different security settings for various files and DCOM. I even tried creating a dedicated user and configuring DCOM to run OpenOffice as the dedicated user when the OpenOffice COM objects were activated. I changed Windows event log settings to log all security violations. Nothing would work and I couldn’t see anything useful in Event Viewer. I tried coding up a simple Asp.Net application and got the same results under the .Net OpenOffice API. Under both PHP/COM and C#/.Net, OpenOffice could be automated by a command line program but would not run under IIS. Right now, I develop on Vista and deploy to Windows Server 2008, so one might think the problem was trying to use IIS 7, but OpenOffice 2 didn’t have similar problems.

Finally, I gave up trying to run OpenOffice directly under IIS. The problem was either some type of security issue or that OpenOffice was trying to pop up registration dialogs. In Windows, the simple way to run a program under a given user account is to run the program as a Windows Service. It turns out that it is pretty easy to create a Windows Service in C#. The basic idea I came up is was:

1. Install OpenOffice
2. Create a Windows user account dedicated to Pdf conversion.
3. Login as the Pdf conversion user and make sure that OpenOffice opens without prompting for registration, crash feedback, etc. (I suspect that the reason OpenOffice 3 won’t run properly under IIS is that it is prompting the user to register).
4. Create a Windows Service with a simple API to convert documents
5. Install the Windows Service and configure it to run as Pdf conversion user
6. Access the Windows Service from my document conversion Web Service

Here is how the API calls flow: Application -> Web Service -> Windows Service

As I mentioned earlier, the API between the application and the web service was standard a HTTP Post. Between the Web Service and the Windows Service, I originally planned to use a simple, proprietary TCP/IP protocol. As long as you aren’t try to get too fancy coding thread pools and asynchronous I/O, at least in C/C++, it’s pretty easy to open a TCP socket and communicate between client and server with a line oriented protocol (like HTTP, POP3, SMTP, etc.). But, since both the Web Service and the Windows Service were going to run on Windows (at least for now), I decided to take a look at .Net Remoting. The .Net remoting solution was easy to code and use so I ended up sticking with it.

Below is some code to show you how everything fits together:

The C# interface IConverter is compiled into a DLL. The DLL is referenced by the client (Web Service) and server (Windows Service).

    public class ConverterEndpoints
    {
        public static int    Port = 7047;
        public static string Uri  = "DocumentToPdf";
    }

    public interface IConverter
    {
        void Convert(string srcDocument, string destPdf);
    }

The class ConverterEndpoints holds a couple of constants that the client and server use to setup the TCP port and service name. The code below demonstrates how ConverterEndpoints is used in the Windows Service. I’ve included code for the entire class so you can see how easy it is to program a Windows Service and access the Windows Event Log in C# – no 3rd party libraries are required.

The original version of this interface passed the files as simple byte[] variables. Theoretically, the files never need to be written to disk (input and output is via HTTP). But, I found that I could easily pass small byte[] variables across .Net remoting but large byte[] variables failed. Since the Web Service and the Windows Service run on the same machine and the file has to be written to disk for OpenOffice to do the conversion, I settled for creating temporary files and passing the names of the files between the Web Service and Windows Service.

   class WindowsService : ServiceBase
    {
        protected EventLog _log;
        protected TcpChannel _channel;

        public WindowsService()
        {
            ServiceName = "Teztech Document Conversion";
            EventLog.Source = "Teztech Document Conversion";
            EventLog.Log = "Application";
            
            // These Flags set whether or not to handle that specific
            //  type of event. Set to true if you need it, false otherwise.
            CanHandlePowerEvent = false;
            CanHandleSessionChangeEvent = false;
            CanPauseAndContinue = true;
            CanShutdown = true;
            CanStop = true;

            if (!EventLog.SourceExists("Teztech Document Conversion"))
                EventLog.CreateEventSource("Teztech Document Conversion", "Application");

            _log = new EventLog();
            _log.Source = "Teztech Document Conversion";

            _channel = new TcpChannel(ConvertInterface.ConverterEndpoints.Port);
            ChannelServices.RegisterChannel(_channel, false);

            RemotingConfiguration.RegisterWellKnownServiceType(typeof(DocumentToPdf), ConvertInterface.ConverterEndpoints.Uri, WellKnownObjectMode.Singleton);
        }

        static void Main()
        {
            ServiceBase.Run(new WindowsService());
        }

        /// <summary>
        /// Dispose of objects that need it here.
        /// </summary>
        /// <param name="disposing">Whether or not disposing is going on.</param>
        protected override void Dispose(bool disposing)
        {
            base.Dispose(disposing);
        }

        protected override void OnStart(string[] args)
        {
            _log.WriteEntry("Service Starting");

            _channel.StartListening(null);

            base.OnStart(args);

            _log.WriteEntry("Service Running");
        }

        protected override void OnStop()
        {
            _log.WriteEntry("Service Stopping");

            _channel.StopListening(null);

            base.OnStop();
        }

        protected override void OnPause()
        {
            _channel.StopListening(null);
            base.OnPause();
        }

        protected override void OnContinue()
        {
            _channel.StartListening(null);
            base.OnContinue();
        }
    }

Below is the code behind file for the Web Service client. As you can see, .Net makes this type of Web Service easy to create.

public partial class Pdf : System.Web.UI.Page
{
    protected TcpChannel _channel;

    protected void Page_Load(object sender, EventArgs e)
    {
        try
        {
            if (Request["Username"] == null || Request["Password"] == null)
                throw new ApplicationException("Username and Password are required");

            HttpPostedFile postedFile = Request.Files["InputFile"];
            if (postedFile == null || postedFile.FileName == null || postedFile.FileName == "")
                throw new ApplicationException("InputFile was not supplied or file name is empty.");

            string inputFileExtension = Path.GetExtension(postedFile.FileName);
            if (inputFileExtension.Length < 2)
                throw new ApplicationException("InputFile does not have a valid file name extension.");
            inputFileExtension = inputFileExtension.Substring(1);

            string outputFileName = Request["OutputFileName"];
            if (outputFileName == null || outputFileName == "")
                outputFileName = Path.GetFileNameWithoutExtension(postedFile.FileName) + ".pdf";

            DbRequest.CheckAuthorization(Request["Username"], Request["Password"]);

            if (_channel == null)
            {
                Dictionary<string, string> channelProperties = new Dictionary<string,string>();
                channelProperties["name"] = "";
                _channel = new TcpChannel(channelProperties, null, null);
                ChannelServices.RegisterChannel(_channel, false);
            }

            string url = string.Format("tcp://localhost:{0}/{1}", ConvertInterface.ConverterEndpoints.Port, ConvertInterface.ConverterEndpoints.Uri);

            ConvertInterface.IConverter converter = (ConvertInterface.IConverter)Activator.GetObject(typeof(ConvertInterface.IConverter), url);

            string path = MapPath(".");
            path = Path.Combine(Path.GetDirectoryName(path), "Temp");

            using (TempFileCollection tempFiles = new TempFileCollection(path))
            {
                string inputFile  = tempFiles.AddExtension(inputFileExtension);
                string outputFile = tempFiles.AddExtension("pdf");

                postedFile.SaveAs(inputFile);
                converter.Convert(inputFile, outputFile);

                Response.Clear();
                Response.ContentType = "application/pdf";
                Response.AddHeader("content-disposition", string.Format("inline; filename={0}.pdf", "sample"));

                byte[] pdfBytes = File.ReadAllBytes(outputFile);

                Response.OutputStream.Write(pdfBytes, 0, pdfBytes.Length);
            }
        }        
        catch (Exception ex)
        {
            Response.Clear();
            Response.Write(string.Format("<HTML><HEAD></HEAD><BODY><H1>Conversion Error</H1><P>{0}</P></BODY></HTML>", ex.Message));
            Response.StatusCode = 500;
            Response.StatusDescription = ex.Message;
        }
    }

For completeness, I have included the code for the actual Web Service’s .aspx file below. As you can see, it’s just standard .aspx. boilerplate – all HTTP output is generated in the code behind file’s Page_Load event (which will be invoked when the Web Service client posts to the page). This is quick and dirty, but it works.

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="Pdf.aspx.cs" Inherits="Pdf" EnableViewState="false" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
    <title>Untitled Page</title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
    
    </div>
    </form>
</body>
</html>

Here And here is the code for a little PHP application I used to test my conversion Web Service:

<?php $ErrorMessage = ''; ?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>
File Conversion Server Demo
</title>
</head>
<body>

<h1>File Conversion Server Demo</h1>

<form name="Demo" method="POST" enctype="multipart/form-data" action="/demo.pdf">

    <?php if($ErrorMessage): ?>
    <p><font size="5" color="#008000"><?php echo $ErrorMessage; ?></font></p>
    <?php endif; ?>
    
    <table>
        <tr>
            <td><b>Username:</b></td>
            <td><input name="Username" value=""></input></td>
        </tr>
        <tr>
            <td><b>Password:</b></td>
            <td><input name="Password" value=""></input></td>
        </tr>
        <tr>
            <td><b>Input File:</b></td>
            <td><input type="file" name="InputFile"></input> </td>
        </tr>
    </table>
    
    <input type="submit" value="Submit"></input> 
    
</html>

If you look the PHP form’s action tag, you’ll see I used /demo.pdf as the action. Using the very nice IIS7 URL Rewrite Module, I map all *.pdf URLs to my conversion Web Service. OK, this might be a little overkill, but this way the URLs look nice and all end in .pdf (which is appropriate).

The final piece of the puzzle is a small wrapper class I created to access the conversion Web Service from within my C# application:

    public class ConvertClient
    {
        public ConvertClient(string convertHost, string username, string password)
        {
            _ConvertHost = convertHost;
            _Username    = username;
            _Password    = password;
        }

        protected string _ConvertHost;
        public string ConvertHost { get { return _ConvertHost; } set { _ConvertHost = value; } }

        protected string _Username;
        public string Username { get { return _Username; } set { _Username = value; } }

        protected string _Password;
        public string Password { get { return _Password; } set { _Password = value; } }

        protected string _Response;
        public string Response { get { return _Response; } set { _Response = value; } }

        protected int _MaxConvertAttempts = 2;
        public int MaxConvertAttempts { get { return _MaxConvertAttempts; } set { _MaxConvertAttempts = value; } }

        protected string _ResponseCode;
        public string ResponseCode { get { return _ResponseCode; } set { _ResponseCode = value; } }

        protected static string _szBoundary    = "SEPARATORSTRINGTEZTECHDOTCOM1";
        protected static string _szBoundary2   = "\r\n--SEPARATORSTRINGTEZTECHDOTCOM1\r\n";
        protected static string _szBoundary3   = "\r\n--SEPARATORSTRINGTEZTECHDOTCOM1--";
        protected static string _szFileSizeHdr = "Content-Disposition: form-data; name=\"MAX_FILE_SIZE\"\r\n\r\n";
        protected static string _szFileHdrFmt  = "Content-Disposition: form-data; name=\"InputFile\"; filename=\"{0}\"\r\nContent-Type: application/octet-stream\r\n\r\n";

        public byte[] ConvertToPdf(byte[] inputFileBytes, string inputFileName, string outputFileName)
        {
            string url = string.Format("http://{0}/{1}?Username={2}&Password={3}", ConvertHost, outputFileName, Username, Password);

            // Calculate upload data size

            string szFileSizeData = string.Format("{0}", inputFileBytes.Length + 50000);
            string szFileHdr      = string.Format(_szFileHdrFmt, inputFileName);

            ASCIIEncoding ascii = new ASCIIEncoding(); // At this time, file names must be ascii
            List<byte> header = new List<byte>(); 
            List<byte> footer = new List<byte>(); 

            header.AddRange(ascii.GetBytes(_szBoundary2));      // MAX_FILE_SIZE field
            header.AddRange(ascii.GetBytes(_szFileSizeHdr));
            header.AddRange(ascii.GetBytes(szFileSizeData));
            header.AddRange(ascii.GetBytes(_szBoundary2));      // userfile field
            header.AddRange(ascii.GetBytes(szFileHdr));

            footer.AddRange(ascii.GetBytes(_szBoundary3));
            
            int cbContent = header.Count + inputFileBytes.Length + footer.Count;

            int attempts = 0;

            while (true)
            {
                try
                {
                    attempts++;

                    HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
                    webRequest.Method        = "POST";
                    webRequest.ContentType   = string.Format("multipart/form-data; boundary={0}\r\n", _szBoundary);
                    webRequest.ContentLength = cbContent;

                    using (Stream request = webRequest.GetRequestStream())
                    {
                        request.Write(header.ToArray(), 0, header.Count);
                        request.Write(inputFileBytes, 0, inputFileBytes.Length);
                        request.Write(footer.ToArray(), 0, footer.Count);
                    }

                    using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
                    {
                        using (BinaryReader reader = new BinaryReader(webResponse.GetResponseStream()))
                        {
                            byte[] pdfBytes = reader.ReadBytes((int)webResponse.ContentLength);
                            return pdfBytes;
                        }
                    }
                }
                catch
                {
                    if (attempts >= MaxConvertAttempts)
                        throw;
                }
            }
        }
    }

I didn’t see an easy way to build a MIME document in .Net, so I’m building up the MIME document (used in the HTTP Post) from scratch.

To debug the Windows Service, as shown in the code above, I added code to log to the Windows Event Viewer. In this case, I started with working document conversion code, so I didn’t end up needing to run the service under a debugger. But, one of the nice things about services running as a dedicated process (unlike Apache modules, ISAPI modules and Control Panel applets) is that, with just a simple change (adding a Main function), you can run the service as a regular command line application and attach the Visual Studio debugger to the running process.

To troubleshoot the Web Service, my first step was to disable “friendly errors”. With IIS7, you have to do it on both the web browser client and in IIS7. Debugging is just a matter of attaching the IIS7 process in Visual Studio (as usual for any web application).

All our production Windows servers use the 64 bit edition of Windows 2008 Server. Right now, I happen to do most of my development on a 32 bit edition of Windows Vista. Both platforms run IIS7 and usually, the 32 vs. 64 bit doesn’t cause any problems. However, the Windows version of OpenOffice only comes in a 32 bit version. This version runs fine on 64 bit Windows, but when programming OpenOffice via .Net, you need a 32 bit application, so you have to set the Target Platform in the Project Build Properties to X86.

For me, the IDE goes back to the days of Borland’s Turbo Pascal and Turbo C. They didn’t have GUIs much less GUI form editors, HTML wasn’t yet invented and I don’t remember the text editor being anything special. What turned them into IDEs was the super efficient work flow… A typical “Hello World” program was mostly a matter of typing in some code and pressing the Run key. If you wanted to debug, you would set your breakpoints right in the editor, then press the Debug key to start your program under the integrated debugger. When your program expanded to more than one file, IDEs had simple project facilities that did a good job of bridging the gap between “Hello World” and tools for large projects like “make” and “nmake”. Later, when the GUI APIs got so big that nobody could keep them all their head, integrated documentation became critical to a smooth workflow. For better or worse, I’ve become addicted to code completion. It doesn’t work as well for dynamic languages like PHP, but some code completion is better than none.

So, when I think IDE, I think of an all purpose application that makes a programmer’s day-to-day workflow smooth and efficient… I find that anything I can do that cuts down my “cycle” time improves my productivity (and my mood).

Every since the days of Microsoft C 6, Microsoft has always had the best IDEs for C/C++ and Windows development. Their documentation is better and better integrated, their compilers are way, way faster and the general work flow is always the smoothest. Visual Studio continues this trend for C/C++ and now C# development – it sure is easy to crank up a new C# web application and start debugging.

As as web host, I end up working on a lot of different sites. These days, there is a lot of PHP code out there and I end up spending a fair amount of time working on PHP projects. For most my really big projects, I typically use C# and Asp.Net, so I have Visual Studio handy. With a few registry tweaks to syntax highly PHP code, Visual Studio works OK as a text editor for PHP, but I wouldn’t call it a true PHP IDE. For one thing, there is no integrated debugging support. You can kind of get project file support (PHP files can be listed), but the only integrated documentation is for HTML and JavaScript (and sadly, the Microsoft documentation on these frequently referenced topics is IE centric).

So, several times, I’ve set out on a mission to find a IDE for PHP programming. Last time around, I gave up and stuck it out using Visual Studio as a text editor. After this last round of reviews, I ended up buying NuSphere’s PhpED.

When you are in the world of “free” software, it is tempting to stick only to “free” tools. However, in the case of IDEs, I just couldn’t find a free one that worked well enough to be better than Visual Studio.

One area where PHP needs a lot of work is support for debugging. The basics are there, but having to fiddle with various debugging DLLs is a hassle – debugging information has been standard for binary programs for years – how hard could it be to figure out a standard debugging API for IDEs to use?  Having to use query strings to initiate a debugging session is a serious design mistake. With all the rewriting and other things going on, fiddling with URLs just doesn’t make sense. Visual Studio is able to attach a debugger to server code in just about any situation – PHP and its IDEs should make this as easy.

The ability to automatically reformatting HTML in both blocks and full pages is very handy… plenty of tools (like content management systems) make a real mess of HTML. And, for better or worse, you end up using some of these tools over and over so it’s not like you can fix the code just once and be done with it.

Below are some notes I made as I tried out various PHP IDEs:

Eclipse PHP

No cost and open source

Cross-platform (Java)

Attractive

Amazingly slow and memory hungry – based on Java and requires a Java Runtime to be installed.

Integrated PHP reference uses the php.net web site so, it requires Internet access and is slow.

No integrated CSS, HTML, JavaScript or DOM reference

No WYSIWYG HTML editor, no integrated page preview, either

HTML reformat is weak and not customizable

Go to definition doesn’t work reliably

Code completion doesn’t work reliably

Code completion doesn’t work in CSS files.

HTML code completion uses upper case for tags and attributes.

Subversion integration is slow and shows incorrect status

Crashes frequently

Debugger worked for a simple page, but did not work in a real project

Komodo IDE

Attractive 

Cross-platform – interesting choice of Firefox engine

Reasonable performance and memory usage

Subversion integration is quick and accurate

Once configured, PHP syntax checking is very handy

No WYSIWYG HTML editor, but the integrated page preview is well done

Go to definition doesn’t work reliably

Code completion doesn’t work reliable (does not seem to follow include paths).

No integrated PHP, CSS, HTML, JavaScript, DOM or MySQL reference

Copy and paste of files in Projects tree fails with no error message

Deluxe syntax PHP + HTML syntax highlighting.

View unsaved changes off File menus is a great idea! The integration of this feature with the recovery of unsaved changes after a crash is nice, too.

Extremely annoying rendering bugs in the text editor

Code completion for CSS and HTML

CSS code completion is not very good. For example, once you type “margin:”, you get a drop list of choices instead of a description of the arguments for the margin style.

HTML code completion is annoying because it defaults to upper-case tags (is anyone still NOT using xhtml for new code?)  and does a poor job of suggesting closing tag (typing “</” does not give a drop list of closing tags).

PHPEdit

Attractive

Reasonable performance and memory usage

Subversion integration does not work out of the box.

Integrated PHP and MySQL reference is nice, no integrated CSS, HTML, JavaScript or DOM reference.

The ultra-deluxe syntax highlighting changes the highlighting rules depending on where the caret is placed – very handy for files with integrated HTML, PHP, CSS and JavaScript!.

No code reformat for any language other than PHP.

No code completion for any language other than PHP (CSS and HTML would be nice).

Go to definition (“Jump to declaration”) doesn’t work across files.

Cannot right click to open a file via a simple require() declaration.

No heroics to resolve PHP include paths for code completion.

DB Explorer is an interesting idea, but I think the most important thing about a database to a programmer is the column definitions (names, types, lengths) and, other than seeing the column names in the DB Explorer tree, there is no way see the column definitions for a table. Manually inputing a “show columns from” query into the SQL tab didn’t work – you end up with an “Invalid SELECT statement” error message in a message box.

Non-standard Alt+Tab behavior is not very useful.

PhpED

The debugger actually works! But, it’s a lot more hassle than it should be.

CSS code completion is OK, but Visual Studio is still the best.

PHP code completion is reasonable and follows include paths in some cases.

Integrated documentation for CVS, CSS, HTML, JavaScript, MySQL, PHP, PostgreSQL  and Smarty – very nice!

The project system works OK for PHP projects, but it is cumbersome and has odd limitations (you can’t copy and paste files and folders, for example). 

Non-standard Alt+Tab behavior is not very useful.

No Subversion integration

No integrated code reformatting (help provides a good explanation of how to integrate 3rd party tools, but these don’t seem to work well, either).

One of the real selling

The ability to edit files via FTP or SSH file transfer is nice.

Integrated support for SSH terminals is nice (but why, oh why is copy and paste so difficult?)

Other Suggestions

A lot of people use Dreamweaver to edit PHP code. I guess if you come from the design world, this makes sense. Syntax highlighting for PHP is pretty poor, but I will say that Dreamweaver has the best support for code reformatting.

Notepad++ has a good syntax highlighting of many languages and there are add-ons to support FTP and integrated spell check.

Here is an article I found interesting: http://politics.slashdot.org/article.pl?sid=07/02/25/1935215

Notice how the high brow, traditional designers favor fixed width layouts with lots of wasted space in the margins. Every single one of these sites is fixed width. Most have top and bottom margins – a waste of valuable screen real estate, IMO

So why do traditional designers favor these kinds of layouts? I’ve seen lots of arguments, some lame and some worthwhile, but who cares what the designers prefer? Most designers are as much a part of the “typical audience” as I am – no part at all. The job of the web site to please it’s audience, not it’s designers. That is easy to say, but how do we really figure out how to best please the “the audience”? It’s something I’m still working on. Listening carefully to the client is one idea – often they have spent years working with their clients. Over those years, they have built up instincts about what will and won’t work. For interactive sites, I’ve got a stack of books that explain, in painful detail, how to create user interfaces that function well. These books are backed up with actual studies of end user actives. But web sites need to be both functional and pleasing to the eye.

Another related thing I’ve realized: Designers prefer smallish fonts. I guess, to their tastes, small fonts look better in the layouts. However, over and over again I’ve found that most end users (well, the ones that actually read the content) prefer larger font sizes. If you ever want to be a hero for a day, find an office with lots of 17″ 1280×1024 LCD monitors and switch everybody from “Small Fonts” to “Large Fonts” (under the Windows, it’s under Advanced Display Properties). They’ll be cheering you as you walk out the door! Overall, I find flow layouts work better with larger font sizes. With a small font size, you can cram lots of stuff into small layout boxes. Flow layouts and larger fonts force you to work out other solutions. Flow layouts also allow you to support the brower’s ability to change font sizes (YUI fonts has nice support for this). This is a great way to easily accomidate low vision and older users.

Most designers probably don’t know how to make flow layouts, so they think they are too hard. Once you learn the skills, flow layouts really aren’t so much harder than the tricks you have to use to brow beat your HTML into a cross-browser fixed layout. They’ll also make translating your site into other languages a whole lot easier, if you think this might ever occur.

I’ve also read that fixed layouts have to be done to get content to fit properly into the boxes in the layouts. For busy sites, like the presidential candidate site home pages and sites with paid advertising, the need for a fixed layout makes some sense, but I’m not sure it’s something that can’t be worked around with a different design philosophy.  Advertisers expect their ads to appear in certain locations. However, Google, Slashdot and many other sites run on ad revenue have showed us that ads don’t have to be flash or bitmaps or be fixed at a pixel precise location. “We have to direct the reader’s attention” is a common cry. “Focus them on the buy button”, they say, as if a clever graphic design could really fool somebody into buying something. And as if amazon.com has not been showing us how to do this properly in a fluid design for the last 10 years.

Yes, a flow layout won’t look like a printed brochure, but so what? Only us geezers and designers that lived through the days of printed sales literature find comfort in layouts with portrait style proportions. Look at the proportions of your monitor – wider than tall, yes? New wide screen TVs and monitors are even wider. My teenage kids spend way more time looking at at the wide, fluid web pages of their favorite interactive web sites than narrow, glossy sales slicks. What do you think they will prefer when they grow up and get jobs (and the money and power to buy things)?

I’ve heard it said that columns of text need to be kept narrow because “wide columns of text are hard to read”. This is just a weak excuse. If I ever find a web page uncomfortable to read because it’s too wide, I’ll just make my browser window narrower. Power to the people, I guess. I’ve heard lame excuses about how users don’t know how to resize browser windows. Maybe users with 800×600 screens don’t know how, but flow layouts (and worries about too wide columns of text) aren’t for them, anyway.

When you design software user interfaces, you have to trade off between first time users and users of intermediate ability. Research shows, for a typical application, there will be far more users of intermediate ability (users that might, for example, know how to resize a window). But the proper audience to design for really depends on the application. For a presidential candidate web site, I could argue either way: Sure, you have the obvious sales brochure aspect of the site, but a good presidential candidate web site will truly be devoted to interactive activities such as volunteer sign up, forums and polls. For a web site that is just a slick brochure with limited interactivity, I can almost grasp the argument that users might not know how to resize their browser windows. Thus, we need to force a width to keep the columns comfortable to read. However, in the particular case of presidential web sites, aren’t the candidates hoping to draw you into the interactive part of their site? Interactive sites are much better with flow layouts. What type of layout is used behind the home page of your favorite interactive site?

A final consideration I’ll leave you with: What happens when your site is viewed on a device like the iPhone (or a Sony PS3, XBox360 or Nintendo Wii)? In the limited space available, it’s important to make the most of screen real estate. Only a fluid design can accomplish this (without having a site dedicated to each device). My brother has an iPhone. I was interested to find out that he uses it often to do the kind of web surfing that I associate with sitting at a desk. His iPhone lets him web surf in a lot of situations where I would be reading a book or magazine. It’s interesting to consider my web sites viewed this way. Fluid designs work now and have the benefit of being ready for the future. 

When I get a chance, I’ll post some pictures of flow sites that I’ve worked on running on the my daughter’s Wii and my brother’s iPhone.  They look great ( considering the limited resolution available).

Logging in Asp.Net

Logging in Asp.Net is confusing because Microsoft uses the word “trace” in two different contexts:

When a page is rendered and HTML generated, you can have the Asp.Net engine add a bunch of additional page state information to the generated HTML. In your application’s web.config file, this is the tracing that is controlled by
<trace></trace> under <web.config></web.config>. In practice, I don’t often find this kind of tracing helpful, but when I do,  I put the following tags in web.config to make it easy to turn on or off:

<system.web>
  <customErrors mode="Off"/>
  <trace enabled="false" requestLimit="10" pageOutput="true" traceMode="SortByTime" localOnly="true" />
  <!-- ... -->
</system.web>

With these tag in place, you can enable and disable page state tracing (more like a dump if you ask me) by changing enabled=”false” to enabled=”true”.

The other use of the word “trace” is related to the more traditional form of logging via .Net Debug.Write and Trace.Write. Essentially, under Asp.Net, you use Debug.Write and Trace.Write as usual. Contrary to the MSDN documentation, the default compiler options for Asp.Net web applications allow Debug.Write output in Debug builds and allow Trace.Write output in both Debug and Release builds. If you run your Asp.Net application under the Visual Studio debugger, you will see that Debug and Trace output is redirected to the debug output window without any additional configuration.  As with any use of .Net logging, you can setup tracing to various outputs either in code or via your application’s configuration file. To configure your Asp.Net web application to log trace output to a file (for use with other tools), place the following into your web.config:

<system.diagnostics>  
  <trace autoflush="true">    
  <listeners>    
  <add name="MyApplicationTracer" type="System.Diagnostics.TextWriterTraceListener, System, Version=2.0.3600.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" initializeData="MyApplicationTrace.log" />
  </listeners>
  </trace>
</system.diagnostics>

This will result in debug (in debug builds) and trace output going into a file named MyApplicaitonTrace.log in your web site’s root folder.

Logging in .Net Command Line Applications

Since .Net applications are easy to create and fairly expressive, I often write small command line utilities that, on other platforms, I might create as shell scripts.  Here are some handy code snips that I use to handle simple logging for these applications:

namespace Program1
{
 ///


 /// Prepends each output line with the time
 ///
 public class TextWriterTraceListenerWithTime : TextWriterTraceListener
 {
  public TextWriterTraceListenerWithTime()
  : base()
  {
  }public TextWriterTraceListenerWithTime(Stream stream)
  : base(stream)
  {
  }public TextWriterTraceListenerWithTime(string path)
  : base(path)
  {
  }public TextWriterTraceListenerWithTime(TextWriter writer)
  : base(writer)
  {
  }public TextWriterTraceListenerWithTime(Stream stream, string name)
  : base(stream, name)
  {
  }

• Simple Tracing and Logging for Asp.Net and .Net Command Line Applications

Wow, some programmers go crazy about logging.  Do a Google search for “.net logging” and you’ll see what I mean. Some of these libraries are huge.  While some applications may really need all the complexity these libraries offer, I have seen it severely abused on more than one occasion. Crank up a copy of Blackberry Enterprise Server (BES) and let it run for a week or two. You’ll find your Windows Event Viewer overflowing with all types of mysterious, confusing and generally useless messages.  In addition to the constant Event Viewer spew, if you let BES run unattended, you’ll eventually find that your BES hard drive is literally overflowing with many different log files all with obscure names and even more obscure contents.  Exchange Server has a similar problem where some logging is done in Event Viewer, other logging is done to text files. And none of the Exchange Server logging options ever seems to enable the simple email flow messages you need for common day to day administration tasks that I take for granted on our Linux email servers.

Is all this really necessary? After 20 years of programming, I’ve found that the following simple API is more than enough for all my needs:

LogMessage(componentName, severity, format[, formatArg1, formatArg2, ...])

Depending on your programming language and coding standards,  this simple function may need to be wrapped in a class. You might also wrap the function call in a macro to allow for conditional compilation and to keep from having to pass the same componentName over and over.

With this API in mind, you also need a fairly simple configuration system to automatically prefix the messages with time, date, and thread ID and direct messages to some combination of syslog, Windows Event Log, a text file (just 1, please!), console, system debugger, email (for Alerts – see below)  and user defined functions.

The componentName argument is best handled as a single string determined by the programmer. This keeps things simple, yet gives the programmer a lot of flexibility in both the presentation of messages and the ability to filter messages.

There is no sense in going overboard with adding lots of flexibility for the severity argument. It’s best to spell things out and keep them simple. You’re never going to get a group of programmers and system administrators to agree on the difference between “error”, “critical” and “alert”. I use an enumerated type with the following values:

• Debug – Low level messages of interest only to programmers. It should be easy to enable and disable output of debug messages. The only reason a user would enable debug messages is to create a log file to be sent to a programmer. Information messages may be sent to log files and/or UI elements, but they should never be sent to system facilities (such as syslog or the Windows Event Log).
• Information – Messages that let users know the program is doing what it supposed to do. Information messages are of most value for services where there is no interactive UI to demonstrate that the program is working properly. However, interactive programs can trap Information messages to display status messages during length operations. Information messages may be sent to log files and/or UI elements, but they should never be sent to system facilities.
• Warning – Messages to let the user know the program encountered a problem, but the problem is not completely unexpected. The program should be able to work around problems that result in warnings (possibly by retrying at a later time).  Warnings should be sent to all log files and system facilities.
• Error – Messages that need the user’s attention.  Errors should be sent to all log files and system facilities.
• Alert – Messages that need the user’s immediate attention.  The user should be woken from bed. Alerts should be sent to all log files and system facilities.

I’ve found that the most difficult choice is between Error and Alert. For example, at an ISP, email server problems are almost always Alert messages, while for home users or small businesses, email problem can wait until morning. If it’s not obvious, make everything an Error and assume the administrator will have some way to filter the messages to decide for himself which Errors are really Alerts.

The data type for the format argument depends on your programming language and I8N requirements and tools. For ease of programming, I much prefer to use String.Format style format strings (printf strings in C++ and PHP code).  

I could write an entire article about choosing the best method for formatting strings for output, but quickly:

For production applications, use formatting rather than composition:

• Do use: string output = string.Format(“A = ‘{0}’”, a)
• Do not use: string output = “A = ‘” +  a + “‘”;

As much as I love C++ and Bjarne Stroustrup (and understand the arguments for the ability to format user defined types in a safe and type safe manner), favoring formatting over composition leads to programs that are easier to write, easier to read and easier to translate.

If you need to translate your program into other languages, there are two schools of thought for handling literal strings that need to be translated:  

• In the Windows/Mac style system, each string is manually assigned a unique integer identifier and all strings are stored in a database of some kind (typically application resources). When the you need a string, you program a call to the database API, passing it the unique integer ID of the string to be retrieved as a lookup key. 
• In the gettext style system (UNIX like platforms),  translated versions of the strings are still stored in an external database (though typically in simple text files). However, the original strings created by the programmer (often English) remain in the code. The original strings are used as the lookup key to retrieve the translated string.

Kind of like the difference between using raw char* vs. a string class for low level string operations, the Windows style system is the most resource efficient. The gettext system is less tedious to program with and, for better or worse, more tolerant of errors. You can find tools to retrofit your old code for either style. Also, there are various kinds of tools for both systems that allow translators to translate application strings without access to program source code.

Since my API is a lowest common denominator, mapping my simple API to syslog and the Windows Event Log is fairly straightforward.

Eventually, my credit card processor teamed up with one of the first companies to offer an Internet credit card processing gateway, Cybercash. I hopped on board as one of the first customers. As with a lot of APIs created by hurried teams of inexperienced programmers, Cybercash was much more complex than was necessary to accomplish the simple task at hand. Credit card processing can be really simple: Pass the credit card processor a list of required fields and get back an Boolean success code and an error message.  HTTPS is a fine protocol for this type of stateless transaction that must occur over a secure channel. Input fields can be passed as a URL encoded query string. The error code and error message could be returned an HTTP response code or via a URL encoded fields in the response body. The modern, fancy name for this type of transaction protocol is REST. Under the hood of its overly complex API, Cybercash used the simple REST protocol I described earlier. Liked a lot of other Cybercash programmers, I eventually figured this out on my own and bypassed the complex and cumbersome API to use the simple REST protocol.

Time passed on and Verisign bought up all kinds of important technology related to e-commerce. They purchased Cybercash and started promoting their own gateway API, PayflowPro. Several times, we researched the fees and APIs offered by the various vendors. PayflowPro was the best and the one we used and recommended. The PayflowPro API was a tad bit easier to use than the official Cybercash API – easy enough that instead of bypassing the API for the REST protocol, I used the actual PayflowPro API function calls.

When the time came to automate the billing for T3city, I started out with PayflowPro. However when it came time to get a new merchant account, I took some time to review my options. On paper, PayPal Website Payments Pro looked pretty good – they have a nice and simple fee structure (unlike PayflowPro with all sorts of fees billed out at different times) and the fees are competitive if you can get your volume over $3,000/month (and not too bad if you can’t). The sign-up process was all online. I figured it couldn’t be too hard to change gateway APIs – I’d already used several others – so I made the switch.

Well, I was wrong about the difficulty level associated with switching from PayflowPro to PayPal. The PayPal API is much more complex. First, there is the fact that the API is SOAP based. In addition to the more complex SOAP protocol, the PayPal programming examples use another layer on top of the native SOAP API. Why they add this extra layer, I really don’t know – maybe it’s just a fact that every junior programmer feels like an extra layer or two will make things simpler. It doesn’t make the API any easier to use. In fact, I found the differences between the new layer and the documentation for the base SOAP protocol very confusing. In the end, I elected to ditch the PayPal examples and use PayPalSvc.wsdl and .Net’s SOAP support tools to generate my own SOAP client for the PayPal API. The resulting API is a little awkward is spots (mostly due to PalPals not so hot API design), but at least it matches the detailed and fairly comprehensive PayPal SOAP protocol documentation.

Once you get over the base complexity of the PayPal SOAP based API, then you have to deal with the stateful nature of PayPal. Credit card processing is normally a simple transaction, but the PayPal Website Payments Pro program requires that you offer your customers the option to pay via PayPal. And, what the heck, allowing payments by PayPal is not such a bad idea.  However, unlike a simple credit card transaction, payments with PayPal naturally flow over a series of pages, some on your site and some on PayPal’s site. Information such as billing and shipping addresses and invoice line items have to be exchanged in a delicate dance. Assuming you are willing to tackle the complexity of proper error handling, avoiding browser complaints about HTTPS redirects and giving the user the ability to cancel cleanly, getting PayPal transactions to flow smoothly turns out to be a fairly complex.

Is the extra complexity worth it? Consider these points:

• Both gateways are very reliable. 
• PayPal fee structure is simpler, clearly specified on their web site and may be less expensive – YMMV. Traditional store fronts may be able to get better rates from their current processors with a different gateway API.
• PayPal charges the same fees for all types of credit cards. AMEX fees are usually higher, so this may save you some money, but, again, YMMV.
• PayPal does not require any type of term or volume commitment
• Accepting PayPal is nice – giving customers more ways to pay you is a good thing!
• Implementing PayPal Website Payments Pro is a good deal harder than using PayflowPro. Depending on your web site/application, adding stateful credit card / PayPal transaction processing may require adding new database fields, new screens and other large scale programming changes.
• Under .Net, PayPal Website Payments Pro does not require any native (non .Net) code. To use PayflowPro in .Net, I load and make native function calls into the native PayflowPro DLL.
• PayPal Website Payments Pro requires full names for states (not the typical 2 letter codes) and proper ISO country codes for every transaction. If you don’t already have validation in place and tables to translate state codes into state names (users will use 2 letter state codes if you let them), this support will require additional programming work.
• With PayPal Website Payments Pro, you have to perform an additional step to transfer money from your PayPal account into your bank account.  Funds from credit card purchases show up in your PayPal account immediately, but transfers to your bank account take multiple business days (similar to the delay you’ll have with a traditional merchant service provider and PayflowPro). In theory, you can use funds in your PayPal account immediately for purchases via PayPal, but you would need a careful strategy to ensure proper accounting.
• To credit a customer via PayPal, you must create a new credit transaction that references a special code from the original sale transaction. Unlike PayflowPro, you cannot void a transaction and you cannot change a sale transaction to a credit transaction.
• PayPal has different and likely much more strict fraud protection measures. Overall, I like this capability, but it did cause some pain during our switch when credit cards on file from long term customers no longer worked because of mismatched a CVV number, a mistyped expiration date, etc..

Most of my projects have a readme.txt file. Typically, a readme file will include an overview of the project and, if it is a code library, a few key source code samples ready to copy and paste. In the past, I have considered using .html files or even .doc files instead of .txt files. I’ve stuck with readme.txt for two reasons: One, I’m mostly working in a programmer’s editor. True, many IDEs have built-in HTML editors, but they are usually slow to load. Two, all the worrying about fonts and styles can be distracting. Overall, both these problems introduce what I’m going to call friction for lack of a better term. When you’ve built up some coding momentum (and this is important to do for reasons I may one day get around to writing about), friction slows you down. Some types of friction are important to the overall goals of a project and just can’t be avoided, but if friction is avoidable, then I say avoid it and gain some coding efficiency. Over time, the overviews and code samples in a readme.txt file can be pretty important to a project.  HTML (or Word) formatting is pretty, but it increases friction for no practical benefit that I can see. Thus, readme.txt instead of readme.html.

So, returning to the subject of browsing a source repository. I started thinking that maybe readme.html was not such a bad idea after all. All you really need, is <h1>, <h2>, <p> and <pre> tags. How much hassle could that be? Tables can be handy, right? So, first I started thinking about getting a global style sheet in place, next I started thinking about creating a template I could use to create a new readme.html.  Then, I started to think about how much friction I was going to add to day-to-day development and what the benefits will be. It’s a tougher choice than it was in the past, but, for now, I’m sticking with readme.txt.

I can understand why they wanted a bit of money for the work that was required for validation that first year, but overall, SSL certificates have long been overpriced for the value they provide. After that first validation, the next year’s renewal costs the CA practically nothing, but they used to give no renewal discounts at all and, even now, renewal discounts don’t exist and multi-year discounts are not as substantial as they could (should?) be.

Then there were (are) silly SSL cert upgrades that supposedly provided stronger encryption. Well, I suppose those upgrades actually could enhance encryption if you just happened to be running an old, obscure version of IE that was only available outside the US, only for a short time and has not been available since early 2000. Funny how even today, you’ll find that even the biggest CA charges extra for Server Gated Cryptography, even though no browser modern enough to be secure needs or supports it.

If you want to spend even more on your SSL cert, CAs will happily add on various types of hyper-specific insurance policies and all manner of “site seals” and  “trust logos”.

The entire verification and trust thing is just silly. The percentage of Internet users that would recognize Verisign, Thawte, Comodo or any other CA is vanishingly small. Even if we were to assume your average Internet buyer were a sophisticated, educated, rational consumer, why would they trust some company they’ve never heard of to tell them how trustworthy amazon.com is? On top of all this, given the year after year rape attempts committed by big CAs with their over pricing of renewals, fake SGC upgrades and other kinds of fake “strong cryptography” upgrades, the only thing I personally trust CAs to do is to make as much money as they possibly can with any means at their disposal.

With the useless state of verification and trust, it’s no wonder that some smaller CAs eventually started verifying less and charging a lot less. Now days, you can buy an SSL certificate that certifies nothing other that you are using SSL. Fair enough – even that tidbit is more than most consumers are interested in knowing. Practically speaking, SSL is only a technology for the vendor. Vendors should use SSL properly because they care about the consumer enough to not transmit their personal information in the clear over the Internet.  Frankly, if my wife, my Mom or one of my kids finds something they want to buy, as long as the browser don’t completely refuse to accept the connection, they will be happy to click through all manner of browser warnings to put in a credit card number. Haven’t we all been trained to ignore these peskey warning messages by now? A few shoppers might be consoled by a a happy, friendly padlock icon, but how many users are fully aware that the pad-lock icon is supposed to be in the browser’s status bar. “What’s a status bar”, you ask?

Given all this, I’ve been using the least expensive SSL certs I can find. Here are a couple of different examples – can you tell what kind of validation was used?

• https://www.searchenginecommando.com/order/  (cheap)
• https://www.t3city.com/ (cheaper)
• https://www.embracegroup.com/index2.html  (cheapest)

I suppose the SSL CAs all got together and decided something just had to be done before everybody started using self-signed SSL certificates. Enter now the “Green Address Bar” SSL Certificate. The real name is the “Extended Validation”  SSL Certificate, but my name would be better. At least with my name, there is a slight chance that consumers (of SSL certs) will notice.  If you have one of these super-duper certificates, IE’s address bar is supposed to turn green (as in the color of money, eh?). I think this is a Vista only feature, though.

The SSL vendors want $500 and more for these “EV” SSL certs. Personally, I think they are pricing themselves out of the market. Yes, a handful of sites like amazon.com will pay the extra $489 for the SSL cert that makes the browser’s address bar turn green (as in the color of envy?). I have to guess, though, that the vast majority of SSL certs are purchased for small e-commerce sites like searchenginecommando.com, t3city.com and embracegroup.com. These small operators will rightly assume that public will pay no attention to the fact that the address bar is not green. Personally, I doubt if many people really even look for the SSL lock icon anymore. If EV were just a $50 up-charge, then a lot of small shops might would go ahead and get it (at least for the first year until they find out it didn’t effect sales). Right now, since the new certs are so expensive, practically nobody will buy them and buyers will just forget all about the green address bar. If they do notice, they’ll probably just have a vague notion that something wrong with their computer (again). “Wasn’t Vista supposed to fix these kinds of problems?” they’ll wonder.

In modern times, Windows 2000 and Windows 2003 closed the gap for fast and reliable web hosting, but Linux still makes a better email server for ISP type email hosting. The Linux/UNIX security model is simpler than Windows while the Linux process model is slightly more featured. It turns out that these qualities make Linux a good fit for web and email hosting. You really don’t need complex ACL based permissions for web hosting. The simpler Linux security model works and because it’s simpler, it is easier to get right.  The Linux lightweight process model (including support for fork with copy on write) is a good match for virtual web hosting – think about all the perl CGI scripts out there. The lightweight process model is also handy for writing email servers. None of the nice email software I use for hosted email is even available for Windows. It’s not that it would be ultra-hard to port these packages to Windows. It’s just painful and really, why bother? DNS hosting is a wash. The only package I like is PowerDNS and it runs on both Linux and Windows.

PHP runs fine on Windows under ISS, but really, why bother? I run Asp.Net on Windows under IIS. It is possible to run .Net applications under Mono. But, from a practical standpoint, Mono still doesn’t fully support Asp.Net 2.0 (though it’s really close now) and Asp.Net 2.0 is so much better than .Net 1.1, I don’t bother with 1.1 for any of my own stuff. Overall, running Asp.Net on Linux seems like a lot of struggling against the machine.

When it comes to developer tools, nothing comes close to Microsoft. I can program in vi, emacs or even notepad if I have to, but Microsoft programming tools are better. The IDEs are faster and more integrated. The debuggers are better. The compilers and linkers are faster. My biggest complaint with Linux programming tools is lack of performance. I need a fast compile/link/debug cycle. For anything bigger than a single file project, I can’t seem to get it on Linux. Last time I had a serious project on Linux, I took the time to try out lots of different IDEs. As I would have guessed, Kdevelop was the best – it’s beautiful and has tons of features, but I still found it slow and awkward for day-to-day use. I figured out how to use “automake and friends”. Talk about weird (and slow). I do like Subversion. I’m already using it for all my cross-platform development projects and I’m thinking about converting everything else over. With Microsoft tools, debugging Asp.Net applications is just like debugging a GUI application. This alone justifies developing all my web application in Asp.Net and hosting them on Windows.

It’s been more than a couple of years since my last serious desktop GUI application. At that time, Windows and MFC were the obvious platform. If I had to run the application on Linux, I would have tried to get something going with Winelib, but MFC support under Winelib has always looked like a PITA. For new development, I would consider using Qt or wxWigits. My big complaint about both these libraries is the deliberate lack of support for exceptions – talk about stuck in the dark ages. I’ve done a little bit of WinForms programming. Overall, I think the library is pretty good (and with Mono, it could be cross-platform). My worry is the distribution requirements (though Windows 9X, ME and 2000 are finally starting to fade away) and the hefty resource requirements.

A couple of weeks ago, Gary started having more problems with his guestbook. It was slow and, more importantly, it wasn’t taking new posts. After some digging around, I found out that it was slow because PYG stored all the entries in a simple text file. THis is OK (and maybe ideal) for a guestbook with a few posts. Gary’s guestbook is active and covers several years. Also, there were still a fair number of spam posts that were not visible (Gary’s guestbook is moderated), but still in the text file. Altogether, Gary has over 3000 posts in his guestbook. The combination of simple text file storage with some inefficient coding in PYG and 3000+ posts was resulting in slow page generation. Gary allows HTML in posts. This is mostly so that this guests can post links to pictures, other concerts and videos (on Youtube, etc.). Allowing HTML can cause problems, but, because his guestbook is moderated, the problems should be correctable. Unfortunately, posts with invalid HTML tripped up the PYG administration interface. So, I started looking for a new guestbook script. PYG doesn’t have a version that supports MySQL. I wanted something that:

• Used PHP 4.x
• Used MySQL
• Used templates to create the Guestbook pages
• Allowed the guestbook to be moderated

Oddly enough, I couldn’t find anything that met these objectives. I searched Google and SourceForge. I came across tons of Guestbook projects. Some required PHP 5. Others didn’t use templates. I found one that used PHP 4 and templates but didn’t support moderation. So, I ended up writing one from scratch. I’ve already got one PHP project on SourceForge, phpHitCounter. phpGuestbook is already taken, so I’m thinking phpMysqlGuestbook will be the name. Here are the highlights:

• Simple configuration (config.php) and installation (edit config.php and run .sql script to create database).
• Excellent error messages and error handing for end users ((JavaScript and in page error reporting).
• Extra feedback for end users via a special page that explains post moderation.
• Fast post display for both end users and administrators
• Emails sent to the moderator include a link to the post to be reviewed
• Simplified administration
• Simple template based customization of HTML output. Templates are valid HTML so they can be maintained in FrontPage, Dreamweaver, etc.
• Option to allow HTML content in end user posts. Careful handling of user supplied HTML content in administration interface.
• Comprehensive support for banned IPs and banned/moderated content

This script is a nice companion to phpHitCounter.