Got to Portland yesterday (Tuesday), got the booth set up, went to the Tuesday night event, and slept.

The Tuesday night event was pretty good. I got there a bit before Larry Wall took the stage. I’d been wanting to see a State of the Onion talk (rather than just read them afterwards), and it was great — many a chuckle. The White Camel awards were good, and Kathy Sierra’s talk on Cognitive Seduction was excellent. Damien Conway’s talk (The Da Vinci Codebase) was amusing, but I couldn’t totally get into it.

Today brought three sessions, all of which got me in the mood to code. The first was Jim Weirich’s Test-Driven Design Meets Design By Contract. Second was Deploying Rails with Capistrano by Mike Clark. Extending Ruby with C by Garrett Rooney finished off the day.

The balance of the day was spent helping at the booth. This was my first time on the vendor side of the booth, and it was pretty interesting. I worked in technical sales for a while when I lived up north, and I found myself relying a lot on skills I learned there — the primary one being going into what I call “sales mode”.

When I’m in sales mode I tend towards being extroverted, friendly, and helpful. I also completely lose track of who I’ve talked to. Sometimes I even lose track of who it is I’m talking to. Today was one of those days.

The afternoon was pretty busy at the booth, and I was fielding questions about just about everything. Most of the questions were in the vein of “What is Komodo?” and “So, what do you guys do?” One gentleman came up to me and surprised me somewhat.

Warning: The following conversation is approximate. The funny part is accurate, though.

Him: So, I hear… Some squabble about PerlApp. The output files are 30 megabytes or something like that on Windows.

Me: Wow. I haven’t heard that one. I’ve never heard of one that big. I think the biggest I’ve seen was 19-point-something megs, and that included a heap of DLLs and graphics for the UI — it was a pretty complex app from one of our enterprise customers.

Him: Oh. So it’s only about 20 megabytes.

Me: It depends — Hello World weighs in at about 3.5 or 4 megabytes, but that includes the Unicode modules and translation tables. If you’re sure you don’t need those you can trim them out, and if I remember correctly, that takes the size down to 2 or 2.5 megs. So that’s really where you start.

[short conversation on PerlApp's module inclusion strategy omitted]

Him: So 20 megabytes is pretty much what you can expect.

Me: Yup, that’s a pretty darn big app, too.

Him: So you won’t hit 30 megabytes?

Me: No, not unless you’re reimplementing Emacs or something like that.

We both had a good laugh, and the gentleman took his leave.

A short time later I was walking through the halls near the exhibition hall, and saw the OSCON Portraits bulletin board. It was then that I realised something that should have been obvious all along.

I’m pretty sure I had been talking to RMS.

More than a few people are surprised when they upgrade to PDK 6.0.x and ActivePerl 5.8.  In addition to being more solid than ActivePerl 5.6 with PDK 5, the freestanding executables are sometimes three or four times larger.

There is one word that is responsible for much of this apparent bloat:  Unicode.

When PerlApp puts together an executable in freestanding mode, it has to be sure to include all required modules and their contingent bits and pieces.  Because PerlApp can’t tell if a conditionally required module will be needed, it includes it to be on the safe side.  Often, this is how the Unicode modules (which aren’t all that big) and the translation tables (which are that big) get involved.

The easiest way to cure this problem is to trim out the Unicode modules and related files using the PerlApp graphical user interface, or –trim arguments on the command line.  Whichever way you choose, you should be aware that some modules generate and use Unicode internally.  If your application begins to fail after removing the Unicode modules and/or translation tables, you will need to play with it a bit to figure out which are needed and which aren’t for your given application.

The best way to get the initial settings correct is to use the PerlApp graphical interface.  There you can see which modules and extra files will be included, and have the opportunity to trim them out.  When you have everything set up correctly, the Output tab of the graphical interface will show you the command line you need to use to reproduce the build with those settings.

This should bring the size down substantially.  If it does the trick for you, let me know and I will give some rough numbers that I hear from people as examples in a future version of this entry.

A very common question is what the difference between the AS and MSI installer packages is.  Here’s the quick answer, with some additional points for MSI packages from TrentM:

The AS package is far less functional than the MSI package.  Here’s a quick run-down of the differences:

AS package:

• zip file
• batch-file installation script
• cannot repair a corrupted installation
• cannot uninstall the package automatically
• requires manual setting of paths, registry entries, etc.

MSI package:

• Microsoft Installer file
• double-click the file to run the graphical installer
• can repair a corrupted installation
• can uninstall the package via the Add/Remove Programs control panel
• sets the required environment variables and registry settings automatically
• can be installed non-interactively (no GUI required)
• can be installed remotely using standard Microsoft tools

As you can imagine, the MSI package is highly recommended.

A couple of weeks ago a user wrote in with an unusual problem.  Although he was checking out his repositories from his Subversion server, Komodo wasn’t recognising them as version-controlled files.  The SCC context menu was showing all entries as disabled, and nothing he could do would change the situation.  To make things a little bit stranger, I couldn’t reproduce the problem on my end.

After some time writing back and forth he wrote back to tell me he had found the problem, and he asked me to post the solution to the web so others having the same problem could benefit from it.  Of course, I am only too happy to do so.

When he installed Subversion, he selected an option called SVN_ASP_DOT_NET_HACK.  The option sets an environment variable that causes the installed Subversion client to create Subversion private directories named _svn instead of the usual .svn (underscore instead of dot).  Problem is, Komodo relies specifically on the presence of a file named with a dot, and when it does see that directory, it assumes no files in the current directory are version-controlled.

The fix is fairly simple — remove the SVN_ASP_DOT_NECK_HACK environment variable and possibly reboot, depending on the version of Windows.

Thanks, Martin!

Yesterday I ran into an interesting problem that took me on a tour of the sources for PerlApp and an old version of Apache.  It’s perhaps more of an uncommon problem than what I cover here, but it’ll come in handy for those who run into it.

A user wrote in and related the following situation:

• CGI using interpreted Perl works great
• CGI using compiled C works just fine
• CGI using PerlApp-wrapped Perl dies with a 500 Internal Server Error

On the surface this looks annoying but innocuous.  The environment is AIX 5, Apache 1.3.29, and PDK 6.0, which isn’t a bad way to go by any means, but AIX has a few oddidities that can keep you on your toes.

The Apache configuration seemed pretty standard, and the .htaccess file the user provided didn’t hold any surprises.  The source code for the CGI was effectively nothing more than a few lines of test code — the problem definitely wasn’t there.

I had the user generate a ScriptLog, and among the output in the log was this little gem:

Panic: Cannot determine full path of ‘foo’

In this case, ‘foo’ was the name I gave the compiled CGI script.  At this point I had more questions than answers.  Time to haul out the sources for PerlApp.

I had never checked out the sources for PerlApp before, but thanks to the PDK team’s efforts to keep the codebase clean it didn’t take me long to find where the error message was being generated.  I expected PerlApp to try to find the path to the program via argv[0], and it does, so it seemed that argv[0] was being set to just foo.

This didn’t make a lot of sense.  Apache not only knows the URI for the CGI script, it has to know the full path to it in order to execute it.  Why wasn’t that being used for argv[0]?  In crawling through mod_cgi.c, I noticed a handy ifdef, and rebuilt Apache with -DDEBUG_CGI.  One quick ‘./httpd -X‘ later, and I was in business — and had it definitively from Apache that it was setting argv[0] to only foo.

A quick chat with Jan Dubois turned up the solution pretty quickly — add the directory containing the CGI to the PATH, which gets walked by PerlApp as a last resort if it can’t find the full path of the program any other way.  Inefficient, but effective.  Sure enough, that solved the problem.

So why the war story, especially on a fairly easy problem?  Two reasons.  Firstly, to make sure there’s some Google food on the topic.  Secondly, to show the cooperation between users, developers and myself, and reiterate how important it is.  Without that cooperation this could have taken a lot longer to figure out.  Thirdly, to show that not every question that comes to us is one that we know the answer to.  The only way to make sure it wasn’t a bug in PerlApp was to track down the reason why it was happening.

This story displays many of the elements of solving a new problem for a user:

1. Get a problem report and description of the problem.
2. Get information on the environment in which the problem happens.
3. Reproduce the problem.
4. Consult the developers if the ideal solution is ambiguous or not immediately apparent.
5. Solve the problem in a test case (reproduction to fixed).
6. Relay information on the solution to the user.
7. Document the problem and its solution.

That’s part of what I do here — and it’s fun.

A favourite trick of many people is to tweak the PERL5LIB environment variable to tell Perl where to look for their modules.  This often works great in testing, but runs into problems when things go into production under mod_perl.  What is one to do then?

Apache with mod_env installed allows use of the SetEnv and PassEnv lines in the configuration file.  PassEnv tells Apache to expose the value of an existing environment variable, and SetEnv allows you to set the value of a given variable explicitly.  I would recommend using SetEnv in this case, however.

If you’ve lost your file associations to Komodo in Windows, here’s how to get them back:

1. go to Preferences | Windows Integration
2. Select Configure Common Associations
3. deselect all entries (clicking "All" in each category once or twice should do it)
4. click "Okay"
5. after the dialog box closes, click Re-Apply Settings to System
6. restart Komodo
7. go to Preferences | Windows Integration
8. select Configure Common Associations
9. select all entries
10. click "Okay"
11. after the dialog box closes, click Re-Apply Settings to System
12. restart Komodo

I recently had a question that went something like this:

I’m trying to wrap an .exe file in my TclApp’d application.  I can test for its presence in the wrapped file, and it’s there — but for some reason, I can’t spawn it from Expect.  Why not?

I’m not a stranger to wrapping binary files in TclApp’d applications.  Indeed, one of my first blog entries discusses it.  I was able to reproduce the behaviour, so I went to the ActiveTcl developers for an answer.

The answer, it seems, is so simple that it took a minute to figure out.  While your TclApp’d application can look into the TclVFS filesystem that holds it, the operating system can’t.  This is why testing for its presence from within Tcl works like a charm, but the OS claims it can’t be found.

The solution is to copy the file to a temporary directory in the regular filesystem and run it from there.  Easy peasy.  Worked like a charm.  You should only have to do it with executables and other files that the operating system needs direct access to.

Here is a quick guide to the steps to take when upgrading any ActiveState software package running on the Windows platform.  Following proper upgrade procedures can keep a variety of problems from occurring — these errors can be fairly tricky to debug.

Without further ado:

1. quit all running instances of the product
2. uninstall the product via the Add/Remove Programs control panel
3. delete all directories where the product was installed
4. install the new version of the product

You should prefer MSI or executable installers to AS packages, as the AS packages offer no uninstallation options.  If you have installed something from an AS package, skip the second step above and simply delete the installation directory.  This may not remove all files from the system (those that have been installed in other directories) and will not remove registry entries associated with that installation.

The third step may seem redundant, but sometimes files do get left behind, and it is good practice to make sure they are gone, as these can also cause difficult-to-diagnose problems.

In general, you don’t need to worry about re-installing a license unless you are upgrading to a new major version of a piece of software.  For example, upgrading from Komodo 3.1 to 3.5.2 will just work — you don’t need to install anything more.

As usual, if you have any questions, let us know at support@activestate.com.

I had an interesting question today, and it turned out to have an interesting answer.  I thought I’d put it here for posterity’s sake, and to give people a little insight as to why decisions get made the way they do behind the scenes.

Someone wrote in asking why we compile ActivePython for Windows with an early version of Visual Studio (VS6 for Python 2.3, VS7 for Python 2.4), and if we were going to change over to Visual Studio 2005, and if so, when.

Not knowing the answer off the top of my head, I went and had a quick conversation with our ActivePython guy, TrentM.

It turns out there is good reason for it.  By using the versions of Visual Studio that we do, we keep binary compatibility with extensions for the Python.org distribution of the language.  Our choice of build environment will continue to follow the lead of the community to ensure this compatibility remains.

There was a thread on the Python-Dev mailing list about switching build environments to VS2005.  After some back-and-forth, this message told at least part of the story why it’s not going to happen right now.

I would imagine that we will keep following the lead of the Python.org community to make sure we are compatible, and that binary extensions will continue to be useful for everyone.