Setting Windows Display Mode From The Command Line

A relative of mine runs Windows ME (yes Microsoft, people are still running 98/ME even though you discontinued support for it this year) and wanted to play an old version of Scrabble. One problem was that this version of Scrabble must be run in 256 color mode, but switching to 256 color mode can make other applications do strange things.

Instead of forcing the relative to manually switch to 256 color mode before running Scrabble and then manually switch back to True Color mode after exiting the game, a bit of Googling found a more automated solution: nircmd is a command-line utility that can do all sorts of Windows admin things that you would normally have to do through the GUI including setting display modes, create shortcuts, modify file properties, empty the recycling bin, mute the speaker, turn off the monitor, ejecting the CDROM and much more.

So I created a batch file that sets the display mode to 256 colors (8 bits) using nircmd and then launches Scrabble:

nircmd setdisplay 800 600 8
scrabble.exe

Just to be sure the display is restored to a better color mode I then created a startup script that sets the display back to True Color (24 bit).

I wish Microsoft would create such useful command-line admin utilities and ship them with the OS. (MS do make some command-line admin utilities, but they ship as optional extras and they are not very consistent across versions of Windows :()

nicrmd can also reboot a PC, so it might come in handy to do a remote reboot of a PC when Remote Desktop starts to play up (as it does if the PC hasn't been rebooted in the last few days) :)

Temporary Email Address Reduces Spam

10MinuteMail gives you an email address that lives temporarily.

This is useful for those websites that require you to sign up with a valid email address to get access to something (usually you have to click on a link in an email they send you to complete registration), but you don't want to continue to receive spam from them forever more.

By default the lifetime of the email addr is 10 minutes, but you can make it live longer.

Sun Java implementation goes open source under the GPL!

I'm sure most of you say the announcement yesterday that Sun will soon be open sourcing their JVM implementation under the Gnu Public License (GPL) v2 license. (Note that the Sun implementation, not the actual Java specs, are being open sourced.) At first I was surprised that they didn't choose an LGPL license, since there was a possibility people might be forced to use a shim in order to protect their intellectual property, but Dalibor Topic, lead on the open source JVM Kaffe (licensed under GPL) clarified this is not the case:

The GPL doesn't require that bytecode classes using a GPLd java.lang.Object be licensed under the GPL as well. That's because neither the bytecode nor the source code using it are derivative works of java.lang.Object, as all that ever ends up in them are interface names and constants, and those remain the same, regardless of the license of the java.lang.Object class. Those symbols are standardised through the JCP, and published as specifications. They are necessary for interoperability. Therefore, the symbol names and constant values can not be claimed as original works by a GPLd java.lang.Object, and accordingly don't meet the bar for copyrightability.

As an added bonus the Dancing Duke has also been open sourced. Party time!

It'll be interesting to see what happens to the GNU Classpath project now that a "fully compatible and complete" Java implementation will be available.

Windows Blue Screen of Death (BSOD) - the screensaver!

Bluescreen cycles between different Blue Screens and simulated boots every 15 seconds or so. Virtually all the information shown on Bluescreen's BSOD and system start screen is obtained from your system configuration - its accuracy will fool even advanced NT developers...

Integrating WebSphere PMI and Spring

Back in the WebSphere 4.x days a light-weight performance monitoring infrastructure named, not surprisingly, Performance Monitoring Infrastructure (PMI) was introduced. The PMI infrastrcture basically consists of some application-upadted statistics "stubs" (i.e. handle objects containing counters or timers or something else) and a "collector" that periodically gathers data from the stubs. Before WebSphere 6.0 you had to use a thick client to view and analyze the PMI data, but starting with WebSpherer 6.0 the admin console generates pretty statistics graphs.

So you have a few different Spring-based applications deployed on WebSphere and after they go to production you think it would be a good idea to get gather some rough performance metrics on the applications (a pretty common scenario). What are your options?

• Use a monitoring package that already has some Spring integration such as JAMon (see JamonPerformanceMonitorInterceptor) or JETM. These packages require a bit of work to integrate, requiring you to add servlets and/or JSPs and/or security to each of the applications you want to monitor.
  
• Integrate the applications with PMI. Use the infrastructure already provided by WebSphere. No extra servlets/JSPs to roll out, no security changes.
  

PMI sounds like a cool option. Where do I find doco on it? Unfortunately this is very badly documented by IBM, but if you dig around the WebSphere InfoCenter you can find a few hints. But let me save you some trouble.

WebSphere uses PMI internally to monitor basic server health. IBM kindly provided "Custom PMI" hooks to allow developers to plug their applications into the PMI infrastructure. The way Custom PMI is supposed to work is as follows:

• Each application needs an XML file which defines a number of stats "stubs" (see below for an example).
  
• At application startup the application registers the stats stubs definition file with the PMI StatsFactory.
• The application code must fetch and update the generated stats stubs at the appropriate places.
  

A stats stub definition file looks something like:

<?xml version="1.0"?>
<!DOCTYPE PerfModule SYSTEM "perf.dtd">

<PerfModule UID="com.ibm.websphere.pmi.jvmRuntimeModule">
<description>jvmRuntimeModule.desc</description>

<BoundedRangeStatistic name="jvmRuntimeModule.totalMemory" ID="1">
<participation>cross-family</participation>
<level>high</level>
<description>jvmRuntimeModule.totalMemory.desc</description>
<unit>unit.kbyte</unit>
<comment>Total memory in JVM runtime</comment>
<resettable>false</resettable>
<statisticSet>basic</statisticSet>
<zosAggregatable>false</zosAggregatable>
</BoundedRangeStatistic>

<CountStatistic name="jvmRuntimeModule.freeMemory" ID="2">
<participation>cross-family</participation>
<level>low</level>
<description>jvmRuntimeModule.freeMemory.desc</description>
<unit>unit.kbyte</unit>
<comment>Free memory in JVM runtime</comment>
<resettable>false</resettable>
<statisticSet>extended</statisticSet>
<zosAggregatable>false</zosAggregatable>
</CountStatistic>
...
</PerfModule>

Extract and have a look at com/ibm/websphere/pmi/xml/perf.dtd from $WEBSPHERE_INSTALL_DIR/lib/pmi.jar for the gory details.

WebSphere supports the following types of stats:

• CountStatistic (a simple counter)
• AverageStatistic (take the average of some type of samples)
• TimeStatistic (extends AverageStatistic, used to take the average of, say, time to execute a method)
• BoundaryStatistic
• BoundedRangeStatistic
• DoubleStatistic
• RangeStatistic
  

Of all of these types I can really only find a use for CountStatistic and AverageStatistic. If you know a good use for any of the others please let me know.

This sucks. The whole process of using the PMI API seems very static and very manual. Well, the PMI API was designed quite a few years ago, before more modern non-invasive techniques such as AOP were invented. Fortunately we can use a bit of Spring AOP magic to integrate your application with PMI without too much trouble.

ok, so after all of that preable what I really want to do is something like the following:

1. Use the Spring BeanNameAutoProxy to wrap PMI interceptors around all method calls (excluding bean setters) of beans defined in Spring.
2. Use the WebSphere admin console to enable/disable/log PMI data for my application.

Interestingly enough 1 is a little tricky. Spring AOP interceptors are usually completely unaware of which bean they are being called for, but in our case the method interceptor has to invoke the correct stats stub for the given bean. What we have to do, therefore, is write an autoproxy that glues interceptors to stats stubs. Fortunately BeanNameAutoProxy delegates most of its work to org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator, so we just extend AbstractAutoProxyCreator to create our own stats autoproxy named StatsAutoProxyCreator. (See the end of this post for the source code.)

Using StatsAutoProxyCreator the Spring config for our autoproxy looks something like this:

<bean name="mystats" class="robertmaldon.stats.StatsAutoProxyCreator">
<property name="beanNames"><value>MyPetStoreController</value></property>
<property name="statTypes"><value>time,count</value></property>
<property name="proxyTargetClass" value="true"/>
<property name="setName" value="basic"/>
</bean>

where

• beanNames is a comma seperated list of beans to proxy
• statTypes is a comma seperated list of stat types to collect. Currently supported values are "time" (average invocation time for a method) and "count" (number of times the method is invoked)
• proxyTargetClass is a true/false to proxy the underlying class rather than interfaces implemented by the underlying class.
• setName is a logical group name for the stats. If you do not specify setName it defaults to "basic". All "basic" stats are collected by default. If you use any other name then you will have to manually enable collection of the stats through the WebSphere admin console (Monitoring and Tuning -> Performance Monitoring Infrastructure (PMI) -> serverX -> Custom).
  

So with the stats autoproxy in place you start the application, log into the admin console, go to Monitoring and Tuning -> Performance Viewer -> Current Activity -> serverX -> Performance Modules , check the box next to MyPetStoreController (or whatever the bean name is), click on View Modules and watch the pretty graphs being generated.



You may also choose to save the PMI stats to a log file for later review.

Groovy stuff.

Appendix A : Special notes

• If you are running WebSphere on a headless server then you will need to set the JVM command line option -Djava.awt.headless=true because the PMI console needs a display in which to draw graphs.
• You will need to compile the code against pmi.jar and management.jar from the $WEBSPHERE_INSTALL_DIR/lib directory.
  

Appendix B : Code

StatsAutoProxyCreator.java

package robertmaldon.stats;

import java.util.Arrays;
import java.util.Iterator;
import java.util.List;

import org.springframework.aop.TargetSource;

import org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator;

/**
* NOTE: We had to reimplement BeanNameAutoProxyCreator instead of
* extending it because the list of bean names to proxy is declared as
* private.
*/
public class StatsAutoProxyCreator extends AbstractAutoProxyCreator {
private List beanNames;

private Integer[] statTypes;

private String setName;

public StatsAutoProxyCreator() {
super();
}

/**
* Set the names of the beans that should automatically get wrapped with
* proxies. A name can specify a prefix to match by ending with "*",
* e.g. "myBean,tx*" will match the bean named "myBean" and all beans
* whose name start with "tx".
*/
public void setBeanNames(String[] beanNames) {
this.beanNames = Arrays.asList(beanNames);
}

public void setStatTypes(String[] statTypes) {
List statTypesList = new java.util.ArrayList();
for (int i = 0; i < statTypes.length; ++i) {
   int statType = StatsConstants.getType(statTypes[i]);
   if (statType == 0) {
       throw new RuntimeException("Unknown stat type ["
                                  + statTypes[i] + "]");
   }

   statTypesList.add(new Integer(statType));
}

this.statTypes = (Integer[])statTypesList.toArray(new Integer[0]);
}

/**
* Give a name to the set of statistics created by this auto-proxy
* instance. The set name is used to enable and disable the set of
* stats as a whole.
*
* If the set name is not configured it defaults to "basic", which adds
* the statistics to the existing set of stats WebSphere PMI collects
* by default.
*/
public void setSetName(String setName) {
this.setName = setName;
}

/**
* Identify as bean to proxy if the bean name is in the configured list
* of names.
*/
protected Object[] getAdvicesAndAdvisorsForBean(Class beanClass,
String beanName, TargetSource targetSource) {
if (this.beanNames != null) {
   if (this.beanNames.contains(beanName)) {
       return StatsBuilder.getSingleton()
              .createTemplate(beanName, beanClass, statTypes, setName);
   }
   for (Iterator it = this.beanNames.iterator(); it.hasNext();) {
       String mappedName = (String) it.next();
       if (isMatch(beanName, mappedName)) {
           return StatsBuilder.getSingleton()
              .createTemplate(beanName, beanClass, statTypes, setName);
       }
   }
}
return DO_NOT_PROXY;
}

/**
* Return if the given bean name matches the mapped name.
* The default implementation checks for "xxx*" and "*xxx" matches.
* Can be overridden in subclasses.
* @param beanName the bean name to check
* @param mappedName the name in the configured list of names
* @return if the names match
*/
protected boolean isMatch(String beanName, String mappedName) {
return (mappedName.endsWith("*")
       && beanName.startsWith(
              mappedName.substring(0, mappedName.length() - 1)))
    ||
      (mappedName.startsWith("*")
           && beanName.endsWith(
                  mappedName.substring(1, mappedName.length())));
}
}

StatsBuilder.java

package robertmaldon.stats;

import java.lang.reflect.Method;

import java.util.List;
import java.util.Map;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import com.ibm.websphere.pmi.PmiDataInfo;
import com.ibm.websphere.pmi.PmiModuleConfig;

import com.ibm.wsspi.pmi.factory.StatisticActionListener;
import com.ibm.wsspi.pmi.factory.StatsFactory;
import com.ibm.wsspi.pmi.factory.StatsInstance;
import com.ibm.wsspi.pmi.factory.StatsTemplateLookup;

import com.ibm.wsspi.pmi.stat.SPIStatistic;

import com.ibm.wsspi.pmi.stat.SPICountStatistic;
import com.ibm.wsspi.pmi.stat.SPITimeStatistic;

/**
* This class generates the "custom" PMI glue code for a given Spring bean,
* including the MethodInterceptors.
*
* See StatsAutoProxyCreator for how the MethodInterceptors are applied to
* the beans.
*
* Design note: The underlying "custom" PMI factory is a singleton, so
* since this class is the main interface to the PMI factory it is also
* implemented as a singleton.
*/
public class StatsBuilder implements StatsTemplateLookup {
private Log logger = LogFactory.getLog(getClass());

private static StatsBuilder singleton = new StatsBuilder();

private Map templates = new java.util.HashMap();

private StatisticActionListener actionListener =
    new StatisticActionListenerImpl();

public static StatsBuilder getSingleton() {
    return singleton;
}

private StatsBuilder() {
    // NOTE: This one line is extremely important. The "custom" PMI
    // infrastructure is intended to create StatsInstances from XML
    // "template" (config) files, but in our case we want to create
    // them dynamically in code. To have the PMI infra look up something
    // dynamically if it can't find a template XML file we need to
    // register a "template lookup" class - this class.
    StatsFactory.registerStatsTemplateLookup(this);
}

public PmiModuleConfig getTemplate(String templateName) {
    if (logger.isDebugEnabled()) {
        logger.debug("getTemplate called for template of name ["
                     + templateName + "]");
    }
    return (PmiModuleConfig)templates.get(templateName);
}

public Object[] createTemplate(String beanName, Class beanClazz,
                               Integer[] statTypes, String setName) {
    String moduleId = beanName;
    if (templates.get(moduleId) != null) {
        if (logger.isInfoEnabled()) {
            logger.info("Statistics for bean of name [" + moduleId
                        + "] have already been generated. Skipping...");
        }
        return null;
    }

    PmiModuleConfig config = new PmiModuleConfig(moduleId);
    config.setDescription(beanClazz.getName());

    // In order to keep the noise down we monitor only public methods.
    Method[] methods = beanClazz.getMethods();

    List methodList = new java.util.ArrayList();

    int id = 0;

    for (int i = 0; i < j =" 0;" data =" getData(methods[i]," id =" id" interceptors =" new" instance =" StatsFactory.createStatsInstance(" i =" 0;" spistatistic =" instance.getStatistic(i);" methodid =" getMethodId(method);" typename =" StatsConstants.getTypeName(statType);" data =" new" setname ="=" methodname =" method.getName();" length ="=" methodid =" new" params =" method.getParameterTypes();" j =" 0;" paramclassname =" params[j].getName();" dotindex =" paramClassName.lastIndexOf('.');"> -1) {
            paramClassName = paramClassName.substring(dotIndex + 1);
        }

        methodID.append(paramClassName);

        if (j < params.length - 1) {
            methodID.append(',');
        }
    }
    methodID.append(')');

    return methodID.toString();
}
}

/**
* Simple listener class for statistic events.
* Note for future design: Maybe we want a StatisticActionListener to
* optionally log stats values to the commons-logging log file?
*/
class StatisticActionListenerImpl implements StatisticActionListener {
private Log logger = LogFactory.getLog(getClass());

public void statisticCreated(SPIStatistic spiStatistic) {
    if (logger.isDebugEnabled()) {
        logger.debug("Created statistic of type ["
                     + spiStatistic.getClass().getName() + "]");
    }
}

public void updateStatisticOnRequest(int i) {
    if (logger.isDebugEnabled()) {
        logger.debug("updateStatisticOnRequest called for statistic with id ["
                     + i + "]");
    }
}
}

StatsConstants.java

package robertmaldon.stats;

/**
* Some constants used in construction of PMI stats definitions.
*/
public final class StatsConstants {
public static final int TYPE_COUNT = 2;
public static final int TYPE_BOUNDED_RANGE = 5;
public static final int TYPE_RANGE = 7;
public static final int TYPE_TIME = 4;
public static final int TYPE_DOUBLE = 3;
public static final int TYPE_AVERAGE = 6;

public static final String TYPE_NAME_COUNT = "count";
public static final String TYPE_NAME_BOUNDED_RANGE = "boundedRange";
public static final String TYPE_NAME_RANGE = "range";
public static final String TYPE_NAME_TIME = "time";
public static final String TYPE_NAME_DOUBLE = "double";
public static final String TYPE_NAME_AVERAGE = "average";

public static final int LEVEL_LOW = 1;
public static final int LEVEL_MEDIUM = 3;
public static final int LEVEL_HIGH = 7;
public static final int LEVEL_MAX = 15;

private StatsConstants() {}

public static String getTypeName(int statType) {
switch (statType) {
  case TYPE_COUNT:
      return "@" + TYPE_NAME_COUNT;
  case TYPE_BOUNDED_RANGE:
      return "@" + TYPE_NAME_BOUNDED_RANGE;
  case TYPE_RANGE:
      return "@" + TYPE_NAME_RANGE;
  case TYPE_TIME:
      return "@" + TYPE_NAME_TIME;
  case TYPE_DOUBLE:
      return "@" + TYPE_NAME_DOUBLE;
  case TYPE_AVERAGE:
      return "@" + TYPE_NAME_AVERAGE;
  default:
      return "@unknown";
}
}

public static int getType(String typeName) {
if (TYPE_NAME_COUNT.equals(typeName)) {
  return TYPE_COUNT;
} else if (TYPE_NAME_BOUNDED_RANGE.equals(typeName)) {
  return TYPE_BOUNDED_RANGE;
} else if (TYPE_NAME_RANGE.equals(typeName)) {
  return TYPE_RANGE;
} else if (TYPE_NAME_TIME.equals(typeName)) {
  return TYPE_TIME;
} else if (TYPE_NAME_DOUBLE.equals(typeName)) {
  return TYPE_DOUBLE;
} else if (TYPE_NAME_AVERAGE.equals(typeName)) {
  return TYPE_AVERAGE;
} else {
  return 0;
}
}
}

AbstractStatsInterceptor.java

package robertmaldon.stats;

import com.ibm.wsspi.pmi.stat.SPIStatistic;

import org.aopalliance.intercept.MethodInterceptor;
import org.aopalliance.intercept.MethodInvocation;

/**
* Base class for our statistics around advice.
*/
public abstract class AbstractStatsInterceptor implements MethodInterceptor {

protected SPIStatistic spiStatistic;

public AbstractStatsInterceptor(SPIStatistic spiStatistic) {
this.spiStatistic = spiStatistic;
}

public abstract Object invoke(MethodInvocation invocation) throws Throwable;
}

TimeInterceptor.java

package robertmaldon.stats;

import org.aopalliance.intercept.MethodInvocation;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import com.ibm.wsspi.pmi.stat.SPITimeStatistic;

/**
* Average invocation time.
*/
public class TimeInterceptor extends AbstractStatsInterceptor {
public TimeInterceptor(SPITimeStatistic spiStatistic) {
super(spiStatistic);
}

public Object invoke(MethodInvocation invocation) throws Throwable {

SPITimeStatistic timeStat = (SPITimeStatistic)spiStatistic;

long start = System.currentTimeMillis();

Object rval = invocation.proceed();

long end  = System.currentTimeMillis();

long diff = end - start;

if (timeStat != null) {
  timeStat.add(diff);
}

return rval;
}
}

CountInterceptor.java

package robertmaldon.stats;

import org.aopalliance.intercept.MethodInvocation;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import com.ibm.wsspi.pmi.stat.SPICountStatistic;

/**
* Simple counter.
*/
public class CountInterceptor extends AbstractStatsInterceptor {
public CountInterceptor(SPICountStatistic spiStatistic) {
super(spiStatistic);
}

public Object invoke(MethodInvocation invocation) throws Throwable {

SPICountStatistic countStat = (SPICountStatistic)spiStatistic;

Object rval = invocation.proceed();

if (countStat != null) {
  countStat.increment();
}

return rval;
}
}

Update: I've had a couple of requests for a ZIP of the source code, so here it is.

Sending email through Google SMTP from Perl

The do no evil people from Google kindly provide a publicly accessible SMTP server to which you can send email. The catch(s)?

• You need a gmail account.
• The client from which you send the email (i.e. the client that connects to the SMTP server) must support Simple Authentication and Security Layer (SASL).

By requiring authentication Google hopes to quickly identify and shut down spammers if they spam from a gmail account.

Since I use Perl to administer my servers I would like to be able to send alerts from Perl scripts to my gmail account when something happens. (And from my gmail account a copy can be sent to my other email accounts, my cell phone, etc.).

To support SASL you need to install the following modules on top of the standard Perl distribution:

• Net_SSLeay.pm (Open SSL)
• IO-Socket-SSL
• Authen-SASL
• Net-SMTP-SSL
  

Unfortunately the Net_SSLeay and IO-Socket-SSL contain C code and need to be compiled. Fortunately you might be able to find a Perl repository that has already built these packages for your platform e.g. http://archive.apache.org/dist/perl/win32-bin/ppms/ for Perl 5.8 and the Win32 platform. So using the Perl Package Manager (PPM) you can install the packages by doing something like:

rep add "Apache Perl 5.8.x" http://archive.apache.org/dist/perl/win32-bin/ppms/
install Net_SSLeay.pm
install IO-Socket-SSL
install Authen-SASL

With Net-SMTP-SSL you can simply extract the SSL.pm file from the distrbution and place it in the <PERL_INSTALL_DIR>/lib/Net/SMTP directory.

Now that you have all of the packages you need installed, a simple Perl script to send email via the Google SMTP server looks something like:

#!/usr/bin/perl -w

use Net::SMTP::SSL;

sub send_mail {
my $to = $_[0];
my $subject = $_[1];
my $body = $_[2];

my $from = 'johnny@gmail.com';
my $password = 'MySecretGmailPassword';

my $smtp;

if (not $smtp = Net::SMTP::SSL->new('smtp.gmail.com',
                            Port => 465,
                            Debug => 1)) {
   die "Could not connect to server\n";
}

$smtp->auth($from, $password)
   || die "Authentication failed!\n";

$smtp->mail($from . "\n");
my @recepients = split(/,/, $to);
foreach my $recp (@recepients) {
    $smtp->to($recp . "\n");
}
$smtp->data();
$smtp->datasend("From: " . $from . "\n");
$smtp->datasend("To: " . $to . "\n");
$smtp->datasend("Subject: " . $subject . "\n");
$smtp->datasend("\n");
$smtp->datasend($body . "\n");
$smtp->dataend();
$smtp->quit;
}

# Send away!
&send_mail('johnny@mywork.com', 'Server just blew up', 'Some more detail');

Sending email with attachments is a little trickier since we have to construct multi-part messages. The Net::SMTP::Multipart module provides a wrapper around Net::SMTP (but not Net::SMTP::SSL) to support attachments, but I don't like the syntax it requires and lack of MIME types guessing so I extracted the core logic from that module into the example below:

#!/usr/bin/perl -w

use Net::SMTP::SSL;
use MIME::Base64;
use File::Spec;
use LWP::MediaTypes;

sub send_mail_with_attachments {
 my $to = shift(@_);
 my $subject = shift(@_);
 my $body = shift(@_);
 my @attachments = @_;

 my $from = 'johnny@gmail.com';
 my $password = 'MySecretGmailPassword';

 my $smtp;

 if (not $smtp = Net::SMTP::SSL->new('smtp.gmail.com',
                              Port => 465,
                              Debug => 1)) {
     die "Could not connect to server\n";
 }

 # Authenticate
 $smtp->auth($from, $password)
     || die "Authentication failed!\n";

 # Create arbitrary boundary text used to seperate
 # different parts of the message
 my ($bi, $bn, @bchrs);
 my $boundry = "";
 foreach $bn (48..57,65..90,97..122) {
     $bchrs[$bi++] = chr($bn);
 }
 foreach $bn (0..20) {
     $boundry .= $bchrs[rand($bi)];
 }

 # Send the header
 $smtp->mail($from . "\n");
 my @recepients = split(/,/, $to);
 foreach my $recp (@recepients) {
     $smtp->to($recp . "\n");
 }
 $smtp->data();
 $smtp->datasend("From: " . $from . "\n");
 $smtp->datasend("To: " . $to . "\n");
 $smtp->datasend("Subject: " . $subject . "\n");
 $smtp->datasend("MIME-Version: 1.0\n");
 $smtp->datasend("Content-Type: multipart/mixed; BOUNDARY=\"$boundry\"\n");

 # Send the body
 $smtp->datasend("\n--$boundry\n");
 $smtp->datasend("Content-Type: text/plain\n");
 $smtp->datasend($body . "\n\n");

 # Send attachments
 foreach my $file (@attachments) {
     unless (-f $file) {
         die "Unable to find attachment file $file\n";
         next;
     }
     my($bytesread, $buffer, $data, $total);
     open(FH, "$file") || die "Failed to open $file\n";
     binmode(FH);
     while (($bytesread = sysread(FH, $buffer, 1024)) == 1024) {
         $total += $bytesread;
         $data .= $buffer;
     }
     if ($bytesread) {
         $data .= $buffer;
         $total += $bytesread;
     }
     close FH;

     # Get the file name without its directory
     my ($volume, $dir, $fileName) = File::Spec->splitpath($file);
  
     # Try and guess the MIME type from the file extension so
     # that the email client doesn't have to
     my $contentType = guess_media_type($file);
  
     if ($data) {
         $smtp->datasend("--$boundry\n");
         $smtp->datasend("Content-Type: $contentType; name=\"$fileName\"\n");
         $smtp->datasend("Content-Transfer-Encoding: base64\n");
         $smtp->datasend("Content-Disposition: attachment; =filename=\"$fileName\"\n\n");
         $smtp->datasend(encode_base64($data));
         $smtp->datasend("--$boundry\n");
     }
 }

 # Quit
 $smtp->datasend("\n--$boundry--\n"); # send boundary end message
 $smtp->datasend("\n");
 $smtp->dataend();
 $smtp->quit;
}

# Send away!
&send_mail_with_attachments('johnny@mywork.com', 'Server just blew up', 'Some more detail', 'C:\logs\server.log', 'C:\logs\server-screenshot.jpg');

A couple of interesting things to note:

• The Google SMTP server is available on port 465, which is the standard port for secure SMTP.
• Google SMTP always uses your fully qualified gmail account name as the "from" (i.e. the account name you used to authenticate with the SMTP server), even if you specify a different from when constructing the message headers and body.
  

Using Spring AOP to log swallowed exceptions

One example of bad programming style is to catch exceptions and then silently do nothing with - them, aka "swallowing" exceptions - which looks something like this:

...
preWork();

try {
doWork();
} catch (Exception e) {
// FIXME
// Dunno what to do with this exception...
}

postWork();
...

If this is your own code then you should fix it (and then severly punish yourself!). If it is someone elses code - for example a third party lib - and you have to use it then it can be very frustrating to have your code die silently; it makes code so much more difficult to debug.

(As a side note, exception swallowing is probably more common in Java because of a feature of the language - typed exceptions. But I've seen swallowed exception code in plently of other C++-like languages.)

I recently had such a problem when I was enhancing a Spring MVC application that used Tiles to lay out pages. Tiles often swallows exceptions. gggrrr.

What to do? It would take a fair amount of effort to fix exception handling in the Tiles code and also a fair amount of effort to step through the application code with a debugger.

Fortunately, Spring - the ultimate software hammer - provided the base infrastructure for a simple solution:

• Use ThrowsAdvice AOP advice to intercept an exception thrown from the application code, log the exception at ERROR level, and let the exception flow continue.
  

Step 1: Create a ThrowsAdvice to that will intercept and log an exception

package robertmaldon.util;

import java.lang.reflect.Method;

import java.io.PrintWriter;
import java.io.StringWriter;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import org.springframework.aop.ThrowsAdvice;

public final class LogThrowsAdvice implements ThrowsAdvice {
 private Log logger = LogFactory.getLog(this.getClass());

 public void afterThrowing(Method method, Object[] args,
                           Object target, Exception ex) throws Throwable {
     logger.error("Intercepted exception of type ["
                  + ex.getClass().getName()
                  + "] thrown by target class ["
                  + target.getClass().getName()
                  + "] and method [" + method.toString() + "]");

     logger.error(getStackTrace(ex));

     // Rethrow the exception in case something else downstream can
     // handle it.
     throw ex;
 }

 private String getStackTrace(Exception ex) {
     try {
         StringWriter sw = new StringWriter();
         PrintWriter pw = new PrintWriter(sw);
         ex.printStackTrace(pw);

         return sw.toString();
     } catch (Exception e) {
         return e.toString();
     }
 }
}

Step 2: Create the bean definition for the ThrowsAdvice and configure an autoproxy to wire the advice to your application code.

<!-- Bean definitions for your application code -->
<bean id="petStoreController1" .../>
<bean id="petStoreController2" .../>

<!-- Define the ThrowsAdvice bean -->
<bean id="logThrowsAdvice" class="robertmaldon.util.LogThrowsAdvice" />

<!-- Wire the advice to the application code -->
<bean class="org.springframework.aop.framework.autoproxy.BeanNameAutoProxyCreator">
<property name="beanNames"><value>petStoreController*</value></property>
<property name="interceptorNames">
 <list>
   <value>logThrowsAdvice</value>
 </list>
</property>
</bean>

Step 3: There is no step 3!

Very simple, very easy.

This approach is good for development, but may introduce too much overhead for production.

CSS changes in IE7

A list of CSS changes in IE7. The UI expert on our team was extremely happy to hear IE finally supports transparent PNGs!

You can test IE7 without installing it (i.e. without blowing away your current IE installation).

Who says Wikipedia is inaccurate?

Wikipedia Celebrates 750 Years Of American Independence

Wikipedia, the online, reader-edited encyclopedia, honored the 750th anniversary of American independence on July 25 with a special featured section on its main page Tuesday...

The special anniversary tribute refutes many myths about the period and American history. According to the entry, the American Revolution was in fact instigated by Chuck Norris, who incinerated the Stamp Act by looking at it, then roundhouse-kicked the entire British army into the Atlantic Ocean. A group of Massachusetts Minutemaids then unleashed the zombie-generating T-Virus on London, crippling the British economy and severely limiting its naval capabilities...

ASP.NET providers now pluggable

A couple years ago I wanted to learn C#, so a good way I thought was to write a medium sized project in C#. For the project I decided to do a Tomcat/Jetty-like "servlet" engine, but using ASP.NET instead of JSPs.

The Microsoft company line at the time was "if you want to do ASP.NET, you have to buy/install/run on IIS". Well, after I did a bit of research that turned out to be only half true. The actual ASP.NET engine was part of the .NET core runtime. Although it wasn't documented how to pass HTTP requests through to the ASP.NET engine Microsoft kindly provided just such an example with the Cassini sample web server.

So far so good. However, when I looked into slightly more complex things like security and session replication I hit a dead end - all of these things were very closely tied to SQL Server, all of the interfaces that would have been useful to extend and make generic were marked "sealed". Bummer. I gave up that pet project because I didn't want to be tied to just SQL Server.

I was therefore plesently surprised to see an infoq article titled "Create Your Own ASP.NET Providers", which tells you how to implement user and session providers in ASP.NET 2.0. I'm very tempted to resurrect that old project and implement some of the new Jetty 6 features like continuations and large scale NIO in C#. (Jetty.net anyone?)

John Davies podcast on the current Investment Banking Technology Stack

In this podcast, taken from a keynote presentation given at TSSJS-Europe, C24 co-founder and CTO John Davies discusses the technology stack that is currently used by most of the investment banks. (Those of us who work in the industry have seen these trends, but it is good to get this type of info out in the open :))

Some of the highlights (in no particular order):

A typical stack

• Excel (in his view is the only part of the Microsoft stack that is 'necessary' because it is the tool of choice by traders. Better not tell that to all of the people building rich .NET clients :))
• Tibco/RV (as well as being high performance, it integrates well with Microsoft technologies)
• MQSeries (aka MQ or WebSphereMQ), and often using JMS to interact with MQ rather than the native MQ API
• A J2EE or more "light-weight" Java application server stack

Application Servers

WebLogic/WebSphere/JBoss are still heavily used but are on the way out. New projects must justify the use of an application server. If they can't justify an application server then they need to use a more light-weight stack such as Tomcat/Spring/Hibernate.

No more EJBs!

Entity Beans in particular have been "banned for years".

Open Source

It's not just the low cost that banks find attractive, its more that the source code is always available of needed - no worries about the company that makes the product going broke or stopping development/innovation/support of the product.

AMQ - Messaging will become a commodity

AMQ, a wire-level protocol which a number of vendors are cooperating on, is set to commoditize messaging - and break the back of Tibco and IBM who charge lots of money for RV and MQ licenses! JP are already running two implementations of AMQ internally. RedHat will ship AMQ implementations with its operating system packages.

Databases are dead - Long live the database!

Your traditional relational database - Oracle, Sybase, etc - are on the way out. Memory is so cheap that finanacial systems are now loading a database into memory at the start of the day and dumping the modified data back out at the end of the day. Mr Davies was very careful to point out that these were in-memory databases and not caches, but didn't go into details about how integrity was preserved if the server power failed.

How many toolbars can you add to IE7?

Funny idea. Find out why someone would want to do this.

Web 2.0 Attacks

A good article on the ways to attack a Web 2.0 application, thanks to the growing number of technologies used in "rich internet applications" (html, flash, applets, javascript, etc):

1. Cross-site scripting in AJAX. A pre-AJAX browser exploit, but AJAX makes it easier to hide such an attack.
   
2. XML poisoning. A denial-of-service attack that tries to exploit weaknesses in server-side XML parsers. Do something nasty like apply a recursive payload to an XML block.
3. Malicious AJAX execution. Some javascript that sits in the background, captures info from a server (e.g. the user has signed into their on-line bank account) and sends that info to a identity theft server. All done silently.
4. RSS / Atom feed injection. RSS is becoming a popular distribution mechanism. Since most people view feeds in their browser it becomes very easy to slip some malicious javascript into a feed.
5. WSDL scanning and enumeration. Looking at the exposed method interfaces of a web service a malicious person may be able to guess at ways to atack the interface. e.g. if you see a "debug" or "override" parameter you might be tempted to see what it does.
6. Client side validation in AJAX. For those people who define validation on the client side but are too lazy to do the equivalent validation on the server side, don't be surprised if someone breaks your application by making a malicious call directly to the server.
7. Web services routing issues. WS-Routing allows SOAP messages to travel in specific sequence from various different nodes on the Internet. Often encrypted messages traverse these nodes. A compromise of any of the intermediate nodes results in possible access to the SOAP messages traveling between two end points.
8. Parameter manipulation with SOAP. Try manipulating the values of the SOAP messages to do things like SQL-injection attacks.
   
9. XPATH injection in SOAP messages. ditto to the point above.
10. RIA thick client binary manipulation. An attacker can reverse engineer the binary file (e.g. flash .swf files) and decompile the code.

Update: DLG asked what are some guidelines to protecting against such attacks? Some of my thoughts are:

• AJAX calls only work cross-site if using iframes. As with javascript the browsers should pop us a security alert asking if cross-site AJAX calls should be allowed on the page.
• XML Poisoning / Parameter manipulation / XPATH injection. You should unit test a number of such attacks to see what happens. You may need to work with your vendor to fix any issues in the XML stack they provide.
• WSDL scanning. A code review should find any methods that should not be exposed to clients. What could make this tricker, however, is that many of the SOAP standards now use a document-centric approach, where the WSDL is a thin wrapper around an XML "blob" payload.
  

Chin up Rover!

It will probably never be a big selling item, but I liked the cute Doggy Car Window Head/Chin Rest I saw in the Newport stores while doing the tourist thing on the weekend. It's a ruff life. Woof!

Clippy gets ported to Linux

Clippy the Microsoft Office Assistant has inspired a Linux clone called Vigor. Need help with the evil six editor (VI)? No problem!

Fixing those AJAX Javascript memory leaks

One thing I've noticed with a number of the recent wave of AJAX applications is that they leak memory. For example, editing googlepages grinds my 512MB computer to a halt in 20 minutes of so.

Jack Slocum has a few tips on avoiding javascript leaks, including using the LeakMonitor Firefox extension, and points out leaks in common AJAX libraries like prototype.

Sarbanes-Oxley squashes IPOs?

Interesting story I saw on Power Lunch while working out at the gym. IPOs are down in Q3 in the US, but steady or up in overseas markets like Europe. The reason? Many business people are finding Sarbanes-Oxley requirements too onerous for listing on the Nasdaq and are preferring to take a company private or list on a UK exchange instead.

The future of Web 3.0: Thick clients?

The New York Times has launched Times Reader, a .NET thick client that provides a more faithful rendering of Times stories than can be achieved with a browser (so they claim), such as:

• the same font and column structure you see in the printed paper; and
  
• no scrolling necessary -- just use the arrow keys on your keyboard to turn the page

Can you do this with a browser? Depends on the browser :) The standard bodies have done there job with the CSS 3 Multi-Column Layout spec. Some clever people have created javascript hacks that display multi-coulmns for pure text. And the Mozilla dudes have preliminary support for multi-column content in the current production version of Firefox 1.5.

So which browser doesn't support multi-column layouts in a meaningful way? You guesses it, Internet Explorer. And font support tends to be operating system related. So why don't the NY Times push Microsoft to make Internet Explorer standards compliant instead of building thick clients that will only run on Windows?

(Lack of CSS support, including the upcoming IE 7, is still the biggest gripe I hear when talking to web design professionals.)

On a related note, what will our interface to the internet look like post-Web 2.0? Will it be thick clients instead of a browser? I guess I'm used to a browser, but the thought of having to learn a different UI to navigate each "web site" I access is frightening. Some of the Finetix UI experts I've discussed this with can only see AJAX + CSS in the near and medium term.

Swing gone crazy

An article I saw yesterday on the SWTSwing project, which attempts to allow you to program to the SWT interfaces but really use Swing underneath, brought back a lot of bad Java GUI memories for me. Here is how I remember the big mess that has been made of Java on the client side:

In the beginning - Java 1.0 - Java supported GUI components as part of its core library. At the time this was fantastic because most of the other competing languages did not provide a "standard" GUI package. The Java GUI framework was called Abstract Window Toolkit (AWT).

Now, AWT had its limitations. It only supported a few of the most common widgets. It also had threading issues - all of the AWT components are updated from a single event thread, so you have to jump through a few hoops to get updates from another thread on to the AWT thread (NOTE: this is common in GUI frameworks, including .NET forms). And most serious from Sun's point of view (and IBM complained a lot about it as well) was that AWT was built on top of "native" widgets, which look different on different operating systems.

Sun's solution - with more than a little help from IBM - was Swing. Swing had lots of widgets - menus, popups, sliders, you name it. Threading was solved by implementing a MVC pattern. And the different look on different operating systems problem was solved by "drawing" (simulated) each Swing widget so that it looked identical on each operating system.

Swing, however, was painfully slow. Most of the blame was attributed to the widgets have to draw themselves line-by-line, pixel-by-pixel, although I reckon some of the internal implementation was just badly done.

So a few years went by without Sun doing much to improve the performance of Swing. There were a few religious wars over native widgets vs simulated widgets, but Sun stuck to their guns when it came to ensuring the same look and feel across all operating systems.

Finally IBM - cohorts in the creation of Swing - went a different direction and created SWT, which was based on top of native widgets. And it didn't have any MVC support. SWT was used as a foundation for Eclipse.

A couple of years ago RedHat decided to support Java in a big way. In particular they wanted to add Java compiler support - actually compile Java programs to assembler - to the fantastic gcc and be able to run Eclipse on Linux. One major stumbling block to this ambition was getting Swing to run natively. Since SWT was implemented on top of native widgets it worked pretty much straight away with gcc. So along comes the SwingWT project which implemented the Swing interfaces on top of SWT.

And as I said at the top of this post SWTSwing implements SWT on top of Swing. And now SWTSwing is integrated with SwingWT, which is "great for those places where you want native if available, but can fall back to Swing if not".

When will this madness end! (Particularly now that Swing in Java 5 is actually pretty speedy and looks ok.)

Desperate to try the new Flaming Big Mac

And I thought this would only happen in America. From the Sydney Morning Herald:

When you've got to eat, you've just got to eat. That was the case at a McDonald's restaurant today when the lunchtime rush continued unabated even though a fire had broken out in the establishment's dumpster bin...

"Someone called the fire brigade [but] in the meantime an endless stream of vehicles continued to drive in through the drive-thru and continued to be served, which is quite amazing because there were huge flames.

Dot com days returning?

From a Wall Street Journal article:

Two and a half years ago, Facebook was a college project run by an undergraduate. Today, in an echo of the 1990s technology boom, it is being chased by large companies with their wallets wide open.

During one series of talks with Microsoft, Facebook executives told their Microsoft peers they couldn't do an 8 a.m. conference call because the company's 22-year-old founder and chief executive, Harvard dropout Mark Zuckerberg, wouldn't be awake, says a person familiar with the talks. Microsoft executives were incredulous.

In an interview, Mr. Zuckerberg declines to comment on any talks. The young entrepreneur says he generally works late -- he recalls eating French fries recently in the parking lot of a local McDonald's restaurant at 3 a.m. -- and doesn't get to work early. "I'm in the office at 10:30 a.m. sometimes," he says.

Who was Winston Churchill?

Status of Churchill at Parliament Square, London

Question on BBC1's Test the Nation: "Who was Winston Churchill?"

-A rapper
-US President
-The Prime Minister
-The King?

Teddy Sheringham (famous English domestic and international football player)'s girlfriend, Danielle Lloyd:

"Wasn't he the first black president of America? There's a statue of him near me - that's black."

Using the WebSphere transaction manager with Spring and Hibernate

It is surprising that there is almost no documentation on how to integrate the common Spring/Hibernate combination with WebSphere's transaction manager. (The Hibernate forums berate IBM for not providing info on WebSpere JTA hooks, and you have to crawl through some of the Spring code to find the correct classes to use.)

For simple applications that only hit a single database you can probably get away with using Hibernate's default simulated transactions, but if your app has to talk to multiple transactional resources (good ol XA) then it's a good idea to hook into WebSphere's transaction manager.

Without further suspense here is how you do it:

First, use the highlighted Hibernate properties when defining the Spring session factory:

<bean id="petStoreSessionFactory" class="org.springframework.orm.hibernate.LocalSessionFactoryBean">
<property name="hibernateProperties">
<props>
 <prop key="hibernate.dialect">org.hibernate.dialect.Oracle9Dialect</prop>
 <prop key="hibernate.show_sql">false</prop>
 <prop key="hibernate.transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</prop>
           <prop key="hibernate.transaction.manager_lookup">org.hibernate.transaction.WebSphereExtendedJTATransactionLookup</prop>
           <prop key="hibernate.transaction.flush_before_completion">true</prop>
           <prop key="hibernate.transaction.auto_close_session">true</prop>
 <prop key="hibernate.cache.use_query_cache">false</prop>
 <prop key="hibernate.cache.use_second_level_cache">false</prop>
</props>
</property>
</bean>

If you are using Spring 2.1 RC1 or later then define a transaction manager using the WebSphereUowTransactionManager bean:

<bean id="wsJtaTm" class="org.springframework.transaction.jta.WebSphereUowTransactionManager"/>

For earlier versions of Spring that do not provide org.springframework.transaction.jta.WebSphereUowTransactionManager, and for versions of WebSphere earlier than V6.0.2.19 or V6.1.0.9 that do not provide com.ibm.wsspi.uow.UOWManager, transaction support in WebSphere is available by using a basic Spring transaction management bean :

<bean id="wsJtaTm" class="org.springframework.transaction.jta.JtaTransactionManager">
 <property name="autodetectTransactionManager" value="false"/>
</bean>

This configuration supports a restricted set of transaction attributes that does not include PROPAGATION_NOT_SUPPORTED and PROPAGATION_REQUIRES_NEW. The Spring class org.springframework.transaction.jta.WebSphereTransactionManagerFactoryBean, which also claims to provide PROPAGATION_NOT_SUPPORTED and PROPAGATION_REQUIRES_NEW capabilities, uses unsupported internal WebSphere interfaces and should not be used.

And finally when you define a TransactionInterceptor or TransactionProxyFactoryBean make sure to use the WebSphereTransactionManagerFactoryBean with the JtaTransactionManager :

<bean id="petStoreTransactionInterceptor" class="org.springframework.transaction.interceptor.TransactionInterceptor">
<property name="transactionManager">
        <bean class="org.springframework.transaction.jta.JtaTransactionManager">
            <property name="transactionManager" ref="wsJtaTm"/>
        </bean>
    </property>
<property name="transactionAttributeSource">
<bean class="org.springframework.transaction.interceptor.NameMatchTransactionAttributeSource">
 <property name="properties">
     <props>
         <prop key="insert*">PROPAGATION_REQUIRED</prop>
         <prop key="update*">PROPAGATION_REQUIRED</prop>
         <prop key="*">PROPAGATION_REQUIRED,readOnly</prop>
     </props>
 </property>
</bean>
</property>
</bean>

And that's it. As my old colleague Stuart Cooper likes to say: "MMMaaagggiiiccc!!!"

Update: The article "Using Spring and Hibernate with WebSphere Application Server" highlights some Spring 2.0 specific integration issues, such as transaction isolation levels and threads.

Windows UI team now blogging

Ever wondered why the Windows UI is the way it is, or want to ask about one of the Vista features? (Or even what ideas they borrowed from Apple for Windows 3.1 and from XWindows for Win 95) ? Well now you can ask the dudes/dudettes directly on the Shell:Revealed blog.

Challenges of running a Managed Service

Over recent years I've seen quite a few companies, particularly IBs, create their own IT Managed Services. They are interesting beasts, and I learnt quite a but about them while working inside a large IB that established a Managed Service based on a J2EE infrastructure. Below are some of my observations of these beasts.

What is a Managed Service?

A Managed Service assumes responsilibity of day-to-day running of important business critical systems. The same Managed Service group can assume responsibility for a number of applications that cross multiple business lines.

The main purpose of a Managed Service is to save money by centralizing operations and infrastructure instead of having to duplicate the same stuff for every new application.

A Managed Service model is very similiar to an ASP or help-desk model, but differs in that it is usually internally run (i.e. not outsourced, but many members of the group may still be Inidian consultants) and closely aligned with the business.

What are the incentives for starting/running/using a managed service?

The main incentives to run a managed service are:

• Reduced cost by hosting as many applications as possible on common servers (i.e. less servers required to host the same number of applications than if each business group had their own servers)
• Reduced number of staff required to support the infrastructure (i.e. each business group does not need to have its own support teams)
• Reduced cost by using common infrastructure (i.e. developers do not have to keep "re-inventing the wheel" every time)
• You do not have the resources to deploy and support applications, so you are happy to use somebody else's infrastructure (as long as the price is right!)

What are some of the barriers to starting a managed service?

• Control. IT people in particular like to have control over their infrastructure and will often re-invent the wheel.
• My empire is bigger than yours. A managed service is likely to be used by many different business groups and therefore has the potential to be larger than a group within one business line. Managers within the business lines may get very jealous.

Keeping it Visible

The most important thing you can do to establish and maintain a managed service is to keep the service visible. i.e. tell everybody about the service and keep telling them.

If people don't know about the managed service they won't use it. If people become complacent about the service they will stop using it.

Some ways you can keep a managed service visible are:

• Regular presentations to managers and developers.
• Comparison of the managed service with similiar services. It is often hard to get information on managed services being run inside competing companies; it is easier to get information on services provided by application hosting companies. The most important comparisons are cost and Service Level Agreements (SLAs).
• Statistics. Managers need to know how the managed service is being used, including:
  • Number of applications deployed
  • Number of outages
  • Resources used by the applications (memory, CPU, disk space, etc)
• Provide feedback mechanisms. Customers need to feel they are being listened to.

Environments

Depending on how much money you have it is generally standard practice to have at least the following three environments:

Environment	Description
Development	A development environment should be similiar to a production environment, but developers should be given the freedom to experiment with it. With freedom comes the potential for disaster. If possible isolate one development environment from another. Make it clear to developers that support for development environments is much less than support for QA and production environments. The worst case should be you may have to destroy and then recreate the environment from scratch.
Quality Assurance (QA)	The QA environment should be as similiar to production as possible. This means you follow the same processes to get an application into the QA environment as you follow to get the application into the production environment. Access to the QA environment should be restricted to the managed service support staff.
Production	The processes for deploying an application into production must be well defined, well understood and - above all else - auditable. Auditability simply means all changes to an environment (e.g. a change to configuration or a new version of an application is installed) have been recorded and are reproduceable. This often means changes are scripted and those scripts are put under version control. Access to the production environment must be restricted to the managed service support staff.

Management, Management, Management

Some of the key things you need to manage in a managed service:

• Well-defined Change Management Processes. The processes of getting an application installed into a managed environment must be well documented and well communicated.
• Change Windows. It must be well reported when changes can be applied to a managed environment. Changes to a production environment have to typically be done out-of-business-hours, such as late at night and at weekends. The processes to get your application into a change window must be well documented.
• Escalation Procedures. Formal procedures to raise issues is a must. If you don't have any escalation procedures then people will do all sorts of ad-hoc and chaotic things to work around problems; often an issue will be reported by a person up through their management chain, then down through your management chain.
• Request / Bug Tracking system. A system to track work requests and bugs gives the impression of a professionally run managed service.
• Patching Policy. One of the trickiest issues to deal with is the issue of patching. Let us say you are running 10 applications on the same server using the same libraries. One of the applications report a bug. A bug fix is made to one of the libraries. The dilema: Do you apply the patch required by the one application to the libraries used by all 10 applications? Do you do regular (e.g. six montly) patch updates to your managed service? If you don't do at least regular updates then your infrastructure becomes difficult to support; big hardware and software vendors do regular updates to their products and can be slow to fix old versions.
• The Stick. People must be given incentive to not abuse the managed service infrastructure. There must also be incentive to agree with the patching policy. The only incentive that really works is to charge people more money if they don't follow the managed service guidelines.

Don't forget the developers!!!

Starting a managed service and continued funding for a managed service will require the support of senior managers within the company, so they should obviously the primary target of your efforts. Developers will (most of the time) use whatever their managers tell them to use.

However, don't neglect the developers. If the developers are not happy with a managed service then they can spread rumours. And some developers will become managers in the future.

Developers are usually interested in the technical details and the processes. Ways to keep developers happy include:

• Simple, brief "Getting Started" documentation. This includes doco on:
  • Guidelines on use of the supported technologies (e.g. "this is how you use Message Driven Beans inside WebSphere over MQ")
  • Using various development tools
  • Processes to build and deploy applications in the managed environment
  • Tricks and gotchas
  An internal website is a good place for this stuff.
• Choice. Developers like to experiment. While a managed service must lock down processes for QA and production environments, you must allow a bit more freedom in the development process. For example, you can recommend and officially support a particular development tool, but you shouldn't try to force developers to use those tools.
• Technology chat channels. Get into the 21st century and install an instant messaging system. Developers love to share insights and opinions with other developers.
• Access to debug information. Since developers won't have open access to QA and production environments they will need access to some sort of debug information so they can support the applications. At a minimum they will need secure access to application log files.
• Request / Bug Tracking system. The ability for developers/managers to sumbit requests for work and bug reports. The submitter can access the system to check the progress of their request, and review the work that was done to complete previous requests.

Beating the freeze - Disabling XP's built-in ZIP support

I'm often downloading and experimenting with all sorts of software and more often than not those downloads are packages as a ZIP file. Out of habit I dump all of those zip files in a single archive folder. A nasty side effect of XP's built-in support for ZIP files is that when I use Windows Explorer to open that archive folder Explorer and often other applications just freeze for a few minutes. The cause is that Explorer is trying to open and examine all of the ZIP files in the folder... gggrrr...

Thanks to this post I now have the solution to disabling the built-in ZIP support (I prefer using 7-zip anyway).

To disable XP's ZIP support:

1. Select Run from the Start menu and enter

regsvr32 /u %windir%\system32\zipfldr.dll

2. Click OK.


To re-enable XP's ZIP support:

1. Select Run from the Start menu and enter

regsvr32 %windir%\system32\zipfldr.dll

2. Click OK.

Transient Documents

An interesting new product from Xerox: reusable paper in the sense that the content is automatically erased after a period of time, ready for fresh printing. This would save a few trees from the occasional thing I print out to read on the train going home.

MS promise not to assert web services patents

In a move to keep web services open and standard Microsoft have pledged not to assert any patents it has on 35 listed web services standards. MS did a similiar thing with regards to the XML schemas defenititions in Office2003.

What is the legal status on XAML patents I wonder?

Deep-fried Coca-Cola

Thanks to the Onion for pointing me to this innovative food that starred at this years Texas State Fair. Don't forget to take the survey!

What's next? Mmmm. Deep-fried water.

Celebrity baby pics really are big business!

I've always been sceptical of exactly how much those celebrity baby pictures actually adds to the bottom line of a newspaper or magazine. Until I saw a reprint of this article in the free (as in beer) amny :

The nationwide obsession with baby TomKat drove a whopping 4.3 million page views on vanityfair.com on Wednesday, according to a spokeswoman, nearly three times more than the previous record for page views when the title posted b-roll of Keira Knightley and Scarlett Johansson from Tom Ford's Hollywood issue in February. VF also signed up 4,000 subscriptions on Wednesday, its largest one-day total, from that day's 1.9 million unique visitors. No doubt a special offer that guaranteed receipt of the October issue to new subscribers (while supplies last) helped pique interest. On Thursday, the Web site attracted 1.1 million page views, 465,000 unique visitors and 1,700 new subscribers. Comparatively, vanityfair.com averages about 60,000 page views and about 20,000 uniques a day.

Let's assume TomKat got $3 million for the photos (I haven't seen the amount reported, but it is safe to assume they got less than the $4 million for the Brangelina photos).

Let's also assume the 5,700 new subscribes chose the bargin 24 issues for $40 deal, which would net Vanity Fair $228,000. Not bad for just two days "work".

Who knows how much other revenue was generated by print and on-line advertising.

.NET CLR faster than JVM for dynamic languages

An interesting post from the dude that originally ported Python to the JVM and who just finished version 1.0 of the .NET Python port:

I started work on IronPython almost 3 years ago. My initial motivation for the project was to understand all of the reports that I read on the web claiming that the Common Language Runtime (CLR) was a terrible platform for Python and other dynamic languages. I was surprised to read these reports because I knew that the JVM was an acceptable platform for these languages. About 9 years ago I'd built an implementation of Python that ran on the JVM originally called JPython and later shortened to Jython. This implementation ran a little slower than the native C-based implementation of Python (CPython), but it was easily fast enough and stable enough for production use - testified to by the large number of Java projects that incorporate Jython today.

I wanted to understand how Microsoft could have screwed up so badly that the CLR was a worse platform for dynamic languages than the JVM. My plan was to take a couple of weeks to build a prototype implementation of Python on the CLR and then to use that work to write a short pithy article called, "Why the CLR is a terrible platform for dynamic languages". My plans quickly changed as I worked on the prototype, because I found that Python could run extremely well on the CLR - in many cases noticeably faster than the C-based implementation. For the standard pystone benchmark, IronPython on the CLR was about 1.7x faster than the C-based implementation...

Selling stocks the subliminal way

Watch the advert at the end of this post. You are getting sleepy. You love this stock...

Apple/Google join forces to take on Microsoft?

Eric Schmidt, CEO of Google, has joined the board of Apple. Hhhmmm.

XML more compressed than MS Office binary format?

An interesting tidbit from an interview on the new "Ribbon" interface in Office 2007:

One likely incentive for that migration will be reduced storage costs. Microsoft claims that file sizes for the new Office 2007 XML-based formats are up to 75 percent less than existing Office formats.

How can Microsoft have been so inefficient with the previous Office binary formats? Or will the new XML formats be impossible for humans to read?

Winning the War on Terror... through PowerPoint

An interesting blog about how the US military has recently been using PowerPoint slides to convey battle strategies instead of clearly written orders:

[Army Lt. General David] McKiernan had another, smaller but nagging issue: He couldn't get Franks to issue clear orders that stated explicitly what he wanted done, how he wanted to do it, and why. Rather, Franks passed along PowerPoint briefing slides that he had shown to Rumsfeld...

Here is one of the slides used to convey how the occupation phase of the Iraq conflict was expected to work:

Reminds me of The Gettysburg Powerpoint Presentation, a what-if Lincoln had access to PowerPoint when giving one of his most famous speeches.

Stargate SG-1 : End of an era

I was disappointed yesterday to see the news that the sci-fi channel will be cancelling Stargate SG-1 at the end of this record-breaking 10th season. I didn't get into SG-1 until I was unemployed for a few months in the wake of the 9/11 economic downturn, when I had the chance to watch SG-1 reruns and view both the good and the bad eprisodes.

I thought the battles with the new uber-enemy the Ori were developing into some good plots, but I guess they were running out of ideas.

At least I have the Peabody-winning Battlestar Galactica to keep my sci-fi interest going. (And the derivative Stargate Atlantis if nothing else is on :))

Finding Memory Leaks in Java : Using HeapSummary

One of the Java questions I like to ask at interviews is: Can you have a memory leak in Java? (or .NET, or any other language with built-in garbage collection). Surprisingly about 70% of people say no... What this can tell me about a candidate (I think) is that they have not worked on a large-scale, heavily used application.

For the record here are my thoughts on causes of leaks and how you track them down.

Symptoms of Leaks

In Java the main symptom is "OutOfMemoryError" exceptions in your log files, followed closely by the application dying.

Causes of Leaks

The most common causes of various memory probelms in Java are:

1. Applications code that holds on to object references. e.g. the application keeps adding objects to global/singleton Maps or Lists and never removing these objects.
2. An Application-level cache is misconfigured or does not behave as the developers expected.
3. (More rare) An application server or third-party library has a leak for the same reasons as above.
4. (Very rare) The C implementation of the JVM has a classic memory leak or suffers from fragmentation.

When a Leak is not a Leak

Sometimes a leak is not really a leak. Sometimes you just need to allocate more memory to your application than you initially guessed/estimated, particulalry if you have more users than expected. So how do you determine the write memory size and garbage collector settings for your application?

Application Load Testing

The best way to both size an application (hopefully before you release to production) and to track down any memory problems is to put the application under a realistic load. The key word in that sentence is realisitc, meaning the load should be approximately the same load pattern you expect to see in production. It's always frustrating to explain to people that you should probably expect an application to fall flat on its face if your first load test is to throw 1 million simultaneous requests at it. Instead you need to measure the application scalability, in throughput per second, at loads approximate to (both above and below, but around) the production load.

What tools can you use to generate load? The most common tools I've seen used are JMeter (my tools of choice, very easy to get up and running), The Grinder and LoadRunner. Those tools simulate client requests, but because they are not web browsers they can't interpret javascript, so may not be suitable to load test applications that are heavily DHTML based. For DHTML types of applications tools that drive a browser like Selenium and WinRunner are more suitable - but you will need a lot of client machines to generate a decent load.

Profiling

To help detect a leak you will need the ability to profile your application.

The most basic information you need from a profiler is how much memory is being used by all of the classes in the JVM. If one class in particular, say a domain object called StockOption, keeps using up more and more memory as your load test continues (if there are more instances of the object that keep getting added to a Map then of course that class will use up more memory) then that is a good leak candidate to investigate.

A really useful profiler should have the ability to show you how instances of objects are related to each other ("heap walking") so you can trace where objects are allocated and where they are referenced. Only the commercial profilers like JProfiler and OptimizeIt have these features. I wish a decent open source profiler was available :(

The most basic memory leak can be found by:

• Start the application
• Record memory usage of classes (number of instances and amount of memory used)
• Apply some load, then stop
• Run a full GC (or two GCs, just to be sure)
• Compare the class memory usage to before. The cause of the leak should be at or near the top of the list of most amount of memory being used.

One thing to note is that profilers tend to slow down the application by at least 10 times, so learn to be patient :)

Application Tuning

This is a really big topic. If I get the energy I'll tackle it in another blog entry.

Gathering Memory Statistics from Production : HeapSummary

One thing I like to recommend when dealing with production memory leaks is to get some basic heap information from the production systems themselves. Ideally you want to be able to reproduce memory leaks in a non-production system such as Staging or QA, but sometimes you just don't know which area of the code to start looking at or don't have enough hardware to mirror production problems. Doing anything with a production system makes some people nervous, but I do recommend it when you are short on time or not having luck reproducing the leak in a non-production environment.

So what are your options for gathering some stats from a production system? You could install a profiler, but that slows the application down 10 to 20 times, which is a no-no. You could use "monitoring tools" like Glassbox Inspector or Introscope that profile only selected classes - and therefore adds minimal overhead - but you could easily miss profiling the class that is the cause of the leak.

My preferred solution is to do a JVM heap dump shortly before the application dies with an OutOfMemoryError. The IBM JVM has a number of "dump heap on exit" features and tools to analyze those dumps - see the Diagnostic Guide for your version of the JVM. The Sun Java 5 JVM has a dump on exit feature, which has only very recenlty been backported to the most recent 1.4.2 patch (-XX:+HeapDumpOnOutOfMemoryError command line option added to the Sun 1.4.2_12 JVM), and very few tools to analyze those dumps (jhat doesn't work on dumps from all versions of the JVM).

One lightweight tool I developed to just dump summary information - total number of instances and total memory used for each class - I call HeapSummary. This is a small C module (see source code at end of this blog) that hooks into the JVM via the JVMPI interface. When you send a SIGQUIT/CTRL-Break signal to the JVM process it will dump a text file summary of the heap usage whose output looks something like this:

       26664400          562392    TOTAL
        9361944          135484    [char
        2201272           24938    [java.util.HashMap$Entry
        1750704           72946    java.util.HashMap$Entry
        1230624           51276    java.lang.String
         733480           18337    java.util.HashMap
         534328           11448    [java.lang.Object
         296928            6186    org.apache.commons.collections.FastHashMap

HeapSummary lays dormant until it receives the SIGQUIT signal, so it adds zero overhead during normal application use. Once it has been activated two full GCs are forced, the heap is "walked" to calculate instance and memory summaries (which can take four or five minutes for heaps of 2GB or greater), the summary file is written and the application goes back to processing as normal.

See the comments at the beginning of heapsummary.c for instructions on compiling and installing.

NOTE: I've only tested HeapSummary on Java 1.4 and 1.3 JVMs, not on Java 5 JVMs yet.

heapsummary.c (source code)

/*

 Copyright (c) 2006, Robert Maldon 

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.


 Based on the original code of heapprofile (http://www.virtualmachine.de/)
 by Matthias Ernst 


 INSTALLING AND RUNNING HEAPSUMMARY

Compiling Win32 /Cygwin:
 gcc -mno-cygwin -I$JAVA_HOME/include -I$JAVA_HOME/include/win32 -Wl,--add-stdcall-alias -shared -o heapsummary.dll heapsummary.c
Compiling Win32/Visual C++ (thanks to Jeppe Cramon ):
 Create a Visual C++ Dynamic DLL project
          Comment out #include 
          Add #define WIN32_LEAN_AND_MEAN and #include 
          Change sleep(5); to Sleep(5);
Compiling Win32/VC++ Toolkit (incomplete):
 Execute %VC_HOME%\VCVARS32 to set up the command line
 compilation environment, then compile with
 cl -LDd -Zi -I. -I%JDK_HOME%\include -I%JDK_HOME%\include\win32 -Tcheapsummary.c -o heapsummary.dll

Compiling Solaris:
 gcc -c -G -O -DUNIX -DSUN -ansi -fPIC -I$JAVA_HOME/include -I$JAVA_HOME/include/solaris heapsummary.c -o heapsummary.o
 gcc -shared -o libheapsummary.so heapsummary.o
Compiling Linux:
 gcc  -o libheapsummary.so -shared -Wl,-soname,libheapsummary.so  -I$JAVA_HOME/include -I$JAVA_HOME/include/linux heapsummary.c  -static -lc
Compiling OS/X:
 cc -bundle -I$JAVA_HOME/include -I$JAVA_HOME/include/linux -I/System/Library/Frameworks/JavaVM.framework/Headers -o libheapsummary.jnilib -framework JavaVM heapsummary.c
Compiling HP/UX 11 PA/RISC:
 cc -b -I$JAVA_HOME/include -I$JAVA_HOME/include/hp-ux -o libheapsummary.sl heapsummary.c
Compiling HP/UX 11 Itanium 2 (for 32 bit) like above, only name the library .so instead of .sl


       Running:
           1) Make sure that the library is in your JVM path either by using
              LD_LIBRARY_PATH under Solaris/Linux or PATH under Win32.
           2) Load the JVM with the parameter -Xrunheapsummary (see optional
              Usage below).

       Usage:

           -Xrunheapsummary[:[dir=SUMMARY_DIR][,][sort=sizecount]]

           where

               SUMMARY_DIR is the directory to write summary files to.
               SUMMARY_DIR may be an absolute path (e.g. /tmp/summarylogs)
               or relative to the JVM start directory (e.g. logs/summarylogs).
               If the "dir" option is not specified then summary files will
               be written to the JVM start directory.

               The "sort" option can have a value of "size" (sorted by total
               memory size of a class) or "count" (sorted by number of
               instances of a class). If the sort option is not specified
               then sorting will default to size.

           The name of the summary file is of the format
           "heapsummary.YYYYMMDDhhmmss" where YYYY is the year, MM is the
           month, DD is the day, hh is the hour, mm is the minute and ss is
           the second the summary file is created.
    
       Generating a Heap Summary:

           kill -QUIT 
           or CTRL-\ or CTRL-Break


      Format of the Heap Summary File:

          MEMORY_SIZE    MEMORY_COUNT    CLASS_NAME

          where MEMORY_SIZE is the total amount of memory in use by a
          class, MEMORY_COUNT is the total number of instances of a
          class and CLASS_NAME is the name of the class. e.g.

       26664400          562392    TOTAL
        9361944          135484    [char
        2201272           24938    [java.util.HashMap$Entry
        1750704           72946    java.util.HashMap$Entry
        1230624           51276    java.lang.String
         733480           18337    java.util.HashMap
         534328           11448    [java.lang.Object
         296928            6186    org.apache.commons.collections.FastHashMap


 Changelog

 1.0    24th July 2006    Robert Maldon

     Initial release version.

*/
#include 
#include 
#include 
#include 
#include 

/* global jvmpi interface pointer */
static JavaVM *jvm;
static JVMPI_Interface *jvmpi_interface;
static JNIEnv *jni;
static jobjectID object;
static jobjectID clazz;
static int objectSize;
static const char *className;
static char buf[1024];
static char toString[64];

static jobjectID clazzCache;
static const char *nameCache;

#define OUTFILENAME_PREFIX "heapsummary."
static char outFileName[256];
static char *outFilePtr;

#define SORT_KEY_SIZE  0
#define SORT_KEY_COUNT 1
static int sortKey = SORT_KEY_SIZE;

/* START - hashtable */

#define HASH_KEY_MAX_SIZE 256
#define HASH_LIMIT 0.5

/* hash table top level data structure */
typedef struct hash_t {
 struct hash_node_t **bucket;        /* array of hash nodes */
 int size;                           /* size of the array */
 int entries;                        /* number of entries in table */
 int downShift;                      /* shift count, used in hash function */
 int mask;                           /* used to select bits for hashing */
} hash_t;

typedef struct hash_node_t {
 long long memSize;                  /* total memory size of java class */
 long long memCount;                 /* total number of instances of java class */
 const char key[HASH_KEY_MAX_SIZE];  /* key for hash lookup - java class name */
 struct hash_node_t *next;           /* next node in hash chain */
 struct hash_node_t *sortNext;       /* next node in sort linked list */
} hash_node_t;

static hash_t *hashTable = NULL;

static hash_node_t *freeNodeList = NULL;

static hash_node_t *sortedList = NULL;

/* calculate the hash number for a given key */
static int hash(const char *key) {
 int i = 0;
 int hashValue;

 while (*key != '\0') {
   i = (i<<3) hashvalue =" (((i*1103515249)">> hashTable->downShift) & hashTable->mask);
 if (hashValue < hashvalue =" 0;" buckets ="=" buckets =" 16;">entries   = 0;
 hashTable->size      = 2;
 hashTable->mask      = 1;
 hashTable->downShift = 29;

 /* make sure buckets are a power of 2 */
 while (hashTable->size <>size <<= 1;     hashTable->mask = (hashTable->mask <<>downShift--;
 }

 /* allocate memory for the table */
 hashTable->bucket = (hash_node_t **)calloc(hashTable->size, sizeof(hash_node_t *));

 return;
}

/* create new hash table when the old one fills up */
static void hashTableRebuild() {
 hash_node_t **oldBucket, *oldHash, *tmp;
 int oldSize, h, i;

 oldBucket = hashTable->bucket;
 oldSize   = hashTable->size;

 /* create a new table and rehash the old buckets */
 hashTableInit(oldSize << i =" 0;" oldhash =" oldBucket[i];" tmp =" oldHash;" oldhash =" oldHash-">next;
     h = hash(tmp->key);
     tmp->next = hashTable->bucket[h];
     hashTable->bucket[h] = tmp;
     hashTable->entries++;
   }
 }

 /* free memory used by old table */
 free(oldBucket);

 return;
}

/* look for an entry of the given key in the hashtable */
static hash_node_t *hashTableLookup(const char *key) {
 int h;
 hash_node_t *node;

 /* find the entry in the hash table */
 h = hash(key);
 for (node = hashTable->bucket[h]; node != NULL; node = node->next) {
   if (!strcmp(node->key, key)) {
     break;
   }
 }

 /* return the entry if it exists, or NULL if it doesn't */
 return node;
}

/* allocate a node for inserting into the hashtable. reuse a node from
  the free list if available, otherwise malloc the space for the node */
static hash_node_t *hashTableAllocateNode() {
 hash_node_t *node;

 if (freeNodeList == NULL) {
   node = (struct hash_node_t *) malloc(sizeof(hash_node_t));
 } else {
   node = freeNodeList;
   freeNodeList = node->next;
 }

 memset(node, 0, sizeof(hash_node_t));
 return node;
}

/* insert an entry into the hashtable */
static hash_node_t *hashTableInsert(const char *key, int memSize) {
 hash_node_t *node;
 int h;

 /* if this was a generic hash table insert method then we would
    first check to see if an entry that matches the given key
    already exists. however, in the case of heapsummary we can
    guarantee the key does not already exist in the hash table, so
    for perf reasons we can bypass this check.
 if ((node = hashTableLookup(key)) != NULL) {
   return node;
 }
 */

 /* expand the table if needed */
 while (hashTable->entries >= HASH_LIMIT * hashTable->size) {
   hashTableRebuild();
 }

 /* insert the new entry */
 h = hash(key);
 node = hashTableAllocateNode();
 node->memSize  = memSize;
 node->memCount = 1;
 strncpy((char *)node->key, key, HASH_KEY_MAX_SIZE - 1);
 node->next = hashTable->bucket[h];
 hashTable->bucket[h] = node;
 hashTable->entries++;

 return node;
}

/* destroy the hash table, moving all of the nodes to a free list for
  future reuse */
static void hashTableDestroy() {
 hash_node_t *node, *next;
 int i;

 /* move the nodes to the free node list */
 for (i = 0; i <>size; i++) {
   next = hashTable->bucket[i];
   while (next != NULL) {
     node = next;
     next = node->next;
     node->next = freeNodeList;
     freeNodeList = node;
   }

   /* null out the bucket pointer */
   hashTable->bucket[i] = NULL;
 }

 hashTable->entries = 0;
}

/* sort the contents of the hashtable into a linked list */
static void hashTableSort() {
 int i, greater;
 hash_node_t *node;
 hash_node_t *listNode;
 hash_node_t *listPrev;

 sortedList = NULL;

 for (i = 0; i <>size; i++) {
   for (node = hashTable->bucket[i]; node != NULL; node = node->next) {
      if (sortedList == NULL) {
          sortedList = node;
      } else {
        for (listNode = sortedList, listPrev = NULL; listNode != NULL;
             listPrev = listNode, listNode = listNode->sortNext) {

          if (sortKey == SORT_KEY_SIZE) {
            if (node->memSize > listNode->memSize) {
              greater = 1;
            } else {
              greater = 0;
            }
          } else {
            if (node->memCount > listNode->memCount) {
              greater = 1;
            } else {
              greater = 0;
            }
          }

          if (greater) {
            node->sortNext = listNode;
            if (listNode == sortedList) {
              sortedList = node;
            } else {
              listPrev->sortNext = node;
            }
            break;
          }
        }
        if (listNode == NULL) {
          listPrev->sortNext = node;
        }
      }
   }
 }
}

/* END - hashtable */

void loadClass() {
 jvmpi_interface->RequestEvent(JVMPI_EVENT_OBJECT_ALLOC, object);
}

void loadClassName() {
 if(clazz == NULL) className = "NULL?";
 else if(clazz != clazzCache) {
   className = NULL;
   jvmpi_interface->RequestEvent(JVMPI_EVENT_CLASS_LOAD, clazz);
   clazzCache = clazz;
   nameCache = className;
 }
}

jobjectID asObject(char *curr) {
 char ptr[4];
 memcpy(ptr, curr, 4);
 return *((jobjectID*)ptr);
}

FILE *openSummaryFile() {
 time_t tm;
 struct tm *tmptr;

 tm = time(NULL);
 tmptr = localtime(&tm);
 strftime(outFilePtr, sizeof(outFileName) - (outFilePtr - outFileName) - 1,
          "%Y%m%d%H%M%S", tmptr);

 return fopen(outFileName, "w");
}

/* function for handling event notification */
void notifyEvent(JVMPI_Event *event) {

 JVMPI_HeapDumpArg arg;
 char* curr;
 long long totalCount;
 long long totalSize;
 int type;
 hash_node_t *node;
 FILE *summaryFile;

 switch(event->event_type) {
   case JVMPI_EVENT_CLASS_LOAD   JVMPI_REQUESTED_EVENT:
     className = event->u.class_load.class_name;
     if (className == NULL) className = "NULL?";
     break;
   case JVMPI_EVENT_OBJECT_ALLOC   JVMPI_REQUESTED_EVENT:
     objectSize = event->u.obj_alloc.size;

     switch (event->u.obj_alloc.is_array) {
     case JVMPI_CHAR :    className = "[char"; break;
     case JVMPI_BOOLEAN : className = "[boolean"; break;
     case JVMPI_FLOAT :   className = "[float"; break;
     case JVMPI_DOUBLE :  className = "[double"; break;
     case JVMPI_BYTE :    className = "[byte"; break;
     case JVMPI_SHORT :   className = "[short"; break;
     case JVMPI_INT :     className = "[int"; break;
     case JVMPI_LONG :    className = "[long"; break;
     case JVMPI_NORMAL_OBJECT: clazz = event->u.obj_alloc.class_id; loadClassName(); break;
     case JVMPI_CLASS:    clazz = event->u.obj_alloc.class_id; loadClassName(); sprintf(buf, "[%s", className); className = buf; clazzCache = NULL; break;
     default :
       sprintf(buf, "type %d ???", event->u.obj_alloc.is_array);
       className = buf;
     }

     break;
   case JVMPI_EVENT_DATA_DUMP_REQUEST:
   case JVMPI_EVENT_JVM_SHUT_DOWN:
     /* we do a full gc twice because, depending on the garbage collector
        settings, the first gc may not collect all of the objects we
        would like it to */
     jvmpi_interface->RunGC();
     sleep(1);
     jvmpi_interface->RunGC();

     (*jvm)->AttachCurrentThread(jvm, (void **)&jni, NULL);
     arg.heap_dump_level = JVMPI_DUMP_LEVEL_0;
     jvmpi_interface->RequestEvent(JVMPI_EVENT_HEAP_DUMP, &arg);

     break;
   case JVMPI_EVENT_HEAP_DUMP  JVMPI_REQUESTED_EVENT: {
     if ((summaryFile = openSummaryFile()) == NULL) {
       fprintf(stderr, "heapsummary> Failed to open dump file %s\n",
               outFileName);
       return;
     }

     if (hashTable == NULL) {
       hashTable = (struct hash_t *)malloc(sizeof(hash_t));
       hashTableInit(512);
     }

     totalCount = 0;
     totalSize  = 0;

     curr = event->u.heap_dump.begin;
     while(curr <>u.heap_dump.end) {
       /* if(((++totalCount)%1000) == 0) fprintf(stderr, "."); */
       totalCount = totalCount + 1;
       totalSize += objectSize;
       type = *(curr++);
       object = asObject(curr);
       if(object == NULL) fprintf(stderr, "object null\n");
       curr += 4;
       loadClass();

       node = hashTableLookup(className);
       if (node != NULL) {
           node->memSize  = node->memSize + objectSize;
           node->memCount = node->memCount + 1;
       } else {
           hashTableInsert(className, objectSize);
       }
     }

     /* print totals */
     fprintf(summaryFile, "%20lld%20lld    TOTAL\n", totalSize, totalCount);

     /* sort the elements of the hash table into a linked list, then
        iterate through each item and print out its values */
     hashTableSort();
     for (node = sortedList; node != NULL; node = node->sortNext) {
       fprintf(summaryFile, "%20lld%20lld    %s\n",
               node->memSize, node->memCount, node->key);
     }

     fclose(summaryFile);

     hashTableDestroy();

     break;
   }
 }
}

/* profiler agent entry point */
JNIEXPORT jint JNICALL JVM_OnLoad(JavaVM *theJvm, char *options, void *reserved) {
 char *token;
 jvmdiError err;

 jvm = theJvm;
 fprintf(stderr, "heapsummary> Initializing...\n");
 err=(*jvm)->GetEnv(jvm,(void**)&jvmpi_interface, JVMPI_VERSION_1_1);
 /* get jvmpi interface pointer */
 if (err != 0) {
   fprintf(stderr, "heapsummary> Error in obtaining jvmpi interface pointer\n");
   return JNI_ERR;
 }

 jvmpi_interface->NotifyEvent = notifyEvent;
 jvmpi_interface->EnableEvent(JVMPI_EVENT_DATA_DUMP_REQUEST, NULL);
 jvmpi_interface->EnableEvent(JVMPI_EVENT_JVM_SHUT_DOWN, NULL);

 *outFileName = NULL;
 outFilePtr = outFileName;

 for (token = strtok(options, ","); token != NULL; token = strtok(NULL, ",")) {
   if (strncmp(token, "dir=", 4) == 0) {
     strcpy(outFileName, token + 4);
     outFilePtr = outFilePtr + strlen(outFileName);
#ifdef WINDOWS
     *outFilePtr = '\\';
#else
     *outFilePtr = '/';
#endif
     outFilePtr = outFilePtr + 1;
   } else if (strncmp(token, "sort=", 5) == 0) {
     if (strcmp(token + 5, "size") == 0) {
       sortKey = SORT_KEY_SIZE;
       fprintf(stderr, "heapsummary> Sorting on memory size\n");
     } else if (strcmp(token + 5, "count") == 0) {
       sortKey = SORT_KEY_COUNT;
       fprintf(stderr, "heapsummary> Sorting on memory count\n");
     } else {
       fprintf(stderr, "heapsummary> Unknown sort key [%s]. Defaulting to sorting on memory size\n", token + 5);
     }
   }
 }

 strcpy(outFilePtr, OUTFILENAME_PREFIX);
 outFilePtr = outFilePtr + sizeof(OUTFILENAME_PREFIX) - 1;
 strcpy(outFilePtr, "YYYYMMDDhhmmss");

 fprintf(stderr, "heapsummary> Running. Will output to summary file %s\n",
         outFileName);

 return JNI_OK;
}