Apache Tomcat 6 and the HSM

Recently I’ve had some real fun with SafeNet HSM’s (Host Security Module or Hardware Security Module) and Tomcat 6 in the never ending process of PCIDSS compliance.  People are probably going to line up to stone me after saying this but PCIDSS is a really great effort to make things safe and secure for all of us.  In a lot of cases its the stick that drives organisations to adopt some good practices.  One of which is the use of HSM’s for all things from the crypt.

Now this little rant won’t walk you through PCI compliance, rather its just about my experiences getting Safenet Eracom Orange or Blue HSM to work within Tomcat.  Deployment is really easy:

  1. Install the latest version of Java,
  2. Install the Eracom Orange C runtime,
  3. Configure the Eracom Orange runtime to connect to the HSM,
  4. Configure the HSM with tokens, keys, certificates, users and whatever else you need,
  5. Install the Eracom Orange Java runtime,
  6. Install Apache Tomcat 6,
  7. Place the Eracom Orange ‘jprov.jar‘ Tomcat’s server shared lib directory ‘CATALINE_HOME/lib‘,
  8. Restart the Tomcat server if its already running or just start it if it is not,
  9. Deploy your webapp.

Apart from steps 7 and 8 simply follow the provider’s directions for all steps.  The key in this thing is that the ‘jprov.jar‘ archive must be loaded when Tomcat starts.  I imagine that it should be the same for other providers like for IBM’s toys.

Now for the development set up.  Simply follow the same steps as above but install the SDK’s for the Eracom and include ‘jprov.jar’ as a library in your development environment.  Typically your development environment will bundle the ‘jprov.jar‘ into the web application directory.  It won’t do any harm there but to get it to work properly it will need to be in the Tomcat server’s common library directory as detailed above.  In fact you don’t really need to install the SDK’s if you have an actual HSM to play with for development purposes, but the SDK’s do provide a software HSM simulator for those of us who don’t want the expense of buying one of these things for every developer on the floor.  Plus the SDK’s will give you a good deal of documentation too.

So you are all set up and ready to build the next great crypt.  Start by building a stand alone application to confirm that everything works before going any further. Lets get to the code:

/*
 * Crypt.java
 *
 * A simple class to provide basic SDES encryption and decryption
 */
package crypt;

import java.io.IOException;
import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.Key;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.NoSuchProviderException;
import java.security.Security;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import java.util.Properties;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.IvParameterSpec;
import org.apache.commons.codec.DecoderException;
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.codec.binary.Hex;
import org.apache.commons.lang.StringUtils;
import org.apache.commons.logging.Log;

/**
 *
 * @author mc
 */
public class Crypt {

  /**
   * Creates a 3DES key cipher with the parameters in the config, valid
   * modes are Cipher.ENCRYPT and Cipher.DECRYPT
   *
   * @param mode set the cipher to either encrypt or decrypt
   * @param config the configuration parameters for this application
   * @param log log to write debug output to
   * @return a cipher that should be used then nulled to be thrown away
   * @throws KeyStoreException
   * @throws NoSuchProviderException
   * @throws IOException
   * @throws NoSuchAlgorithmException
   * @throws CertificateException
   * @throws UnrecoverableKeyException
   * @throws NoSuchPaddingException
   * @throws DecoderException
   * @throws InvalidKeyException
   * @throws InvalidAlgorithmParameterException
   */
  private Cipher createCipher(int mode, Properties config, Log log)
    throws
      KeyStoreException,
      NoSuchProviderException,
      IOException,
      NoSuchAlgorithmException,
      CertificateException,
      UnrecoverableKeyException,
      NoSuchPaddingException,
      DecoderException,
      InvalidKeyException,
      InvalidAlgorithmParameterException {

    log.debug("Adding provider.");
    // will gracefully not add provider if provider already loaded
    Security.addProvider(
        new au.com.eracom.crypto.provider.ERACOMProvider());

    log.debug("Getting key store instance.");
    KeyStore store = KeyStore.getInstance(
        config.getProperty("crypt.keystore.type"),
        config.getProperty("crypt.keystore.provider"));
    log.debug("Loading key store.");
    store.load(null, null);

    log.debug("Retrieving key.");
    Key key = store.getKey(
        config.getProperty("crypt.keystore.entry"), null);

    log.debug("Generating cipher.");
    Cipher cipher = Cipher.getInstance(
        config.getProperty("crypt.transformation"),
        config.getProperty("crypt.provider"));

    log.debug("Creating initialisation vector.");
    IvParameterSpec i = new IvParameterSpec(Hex.decodeHex(
        config.getProperty("crypt.vector").toCharArray()));

    log.debug("Initialising cipher in the given mode.");
    cipher.init(mode, key, i);

    return cipher;
  }

  /**
   * Takes a string and returns its equivalent Base64 encrypted string.
   *
   * @param data string to be encrypted
   * @param config parameters for this application's encryption
   * @param log log to write debug output to
   * @return encrypted Base64 encoded form of the data string parameter
   * @throws NoSuchAlgorithmException
   * @throws NoSuchProviderException
   * @throws NoSuchPaddingException
   * @throws InvalidKeyException
   * @throws IllegalBlockSizeException
   * @throws BadPaddingException
   * @throws InvalidAlgorithmParameterException
   * @throws KeyStoreException
   * @throws IOException
   * @throws CertificateException
   * @throws UnrecoverableKeyException
   * @throws DecoderException
   */
  public String b46Encrypt(String data, Properties config, Log log)
    throws
      NoSuchAlgorithmException,
      NoSuchProviderException,
      NoSuchPaddingException,
      InvalidKeyException,
      IllegalBlockSizeException,
      BadPaddingException,
      InvalidAlgorithmParameterException,
      KeyStoreException,
      IOException,
      CertificateException,
      UnrecoverableKeyException,
      DecoderException {

    log.debug("Encrypting data.");
    byte[] encrypted =
        createCipher(Cipher.ENCRYPT_MODE, config, log).doFinal(
            data.getBytes());

    log.debug("Base64 encoding data.");
    String encoded = new String(Base64.encodeBase64(encrypted));

    return encoded;
  }

  /**
   * Takes a encrypted Base64 encoded string and returns its equivalent
   * decrypted string.
   *
   * @param data string to be encrypted
   * @param config parameters for this application's encryption
   * @param log log to write debug output to
   * @return encrypted Base64 encoded form of the data string parameter
   * @throws NoSuchAlgorithmException
   * @throws NoSuchProviderException
   * @throws NoSuchPaddingException
   * @throws InvalidKeyException
   * @throws IllegalBlockSizeException
   * @throws BadPaddingException
   * @throws InvalidAlgorithmParameterException
   * @throws KeyStoreException
   * @throws IOException
   * @throws CertificateException
   * @throws UnrecoverableKeyException
   * @throws DecoderException
   */
  public String b64Decrypt(String data, Properties config, Log log)
    throws
      NoSuchAlgorithmException,
      NoSuchProviderException,
      NoSuchPaddingException,
      InvalidKeyException,
      IllegalBlockSizeException,
      BadPaddingException,
      InvalidAlgorithmParameterException,
      KeyStoreException,
      IOException,
      CertificateException,
      UnrecoverableKeyException,
      DecoderException {

    log.debug("Base64 decoding data.");
    byte[] decoded = Base64.decodeBase64(data.getBytes());

    log.debug("Decrypting data.");
    byte[] decrypted =
        createCipher(Cipher.DECRYPT_MODE, config, log).doFinal(decoded);

    return new String(decrypted);
  }

}

So that’s it, very straight forward but lets take time to look at one or two points of interest.

In a nutshell functionally this is a class that can be used to encrypt and decrypt data using stored 3DES keys. Note that the configuration information is all passed in via the properties parameter. Also note that the class uses base64 encoding because its easier to play with and store strings. Generally you would also want to replace unsafe base64 characters too if the storage unit required it.

On the encryption front there are a few things to note too. Take a look at the includes, as far as the crypt mechanisms it uses only the standard Java imports and practices. There is only one thing that is specifically from the Safenet SDK, the loading of the provider class in the following snippet:

    // will gracefully not add provider if provider already loaded
    Security.addProvider(
        new au.com.eracom.crypto.provider.ERACOMProvider());

To complete the picture here is the main class and the configuration file to go with it. That’s the lot; three simple files and it should be all go to test your configuration.

/*
 * main.java
 *
 * Test application for the Eracom encryption devices.
 */
package crypt;

import java.io.FileInputStream;
import java.io.IOException;
import java.util.Properties;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

/**
 *
 * @author mc
 */
public class Main {

  /**
   * @param args the command line arguments
   */
  public static void main(String[] args) throws IOException {

    Properties config = new Properties();
    Log log = LogFactory.getLog(Main.class);
    boolean encrypt = "encrypt".equalsIgnoreCase(args[0]);

    config.loadFromXML(new FileInputStream(args[1]));

    try {

      if(encrypt)
        System.out.println(
          new Crypt().b46Encrypt(args[2], config, log));
      else
        System.out.println(
          new Crypt().b64Decrypt(args[2], config, log)); 

    } catch(Exception e) {

      log.error(e.getLocalizedMessage(), e);
    }

  }

}




  
  ERACOM
  au.com.eracom.crypto.provider.ERACOMProvider
  DESede/CBC/PKCS5Padding
  AAAAAAAA00000000
  ERACOM
  CRYPTOKI
  KeyName

Once this is working it should be possible to use the same crypt class file in a Tomcat hosted web application with one slight adjustment. As the Tomcat service will load the provider you should not attempt to load the provider in your application. In fact if you try it will almost certainly fail as your application is unlikely, unless you have specifically configured it, will not have the privileges need to load classes. So just comment out the ‘Security.addProvider‘ call in the class.

So that’s it; all you needed to know to configure an HSM to start working with Tomcat. Simple wasn’t it? Now all you need to do is figure out how to use it to help you with PCIDSS and how to change the crypt file to suite your specific needs.


To Mail a File Elegantly – Python or Perl?

People fixate on particular languages and stir hot debates over the entrails of code compared in the cold hard light of a walk through but is the debate over which language is better worth even the energy of debate any more? Take for example a trivial task such as regularly sending a file via email, a simple administrative task. There was a day in which Java didn’t have Java mail and Perl didn’t have its elegant MIME::lite so the only way to so things was via bash and some mail client written in C. But today every language has mail client tools built into it, in fact so many of the modern languages like Python and PHP have so many solutions to the common problems worked out in them is there any point over personal preference?

Personally I still like to gut the compared implementations like a fish and have a look. Partly because these questions are fascinating to me, partly because I like to take the opportunity to wind up the religious proponents of each language, and partly because I prefer to gut code rather than fish. Your code might stink but never as much as fish guts.

There is one other reason that I like to perform comparative code autopsies and that is to get into the mind of the people it affects. So lets talk Python, the language of all things Gnome and Perl the language of the data miner. These two languages and their communities are very interesting. Our gnomes have only been, comparatively, around for a short period of time and tend to look at code for code’s sake. Yes python might be the language of GUIs and building blocks are important; but they’re not everything. On the other hand we have the gristled old data miners emerging from their day at the data face deep in their mine chiselling out useful information from the earthy data buried three fathoms deep in some odious data warehouse. They’re all about getting the task done, producing the goods for their team and finding the information that their users want. So there’s the stage a constructive pantheon of building block gnomes and the old data hounds seeking useful information.

Let’s set a comparison task now and see how their favourite tools stake up. But we have to be careful. If we choose a GUI task the miners will go on strike; alternatively if we select a data parsing task the gnomes will sit on their treasures and grump. Hence the task that I’ve chosen favours neither group, it’s stupidly simple and it is something any system administrator can relate to; creating a script to email a file. The rules of the game are:

  • Each language must use its own mail tool set, no invoking mutt or any other such tool,
  • For input the script must take all of the information: SMTP email server address, from address, to address, and fully qualified file name from the command line,
  • A help text and minimal useful input validation along with error reporting must be included, but trivially,
  • Only one file at a time is to be handled,
  • Simple text in the body and subject must be hard coded in the script body with the exception of file name,
  • The file must be in an attachment, not in-line Base64 and should be able to be of any type, not necessarily specified by MIME type,
  • This task is stupidly simple, so so should the script be.

So to the breach!  First up the Perl.

#!/usr/bin/perl

use MIME::Lite;

$address=$ARGV[0];
$from=$ARGV[1];
$to=$ARGV[2];
$path=$ARGV[3];

$usage='filemail.pl smtp from to path';
die "Usage: $usage\n" if $#ARGV != 3;

( $dir, $name ) = ($path=~m/(.*)[\\\/](.+)/) ? ( $1, $2 ) : ( undef, $path );
open FILE, $path or die "Could not find file '$path'";
close FILE;

$msg = MIME::Lite->new(
    From     => $from,
    To       => $to,
    Cc       => '',
    Subject  => "File '$name'.",
    Data     => "Please find file '$name' attached."
  );

$msg->attach(
    Type        =>  'application/octet-stream',
    Path        =>  $path,
    Filename    =>  $name,
    Disposition =>  'attachment'
  ); 

MIME::Lite->send('smtp', $address, Timeout=>60);
$msg->send();

Next the Python.

#!/usr/bin/python

import os
import smtplib
import email
import email.mime.text
import email.mime.multipart
import optparse

# set up the command line parser and grab the args
parser = optparse.OptionParser(usage="""\
Send file attached to a MIME message.

Usage: %prog [options]
""")
parser.add_option('-a', '--address',
                  type='string', action='store', metavar='ADDRESS',
                  help="""Address of the SMTP server (required)""")
parser.add_option('-p', '--path',
                  type='string', action='store', metavar='PATH',
                  help="""Location of the file to send (required)""")
parser.add_option('-s', '--sender',
                  type='string', action='store', metavar='SENDER',
                  help='The value of the From: header (required)')
parser.add_option('-r', '--recipient',
                  type='string', action='append', metavar='RECIPIENT',
                  default=[], dest='recipients',
                  help='A To: header value (at least one required)')
opts, args = parser.parse_args()

# bomb with help if the required arguments are not present
if not opts.path \
        or not opts.sender \
        or not opts.recipients \
        or not opts.address \
        or not opts.path:
    parser.print_help()
    os.sys.exit(1)
if not os.path.isfile(opts.path):
    print 'File \'' + opts.path + '\' not found.'
    os.sys.exit(1)

# create the enclosing (msg) message
filename = opts.path.split('/')[-1]
msg = email.mime.multipart.MIMEMultipart()
msg['Subject'] = 'File \'' + filename + '\'.'
msg['To'] = ', '.join(opts.recipients)
msg['From'] = opts.sender
msg.preamble = 'Please find file \'' + filename + '\' attached.\n'

# message body
body = email.mime.multipart.MIMEMultipart('alternative')
body.attach(email.mime.text.MIMEText('Please find file \'' + filename + '\' attached.\n'))
body.attach(email.mime.text.MIMEText('Please find file \'' + filename + '\' attached.\n', 'html'))

msg.attach(body)

# attach the file
fp = open(opts.path, 'rb')
att = email.mime.base.MIMEBase('application', 'octet-stream')
att.set_payload(fp.read())
fp.close()
email.encoders.encode_base64(att)
att.add_header('Content-Disposition', 'attachment', filename=filename)

msg.attach(att)

# send the message
composed = msg.as_string()
s = smtplib.SMTP(opts.address)
s.sendmail(opts.sender, opts.recipients, composed)
s.quit()

Now for my taste there is no contest. I find myself in the company of wrinkled old miners covered in the dust of ages of data mining experience and dutiful determination. So why did I choose to use the tool of gnomes? Well several reasons really but first lets just make a couple of observations about the code. Neither good or bad, just observations. Then we’ll come back to the decision over choice and the ultimate answer to the ultimate question, is there really any difference of substance between the languages?

  • For shear size Perl has the smaller amount of code.
  • In detail Python is very revealing, it shows some of the nature and structure of the thing that it is building.
  • Considering layout Perl provides us with more opportunities to lay things out for current and future reviewers.
  • In terms of handiness Python provides a wonderful tool for handling standard command line parsing and help generation.
  • Results from either implementation are near enough to identical where it counts.

Now the key here in this list is the last item, where it counts, in other words the results of running both of these scripts, is near enough to identical. The function for which these scripts was written has been achieved within the rules of the game. The only things that we are left debating are the non-functional aspects of the implementation, size, process description, navigation and flexibility. To me the elegance of the Perl solution simply trumps the Python implementation cold. But that’s personal preference based on ascetics. Python proponents will argue that the elegance of Perl is simply the result of a library which could quite easily have been created in Python. Which is true. But you see it isn’t. If it was it would have been. To expand; this is symptomatic of the perspective of the two different communities. Those playing with perls of wisdom know that scripting is all about getting things done, so they ask themselves what is the minimum amount of information that I can get away with providing to get the results that I want. Speed to an end. Focus on results. In the garden of code we find our gnomes building beautiful frameworks which can be used for all sorts of things, generalisation is the key, making it work for the world. Of course you can do mail with Python, but we would rather have a flexible solution for the masses than elegant solutions for the minorities.

To answer the question is there any point to using one language over another other than preference? No. However if you are sitting in a camp of gnomes armed to the teeth with pick axes then all hail the magnificent Python, but if you don’t want to get a shovel through your head after a day in the data mines polish your Perl. At the end of the day there is no real big difference between all of these languages, and if you are like me and sit in the camp with gnomes and miners just pick the one that means the least work for you. In my case it was just more expedient to deploy Python.


Flatening Java Servlet Request Maps

Some times it is necessary to flatten a map into a string. For example when trying to log a servlet request onto a single log line without loosing too much info. In this case each value is an array of strings. So here it is, short and effective. Enjoy!

  private String mapToString(Map map) {

    StringBuffer buffer = new StringBuffer();
    Iterator it = map.keySet().iterator();

    while (it.hasNext()) {

      String key = (String) it.next();
      String[] value = (String[]) map.get(key);
      buffer.append(key + "=");

      for (int c = 0; c < value.length; c++) {

        buffer.append("[" + value[c] + "]");
      }

      if (it.hasNext()) {

        buffer.append(":");
      }

    }

    return buffer.toString();
  }

Timing Issues with pyinotify


Ok so lets look a that little issue with pyinotify, and it really is only little, but enough to keep you on your toes. So this is the issue in brief.

START LOOP 2009-05-11 21:55:39.733393
Result of notifier.check_events() = TRUE
Calling notifier.process_events()
<Event dir=False mask=0x20 maskname=IN_OPEN name=foo path=/tmp pathname=/tmp/foo wd=1 >
END LOOP 2009-05-11 21:55:40.339782
START LOOP 2009-05-11 21:55:40.339807
Result of notifier.check_events() = TRUE
Calling notifier.process_events()
<Event dir=False mask=0x4 maskname=IN_ATTRIB name=foo path=/tmp pathname=/tmp/foo wd=1 >
END LOOP 2009-05-11 21:55:40.340403
START LOOP 2009-05-11 21:55:40.340426
Result of notifier.check_events() = TRUE
Calling notifier.process_events()
<Event dir=False mask=0x8 maskname=IN_CLOSE_WRITE name=foo path=/tmp pathname=/tmp/foo wd=1 >
END LOOP 2009-05-11 21:55:40.340971


So here is the culprit notifier = pyinotify.Notifier(m, e(), 0, 0, 1000) (I’ve altered the timeout to accentuate the issue). Take a careful look at the timings between the event output lines and you can see that they are uneven. This is because the timeout isn’t a spacer, its just a wait and the events are not linearly spaced.


My error was my own of course. The lesson here is read the code through twice and check it more for design flaws.


Simple Custom Event Processor for pyinotify

In my previous post on pyinotify I took a short look at the basic use of the pyinotify watcher using mostly default settings. Here I take a brief look at building a custom event processor. The event processor is called to retrieve the event data and, well, process it. The default processor simply prints the output to ‘stdout‘; mine will too. But it is just there to show how simple it is. Note that the processor actually processes the events in line so there’s no need for messy threading.


#!/usr/bin/python

import os
import pyinotify
from datetime import datetime

m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, pyinotify.ProcessEvent(), 0,  0, 100)
m.add_watch('/tmp', pyinotify.ALL_EVENTS, rec=True)

while True:

    print "START LOOP %s " % datetime.now()

    try:
        if notifier.check_events():
            print "Result of notifier.check_events() = TRUE"
            notifier.read_events()
            print "Calling notifier.process_events()"
            notifier.process_events()

    except KeyboardInterrupt:
        notifier.stop()
        break

    print "END LOOP %s " % datetime.now()


This is just the same code as we saw in the earlier post except that I’ve put the ‘notifier.process_events()‘ in its proper place so that it doesn’t get called in every loop execution. It is restricted by the ‘notifier.check_events()‘. To add a little info to the whole process I’ve included some print statements to illustrate the flow of execution. With this we get results that look like:


START LOOP 2009-05-07 23:02:25.025048
Result of notifier.check_events() = TRUE
Calling notifier.process_events()



END LOOP 2009-05-07 23:02:25.104546


Which is what you’d expect. However you will also get results that look like.


START LOOP 2009-05-07 23:02:18.808039
Result of notifier.check_events() = TRUE
Calling notifier.process_events()

END LOOP 2009-05-07 23:02:18.863855
START LOOP 2009-05-07 23:02:18.863912
Result of notifier.check_events() = TRUE
Calling notifier.process_events()

END LOOP 2009-05-07 23:02:18.864656
START LOOP 2009-05-07 23:02:18.864710
Result of notifier.check_events() = TRUE
Calling notifier.process_events()

END LOOP 2009-05-07 23:02:18.865415


So you can see that we get some reasonably unpredictable results.

Now let’s do something a little different and collect the event and print it ourselves a little differently. In this case we’re still collecting all events, we just wanted to mark up the output with something extra.


#!/usr/bin/python

import os
import pyinotify
from datetime import datetime

class e(pyinotify.ProcessEvent):
    def process_default(self, event):
        print(repr(event))

m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, e(), 0,  0, 100)
m.add_watch('/tmp', pyinotify.ALL_EVENTS, rec=True)

while True:

    print "START LOOP %s " % datetime.now()

    try:
        if notifier.check_events():
            print "Result of notifier.check_events() = TRUE"
            notifier.read_events()
            print "Calling notifier.process_events()"
            notifier.process_events()

    except KeyboardInterrupt:
        notifier.stop()
        break

    print "END LOOP %s " % datetime.now()


Easy huh? But there’s still a little more, there’s a problem! More on that later.


Python and pyinotify

A friend of mine posted a really good article over at blogspot about using pyinotify; after reading one of the comments there I started playing around with it. So if you want the original have a look at David Latham’s original post. I’ve simplified the code just a little to get to the essence of pyinotify. Incidentally if you’re working from Fedora like I am you might like to ‘yum install python-inotify*‘ and confirm that the module and documentation are installed. Here’s the base version.


#!/usr/bin/python

import os
import pyinotify

m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, pyinotify.ProcessEvent())
m.add_watch('/tmp/', pyinotify.ALL_EVENTS, rec=True)

while True:

    try:
        notifier.process_events()
        if notifier.check_events():
            notifier.read_events()

    except KeyboardInterrupt:
        notifier.stop()
        break


It works really well for this example but has one generally unwelcome side effect. It also blocks on the ‘notifier.check_events()‘ call. A blocking call is not really what you want in an event loop so it doesn’t really work too well for the core of the application. However this is overcome very simply by adding the three extra parameters defining the wait timing of the ‘pyinotify.Notifier(m, pyinotify.ProcessEvent(), 0, 0, 10)‘. In this case the block is turned to a 10 second wait courtesy of the last parameter as shown in the next code snippet below. In an application event loop you would use a value of 0 meaning that the call would not block at all.


#!/usr/bin/python

import os
import pyinotify

m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, pyinotify.ProcessEvent(), 0, 0, 10)
m.add_watch('/tmp/', pyinotify.ALL_EVENTS, rec=True)

while True:

    try:
        notifier.process_events()
        if notifier.check_events():
            notifier.read_events()

    except KeyboardInterrupt:
        notifier.stop()
        break


In this case the default event handler just prints XML formatted event lines to stdout. But there’s just one more unexpected side effect. You may find there are a few timing issues and how do you get the notification back to the main thread? But more on that later.


PI

4 * (1 - 1/3 + 1/5 - 1/7 + 1/9 - 1/11 + . . .)

LDAP Attributes in JNDI

Almost everything that you do with LDAP concerns the retrieval and manipulation of attributes. Due to the extensible nature of attributes finding out their types and content can be quite involved. Here is a simple sample code snip to the very basic of attribute look up including type information.

    /**
     * printAttributes
     *
     * Prints the attributes listed in ats to the stream defined by s.  If ats
     * is null all attributes will be listed.  If ats is empty none will be
     * listed.
     *
     * example:
     *      String obj = "uid=admin,ou=system";
     *      String[] ats = {"uid","displayName"};
     *      printAttributes(ctx,obj,ats,System.out);
     *
     *
     * @param ctx initial directory context
     * @param obj object in directory to retrieve attributes for
     * @param ats attribute list of attribute names to retrieve, null for all
     * @param s stream to write the attributes to
     * @throws javax.naming.NamingException
     */
    public static void printAttributes(InitialDirContext ctx, String obj,
            String[] ats, PrintStream s) throws NamingException {

        Attributes a = ctx.getAttributes(obj, ats);
        NamingEnumeration e = a.getAll();

        while (e.hasMore()) {

            Attribute t = (Attribute)e.next();

            s.println("<attribute name=\"" + t.getID() + "\">");
            s.println("      <syntax>" + t.getAttributeSyntaxDefinition().getAttributes("") + "</syntax>");
            s.println("      <schema>" + t.getAttributeDefinition().getAttributes("") + "</schema>");
            NamingEnumeration f = t.getAll();

            while (f.hasMore()){

                Object z = f.next();
                s.println("      <value>" + z + "</value>");
            }

            s.println("</attribute>");
        }

    }

This produces output that looks something like this:


    <attribute name="cn">
      <syntax>{x-schema=X-SCHEMA: system, x-is-human-readable=X-IS-HUMAN-READABLE: true, numericoid=NUMERICOID: 1.3.6.1.4.1.1466.115.121.1.15}</syntax>
      <schema>{name=NAME: cn, commonName, substr=SUBSTR: caseIgnoreSubstringsMatch, x-schema=X-SCHEMA: system, syntax=SYNTAX: 1.3.6.1.4.1.1466.115.121.1.15, numericoid=NUMERICOID: 2.5.4.3, sup=SUP: name, desc=DESC: RFC2256: common name(s) for which the entity is known by, usage=USAGE: userApplications, equality=EQUALITY: caseIgnoreMatch}</schema>
      <value>developer</value>
    </attribute>

Simple Berkeley Sockets C Server Example

About Apache and Bloat

Every now and then I look at Apache and wonder about its size and complexity. Its a huge beastie now but absolutely brilliant too. I have quite a respect for it as do many around the world. But then I don’t really use very much of its functionality.

Sure its been re-factored into a modular form so that you only need to use the bits that you want but it still presents a mental preponderance thatmany would consider bloat.  But bloat is a term used to describe software that has vast wads of unnessecary code that could be accomplished more simply.  However Idon’t think that Apache has that.  All of its code seems very purposeful, and as I indicated earlier it is reasonably easy to include or exclude as you will.

Introducing the Example

So this example is not here to show you the right way of doing things, but rather how simple the solution to you particular problem could be.  In this case the example server simply serves up the time on the server machine to port 6543 for any client and then closes the stream.  It can do this for many concurrent connections at once given the only complexity that is added to the server is client handling by forked process.

Now to the Code

/* -----------------------------------------------------------------------------

	simple example of a server in C using very simple Berkeley sockets 

	This is just an example for looking at.  Not really how you would actually
	implement a server.  For a start it does not take care of signals and isn't
	capable of being implemented as a service.

----------------------------------------------------------------------------- */

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <strings.h>
#include <arpa/inet.h>

/* we're not going to use arguments for this example */

#define SERV_UDP_PORT 6543
#define SERV_TCP_PORT 6543
#define SERV_HOST_ADDR "192.168.2.200"

int main (int argc, char ** argv, char ** env) {

	int socket_server, socket_client, child_pid;
	struct sockaddr_in address_client, address_server;
	socklen_t client_address_size;
	FILE *stream_client;
	time_t now;
	struct tm *tm;

	/* open a tcp socket to listen on */

	if ( (socket_server = socket(AF_INET, SOCK_STREAM, 0) ) < 0)
		err_dump("server: can't open stream socket");

	/* bind to our local address  ans start listening so that clients can find us */

	bzero((void *) &address_server, (size_t)sizeof(address_server));

	address_server.sin_family = AF_INET;
	address_server.sin_addr.s_addr = htonl(INADDR_ANY);
	address_server.sin_port = htons(SERV_TCP_PORT);

	if (bind(
		socket_server,
		(struct sockaddr *) &address_server,
		sizeof(address_server))
			< 0)
		err_dump("server: can't bind local address");

	listen(socket_server, 5);

	for ( ; ; ) {

		/* wait for client connection and accpt it when it comes */

		client_address_size = sizeof(address_client);
		socket_client = accept(
				socket_server,
				(struct sockaddr *) &address_client,
				&client_address_size);

		if (socket_client < 0)
			err_dump("server: accept error");

		if ( (child_pid = fork() ) < 0)
			err_dump("server: fork error");

		/* specific client process handling starts here */

		else if (child_pid == 0) {

			/* client process has no need of the server socket */

			close(socket_server);

			/* generate a file stream from the client socket */

			if ((stream_client = fdopen(socket_client, "w")) == NULL) {
				perror("daytimed fdopen");
				return 5;
			}

			/* write to the client stream as per normal then close it */

        		if ((now = time(NULL)) < 0) {
            			perror("daytimed time");

            			return 6;
        		}

        		tm = gmtime(&now);
        		fprintf(stream_client, "%.4i-%.2i-%.2iT%.2i:%.2i:%.2iZ\n",
           			 tm->tm_year + 1900,
           			 tm->tm_mon + 1,
            			tm->tm_mday,
           			 tm->tm_hour,
           			 tm->tm_min,
            			tm->tm_sec);

        		fclose(stream_client);

			exit(EXIT_SUCCESS);

			/* client process has left the building */
		}

		/* the server has no need to keep the client socket open */

		close(socket_client);
	}

	return 0;
}

Batch Posting to a Website

Ok so this is mostly about me playing with the Google syntax highlighter.  Its basically some code I rattled up to post data from a | seperated list of post_variable|post_value pairs.  Read the code to find out more.  Its not a real solution to anything if you want to run tests or batch stuff use Junit, Maven or Ant.  But you can have fun with my limited toy if you like – but remember as always the code is GPL and will probably kill you so watch out!

/*
 * Batch processor that takes | delimitied lines of fields in a file and posts
 * them to an HTTP server printing out the message and response to std out.
 */

/*
 * TODO:
 * Could be extended to use a configuration file for inputs, logging and could
 * bennefit from SSL and proxy configuration options.
 */
package batchpost;

import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;

import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.ProtocolException;
import java.net.URL;
import java.net.URLEncoder;

import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author mc
 */
public class batchpost {

  /**
   * General output device
   */
  static private Logger log = Logger.getLogger(batchpost.class.getName());

  /**
   * main
   *
   * Main entry point.
   *
   * @param args the command line arguments
   *   args[0] - is the fully qualified name of the batch file to read
   *   args[1] - is the URL to post the transactions to
   */
  public static void main(String[] args) {

    /*
     * TODO:
     * I should validate arguments  and do other things but I am skipping doing
     * them because of time.
     */

    processFile(args[0], args[1]);
  }

  /**
   * processFile
   *
   * Takes a file apart one line at a time and hands processing off to a
   * message processing method.  Logs errors if the file is not found or there
   * is an IO error while reading the file.  Assumes that each line contains a
   * complete message.
   *
   * @param url - path to the web server to send the messages to
   * @param file - file to retrieve the message lines from
   */
  static private void processFile(String url, String file) {

    try {

      String line;
      BufferedReader br =
          new BufferedReader(
          new InputStreamReader(
          new DataInputStream(
          new FileInputStream(file))));

      try {

        while ((line = br.readLine()) != null) {
          processMessage(url, line);
        }

      } catch (UnsupportedEncodingException ex) {

        log.log(Level.SEVERE, "processFile - processing halted because it is " +
            "not possible to encode  the post string in UTF-8.", ex);
      } catch (IOException ex) {

        log.log(Level.SEVERE, "processFile - IO exception has halted " +
            "processing of the file \"" + file + "\".", ex);
      }

      try {

        br.close();

      } catch (IOException ex) {

        log.log(Level.SEVERE, "processFile - IO exception has interrupted " +
            "closing of the file \"" + file + "\".", ex);
      }
    } catch (FileNotFoundException ex) {

      log.log(Level.SEVERE, "processFile - unable to locate the file \"" +
          file + "\".", ex);
    }
  }

  /**
   * processMessage
   *
   * Takes a line of | delimited message fields and posts them to a web server.
   * Each line is composed of a list of field_name|field_value pairs such that
   * the line looks like:
   *
   *    name_1|value_1|name_2|value_2|...
   *
   *    or
   *
   *    name|bob charles|age|62|occupation|pro golfer
   *
   * The total number of | delimited fields must be a multiple of 2.  The
   * message fields must not contain a |.
   *
   * @param url - address of the web server to post this message to
   * @param line - string containing a | delimited set of fields
   * @throws java.io.UnsupportedEncodingException when the platform does not
   *    support utf-8
   */
  static private void processMessage(String url, String line)
      throws UnsupportedEncodingException {

    String[] fields = line.split("\\|");
    if (fields.length % 2 == 1) {
      log.log(Level.SEVERE, "processMessage - unable to process the following" +
          " message as it did not contain a multiple of 2 fields: message = " +
          "\"" + line + "\"");
      return;
    }
    String postData = createPostData(fields);

    log.log(Level.INFO, "START OF DATA TO BE POSTED");
    log.log(Level.INFO, postData);
    log.log(Level.INFO, "END OF DATA TO BE POSTED");

    String responseData;
    responseData = doHTTPPost(url, postData);

    log.log(Level.INFO, "START OF DATA RECEIVED FROM POST");
    log.log(Level.INFO, responseData);
    log.log(Level.INFO, "END OF DATA RECEIVED FROM POST");
  }

  /**
   * createPostData
   *
   * Takes an array of field names and their values such that each two
   * consecutive strings are a field name then value pair and prepares a POST
   * data string for sending to a web server.  Throws the encoding error as it
   * is pointless to continue processing as this error is going to reccur for
   * all post attempts if it takes place for one.
   *
   * @param fields - a string array of field name and value pairs.
   * @return string containing the post data.
   * @throws java.io.UnsupportedEncodingException
   */
  static private String createPostData(String[] fields)
      throws UnsupportedEncodingException {

    StringBuffer buf = new StringBuffer();
    String charEnc = "UTF-8";

    try {

      buf.append(URLEncoder.encode(fields[0], charEnc));
      buf.append('=');
      buf.append(URLEncoder.encode(fields[1], charEnc));

      for (int i = 2; i < fields.length; i = i + 2) {

        buf.append('&');
        buf.append(URLEncoder.encode(fields[i], charEnc));
        buf.append('=');
        buf.append(URLEncoder.encode(fields[i + 1], charEnc));
      }

      return buf.toString();
    } catch (UnsupportedEncodingException ex) {

      log.log(Level.SEVERE, "createPostData - unable to encode the post data " +
          "as " + charEnc + " is not available on this platform.", ex);
      throw ex;
    }

  }

  /**
   * readAll
   *
   * Reads all of the input from the given stream and returns it in a byte array
   * output stream, the stream contents can then be retrieved as a byte array or
   * a string using either toByteArray or toString respectively.
   *
   * @param is - an open input stream to read the data from
   *
   * @return a byte array output stream that contains the data read from the is
   * @throws IOException if there is an error reading from the input stream
   */
  static private ByteArrayOutputStream readAll(InputStream is)
      throws IOException {

    ByteArrayOutputStream result = new ByteArrayOutputStream();
    byte[] buf = new byte[1024];

    int len = is.read(buf);

    while (!(len < 0)) {

      result.write(buf, 0, len);
      len = is.read(buf);
    }

    return result;
  }

  /**
   * doHTTPPost
   *
   * performs the https post operation of the given data to the given url
   * returning the result in a byte array output stream, throws an exception if
   * an error takes place while performing io operations, does not deal with
   * proxy servers
   *
   * @param data - a post formatted data stream for sending to the url server
   * @param url - the target to send the post to
   *
   * @return a data stream of the response from the server at the url
   * @throws IOException on failure of any io operation
   */
  static private String doHTTPPost(String url, String data) {

    String result = "";
    HttpURLConnection conn = null;

    try {

      // set up the url connection
      conn = (HttpURLConnection) (new URL(url)).openConnection();
      try {

        // write out the data to the server
        conn.setDoOutput(true);
        conn.setRequestMethod("POST");
        conn.getOutputStream().write(data.getBytes());
        System.out.println(conn.getResponseMessage());

        // disconnect and recycle the connection, return the response data
        result = readAll(conn.getInputStream()).toString();
      } catch (ProtocolException ex) {

        log.log(Level.SEVERE, "doHTTPPost - unable to marshal the required " +
            "protocol to establish a connection to \"" + url + "\"", ex);
      } catch (IOException ex) {

        log.log(Level.SEVERE, "doHTTPPost - IO error while attempting to " +
            "perform the data post operation.", ex);
      } finally {

        conn.disconnect();
      }

    } catch (MalformedURLException ex) {

      log.log(Level.SEVERE, "doHTTPPost - the URL currently being attempted " +
          "to connect with is malformed \"" + url + "\"", ex);
    } catch (IOException ex) {

      log.log(Level.SEVERE, "doHTTPPost - IO error while attempting to set up" +
          " the data post operation.", ex);
    } finally {

      conn = null;
    }

    return result;
  }

}

Copyright © 1996-2010 Code Snips. All rights reserved.
iDream theme by Templates Next | Powered by WordPress