Code Snips

Recently I wrote about how to use OpenSSL to connect to a plain data server, this time I’m modifying the same code to perform encrypted connections.  Naturally this is more of an example for how to use the API than production ready code.  It’s main purpose is to show the very small difference between using the library as I did last time and how that example can be altered to create a basic SSL client.

The essential changes to the code below are the replacement of the connection function ‘connect_unencrypted(host_and_port)‘ with ‘connect_encrypted(host_and_port, store_path, store_type, &ctx, &ssl)‘ and the introduction of the SSL cleanup step ‘SSL_CTX_free(ctx)‘.  All other changes are purely cosmetic; which really shows how simple adding SSL to your application connections can be.  Externally you need to provide the root CA certificate for the connection to be verified by.  That’s it.

At this point I could warble through the connection function, but you should just read through it yourself and consult the SSL man pages.  Note that there is a dreadful buffer overflow possibility in this code and no real error handling, just a bit of logging.  This is to keep the example short and also because only you will know what valid handling should take place for each situation when you write your own code.

So take a look and enjoy.  To try this out yourself:

1. Make sure that you have Firefox, GCC and OpenSSL (development sources and libraries) installed.
2. Copy the following code to a file called ‘main.c‘ in a directory that you will be playing around in.
3. Compile the code using ‘gcc main.c -o sslclient -lssl‘ if you are on Linux or ‘gcc main.c -o sslclient -lssl-lcrypto‘ if you are on OSX.
4. Select an SSL (https) web site to connect to and find the Root CA’s certificate name in the site’s certificate.
5. Either export the appropriate root CA from Firefox or obtain it directly from the CA online in pem format and copy it to a file ‘certificate.pem‘ in the same directory as the ‘sslclient‘ file.
6. Run the following command:

'./sslclient servername:443 "GET / \r\n\r\n" certificate.pem f e'

/*
 * File:   main.cpp
 *
 * Licence: GPL2
 */

/* Standard headers */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

/* OpenSSL headers */
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>

/**
 * Simple log function
 */
void slog(char* message) {
    fprintf(stdout, message);
}

/**
 * Print SSL error details
 */
void print_ssl_error(char* message, FILE* out) {

    fprintf(out, message);
    fprintf(out, "Error: %s\n", ERR_reason_error_string(ERR_get_error()));
    fprintf(out, "%s\n", ERR_error_string(ERR_get_error(), NULL));
    ERR_print_errors_fp(out);
}

/**
 * Print SSL error details with inserted content
 */
void print_ssl_error_2(char* message, char* content, FILE* out) {

    fprintf(out, message, content);
    fprintf(out, "Error: %s\n", ERR_reason_error_string(ERR_get_error()));
    fprintf(out, "%s\n", ERR_error_string(ERR_get_error(), NULL));
    ERR_print_errors_fp(out);
}

/**
 * Initialise OpenSSL
 */
void init_openssl() {

    /* call the standard SSL init functions */
    SSL_load_error_strings();
    SSL_library_init();
    ERR_load_BIO_strings();
    OpenSSL_add_all_algorithms();

    /* seed the random number system - only really nessecary for systems without '/dev/random' */
    /* RAND_add(?,?,?); need to work out a cryptographically significant way of generating the seed */
}

/**
 * Close an unencrypted connection gracefully
 */
int close_connection(BIO* bio) {

    int r = 0;

    r = BIO_free(bio);
    if (r == 0) {
        /* Error unable to free BIO */
    }

    return r;
}

/**
 * Connect to a host using an unencrypted stream
 */
BIO* connect_unencrypted(char* host_and_port) {

    BIO* bio = NULL;

    /* Create a new connection */
    bio = BIO_new_connect(host_and_port);
    if (bio == NULL) {

        print_ssl_error("Unable to create a new unencrypted BIO object.\n", stdout);
        return NULL;
    }

    /* Verify successful connection */
    if (BIO_do_connect(bio) != 1) {

        print_ssl_error("Unable to connect unencrypted.\n", stdout);
        close_connection(bio);
        return NULL;
    }

    return bio;
}

/**
 * Connect to a host using an encrypted stream
 */
BIO* connect_encrypted(char* host_and_port, char* store_path, char store_type, SSL_CTX** ctx, SSL** ssl) {

    BIO* bio = NULL;
    int r = 0;

    /* Set up the SSL pointers */
    *ctx = SSL_CTX_new(SSLv23_client_method());
    *ssl = NULL;

    /* Load the trust store from the pem location in argv[2] */
    if (store_type == 'f')
        r = SSL_CTX_load_verify_locations(*ctx, store_path, NULL);
    else
        r = SSL_CTX_load_verify_locations(*ctx, NULL, store_path);
    if (r == 0) {

        print_ssl_error_2("Unable to load the trust store from %s.\n", store_path, stdout);
        return NULL;
    }

    /* Setting up the BIO SSL object */
    bio = BIO_new_ssl_connect(*ctx);
    BIO_get_ssl(bio, ssl);
    if (!(*ssl)) {

        print_ssl_error("Unable to allocate SSL pointer.\n", stdout);
        return NULL;
    }
    SSL_set_mode(*ssl, SSL_MODE_AUTO_RETRY);

    /* Attempt to connect */
    BIO_set_conn_hostname(bio, host_and_port);

    /* Verify the connection opened and perform the handshake */
    if (BIO_do_connect(bio) < 1) {

        print_ssl_error_2("Unable to connect BIO.%s\n", host_and_port, stdout);
        return NULL;
    }

    if (SSL_get_verify_result(*ssl) != X509_V_OK) {

        print_ssl_error("Unable to verify connection result.\n", stdout);
    }

    return bio;
}

/**
 * Read a from a stream and handle restarts if nessecary
 */
ssize_t read_from_stream(BIO* bio, char* buffer, ssize_t length) {

    ssize_t r = -1;

    while (r < 0) {

        r = BIO_read(bio, buffer, length);
        if (r == 0) {

            print_ssl_error("Reached the end of the data stream.\n", stdout);
            continue;

        } else if (r < 0) {

            if (!BIO_should_retry(bio)) {

                print_ssl_error("BIO_read should retry test failed.\n", stdout);
                continue;
            }

            /* It would be prudent to check the reason for the retry and handle
             * it appropriately here */
        }

    };

    return r;
}

/**
 * Write to a stream and handle restarts if nessecary
 */
int write_to_stream(BIO* bio, char* buffer, ssize_t length) {

    ssize_t r = -1;

    while (r < 0) {

        r = BIO_write(bio, buffer, length);
        if (r <= 0) {

            if (!BIO_should_retry(bio)) {

                print_ssl_error("BIO_read should retry test failed.\n", stdout);
                continue;
            }

            /* It would be prudent to check the reason for the retry and handle
             * it appropriately here */
        }

    }

    return r;
}

/**
 * Main SSL demonstration code entry point
 */
int main(int argc, char** argv) {

    char* host_and_port = argv[1]; /* localhost:4422 */
    char* server_request = argv[2]; /* "GET / \r\n\r\n" */
    char* store_path = argv[3]; /* /home/user/projects/sslclient/certificate.pem */
    char store_type = argv[4][0]; /* f = file, anything else is a directory structure */
    char connection_type = argv[5][0]; /* e = encrypted, anything else is unencrypted */

    char buffer[4096];
    buffer[0] = 0;

    BIO* bio;
    SSL_CTX* ctx = NULL;
    SSL* ssl = NULL;

    /* initilise the OpenSSL library */
    init_openssl();

    /* encrypted link */
    if (connection_type == 'e') {

        if ((bio = connect_encrypted(host_and_port, store_path, store_type, &ctx, &ssl)) == NULL)
            return (EXIT_FAILURE);
    }
        /* unencrypted link */
    else if ((bio = connect_unencrypted(host_and_port)) == NULL)
        return (EXIT_FAILURE);

    write_to_stream(bio, server_request, strlen(server_request));
    read_from_stream(bio, buffer, 4096);
    printf("%s\r\n", buffer);

    if (close_connection(bio) == 0)
        return (EXIT_FAILURE);

    /* clean up the SSL context resources for the encrypted link */
    if (connection_type == 'e')
        SSL_CTX_free(ctx);

    return (EXIT_SUCCESS);
}

Strange as it seems but you can make non-SSL connections using the OpenSSL libraries.  The following code is a brief example.  I like it because it appears to simplify (arguably) the rather more involved Berkley sockets.  It still lets you get at the socket if you want to mangle it yourself.  But ultimately its kind of handy to have a library that does SSL and plain transfers without too much of a code change.

Here is a simple example of using the OpenSSL BIO interface to retrieve a root web page.  Naturally most of the exception handling is not included so don’t going using it for production purposes as is.  Not that it’d be a lot of use as anything other than an exaple of the OpenSSL BIO API usage.

Have fun.

/*
 * File:   main.cpp
 *
 * Licence: GPL2
 */

/* Standard headers */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

/* OpenSSL headers */
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>

/**
 * Initialise OpenSSL
 */
void init_openssl() {

    SSL_load_error_strings();
    ERR_load_BIO_strings();
    OpenSSL_add_all_algorithms();
}

/**
 * Close a BIO connection gracefully
 */
int close_connection(BIO* bio) {

    int r = 0;

    r = BIO_free(bio);
    if (r == 0) {
        /* Error unable to free BIO */
    }

    return r;
}

/**
 * Connect to a host using an unencrypted stream
 */
BIO* connect_unencrypted(char* host_and_port) {

    BIO* bio = NULL;

    /* Create a new connection */
    bio = BIO_new_connect(host_and_port);
    if (bio == NULL) {

        printf("Error: %s\n", ERR_reason_error_string(ERR_get_error()));
        printf("%s\n", ERR_error_string(ERR_get_error(), NULL));

        return NULL;
    }

    /* Verify successful connection */
    if (BIO_do_connect(bio) != 1) {

        printf("Error: %s\n", ERR_reason_error_string(ERR_get_error()));
        printf("%s\n", ERR_error_string(ERR_get_error(), NULL));

        close_connection(bio);
        return NULL;
    }

    return bio;
}

/**
 * Read a from a stream and handle restarts if nessecary
 */
ssize_t read_from_stream(BIO* bio, char* buffer, ssize_t length) {

    ssize_t r = -1;

    while (r < 0) {

        r = BIO_read(bio, buffer, length);
        if (r == 0) {

            /* Handle closed connection */
            continue;

        } else if (r < 0) {

            if (!BIO_should_retry(bio)) {

                /* Handle failed read here */

                printf("Error: %s\n", ERR_reason_error_string(ERR_get_error()));
                printf("%s\n", ERR_error_string(ERR_get_error(), NULL));

                continue;
            }

            /* It would be prudent to check the reason for the retry and handle
             * it appropriately here */
        }

    };

    return r;
}

/**
 * Write to a stream and handle restarts if nessecary
 */
int write_to_stream(BIO* bio, char* buffer, ssize_t length) {

    ssize_t r = -1;

    while (r < 0) {

        r = BIO_write(bio, buffer, length);
        if (r <= 0) {

            if (!BIO_should_retry(bio)) {

                /* Handle failed write here */
                printf("Error: %s\n", ERR_reason_error_string(ERR_get_error()));
                printf("%s\n", ERR_error_string(ERR_get_error(), NULL));

                continue;
            }

            /* It would be prudent to check the reason for the retry and handle
             * it appropriately here */
        }

    }

    return r;
}

/**
 * Main SSL demonstration code entry point
 */
int main(int argc, char** argv) {

    BIO * bio;
    char *addr = "GET / HTTP/1.1\r\nHost: www.home.com\r\n\r\n";
    char buffer[4096];
    buffer[0] = 0;

    init_openssl();

    if ((bio = connect_unencrypted(argv[1])) == NULL)
        return (EXIT_FAILURE);

    write_to_stream(bio, addr, strlen(addr));
    read_from_stream(bio, buffer, 4096);
    printf("%s", buffer);

    if (close_connection(bio) == 0)
        return (EXIT_FAILURE);

    return (EXIT_SUCCESS);
}

In late January I suffered a rather nasty medical event which had me on my back for a few weeks and will probably not have me back to full health for some time.  One of the things that helped me was Chess and programming.  Help meant proving to myself that my mental faculties were completely intact.  My results in Chess were no more rubbish than usual so that proved a little but programming has helped a bit more.

Any way we have been trying different things to encourage our children to learn maths so I wrote a little maths testing tool to help us.  Its plain, not fancy, just asks questions and keeps count of the results.  The idea is that we can get one or other of the urchins to run a few tests when they ask for something.  So we get them to get 200 questions correct and then they can play their game, watch TV or whatever.  It has no rules so you can make them up as you want.

So it’s here, if you want to play.  Real simple.  An example of what you can do with a little HTML, CSS and some simple JQuery.

Using SELinux is reasonably straight forward once you get used to it.  It protects OS objects and configuration from potentially dangerous manipulation.  If an exception to its current policy is found then it is logged and either denied or allowed.  The reasons that it may be allowed depend on the policy’s construction.

For me I use a targeted, as opposed to blanket, policy and set the SELinux level to enforcing.  This means that not everything unplanned will be blocked, but it does log and alert me when something crosses the policy line and allows me to block it if I feel it is necessary.  Naturally it also can block services that I want to use too.  So in this case I will look at setting up vsftpd to access home directories.

If you run this command (you need the selinuxtools package installed) you will generate a new policy straight to the console to allow all of the exceptions found in the SELinux log ‘audit2allow < /var/log/audit/audit.log’.  Naturally this is not what we want as it will generate a policy that will allow all previously caught exceptions to proceed.  We just want to allow ftp; which is what the following statement does: ‘grep ftp /var/log/audit/audit.log | audit2allow -R’.

The result:

[foo@bar ~]# grep ftp /var/log/audit/audit.log | audit2allow -R

require {
type ftpd_t;
type home_root_t;
class capability net_raw;
class dir search;
}

#============= ftpd_t ==============
#!!!! This avc can be allowed using one of the these booleans:
#     allow_ftpd_full_access, allow_ftpd_full_access

allow ftpd_t home_root_t:dir search;
allow ftpd_t self:capability net_raw;

What is interesting here is that the audit2allow application alerts us to the fact that the same thing being achieved through the creation of this policy could also be accomplished using SELinux booleans; in this case  ‘setsebool -P allow_ftpd_full_access true’ would do the trick.  However we’re going to do this the long way as booleans are not always available, this is just the first step.

Now that we have inspected the policy we need to build it and integrate it into the active policy.  First we need to construct the new policy file using ‘grep ftp /var/log/audit/audit.log | audit2allow -M ftp’.  This will create a ‘ftp.te’ policy file and a ‘ftp.pp’ compiled policy file.  To make the new policy active simply run ‘semodule -i ftp.pp’ and its done.  This of course may reveal another denial or other OS configuration issue so you just need to perform an allow then test to see if there is anything else that needs sorting out.

For a little while I’ve been playing with sockets in C now and have come up with the following succinct example of connecting.  Note that the connection is fairly flexible with regards to protocol and transport type.  It really is simply there to make the connection to somewhere else with as few questions asked.  It will involve DNS and service lookups if you provide names instead of the network address and port.  If you want to catch exceptions you need to clear the standard error variable and check it yourself after receiving a fail return value.  It won’t, or at least shouldn’t, report any exceptions because for my current purposes failures are not really exceptions just dead ends on some of many paths.  I tried to keep it simple too, have fun.

/* connect_to_socket
 *
 * connect to a socket using an initialised addrinfo structure,
 *
 * info is an initialised addrinfo structure
 * sock is a pointer to the location to store the socket descriptor when opened
 *
 * returns 0 if successful
 */
int connect_to_socket(const struct addrinfo *info, int* sock) {

    /* open socket */
    *sock = socket(info->ai_family, info->ai_socktype, info->ai_protocol);
    if (*sock < 1) return -1;

    /* connect to the server*/
    if (connect(*sock, info->ai_addr, info->ai_addrlen) == 0)
        return 0;

    /* clean up on failure */
    close(*sock);
    return -1;
}

/* connect_to_server
 *
 * connect to a server using the server name or adress and port or service name
 *
 * server is a string containing either the name or address of the server
 * port is a string containing either the service name or port number to use
 * sock is a pointer to the location to store the socket descriptor when opened
 *
 *  returns 0 if successful
 */
int connect_to_server(const char* server, const char* port, int* sock) {

    struct addrinfo hints, *info = NULL, *list = NULL;
    int e = 0;

    /* initialise the hints for retrieving the address details */
    memset(&hints, 0, sizeof (hints));
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = SOCK_STREAM;
    hints.ai_flags = AI_PASSIVE;
    hints.ai_protocol = 0;

    /* get the address if the server to connect to */
    e = getaddrinfo(server, port, &hints, &list);
    if (e != 0) return -1;

    /* connect to the first socket in the list */
    for (info = list; info != NULL; info = info->ai_next)
        if (connect_to_socket(info, sock) == 0)
            break;

    /* clean up and return */
    if (list != NULL) freeaddrinfo(list);
    if (sock > 0) return 0;
    else return -1;
}

Normally I do most of my network programming in Java; life is just easier that way. But some times, *very* rarely, you need to carve up a little C.  So I’ve not touched socket networking for a while and all its foibles.  Hence a few problems with reading from a socket.

When socket programming with the ‘read’ function it does not always return the length read into the buffer.  In fact if you are talking to an HTTP server a ‘GET /‘ will nearly always end with read returning 0.  This is because the socket will have been closed on your last read.  This of course begs the question, how do you know how much was read in the last read?  For me this was simple; I just zero filled the buffer prior to reading.

ssize_t r_read(int sock, void* buf, size_t size)
{
        ssize_t ret;
        bzero(buf, size);
        while (ret = read(sock, buf, size - 1), ret == -1 && errno == EINTR);
        return ret;
}

Which does work because now its a simple matter of counting until the first 0 shows up, but I’d still like to know how to tell how many characters were read without having to do the count. This becomes more important when transferring binary files as you won’t know if the 0 is validly part of the file or not.

Using netfilter as a loopback device is surprisingly simple and can let you tap into that network conversation between components on a platform that tends to be hidden by the OS in its own loopback device. Here’s how:

iptables -t nat -A PREROUTING -s 192.168.122.2 -d 192.168.2.7 -p tcp --dport 80 -j DNAT --to 192.168.122.2:80
iptables -t nat -A POSTROUTING -s 192.168.122.2 -d 192.168.122.2 -p tcp --dport 80 -j SNAT --to 192.168.2.7

Well that’s the netfilter bit done! But of course there’s a bit more setup than this. Firstly you need to have your netfilter, soon to be loopback, host on the same subnet as the box that we are working with. Next you need to configure the client that makes the connection to the other component on the same box point to the netfilter host. Finally you need to work out which addresses and ports go where in the statements above when you execute them.

In the case above we are intercepting the communication between the HTTP client and server that are both hosted on the same box 192.168.122.2.  The netfilter loopback is being hosted on 192.168.122.1.  For my worked example here the client/server box is actually a virtual machine guest and the netfilter configuration is being placed on the virtual machine host.  Note the third address I used here 192.168.2.7 is bound to another ethernet card in the virtual machine host.  The third address is necessary, for no other reason than to help with the clarity of this example and to get the routing right.

Our first rule catches the inbound requests from our guest and redirects them back to the guest.  Note that the rule is restricted to a specific initial destination, source, protocol and port so that no other hosts will be affected.  In fact the rule will only catch the exact scenario that we want.  However this is not the complete picture.  If we were to try to browse from the client to 192.168.2.7 on port 80 and capture the chatter on 192.168.122.1 using tcpdump we would get the following:

16:54:37.800369 IP (tos 0x0, ttl 64, id 17896, offset 0, flags [DF], proto TCP (6), length 60)
192.168.122.2.41418 > 192.168.122.2.http: S, cksum 0x496b (correct), 901912667:901912667(0)
win 5840  E..
Here you see the first syn of the TCP three part handshake leaving the host and going back to the guest with the guest's IP address as the source!  It is the packet going out of the host to the guest but the guest is not going to be able to route it back.  This is where the second rule steps in.  It is applied after the packet is routed by netfilter and it replaces the source address with 192.168.2.7.  Now the client on the guest can communicate with the server on the guest and you can use tcpdump on the host to look at their conversation.

Some time ago I wrote up a hex2bin and a bin2hex utility in this blog.  This is an example of my hint in the comments in that blog as I’ve seen quite a few rather hashed attempts and it is really dauntingly simple.  I didn’t think that I needed to write it!  Note there’s no real error checking here because it has not been written to fit a particular purpose, so be careful. Somewhat interestingly this code runs faster compiled by a C compiler as opposed to a C++ compiler.

/**
 * char_to_hex_buf
 *
 * Takes a char buffer (s) of length l and inserts the ascii representation
 * of the hex values into the provided hex char buffer (h) of length lx2+1
 * terminating the hex buffer with '\0' in the final position of h[l].
 *
 * h - preallocated char buffer of length lx2+1 to put the translated source
 *     buffer hex character contents into
 * s - source char buffer of length l containing the characters to return as
 *     a hex character string the '\0' character is not a terminator for the
 *     purposes of this string
 * l - length of the source string
 *
 * returns EXIT_SUCCESS no other return value makes sense
 */
int char_to_hex_buf(char *h, const size_t l, const char *s) {

	size_t c;

	for (c=0;c<l;c++)
		sprintf(&(h[c*2]),"%02X",(unsigned char)(s[c]));

	return EXIT_SUCCESS;
}

If you want to encode video to PSP format on Linux install ffmpeg and try this:

ffmpeg -i my_home_movie.mpg -f psp -r 29.97 -b 512k -ar 24000 -ab 64k -s 368x208 my_home_movie.mp4

Or if you prefer the other style you can try this:

ffmpeg -i my_home_movie.mpg -f psp -r 29.97 -b 512k -ar 24000 -ab 64k -s 320x240 my_home_movie.mp4

For me the first option is the best.  But naturally its up to your taste as to what you prefer and you can muck about with all of the options but these worked well enough for me.

We use office communicator at work which is a bit of a pain because it doesn’t log anything. If you want to save your conversation you will need to resort to the old copy and paste trick. Nothing as nice as Pidgin’s log option.

So I’ve gone back to Pidgin and installed the SIPE extension. Its a really usable extension but sorting out the configuration and dependancies can be a bit of a pain as it is not yet in a repository. Basically its an old fashioned download – configure – compile – install problem.

First getting the latest version can be a little tricky as it appears that the project web site is not always in sync with the source. When I looked the latest vesion advertised on the project home page was 1.5.0 but 1.6.1 was available from here. Admittedly 1.6.1 is only alpha but hey you’d think they’d advertise the bleeding edge for those of us who live there.

At least all of the dependencies are available through yum for Fedora. There are no way out libraries required. But to find out all of the libraries to install you may have to run configure a few times. Just remember you may have the run-time of any requirement installed but you’ll also need the development version installed too to compile against.

For me I also needed to add a couple of switches to get the beastie to compile properly. So the configure command ended up looking like this:

./configure --prefix=/usr --enable-quality-check=no

Once this is complete then you can make and make install the plugin.  Note that you will need to run the make install under root.

Creating an account was a matter of using the add functionality from the modify account dialog.  Choose the ‘Microsoft LCS/OCS’ variant and set up the account details.  In my case I didn’t need a proxy and by default the connection trys to use SSL/TLS which I didn’t want so I switched it off to TCP, the Microsoft default.  The username is your base email address, not your alias and the login is your domain login details.  Remember to set the server too and it should be all ready to go.