Posts Tagged design
Simple Custom Event Processor for pyinotify
In my previous post on pyinotify I took a short look at the basic use of the pyinotify watcher using mostly default settings. Here I take a brief look at building a custom event processor. The event processor is called to retrieve the event data and, well, process it. The default processor simply prints the output to ‘stdout‘; mine will too. But it is just there to show how simple it is. Note that the processor actually processes the events in line so there’s no need for messy threading.
#!/usr/bin/python
import os
import pyinotify
from datetime import datetime
m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, pyinotify.ProcessEvent(), 0, 0, 100)
m.add_watch('/tmp', pyinotify.ALL_EVENTS, rec=True)
while True:
print "START LOOP %s " % datetime.now()
try:
if notifier.check_events():
print "Result of notifier.check_events() = TRUE"
notifier.read_events()
print "Calling notifier.process_events()"
notifier.process_events()
except KeyboardInterrupt:
notifier.stop()
break
print "END LOOP %s " % datetime.now()
This is just the same code as we saw in the earlier post except that I’ve put the ‘notifier.process_events()‘ in its proper place so that it doesn’t get called in every loop execution. It is restricted by the ‘notifier.check_events()‘. To add a little info to the whole process I’ve included some print statements to illustrate the flow of execution. With this we get results that look like:
START LOOP 2009-05-07 23:02:25.025048 Result of notifier.check_events() = TRUE Calling notifier.process_events()END LOOP 2009-05-07 23:02:25.104546
Which is what you’d expect. However you will also get results that look like.
START LOOP 2009-05-07 23:02:18.808039 Result of notifier.check_events() = TRUE Calling notifier.process_events()END LOOP 2009-05-07 23:02:18.863855 START LOOP 2009-05-07 23:02:18.863912 Result of notifier.check_events() = TRUE Calling notifier.process_events() END LOOP 2009-05-07 23:02:18.864656 START LOOP 2009-05-07 23:02:18.864710 Result of notifier.check_events() = TRUE Calling notifier.process_events() END LOOP 2009-05-07 23:02:18.865415
So you can see that we get some reasonably unpredictable results.
Now let’s do something a little different and collect the event and print it ourselves a little differently. In this case we’re still collecting all events, we just wanted to mark up the output with something extra.
#!/usr/bin/python
import os
import pyinotify
from datetime import datetime
class e(pyinotify.ProcessEvent):
def process_default(self, event):
print(repr(event))
m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, e(), 0, 0, 100)
m.add_watch('/tmp', pyinotify.ALL_EVENTS, rec=True)
while True:
print "START LOOP %s " % datetime.now()
try:
if notifier.check_events():
print "Result of notifier.check_events() = TRUE"
notifier.read_events()
print "Calling notifier.process_events()"
notifier.process_events()
except KeyboardInterrupt:
notifier.stop()
break
print "END LOOP %s " % datetime.now()
Easy huh? But there’s still a little more, there’s a problem! More on that later.
Python and pyinotify
A friend of mine posted a really good article over at blogspot about using pyinotify; after reading one of the comments there I started playing around with it. So if you want the original have a look at David Latham’s original post. I’ve simplified the code just a little to get to the essence of pyinotify. Incidentally if you’re working from Fedora like I am you might like to ‘yum install python-inotify*‘ and confirm that the module and documentation are installed. Here’s the base version.
#!/usr/bin/python
import os
import pyinotify
m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, pyinotify.ProcessEvent())
m.add_watch('/tmp/', pyinotify.ALL_EVENTS, rec=True)
while True:
try:
notifier.process_events()
if notifier.check_events():
notifier.read_events()
except KeyboardInterrupt:
notifier.stop()
break
It works really well for this example but has one generally unwelcome side effect. It also blocks on the ‘notifier.check_events()‘ call. A blocking call is not really what you want in an event loop so it doesn’t really work too well for the core of the application. However this is overcome very simply by adding the three extra parameters defining the wait timing of the ‘pyinotify.Notifier(m, pyinotify.ProcessEvent(), 0, 0, 10)‘. In this case the block is turned to a 10 second wait courtesy of the last parameter as shown in the next code snippet below. In an application event loop you would use a value of 0 meaning that the call would not block at all.
#!/usr/bin/python
import os
import pyinotify
m = pyinotify.WatchManager()
notifier = pyinotify.Notifier(m, pyinotify.ProcessEvent(), 0, 0, 10)
m.add_watch('/tmp/', pyinotify.ALL_EVENTS, rec=True)
while True:
try:
notifier.process_events()
if notifier.check_events():
notifier.read_events()
except KeyboardInterrupt:
notifier.stop()
break
In this case the default event handler just prints XML formatted event lines to stdout. But there’s just one more unexpected side effect. You may find there are a few timing issues and how do you get the notification back to the main thread? But more on that later.
Blinded by the Wobbly Wobbly Web
Its really hard to make a great ascetically pleasing web site, which is probably why this one is so dull, mostly because of your monitor. You might think its rude of me to blame your monitor, perhaps it is. Or is it? Actually it looks really good on my 1080p wide screen monitor with its thousands of shades of black. It even looks really good on my wife’s cool little Mac Book. But it really sucks on the screens at work.
What I’m getting at here is not really how the monitor makes it look but the challenge that we as designers face in conveying our creations to the outside world through this uncooperative medium of the web. Today we constantly fight with:
- Browser reproduction – is the box model in IE really broken, if so which one in which version of IE? Is Safari the benchmark or Opera? What about Firefox and Chrome, surely they could get it right?
- Display characteristics – how do you know what black really looks like on my screen, or someone else’s? Will your carefully shaded images show the lovely gradients of a baby’s skin or will it be uglified by the course grades available? Are the dimensions just right? How about the resolution?
- Plugin availability – all my browsers have flash block on them to stop Internet bandwidth thieves stealing what I’ve paid for with their extravagant advertising. Are all the fonts that you need there? JavaScript anyone? Will that DHTML find everything that it needs?
Of course there are lots more things that can seemingly go astray that aren’t mentioned here, so how do we solve it? Some hail standards as the way to go. Perhaps they are right, I certainly like standards. But the problem with standards is that they need to cater for every situation; and they need to be created, grow and die gracefully. Very hard to do well. Other’s take the minimalist approach claiming that it’s easier to be consistent accross diverse transmission and presentation media by making it all to simple to stuff up. But as a friend has a habit of saying this is proved wrong by the all too true adage ‘when one makes a thing idiot proof, one will only find a deeper class of idiot’. It is so easy to disprove the idea that a central control system or monopoly can render things perfect through ultimate control with one word: Microsoft.
Ultimately there’s one thing that we can’t protect ourselves from, idiots who think that this problem can be universally solved. The problem is not that the situation need be solved but that we need to recognise that it doesn’t need to be solved. You really only want to talk to those of a similar mind or a specific group so just design for them. People who want to talk to the world fail because the world quite frankly isn’t interested and isn’t listening. So don’t solve that problem.
Solve the problem of communicating with your target market. Our only concern should be with our target audience and communication with them. Smaller number of people means a smaller number of problems and perhaps we can manage that. So take this web site, its target market is me. I blog mostly to see my thoughts on the screen because it helps me think about them. Some times it helps me understand the rubbish that I think. Other times it helps me develop some really neat ideas and concepts. If other people read it that’s fine, I might even refer people to it to understand something that I don’t have time to explain but that I’ve written about. But at the end of the day its just a place for my thoughts and ramblings. Do the same with your market, are they old people with simple set ups they bought from Wall Mart? Are they techno-junkies how are likely to have the latest in colour reproducing screens? Will they actually consume your content how you want them too?
At this point I need to wrap up with a tie to the opening about aesthetics and the variances that make it hard with one last and the most significant point, the people factor. Every now and then someone says to me, ‘it shouldn’t be black with white ’cause its hard to read’ or ‘if you don’t make it colourful it won’t be a site that people like to spend time on’ or my favourite ‘I’m an artist / designer / architect and I know what looks good and that doesn’t’. But even the so called professionals are really just espousing their own opinions and preferences unless they can come up with concrete reasons that demonstrates that your target market will be affected by what they have observed. So take the hints from above and comments of others into consideration when designing your site, but most importantly survey your target market to make sure your web site works for your audience.
All good points, however there is a point that needs to be understood; I’m sad if you don’t like my web site but that’s OK because it works for my audience, me. Hopefully everybody else has just as much success designing for their audience too.
Best of luck to all of you, M.C.
LDAP vs RDBMS is the war over?
Perhaps the question is better put, ‘Did the war even happen?’ What war you might well ask; well when different ways of looking at potentially the same data are viewed then there are bound to be zealots on either side fanning the flames of FUD on the opposition and extolling the vitues of their chosen creed. However although both RDBMS and LDAP are commonly used to host the same data for applications there appears to be little controversy.
Mostly, I think, there is no conflict because in general LDAP is not even an option for the vast masses of application developers out there. Especially as the bulk of application development appears to be done facing the Internet. Normally hosting environments provide a choice of RDBMS or RDBMS. LDAP is just too difficult a concept, and that’s probably the first reason that it has fallen out of contention for storing the masses of organisational data out there. Yet it is probably the best candidate that I know of for holding operational control of that data.
To illustrate this conceptual difficulty lets consider the contention between software developer and database architecht perspectives. Typically the developer does not really understand the various tools that the RDBMS offers to manage data. The database expert loaths the way that developers start implementing relational contraints in code instead of applying the appropriate design concepts of foriegn keys, triggers, stored procedures etc. I’ve seen this many times. Often I’ve seen the opposite where database folk have decided to embed code into their domain. To really get this right each party needs to understand the value that the other brings to the table. But I continually read laments from either side criticising the way that this or that should really be done over here and not there.
Which brings us into LDAP. This underused gem is typically used by the development community simply as a convienient pre-built authentication tool. Even when LDAP is used a lot of information is then stored in an RDBMS that it really does not do well; or at least as well as LDAP. I’ll spare you the history of why this is so and how it came to be but the fundamental difference between the two is that RDBMS generally is a collection of flat file tables that are related by loose rules; whereas the LDAP server is a tightly coupled hierarchy of objects (called the Directory Information Tree – DIT) similar in nature to that of other concepts like XML.
LDAP servers are generally built for high speed retrieval and mass replication focussed on the enterprise, where RDBMS is a general ledger store house, and pretty primative too. The kind of data then that you should store in the LDAP directory is structural data. Like say configuration data for a telephone system or contact information for a customer relationship management (CRM) type of application. Even the configuration and preferences information of Wordpress for this blog should really be stored in an LDAP server. But its not.
If you want to know if the data that you are looking at is suited to LDAP then consider.
- Is the data dynamic or relatively static?
- Does the data need to be distributed?
- Can the data be used by more than one application?
- Is the data multi-valued?
- Can your data or application take advantage of a hierarchical relationship?
- Do you need flexible security options?
- Do you need single sign-on?
- Do you need distributed or delegated administration capabilities?
If you can answer yes to some or all of these questions, then directories and directory-based applications would likely be useful to your application or project.
So why did I write this post? Basically it is a pointer into the world of LDAP for those who’ve not thought of it. A start. If you really want a better introduction go here. Have fun.