Python UTF-8 for non english characters

Configure module
First make sure your python module is saved as UTF-8

Next, in the top of your python module add the setting:

# -*- coding: utf-8 -*-

this will tell the compiler to use UTF-8 instead of OS specific

Prefix a 'u' for all strings
print u"rødgrød med fløde"

- will work

print "red porridge with cream"

- wil work because no non-english characters are present
(or whatever difference between your OS and UTF-8)

print "rødgrød med fløde"

- will not work because the encoding is misinterpreted

Entire python script

# -*- coding: utf-8 -*-
print u"rødgrød med fløde"


KHAN's ladder to programming excellence!

In Maxwell Gladwells book 'Outliers' he mentions that in order to excel in anything you have to put atleast 10000 hours in to it, which is the equivalent of 6 hours each day in 5 years. 356 * 5 * 6 = 10680, well almost.

Here I present the ladder of programming excellenceu as perceived by me :)
Legend: HoP = Hours of Programming

KHAN's ladder to programming excellence!

I reckoned I hit the programming experience mark of 5000 hours 5 years ago.

Programming in Basic and DOS before my education = 1000 HoP

beginning programmer - 1000 HoP

Programming in high school Assembler, electronics, VB, Java, ASP, HTML: 1000 HoP

greenhorne programmer - 2000 HoP

Higher education as datamatician(mostly Java) 2 1/4 years, with an average of 140 hours a month: 4000 HoP

- at this point I broke the 5000 HoP

jorneyman programmer - 4000 HoP

After the bulk of work with non-contractor jobs 3 years, with an average of 180 hours a month: 6500 HoP
This involved mostly Java with J2EE, but I began on more complex tasks like configuring WebSphere servers, designing parts of enterprise wide technical solutions. Here I picked up many new complex skills, Eclipse RCP, Version control, functional programming, unit testing, server administration etc.
- here i accelarated because I had to break into many new concepts and frameworks, working after hours and reading a lot while experimenting.

veteran programmer - 8000 HoP

At this point I started my own company and began working as a freelancer. I had no products, no particular approach or plan, only my skill as a programmer. Though I discovered I also needed economical and business skills.
In this work I now have 2 years of experience, my pace has slowed down and I would now estimate to having 170 hours per month of IT experience: 4000 HoP, making a total of 16500 HoP.

expert programmer - 16000 HoP

master for me is 5 years away

master programmer - 32000 HoP

I might achieve programming guru before retirement

programming guru - 64000 HoP

//TODO insert lots of effort

deity programmer - 128000 HoP

I think of this scale as a logarithmic one. Each step requires the double amount of effort and I find this to be quite true. At a time while being greenhorne I experienced discovering new things more often, towards becoming a guru I can only guess to what I learn.

Automatically testing advanced web applications (revisited)

More than one and a half years ago I first posted a blog article 'high automatic test coverage of web applications'. While working with this and later projects I realized that in order to have complete coverage and a more dynamic testing environment I had to take it a step further and therefore started inventing my own test framework, described in the article 'full testing with javascript'.

Soon thereafter I discovered the Selenium Core framework which was still rather immature. Since it functions similar to the framework I was working on, I gave it a go and never looked back.

Testing priorities in a web application
When testing you have to first plan what areas need the most testing. Test driven development requires a lot of focus on unit testing and functional testing. I will not discuss unit testing here, since it depends heavily on the platform of the web application.
Aside from unit testing, functional testing is the most important testing for web applications, because it can be used to directly measure progress and prohibits regression.
On a second place comes integration testing, because alot of problems tend to occur in communication with other systems.
In third comes performance testing. This tends to be the focus of automatic tests, but that is an ancient view imo. Performance testing is important, but it's rather easy once functional testing has been done, because you can leverage the functional tests and scale them to performance tests.
On a fourth place comes installation testing. Redeploying a web application is important, both for testing environments, but also for reestablishing an environment.

Functional testing
this is exactly what Selenium is for. It's fast and works for Python, Java and several other popular platforms. I would go for Python because the interactive compiler is killer for writing test cases. Java is not a bad alternative with Seleniums excellent integration with JUnit.
You should not need any other tool for this task.

Performance testing
For performance testing you could use HTTPUnit, which also integrates well with JUnit. It's a rather simple, but powerful framework for controlling all parameters of requests and monitoring the responses.
Functional test cases written for functional testing using Selenium could be rewritten easily and scaled to do performance testing.

Integration testing
This depends on what technology is used for communication, I'll only present the most common one here. Integration in web applications are typically over Web Services, therefore testing could be done with a framework for easily creating web clients, like Axis 2 and using JUnit to execute it, but HTTPUnit is also viable.

Installation testing
Installation testing involves two parts. Scripts to deploy the application and verification of the function.
Scripts would typically depend on the platform of the web application. For WebLogic or WebSphere, you would use ant targets available from the vendor. How you manage to deploy the application automatically is rather irrelevant, since you needn't verify anything during the process.
The second part of verifying function after installation is rather easy, since it's just a matter of running the functional tests.

Conclusion
Using a few open source tools is an easy, fast and effective way to test advanced Web Applications with lots of javascript and complexity.
Standardizing the executing can be achieved using a common executing framework like JUnit.

KHANDI the temporal database manager

Introduction
On contract for a customer I had to develop a temporal database manager.

The problem is very simple. You have an entity that exists from a data to another date and is dependent on another entity to exist in the same period.

Consider the example below, this is a valid dependency, because, the dependency exists in atleast the same period as the entity.

Entity
     I-------------I

Dependency Entity
   I-------------------I

Legend:
lines represent the existance of the entities

Considering you have to edit an entity, you have two basic changes you can make.
Edit attributes of an entity, creating a new record from that point, or
Edit the period in which the entity exists

Editing attributes
Entity
     I-------I (attribute changed on this date)
              I----I

Dependency Entity
   I-------------------I

 - dependency is still valid

Editing period
Entity
     I-------I (end date of entity is sooner)

Dependency Entity
   I-------------------I

 - dependency is still valid

Now consider the situation where a user makes an invalid edit.

Invalid edit
Entity
     I-------------------I (end date of entity is later)

Dependency Entity
   I-------------------I

 - dependency is NOT valid

The main point of this last example is that there's a lot of checks that need to be made before editing something.

The database I've made with the customer consists of multiple tables 20+, that are all interdependent.

The component I've written is an ORM with temporal management built-in and transparently handles these relation restrictions as defined in the business logic.

Although very clean, it poses some challenges in a UI. If you need to edit several types of entities in the same submit, you could end up having dependencies, that may put the user in a deadlock. Therefore, one should consider if temporal management of an entitty is really necessary to store temporally.

KHANDI Component short:
temporal ORM manager
compliable with java 1.4
Connects with SAS and MySQL out of the box, easily integratable with other dbs

Write me at kennethhn@gmail.com, if interested in knowing more.

TOMCAT UTF-8 gotcha

If you are having problem with your encoding running on a platform with other than UTF-8 as a default, try going through your code and make sure request.getParameter() is not called before setting request.setCharEncoding().

example:

request.setCharacterEncoding("UTF-8");
String adUser = request.getParameter(TAM_AD_USER);


If you have no hook for it, consider making a filter: http://cagan327.blogspot.com/2006/05/utf-8-encoding-fix-tomcat-jsp-etc.html