Why are the stem-and-leaf plots useful in that case? This kind of plot is a method for showing the frequency with which certain classes of values occur. You could achieve the same by making a frequency distribution table or a histogram for the values, or you can use a stem-and-leaf plot and let the numbers themselves to show pretty much the same information.

For our case, we have a train timetable that can be displayed in that way:



Pretty boring right? Are you able to see how frequently trains leave at a given hour? which are the rush hours? how many trains leave at 11pm? In fact, it’s really hard to detect any pattern here.

Using a stem-and-leaf plot we can show the train schedule below, where for each departure time, the hour-digits are the stem (left column) and the minute-digits are the leaves (right columns). So hour-digits are not repeated over and over and minute-digits can be stacked, so we obtain kind of an histogram of the departure times.



We could reduce more the amount of data to display by grouping the departure-hours that have the same minute-departures frequency, obtaining this plot:



A great example of this technique can be found at many train stations in Japan, where this plot is widely used to display the timetables. The resulting plot is useful, as it gives a quick overview of the departures distributions and it lets also to find an specific departure really fast, which is the main purpose of a train timeable.

Time table at Yokohama\’s Minato Mirai train station in Japan illustrating the widespread use of stem-and-leaf graphs in the country. Author: Eliazar

However, the data could be displayed in a different way, so I’ve spent some time playing with the plot to find another way to visualize the data. Looking at the plot, there are a lot of departure hours that share several departure minutes, so a lot of minute-digits are repeated over and over… Also, with the stem-and-plot is easy to detect the density of departures based on hours, but, what about the density of departures based on the minutes? Maybe that does not seems really useful, but who knows? So after playing some time with these questions, I end up with this table below:



The graphic is nothing new, and the result is not a stem-and-leaf plot anymore, but it has its roots on it. As you can see, hour-digits and minute-digits are displayed only once, so here the black dots are the departure times. Also the color is used as a visual clue to communicate the density of the departures for each hour/minute departures. Althought the distance between the points is not precise at all (the spatial distance between the minutes is not proportional) , the distance between dots gives you and idea about the time to wait until the next departure.

Still some improvements can be added to the graphic, but it illustrates how data can be displayed in a graphical format in order to communicate better the information contained in the data itself.

The first one demonstrates how the catalan government is sensible about the value of providing data to the general public, businesses and other organisations so they would be able to re-use public sector data and create value.

The second one is from my point of view such an awesome project that needs an entire post but for the moment, remember that name and its concept: Impure, a visual programming interface designed to give non-programmers access to professional tools for data visualization. Users can use it to process and display data from social media feeds, financial information and more.

Let’s go back to the first initiative:

Government of Catalonia Open Data Project

The government of Catalonia has launched Gencat Open Data, an open government data portal where information of a public nature is published with the goal of fostering the use and reuse of information that comes from the administration. This portal groups together all the Government’s open data initiatives into a single catalogue, and adds the most important information associated with them for reuse purposes.

The offered data set comprises the database of the 26,000 official facilities of Catalonia, the 1,400 procedures handled in the Government’s offices and some of its multimedia archives.

The catalogue is divided into this categories:

• Administrative procedures and services
• Statistical data
• Geographic and cartographic data
• Visual data
• Language data
• Weather data
• News
• Transport and mobility

Also delivers the data in different types of formats (doc, xls, shp, pdf, odf, csv, xml, tmx, json, rdf-xml, etc…). The data is available under 4 possible licences, among which two creative commons licences.

So, what can we do with this data? you can plug it to Impure!

Impure

Impure has been created by Bestiario, a Barcelona Spain based start-up, has been exploring and inventing new interactive visualization paradigms over the past 5 years.

Impure is a visual programming language aimed to gather, process and visualize information. With impure it is possible to obtain information from many different sources; from user owned data to diverse feeds in Internet, including social media data, real time or historical financial information, images, news, search queries and many more. It is a tool to be in touch with data around the Internet, to deeply understand it. Within a modular logic interface you can quickly link information to operators, controls and visualization methods, bringing all the power of the comprehension of information and knowledge to the non programmers that want to work with information in a professional way.

Among other possibilities, impure allows you to:

• Easily read data from diverse sources and repositories
• Load your own data locally or remotely
• Visualize it in a wide range of ways (more than 100 visualization methods so far)
• Process it, compare it, mix it, filter it, (more than 300 controls and operations so far)
• Publish and share your projects

The application provides a set of examples to getting started, as well as good documentation and a wiki. Don’t forget to visit also its impure gallery’s photostream on flickr.

Definitively, a great tool built by the guys of Bestiario. Oh! did I mention that it’s built in flash? ;)

I’ve really got impressed about the posibilities that Flash and P2P can offer together. Specially interesting it has been the demo of Mark Doherty, showing how comunicate via p2p a laptop with an Android mobile device, using a second Android mobile device as a net access point. It was just a demo built with the AIR 2.5 beta but it shows what this technology can offer in reponse to other similiar technologies (the video chat FaceTime for Iphone). Mark shows this in a video on his Flash Mobile Blog.

P2P Video Demo – AIR2.5 on Android from Mark Doherty on Vimeo.

Another Adobe Platform evangelist, Tom Krcha, also has spoken about P2P and Flash, explaining what is peer-to-peer, differences from a centralized server-side model, using Stratus as a hosted rendezvous service, etc. Tom Krach’s blog contains tons of information and tutorials about these technologies and is a perfect starting point if you are interested in real time RIA’s.

Finally, Mihai Corlan have shown another nice example of what can be done using the peer to peer features. This time using several Android mobile phones to manage a desktop MP3 player.

Multi-screen apps for Android Phones and Desktops with Adobe AIR from Mihai Corlan on Vimeo.

Recently I’ve been playing with FLARTooKit and I found this function into the class ArrayUtils.as.

public function createJaggedArray(len:int, ...args):Array
{
     var arr:Array = new Array(len);
     while (len--)
     {
          arr[len] = args.length ? createJaggedArray.apply(null, args) : 0;
     }
     return arr;
}

The function made me smile due to his simplicity. 5 lines of code, use of the … (rest) parameter and a bit of recursion and you have it, a useful function to create arrays with as many dimensions as you want.

So, do you want a 3 x 5 x 2 array? Just call the function as follows:

var jaggedArray:Array = createJaggedArray(3,5,2);

The author is Ben Fry, the founder (with Casey Reas) of Processing, a programming language, and development environment. Initially created to serve as a software sketchbook and to teach fundamentals of computer programming within a visual context, Processing quickly developed into a tool for creating finished professional work as well.

Don’t expect to find awesome graphics or great programming lessons to represent huge sets of data. It’s not a technical book from the point of view of a programmer.

The book is targeted at a wide range of backgrounds, giving the chance the understand the process of getting data from to presenting it usefully, in a way that can be understood and interacted with. The examples in this book are simple enough to be understood for people with a minimal technical knowledge, and for those who have a solid programming background puts some light on the process of manage a data set in order to be assimilated and interacted, and more important, be able to communicate something with it.

More than the technical details, it’s pretty interesenting the way as how data sets have to be managed:

• Each data has particular display needs, and the purpose for which you are using the data set has just as much of an effect on those need as the data itself.
• Lots of data is out there, but it’s not being used to its greatest potential because it’s not being visualized as well as it could be.
• The most important part of understing data is identifying the question that you want to answer.

And so on… Just the ideal way to work when managing data in a creative way, far from just using the existing software to generate typicall pie charts ;)

OpenCV comes with over 500 functions that cover many areas in vision, and its goal is to provide a simple-to-use computer vision infraestructure to build fairly sophisticated vision application quickly. The library is written in C and C++ and runs under Linux, Windows and MAC OS X. There is active development on interfaces for Phyton, Ruby, Matlab, and other languages.

How to find faces

Finding faces means finding complex objects, so OpenCV uses a statistical model (often called classifier), which is trained to find the object we are looking for. The training consists in a set of images, divided into “positive” samples and “negative” samples. The positive samples are instances of the object class of interest and the “negative”, images that don’t contain the object of interest.

From training the statistical model with the images explained above, a set of features are extracted and distinctive features that can be used to classify the object are selected. At the end you have a detector with a set of features that give that chance to find the desired object. The detector used by openCV is called Haar Cascade Classifier and uses simple rectangular features, called Haar features. The word “cascade” in the classifier means that the resultant classifier consists of several simpler classifiers that are applied subsequently to a region of interest until at some stage the candidate is rejected or all stages are passed. These classifiers use Haar features as these:

The feature used in a particular classifiers is specified by its shape, position within the region of interest and the scale and for a basic understanding the idea is that these features encode the existence of oriented constrasts between regions in the image so a set of these features can be used to encode the constrast exhibited by a human face and their spacial relationships.

Basically, the process of face detection slides a “search window” through the image, checking whether an image region can be considered as a “face object” or not. The detector asumes a fixed scale for the object, but since face in an image can be different from the asumed scale, the “search window” goes trough the image several times, searching for the object across a ranges of sizes.

The last openCV download offers a completes set of classifiers as XML files, which include, among others, classifiers for frontal faces, profile faces, eyes, mouth, nose, upper body, lower body, etc…

If you have downloaded OpenCV 2.0, you will find a nice example to start playing with, it’s located at [installation_directory/OpenCV2.0/samples/c/facedetect.exe, and the C source code at [installation_directory/OpenCV2.0/samples/c/facedetect.c.

For the beginners, the code is not enough simple as it should be, so before digging into the code, I recommend start looking at the face detection page in the openCV Wiki. Anyway, it's worth to add some comments to the original example, so let's have a look at the function detectAndDraw of facedetect.c, where all the stuff regarding face detections happens:

void detectAndDraw( Mat& img, CascadeClassifier& cascade, CascadeClassifier& nestedCascade, double scale)
{
    int i = 0;
    double t = 0;
    vector<Rect> faces;
    const static Scalar colors[] =  { CV_RGB(0,0,255),CV_RGB(0,128,255),CV_RGB(0,255,255),CV_RGB(0,255,0), CV_RGB(255,128,0),CV_RGB(255,255,0),CV_RGB(255,0,0),CV_RGB(255,0,255)} ;
    Mat gray, smallImg( cvRound (img.rows/scale), cvRound(img.cols/scale), CV_8UC1 );

The classifier works on on grey scale images, so the incoming BGR image img is converted to greyscale and then optionally resized.

    cvtColor( img, gray, CV_BGR2GRAY );
    resize( gray, smallImg, smallImg.size(), 0, 0, INTER_LINEAR );

    equalizeHist( smallImg, smallImg );

Now the histogram is equalized, that means to spread out the intensity values (the brightness) of the image histogram. This image will help to get the idea:

Now it's time to detect the faces in the input image, so the function detectMultiscale does exactly that. This function returns the detected objects as a list of rectangles. In this case, the vector faces will save the returned data. This function also receives some params to configure the detection:

• scaleFactor: how much the image size is reduced at each image scale
• minNeighbors: how many neighbors should each candidate rectangle have to retain it
• the flag CV_HAAR_SCALE_IMAGE, which tells the algorithm to scale the image rather than the detector
• minSize(the minmum possible object size, object smaller are ignored).

t = (double)cvGetTickCount();
    cascade.detectMultiScale( smallImg, faces,
        1.1, 2, 0
        //|CV_HAAR_FIND_BIGGEST_OBJECT
        //|CV_HAAR_DO_ROUGH_SEARCH
        |CV_HAAR_SCALE_IMAGE
        ,
        Size(30, 30) );
    t = (double)cvGetTickCount() - t;
    printf( "detection time = %g ms\n", t/((double)cvGetTickFrequency()*1000.) );

From here on, everythings is pretty straight forward, we loop through each face and drawing a circle in each face location.

    for( vector<Rect>::const_iterator r = faces.begin(); r != faces.end(); r++, i++ )
    {
        Mat smallImgROI;
        vector<Rect> nestedObjects;
        Point center;
        Scalar color = colors[i%8];
        int radius;
        center.x = cvRound((r->x + r->width*0.5)*scale);
        center.y = cvRound((r->y + r->height*0.5)*scale);
        radius = cvRound((r->width + r->height)*0.25*scale);
        circle( img, center, radius, color, 3, 8, 0 );

Here we check if the we have loaded a second classifier. In our case, facedetect.c defines a second classifiers, which has been trained to detect eyeglasses, so what we are doing here is to repeat the process explained above, but instead of looking for faces in the input image, we are looking for eyeglasses only in the regions of the images that are considered as faces.

        if( nestedCascade.empty() )
            continue;
        smallImgROI = smallImg(*r);
        nestedCascade.detectMultiScale( smallImgROI, nestedObjects,
            1.1, 2, 0
            //|CV_HAAR_FIND_BIGGEST_OBJECT
            //|CV_HAAR_DO_ROUGH_SEARCH
            //|CV_HAAR_DO_CANNY_PRUNING
            |CV_HAAR_SCALE_IMAGE
            ,
            Size(30, 30) );
        for( vector<Rect>::const_iterator nr = nestedObjects.begin(); nr != nestedObjects.end(); nr++ )
        {
            center.x = cvRound((r->x + nr->x + nr->width*0.5)*scale);
            center.y = cvRound((r->y + nr->y + nr->height*0.5)*scale);
            radius = cvRound((nr->width + nr->height)*0.25*scale);
            circle( img, center, radius, color, 3, 8, 0 );
        }
    }
    cv::imshow( "result", img );
}