the written word remains

It's been more than a week since I got back from Kate and KDevelop Hacksprint we had in Berlin, but I feel it's never late to write a blogpost about it.

The hacksprint was full of hacking, drinking awesome German beers, socializing, sightseeing and many more. I have to say I enjoyed every minute of it. Many thanks for Milian for organizing this event!

This time I set up a bold goal for myself. I had to make KDevelop 4 good for my everyday work and finally stop using KDevelop 3. For that to happen I had to do three simple things:
1) don't let C++ parser to take all my memory when opening Ruby interpreter sources
2) have proper X session management support to correctly reopen all my running KDevelop sessions at X startup
3) implement Ruby language support (at least) with the same feature set as in KDevelop 3

Thanks to a good beer and to a good company, I did everything I planned!

Ruby Support

Ruby support features we have so far in KDevelop 4 are:

• Ruby application template
• Ability to open existing projects with Generic Project Manager


• Quick Open Files/Classes/Functions and a Class View
• Outline and Previous/Next Function navigation


• Rails-specific switching between controllers/models/views/tests


• Run Current File feature with automatically created launch configuration
• Run Current Test Function (works only for Test::Unit framework), again with automatically created launch configuration

I've imported Gitorious project (yeah, it's Ruby on Rails!) to illustrate my words.

Quick Open Function:



Rails-specific "Switch to View" navigation using regular Quick Open widget:



Run Current File:



See more screenshots at the Ruby support wiki page.

To try Ruby support yourself, you'll need to compile the plugin from playground yourself. I have no plans to release it yet, but if you'd like to test it or use it for everyday work, please do. I'm writing Ruby code with KDevelop4 every day now, so the Ruby support is stable and I intend to keep it that way in the future.

This is only the beginning. Ahead is the long way to a best possible Ruby support in KDevelop 4. Stay tuned!

Two important things made me busy lately. One is the preparation of the annual KDevelop Team Meeting which I'll write about later and another one is preparing for my RailsConf 2009 presentation called "Advanced Performance Optimization of Rails Applications".

I've been working professionally with Ruby and Rails for 3 years and always found it funny when my applications turned out to be slow. And when I say slow, I don't mean anything like I complained about in my previous posts on KDE performance.

The performance of normal Rails application is measured in seconds :) The performance of slow Rails application - in dozens of seconds. You simply can't write a KDE/C++ application that would be that slow.

But that doesn't mean your Rails application is doomed to be slow. Contrary to that, it can be snappy and feel more like a desktop application rather than like a slow web site. But you do need to know how to do that.

So, after all those years of fighting for performance I decided to tell the world on how to make Rails applications fast. If you're interested in the topic, please see my presentation abstract and other blog post for details.

Someday I believe we'll be optimizing Ruby applications in the KDE world as well ;) QtRuby and Korundum are the great frameworks. They have C++ code behind them so performance will never be as much of a problem as it is with Rails. And they make developers extremely productive. Once just out of curiosity I rewrote my Sublime UI library from KDevelop in Ruby and spent 3 days for that. C++ version took me 3 months (excluding the time I didn't work on it) and it's not even finished yet!

KDE as a project was always keen on using the best tools for the job. I hope one day we'll seriously pick up dynamic languages. I dare to imagine how that will boost the already robust and speedy KDE development ;)

I've recently became quite confident that choosing programming language is a process that is very similar to choosing what to cook. First you taste something. If you like it, you eat it. If you really really like it you look for a recipe and try to cook it. Same for the programming language. First you see some code in that language. If you like it, you read more about the language, learn its concepts and ideas. If you really really like what you found, you start programming in that language.

What's interesting is that your consciousness is not usually involved in the tasting phase. On the surface it's simple - you either like new thing or not. But I do believe that there's a serious subconscious process involved here that takes into account all your previous experience and tells you "yes" or "no" to this new thing.

I've made several failed attempts to ignore this subconscious decision and force myself into learning things I don't like. Ironically, such first attempt was Lisp and the last one was Haskell. Needless to say that Lisp still means to me nothing more than lots of isolated, sil^^^... (you know what ;)) and Haskell is still for me nothing more than a pure product of pure mind. Once again, no offense to these languages and people using them. That's just my mind that doesn't allow me to like these great languages ;)

Contrary to that, I made quite a success learning languages I liked. The best examples would be Prolog and Ruby. And yesterday I discovered for me another language I liked from the first look - Scala. So instead of working hard on my projects and my PhD, I spent last two days reading about Scala.

And I should say the language is very cool. It implements an interesting mixture of functional and object-oriented programming paradigms but keeps static typing and very clean syntax. For those familiar with Ruby, I'd say Scala often feels like Ruby that is statically typed, has C-ish syntax and implements more functional programming goodies. I think next time I'll come up with a side-by-side Ruby vs Scala comparison.

And finally a little code snippet for you to taste ;) (from excellent Martin Odersky's "Scala by Example" book)

abstract class Tree
case class Sum(l: Tree, r: Tree) extends Tree
case class Var(n: String) extends Tree
case class Const(v: int) extends Tree

type Environment = String => int

object MyProg {
    //uses pattern matching to evaluate the expression
    def eval(t: Tree, env: Environment): int = t match {
        case Sum(l, r) => eval(l, env) + eval(r, env)
        case Var(n)    => env(n)
        case Const(v)  => v
    }

    def main {
        val env: Environment = { case "x" => 5; case "y" => 10 }

        //(x+x) + (7+y)
        var expr: Tree = Sum (Sum(Var("x"), Var("x")), Sum(Const(7), Var("y")))
        //(5+5) + (7+10)
        println(eval(expr, env))    //prints: 27
    }
}

MyProg.main

RailsConf this year was huge. About 1600 people were there who paid incredibly high amount of money for the entry. And (like last year) it was again a show with Chad Fowler playing balalaika (or what was that instrument), Ze Frank amusing auditorium with his keynote and of course Dave Thomas again talking nonsense ;)

What is really noticeable is that Rails community became bigger and more pragmatic. We've got only two talks of "is Rails ready for the Enterprise" kind and both of them were not really about the subject. That's surely a good sign, but that's also a sign that the whole Ruby/Rails grew and will start loosing this aura of cool hacker's toolchain.

Not that I care about that much because I still think Ruby and Rails are powerful technologies that allow us to enjoy what we're doing but... this year is about time to start looking at others. People following Reddit might already have the impression Haskell is going to be the new hacker's toy. And maybe it will, who knows... It looks like in the next year we'll see some articles about "is Haskell ready for the Enterprise?" :) Heh... I'd better start fighting with my distaste for functional programming ;)

I'm sitting here at one of the boring talks at RailsConf and have nothing to do, so I decided to blog a bit :)

Basically Ruby/Rails people are great Mac fans (this Flickr picture). Some of them use Windows. But what's really interesting that there's a number of people who're using Linux here and all those people I've seen have KDE on their desktops. I don't know what is the most popular distro among them, but two Kubuntu addicts are sitting near me at this moment. And that's cool, isn't that :)

I thought I'd write some words about RailsConf Europe here. I'll do that now but there's one more important thing I want to tell people here apart from just giving an overview of the conference.

Speaking about RailsConf. There were a lot of interesting technical talks there covering present and future trends in Rails development. I especially liked the new movement towards REST and new developments on ActiveResource RESTful ruby client library which would make interaction between web resource (service) and clients a lot more straightforward than in usual SOAP/XMLRPC case. I even wrote a simple C++ library to do the same thing ActiveResource does. Also interesting was Thomas Fuchs' presentation about javascript testing framework (which supports Konqueror among other browsers btw). There were more cool talks but this blog article is not really about them.

We had two "political" talks at RailsConf both telling us that Rails is cool, Rails is great, Rails is fast, whatsoever. We were also told that any information about Rails' weakness, slowness is just plain FUD. For people not involved in Rails community I'll give several links.

Here and here Joel Spolsky says that Ruby and Rails are slow. This and this are the replies from David Hanssen.

You say, ok, this is just another "language war".

Yesterday I discovered another battlefield. Steve Yegge tries to say that Agile software development can be bad and Agile is basically egomania and religion.

You say, ok, this is not a language but rather methodology war.

But there's a common pattern between those political RailsConf talks, language and methodology wars. In those cases people are presenting things as the fight between Good vs Evil, Right Thing vs Wrong Thing, etc. And when people in both camps do that we have this ridiculous situation when thing "A" can be "Good" and "Evil" at the same time! And more importantly, if we listen to them we naturally hear that things can only be White or Black, there's nothing in between.

So Agile methodology can be "the only true way" if we listen to agilists or "egomania" if we listen to their counterpart. Now I want to ask you, people, is it really so? Is thing "A" doomed to be either White or Black? I really doubt that. "A" can work (or be good) for one group of people, can be bad for another and (sic!) there's a third group of people that don't care of "A" at all.

So there are always situations when you either like something or dislike or don't care. And that doesn't automatically make "something" Good or Evil. Simply put, there're people who want you to think of White vs Black, Good vs Evil and they usually have good reasons to do so (like to get you or your money). Most importantly their reasons and goals may not coincide with yours. Therefore I developed a habit of recognizing this pattern and analyzing all kind of information that comes in that "White vs Black" shell to assure I have my own opinion and make my own decisions and achieve my own results.

Certainly not all people have "Black or White" vision. When it comes for software development methodologies I always remember Barry Boehm and Richard Turner's "Balancing Agility and Discipline" book which is the good example of how can we extract the best thing from all methodologies by avoiding polar "White vs Black".

Same applies for programming languages. Instead of fighting the wars we'd better benefit from all the languages. And here I must admit I'm really sad to see the proposal to choose the only one scrtipting language for KDE. Do you recognize the pattern of "White vs Black" here? I do...

That's what I heard from manyoso lately and he was damn right. This indeed happens and I see I'm not the one. Well, my day this week consisted of 16 hours and if I substract two hours I spend to have some food and read my mail I'll get same 14 working hours as geiseri had.

But I'm having that since February and I can't see the way to avoid this in the next 6 months too so I should live with that. But there's one Cicero's statement that consoles myself. He said once that if we avoid the weight of cares then we miss the prowess. And indeed what would I have done and what kind of man would I be without my cares ;)

Speaking about cares, I had plenty of them recently. Just to mention the first Ruby/Rails meetup I and my fellow Anton organized here in Mykolayiv. The cool thing is that event was financed by the company I work for now and we managed to rent the good conference hall and gather the most interesting people there (we have the meetup site in russian and some pictures).

Anton did extremely well at organizing so my job was only to enjoy and educate people who came to this event. Hopefuly I did it not so bad ;). I was speaking on Ruby and Rails, showed live demos writing some real Ruby and Rails code and I was overwhelmed with questions during the whole event. I thought 3 hours would be enough and I was surprised to be proven wrong :) Four hours the people tortured me asking more and more (especially about Ruby language itself) and then after the event was finished they were continuing torturing me in the bar nearby till late evening :)

As far as I can see, people who work in software development industry in my town are really interested in something new and innovative. This is nothing surprising.

What's more surprising that engineers who worked in the industry at least for two years are more interested in technologies than students in my university. I've been working at the university for four years and I can see this tendency growing in last two years. Most CS students pick one thing they think will bring them money and get stuck (no it's not Java, here in Ukraine we all like Delphi ;)). On their second year they already know (or think they already know) how to program in Delphi/ObjectPascal/whatever and stop listening to their teachers. Of course they miss algorithms, parsing theory, OO programming modelling and lots, lots of really necessary things including two Unix courses we have in our university. Of course they don't hear me telling them about the beauty of Unix, Ruby, whatever else.

And I now kinda enjoy when I see same people to have interest in all those things they missed at the university after spending a couple of years doing real life work. Why does that happen? Only recently I found out an answer. The very simple answer. Money is the answer.

Typical ukrainian student grows in the environment where he constantly lacks money. And after they are able to develop another cool accounting application in Delphi they find out they already can earn quite a lot of money. In fact, that amount can be already bigger their teachers get from the university. This convinces them the university teachers are worthless and they don't believe in their authority anymore. Heh.

Of course thereafter they grow, they start think more about their profession, maybe read blogs like this one (I was shown this several days ago). And their interest level actually starts growing and they start growing themselves at the same time. I wish our students looked one step further or just believed their teachers.

Hmm, why am I writing all that? Maybe I'm just ranting. But maybe that's just my own discourse on duties. Do not neglect your duties, even if they sound wrong if you're student and think the world domination is one step ahead. Just believe me, the real life happens and you will not have enough time to educate yourself after the university. I know it. Don't repeat my mistakes. I know how hard is now to extract just two hours from your daily schedule to learn new language or to read new book.

I hope people are not yet bored with my Ruby blog posts ;) It's just so cool to get into the new environment that makes you care about things you've never cared before. Ruby was definitely something I knew about but never enjoyed as much as I do now. Same applies to my new Ruby/Rails job. I don't think I would read a vendor branches chapter from the svn book without it :) Of course, the chapter is only the example, there were (and are) more interesting things ;)

I started to write long posts not so long time ago, so let me keep this tradition and mention my search for elegant LALR grammar for lists of nested lists.

Let's say we need to parse token stream:

BBABBBABB

as the list of two items with one nested list inside. The ideal result would look like

(B (BAB) B) (B (BAB) B).

The grammar that comes to mind is:

list: item
    | item list
    ;

item: B list B
    | A
    ;

This grammar is ambiguous and has one shift-reduce conflict (it's not clear whether we shall reduce finishing the list or shift starting it). All LALR parser generators would resolve the conflict to shift over reduce and would in fact make our parser fail. So we need to say "prefer reduce". I have several solutions but the cleanest (from the parser's viewpoint) is to add an artificial terminal, say X, denoting the beginning of the list. The grammar

list: item
    | item list
    ;

item: X B list B
    | A
    ;

has no conflicts at all. This solution is good for parser but hacky for lexer which has to supply this fake terminal;) Well, I'm sure there are better solutions but I'm completely missing them today.

Addon: of course, if we allow to modify the lexer, something like

list: item
    | item list
    ;

item: B_START list B_END
    | A
    ;

would be even better.

As work on my ruby-based wiki parser continues, I started to have more and more fun with it. Today I travailed, trying to get RACC to fit my needs and discovered lots of interesting things about it. If you don't know the RACC - this is YACC for ruby (but without Lex/Flex companion ;) to make you get more fun while you write hand-made lexer).

Some time ago I discovered that RACC has one trouble. When a rule has a code block executed before entering the rule, usual numbering for ''val'' array becomes broken.

For example, if we have a rule:

foo: moo boo
     { result = val[0] + val[1] }
     ;

The result of the rule is computed by calculation of a sum of right part nonterminal's return values.

But if we use the rule like this:

foo:
     { result = "I break " }
     moo boo
     { result = val[1] + val[2] }
     ;

then we suddenly break ''val'' numbering scheme. Instead of ''val[0]'' and ''val[1]'' we should reference nonterminal's return values using ''val[1]'' and ''val[2]''. Apparently the code block before nonterminals breaks. Not a grave bug... but costed me several minutes to get an idea of what is happening.

But today I glanced over racc output file and was stunned! RACC effectively considers:

 foo: {} moo boo ;

as

foo: _nonterminal_ moo boo;

and adds extra rule that breaks not only val[] numbering ;)

Example 1:
 a: {} a B
     | A
     |
     ;
Example 2:
 a: a B
     | A
     |
     ;

I implemented two examples (first is broken and second is ok) and studied them. I found that the source of the problem was that for {} a special nonterminal @1 will be created and artificial rule like

@1 -> ε

will be added. But given input token A the parser allows both reducing the rule above and shifting A. Thus we have shift/reduce conflict that RACC hapily reports.

Another issue is endless loop: if the input is B then the parser reduces using the artificial rule and keeps reducing it forever. That happens because it always observes the empty symbol(together with B) and has the rule to reduce it. Ironically, another competing rule to reduce (say hello to reduce/reduce conflict!!!)

a -> ε

stays after the first one and is not selected for reduce.

To summarize, we have two shift/reduce conflicts and one reduce/reduce conflicts just because of the block. RACC is either was not designed to have before-rule blocks or the developers had something else in their mind. Alas, I haven't found any references to this issue yet.

As probably some of you already know, I've got a new cool job recently (no, the university is still my employer, that's the job I do after the Uni). My new duties are Ruby and Rails application development :) Some things I do now are free software, some not. One of the free software things is MediaCloth - MediaWiki syntax parser and HTML generator which I started to implement last week.

I don't know if there are true mediawiki parsers out there, but this one is "true". It has handwritten lexer, LALR Yacc-generated parser, AST and HTML generator (which is just another AST walker). AFAIK the "official" parser is regexp-based.

Development of the parser was much fun for me and made me think about some apparently simple things that suddenly became more and more complex. The one of such things was the discovery that MediaWiki syntax is actually context-sensitive. Heh, that's why people like it! We people hate html and xml in general. It's complex for us but simple for our computers because it's context-free. But we do like context-sensitive languages, we apparently have good parsing mechanisms for them.

Let's look at simple list written in wikimedia syntax:

 * a
 * b
 * c

This is normal flat bullet list and its parsing does not imposes any difficulty. The context-insensitive grammar below will parse the list:

List -> Item List
    | ε

Item -> * ID

Here ID is any terminal symbol like ''a'', ''b'' or ''c'' above.

The left derivation of the list from the root nonterminal will look like:

List => Item List => *a List => *a Item List =>
=> *a *b List => *a *b *c List => *a *b *c

The example above would have persuaded you that wiki language is context-insensitive and thus simple (at least in computer's sense of simple). But we all know lists can be nested like this:

 * a
 ** 1
 ** 2
 * b

Ok. Now we have a parsing problem. The grammars that come to mind are not context-insensitive anymore. One of such possible grammars is shown below:

List -> Item List 
    | Sublist List
    | ε

Item -> * ID Level1
Level1 Sublist -> ** ID Level1
Level1 Item -> Item
Level1 List -> ε

With this grammar we can derive our list from the root nonterminal with left derivation like this one:

List => Item List => * a Level1 List => * a Level1 Sublist List =>
=> * a ** 1 Level1 List => * a ** 1 Level1 Sublist List =>
=> * a ** 1 ** 2 Level1 List => * a ** 1 ** 2 Item List =>
=> * a ** 1 ** 2 * b List => * a ** 1 ** 2 * b

If we look at the grammar closely, we would notice it's actually context sensitive. Even more closer look would reveal that the grammar is recursively enumerable because the right part of production

 Level1 Sublist -> ** ID Level1

is longer than its left part.

From the all discussion above it turns out that the mediawiki language is not only context-sensitive, but recursively enumerable. In computer world such languages are usually considered ''hard'' and ''complex''. The interesting thing is that people obviously disagree with such definition for complexity. A person does have an effective mechanism to parse context-sensitive languages and therefore loves wiki formatting!

Heh, probably for the time we use Turing machines our computers will still fail at those apparently simple tasks like language parsing. You can believe in Gödel's hypothesis or not, but people still have more computation power than their computers ;)

Oh, Ruby has become a hot topic on KDE blogs again :) Let me also join with my 2 kopiykas.

My taste of programming languages tells me Python is not good for me. I don't want to claim here Python is bad, it's just not the language of my choice.

There are two good things about Ruby. First - when I've started learning both Python and Ruby year ago Ruby was perceived "cool". Today I work on Ruby (and Rails) projects half-time and the language is still "cool". There's nothing like this feeling to be commited to something cool :) Good advertising you say. Yep, pretty good but there's also something inside me telling Ruby is cool ;) Cornelius described that feeling in a more precise way.

The second good thing I found about Ruby is syntax. Surprise! Looking at the blogs and comments most people complain about perlish syntax. It's probably possible to write perlish code in Ruby but I barely see anyone does it. For example, what's wrong with Richard Dale's code?

Honestly, syntax matters were the first to take my attention away from Python. I like def initialize more than def __init__(self):. I also like code blocks ending with end and not just backward indentation. Writing self.foo disturbs me more than @foo. My list of preferences is long :)

Ok, I'm sure Python is a great language but I like Ruby more!