Felt like doing an update to my outdated website and ended up changing the whole generation of the static pages.

Why Parcel

I've been working with parcel a lot at my daily job. Initially tried v1 out as a no config bundler and stuck with it. It just works. At least most of the times. Then I saw the following:

I've been working on a documentation site powered by Parcel 2. Turns out, Parcel is a pretty awesome static site generator! 😲

🚀 Super fast!
😍 Statically rendered MDX at build time
✨ Interactive client rendered examples
🔄 Live reloading type docs extracted from the code pic.twitter.com/dhzVz15W8V

— Devon Govett (@devongovett) February 10, 2020

And this is what made me want to try such a solution. On top of that, it would also give me the advantage of familiarizing myself with it's new plugin api.

The problem I am trying to solve is to transform a bunch of md files to html pages in a way that I can reuse the javascript that does the rendering on the server side. So, what I am looking for is an isomorphic static site generator. In order to do that, I will be using react to render the pages.

Entry point

Initially I just created a transformer for my md files and something like parcel articles/**/*.md was enough to create the http files. On top of that the generated http file would go through the parcel pipeline and get optimized. Additionally any script referenced in it would also get bundled/hashed correctly and common scripts across files would get properly shared. Suprisingly what seems to be complex enough by describing it, was fairly straight forward with parcel. All I had to do was to create a custom transformer that could accept md files and spit html out of it.

After I was done with that, I stumbled upon the actual problem. How to create an index file containing links to the generated html pages? In general I needed a way to create new pages making use of metadata of the pages that have been rendered. Went through some github issues and ended up with the following alternatives.

Custom parcel reporter

That can listen to pretty much any parcel event and at the very end it can create the index files that I am interested in. Yet, since plugins should be side effect free, I felt like that was not the right solution. Additionally I had doubts of the proper bundling of the exported file. For example I create an index.html referencing index.ts. Would that index.ts get properly transformed by parcel?

Custom index.md file

Another solution would be to have a custom (most probably empty) index.md file and instruct parcel to treat it differently than the rest of the md files. That is what react-spectrum's doc site is doing for example. Yet, I had to also add more files to the final bundle like robots.txt and sitemap.txt and would prefer a solution where parcel would also know about them. And that made me think of the following.

robots.txt as entry

sitemap.txt that contains a list of all web pages of the specific domain. Basically in my case, all of the generated pages. Therefor all I need is a transformer for it to get a glob pattern as an input (in my case a **/*.md), resolve the files that are described by that, add them as a dependency to the build and adapt the content of sitemap.txt to contain the resolved files.

Since the files have been added as a dependency, parcel will then pick them up and transform them. So I would like to have a sitemap that looks like the following:

src/**/*.md

and for each .md file get an output like

http://valotas.com/page1/
http://valotas.com/page2/
http://valotas.com/

with src/page1.md and src/page2.md added to the bundle graph.

This leaves me though with another file that I need published but should not be included to sitemap.txt. That is robots.txt. Of course since I am anyway have to create a transformer for sitemap.txt, I can make it able to skip files referenced there. Yet, that did not feel good especially when robots.txt should reference sitemap. So, ended up using it as the entry point. So, I had the following:

Sitemap: ./sitemap.txt

that should be transformed to

Sitemap: https://valotas.com/sitemap.txt

adding sitemap.txt to the bundle graph, and from there on, continue with the transformation of sitemap.

Implementation

The implementation finally needed a transformer for robots.txt, one for sitemap.txt, an md transformer and a transformer that could get the rendered body of a the last one and wrap it within a page skeleton.

That last thing, needed also a resolver in order to let parcel know of files referenced to the html template used for wrapping the body.

Since I needed the whole thing to be isomorphic, I had to also create some metadata of each page that I was rendering available as a resource so that I could fetch it and render it without having the browser to do a full page request. That means that the sitemap transformer was also adding a json file to the dependency. So for each name.md, I had /name/index.html and name/meta.json added to the bundle graph.

Finally since I also wanted to make a folder for each given md file, a namer also needed. Was kinda afraid that this thing was not going to end, but eventually didn't need anything else.

With a .parcelrc that looked like:

{
  "extends": "@parcel/config-default",
  "transformers": {
    "*.txt": ["@valotas/parcel-transformer-staticsite/txt"],
    "*.md": ["@valotas/parcel-transformer-staticsite/md"],
    "*.meta": ["@valotas/parcel-transformer-staticsite/meta"],
    "*.htmlbody": ["@valotas/parcel-transformer-staticsite/htmlbody"]
  },
  "resolvers": ["...", "@valotas/parcel-resolver-staticsite"],
  "namers": ["@valotas/parcel-namer-staticsite", "..."]
}

was able to have a static site under /dist ready to be served.

AWS uploading

The only thing left was to upload the created assets to aws. That was the case where a reporter seemed the right thing to do.

I was hoping to come up with something abstract enough but did not make it. Yet, the solution if it can inspire someone is available here.

Lately I came across a problem for which the standard Promise's api wasn't enough. I needed something like Promise.all but I had to make sure that the actions array will get executed in sequence

Prerequisites

There is no reason reading more if you can't answer the question of this blog post

Promise.inSequence

Ideally, I need a utility method that get as an input an array of actions. It executes the given actions one after the other and every execution should wait for the previous one:

inSequence([action1, action2, action3])
  .then(([result1, result2, result3]) => {
    // action1() should have been called first
    // action2() should have been called only after action1 finished
    // action3() should have been called only after action2 finished
  });

The solution was very simple googling, but I am not fan of copy & paste development if I do not understand what am I copying, so as an exercise I thought it would be a good idea to implement a solution myself. In order to attack the problem, I'll try to solve it with just 2 promises in the dumbest possible way:

var result = action1()
  .then(function() {
    return action2();
  })

Now, the question is, how can I make that work with more than 2 actions. In order to do that, let's start with the last action, wait for it and once done execute the next one. Update the last reference with the combination of both promises:

var last = Promise.resolve();
actions.forEach(function (next) {
  last = last.then(function () {
    return next();
  });
});
return last;

This is the most common way of executing actions in a sequence with promises, Let's create a function for it:

function simpleInSequence(actions) {
  var last = Promise.resolve();
  actions.forEach(function (next) {
    last = last.then(function () {
      return next();
    });
  });
  return last;
}

Now the question is, how am I going to keep track of the results too so that in the end I'll get a Promise which's result will be something like [result1, result2, ...]. Since our actions are now executed in sequence, we can just keep track of the results in an array:

function inSequence(actions) {
  var last = Promise.resolve();
  var results = [];
  actions.forEach(function (next) {
    last = last.then(function () {
      return next();
    })
    .then(function (result) {
      results.push(result);
    });
  });
  return last.then(function () {
    return results;
  });
}

Now we can slightly improve this by initializing a promise with an array and make sure that last will always resolve to an array of the results:

function inSequence(actions) {
  var last = Promise.resolve([]);
  actions.forEach(function (next) {
    last = last.then(function (results) {
      return next()
        .then(function (result) {
          results.push(result);
          return results;
        });
    });
  });
}

or even make use of Promise.all to avoid nested .then calls:

function inSequence(actions) {
  var last = Promise.resolve([]);
  actions.forEach(function (next) {
    last = last.then(function (results) {
      return Promise.all([next(), results]);
    })
    .then(function (combination) {
      var results = combination[1];
      var result = combination[0]
      results.push(result);
      return results;
    });
  });
}

Working with arrays

Now that we have a solution we can improve a little bit. When we have an array and we want to create a single value out of it, we can use the reduce function of the array. We would like to reduce our array of actions into one promise of arrays:

function reduceInSequence(actions) {
  return actions.reduce(function (last, action) {
    return last.then(function (results) {
      return action()
        .then(function (result) {
            results.push(result);
            return results;
        });
    });
  }, Promise.resolve([]));
}

Thinking more complex

When I first tried to implement a solution I had in mind that I needed an implementation that could give me back a Promise with an array of the results. I ended up using recursion for implementing it:

function recursiveInSequence(actions, results) {
  results = results || [];
  var action = actions[0];
  return action()
    .then(function (result) {
      results.push(result);
      if (actions.length === 1) {
          return results;
      } else {
          return recursiveInSequence(actions.slice(1), results);
      }
    });
}

while the solution works just fine, it looks a little bit more complex, but I still have it written here as a reminder to always split problems as much as possible.

How to test it

Really awesome work but without being able to test what we did, we are not falling in to the software engineers category. To be honest, testing Promises is another challenge, but let's give it a try. So what I want to test is that two given actions has been called in sequence. Let's create a factory of actions that can log their result before returning a resolved promise:

function action(delay, seq) {
  return function () {
    var result = "delay-" + delay;
    return new Promise(function (resolve) {
      seq.push(result);
      resolve(result);
    }, delay);
  }
}

Now all I have to do is check the seq array:

it("should work with an array of more than 2 actions", function (done) {
  var seq = [];
  inseq([
    action(50, seq),
    action(10, seq),
    action(11, seq),
    action(5, seq)
  ]).then(function () {
    expect(seq).toEqual([
      "delay-50", // first action resolves after 50ms
      "delay-10", // second action resolves after 10ms
      "delay-11",
      "delay-5",
    ]);
    done();
  });
});

TLDR

Since code speaks itself, you can play with it or just have a look at it:

function forEachinSequence(actions) {
  var last = Promise.resolve();
  actions.forEach(function (next) {
    last = last.then(function () {
      return next();
    });
  });
  return last;
}

function inSequence(actions) {
  var last = Promise.resolve();
  var results = [];
  actions.forEach(function (next) {
    last = last.then(function () {
      return next();
    })
    .then(function (result) {
      results.push(result);
    });
  });
  return last.then(function () {
    return results;
  });
}

function recursiveInSequence(actions, results) {
  results = results || [];
  var action = actions[0];
  return action()
    .then(function (result) {
      results.push(result);
      if (actions.length === 1) {
          return results;
      } else {
          return recursiveInSequence(actions.slice(1), results);
      }
    });
}

function reduceInSequence(actions) {
  return actions.reduce(function (last, action) {
    return last.then(function (results) {
      return action()
        .then(function (result) {
            results.push(result);
            return results;
        });
    });
  }, Promise.resolve([]));
}



describe('Promise.inSequence', function () {
  function action(delay, seq) {
    return function () {
      var result = "delay-" + delay;    
      return new Promise(function (resolve) {
        if (seq) {
          seq.push(result);
        }
        resolve(result);
      }, delay);
    }
  }
  
  function test(inseq, name, withResults) {
    describe(name, function () {
      it("should work with an array of 2 actions", function (done) {
        var seq = [];
        inseq([action(50, seq), action(10, seq)])
          .then(function () {
            expect(seq).toEqual(["delay-50", "delay-10"]);
            done();
          });
      });
      
      it("should work with an array of more than 2 actions", function (done) {
        var seq = [];
        inseq([
          action(50, seq), 
          action(10, seq),
          action(11, seq),
          action(5, seq)])
          .then(function () {
            expect(seq).toEqual([
              "delay-50", 
              "delay-10",
              "delay-11",
              "delay-5",
            ]);
            done();
          });
      });
      
      if (withResults) {
        it("should resolve to an array with all the results", function (done) {
          inseq([
            action(1), 
            action(2),
            action(3)
          ]).then(function (results) {
            expect(results).toEqual([
              "delay-1", 
              "delay-2",
              "delay-3"
            ]);
            done();
          });
        });
      }
    });
  }
  
  test(forEachinSequence, "forEachinSequence (ignoring results)");
  test(inSequence, "inSequence", true);
  test(recursiveInSequence, "recursiveInSequence", true);
  test(reduceInSequence, "reduceInSequence", true);
});

References

• Promise
• http://stackoverflow.com/questions/24586110/resolve-promises-one-after-another-i-e-in-sequence

This website is for me a continuous effort of skills improvement. It started with a ready static website generator to end up to something completelly custom made. One thing I did not touch though was the css stuff.

The CSS journey

When I first started this blog, I just used the blogger platform. After some years of inactivity I felt like moving forward using bootstrap that a friend of mine adapted for me.

As a "proper" developer I did not really touched css. Going a step further I moved to a rewrite of the backend. Still the only affair I had with css was to optimise the bootstrap imports in order not to have to curry over all the thing that bootstrap provides but I never used.

After all the javascript related changes, I felt more obligated to work a little bit closer with css. Just like everytime I feel like improving somewhere, I set a simple goal. That was to get rid of bootstrap. Ofcourse I didn't feel like creating something from the beginning, I ended up just trying to integrate ZURB Foundation for sites.

All in all, this site is now using a bootstrapized version of foundation framework, dropping the css file size from 61kb to 42kb.

How is it done?

Well, having an online version of the website, I just tried to make use of the foundation basics to make it look the same. So a new pipeline has been added to compile sass. The old less pipeline has been deleted and made use of the newly empty now main.css.

After the funny and socking time of working with a fully functional unstyled single page application, I started styling parts of the already structured site. Just by styling the header and the footer, I had something that I could see without getting scared. After that, all I had to do was to adapt the grid breakpoints to the ones bootstrap uses. On top of that I managed to also bring some stuff visually closer to what I liked more, without fighting with a frameworks defaults.

Most of the time the adaptation process for each block to be styled was like the following:

• Simplify html
• Import needed foundation module
• Provide a sass module with the rules needed for corresponding block

Needles to say, I ended up using way less stuff of the framework. More specific, out of 40 sass modules I only used 5.

What I learned

Although I did not become an expert on anything and did not apply every single bit of what I read, I feel I improved my knowledge in terms of grid layout, measurement units, media queries and a little bit of atomic design systems.

It is really fantastic the amount of stuff that you can learn just by trying to understand some how something works with in a framework. For example you hear responsive and breakpoints. You have the classes you need to apply in order to be responsive but how does this work, will make use understand media queries and layouts (in general, or a grid in particular).

To bootstrap or not

There are a lot of articles out there prasing or demystifying bootstrap. The truth is you have something good looking with minimum effort. You pay a price though. That is, minimum knowledge input, small flexibility of customization and a website that looks like... bootstrap.

Sooner or later you'll want something different. You can of course try to customise bootstrap. My main problem though was that I had to write html in a way that I can apply bootstrap. And this is how you end up having an html that is anything but semantic. I would love to provide a css with the very basics, but I did not want to rewrite a framework nor I feld that I could swim good enough in to the css world so that I could finish within a couple of hours. The intermediate solution was to use a customisable framework. I have to note though that bootstrap is customizable enough, but this time I wanted to learn something new and still I did not know if I could get rid of the html part of the bootstrap.

To be honest, I do not know if I would ever propose bootstrap as a framework for the frontend part of a website. Have in mind that twitter itself does not use it for their public website. Of course if within a team there is no person with serious css experience, I wouldn't hesitate. I also believe that in any project with an absence of a styleguide, a framework (no matter how bad it is) should be used. In that case though just make sure that you do not fall in to the "Not invented here syndrom" category.

TL;RD

Back to where I started. If you would like to improve your css skills, you have to write css. It is ok to use a framework or to start with it, but you must be willing to go above it, customise it, adapt some things and finally replace parts of it with custom made css.

Rule of thumb: replace framework stuff that can be implemented with a simple straight forward way.

When you hear for a new technology, all you want to do is just to use it. This is how my experience with RxJS started a while ago. It is for sure cool, but also a new way of thinking. Just like anything else new, you are going sooner or later to hit a wall. After grasping myself to find which sequence of operators would do the trick, I tried to just test the thing.

Async testing

Well, no matter how close should streams be to normal iterables, they are in nature async. You have a stream and at a moment you subscribe to it. This "one moment" plays a crucial role to your testing code. Most of the times you would like to test the behaviour right before or/and right after a specific event. Being careful, will make you pass the first test successfully, but as soon as you might need a little bit more, you will pretty soon find yourself struggling with the setup of you test just to have something like the following at the very end of it.

someStream.subscribe((val) => {
    expect(val).toBeDefined();
    //any other expectation
});

Marble tests

If you have that problem, let me tell you a well hidden secret: marble tests. After finding that out, you are going to have yet another problem and that is the lack of documentation on how to use them. That means more baby steps, more source code reading(thanks god, the library is written in typescript) and more experimentation. Given that, I thought it would be better to also write down experiments.

After some reading and searching, I found out that it is all about the TestScheduler class that is provided with the framework. Still not that clear on how to use it.

Stream creation

While in the docu, you have the hot and cold functions, there is no info on where to find them. It looks like they are actually part of the TestScheduler's api:

• createHotObservable<T>(marbles: string, values?: any, error?: any): Subject<T> creates a hot observable
• createColdObservable<T>(marbles: string, values?: any, error?: any): Observable<T> creates a cold observable

My first attempt was to create a simple hot or cold observable and play with it:

/*eslint-env jasmine*/

import Rx from 'rxjs';

fdescribe('TestScheduler', () => {
  
  describe('createHotObservable()', () => {
    it('should create a hot observable emmiting the values given as marble strings', () => {
      //given
      const scheduler = new Rx.TestScheduler(null);
      const source = scheduler.createHotObservable('--a---b-cd-|');
      const expected = ['ab', 'cd'];
  
      //when
      source
        .bufferCount(2)
        .map((array) => array.join(''))
        .subscribe((val) => {
          expect(val).toEqual(expected.shift());
        });
        
      scheduler.flush();
      expect(expected.length).toBe(0);
    });
  });
  
  describe('createColdObservable()', () => {
    it('should create a cold observable emmiting the values given as marble strings', () => {
      //given
      const scheduler = new Rx.TestScheduler(null);
      const source = scheduler.createColdObservable('--a---b|');
      const expected = ['a', 'b'];
  
      //when
      source
        .subscribe((val) => {
          expect(val).toEqual(expected.shift());
        });
        
      scheduler.flush();
      expect(expected.length).toBe(0);
    });
  });
});

Being happy about the first achievement, let's try to get one level up and subscribe to an observable after a period of time. Speaking in marbles, let's say that I subscribe to the following sequence: --a-^-b-c-|. That means that subscription should be notified only for b and c:

it('should schedule the subscription on the right time', () => {
    //given
    const scheduler = new Rx.TestScheduler(null);
    const source = scheduler.createHotObservable('--a-^-b-c|');
    const results = [];

    //when
    source.subscribe((val) => {
        results.push(val);
    }, null, () => {
        results.push('done');
    });

    scheduler.flush();

    //then
    expect(results).toEqual(['b', 'c', 'done']);
});

Guess what. This does not work. TestScheduler initialization alone can not cope with time. After some searching around I found out how to let the scheduler schedule stuff and that is the schedule function. This accepts a closure where your time based part should be placed. Let's see how to achieve that:

/*eslint-env jasmine*/

import Rx from 'rxjs';

describe('TestScheduler', () => {
  describe('schedule()', () => {
    it('should schedule the subscription on the right time', () => {
      //given
      const scheduler = new Rx.TestScheduler(null);
      const source = scheduler.createHotObservable('--a-^--b-c|');
      const results = [];

      //when
      scheduler.schedule(() => {
        source.subscribe((val) => {
          results.push(val);
        }, null, () => {
          results.push('done');  
        });
      });
              
      scheduler.flush();
      expect(results).toEqual(['b', 'c', 'done']);
    });
    
    it('should allow scheduling of non ending streams', () => {
      //given
      const scheduler = new Rx.TestScheduler(null);
      const source = scheduler.createHotObservable('--a-^--b-c');
      const results = [];

      //when
      scheduler.schedule(() => {
        source.subscribe((val) => {
          results.push(val);
        }, null, () => {
          results.push('done');  
        });
      });
              
      scheduler.flush();
      expect(results).toEqual(['b', 'c']);
    });
  });
});

Simplifying the expectations

Although straight forward, it is a bit of code that you have to write in order to test a sequence of events and we haven't a way of testing the actual time (frame) that an event took place. In the docu, you see the existence of expectObservable function, but still no info on where to find it. This time I went straight to TestScheduler and it looked like it had what I needed:

• expectObservable(observable: Observable<any>, unsubscriptionMarbles: string = null): ({ toBe: observableToBeFn })

With that we can write expectations with marble strings just like we create the hot/cold observables, making testing much more easier.

It is though a little bit trickier to use, as we should first instanciate our scheduler with an assertion function. This will be used in order to assert the sequence of the events when flushing the scheduler. That means that it should make our test fail. So either throw an exception or just use your testing framework:

function assertEquals (actual, expected) {
    //we will use jasmine's api for the assertion:
    expect(actual).toEqual(expected);
}

const scheduler = new Rx.TestScheduler(assertEquals);

a full example would look like the following:

/*eslint-env jasmine*/

import Rx from 'rxjs';

function assertEquals (actual, expected) {
    //we will use jasmine's api for the assertion:
    expect(actual).toEqual(expected);
}

describe('TestScheduler', () => {
  describe('expectObservable()', () => {
    it('should simplify the result check of given observables', () => {
      //given
      const scheduler = new Rx.TestScheduler(assertEquals);
      const source = scheduler.createHotObservable('--a-^--b-c');

      //then
      scheduler.expectObservable(source).toBe('---bc');
      
      scheduler.flush();
    });
  });
});

Experimenting

Finally if you feel like experimenting, which by the way is the only way of getting used to the library this jsbin should get you up and running as it contains all the test cases we've discussed on this post. If you are only interesting in the code, here it is:

describe('rxjs TestScheduler', function () {
  var rx = Rx.KitchenSink;
  
  it('should be part of the Rx namespace', function () {
    expect(rx).toBeDefined();
    expect(rx.TestScheduler).toBeDefined();
  });
  
  describe('createHotObservable()', function () {
    it('should create a hot observable emmiting the values given as marble strings', function () {
      //given
      var scheduler = new rx.TestScheduler(null);
      var source = scheduler.createHotObservable('--a---b-cd-|');
      var results = [];

      //when
      source
        .bufferCount(2)
        .map(function (array) {
          return array.join('');
        })
        .subscribe(function (val) {
          results.push(val);
        }, null, function () {
          results.push('done');  
        });
        
      scheduler.flush();
      expect(results).toEqual(['ab', 'cd', 'done']);
    });
    
    it('should create a hot observable emmiting the values given as marble strings', function () {
      //given
      var scheduler = new rx.TestScheduler(null);
      var source = scheduler.createHotObservable('--a-^--b-c|');
      var results = [];

      //when
      scheduler.schedule(function () {
        source.subscribe(function (val) {
          results.push(val);
        }, null, function () {
          results.push('done');  
        });
      });
              
      scheduler.flush();
      expect(results).toEqual(['b', 'c', 'done']);
    });
  });
  
  
  describe('createColdObservable()', function () {
    it('should create a cold observable emmiting the values given as marble strings', function () {
      //given
      var scheduler = new rx.TestScheduler(null);
      var source = scheduler.createColdObservable('--a---b|');
      var results = [];

      //when
      source.subscribe(function (val) {
        results.push(val);
      });
        
      scheduler.flush();
      expect(results).toEqual(['a', 'b']);
    });
  });
  
  
  it('should be reusable with time related operators', function () {
    //given
    var scheduler = new rx.TestScheduler(null);
    var source = scheduler.createHotObservable('a----bcd-f|');
    var results = [];
   
    //when
    source
      .bufferTime(40, null, scheduler)
      .map(function (buffer) {
        return buffer.join('');
      })
      .subscribe(function (val) {
        results.push(val);
      });
      
    scheduler.flush();
    
    //then
    expect(results).toEqual(['a', 'bcd', 'f']);
  });
  
  
  describe('schedule()', function () {
    it('should schedule the subscription on the right time', function () {
      //given
      var scheduler = new rx.TestScheduler(null);
      var source = scheduler.createHotObservable('--a-^--b-c|');
      var results = [];

      //when
      scheduler.schedule(function  () {
        source.subscribe(function (val) {
          results.push(val);
        }, null, function () {
          results.push('done');  
        });
      });
              
      scheduler.flush();
      expect(results).toEqual(['b', 'c', 'done']);
    });
    
    it('should allow scheduling of non ending streams', function () {
      //given
      var scheduler = new rx.TestScheduler(null);
      var source = scheduler.createHotObservable('--a-^--b-c');
      var results = [];

      //when
      scheduler.schedule(function () {
        source.subscribe(function (val) {
          results.push(val);
        }, null, function () {
          results.push('done');  
        });
      });
              
      scheduler.flush();
      expect(results).toEqual(['b', 'c']);
    });
  });
  
      
  function assertEquals (actual, expected) {
    expect(actual).toEqual(expected);
  }
  
  describe('expectObservable()', function () {
    it('should simplify the result check of given observables', function () {
      //given
      var scheduler = new rx.TestScheduler(assertEquals);
      var source = scheduler.createHotObservable('--a-^--b-c');

      //then
      scheduler.expectObservable(source).toBe('---b-c');
      
      scheduler.flush();
    });
  });
  
  describe('operators', function () {
    describe('fromPromise()', function () {
      it('should transform a Promise to an Observable', function (done) {
        var promise = Promise.resolve('promise value');
        rx.Observable.fromPromise(promise)
          .subscribe(function (e) {
            expect(e).toEqual('promise value');
          }, done.fail, done);
      });
    });
    
    describe('mergeMap()', function () {
      it('should transform a Promise to an Observable', function () {
        //given
        var scheduler = new rx.TestScheduler(assertEquals);
        var source = scheduler.createHotObservable('--a-^--b-c');
        var other = scheduler.createColdObservable('-d--e');

        //then
        scheduler.expectObservable(source.mergeMap(function () {
          return other;
        })).toBe('----d-de-e');
      
        scheduler.flush();
      });
    });
  });
});

References

• RxJs test api
• RxJs github repo
• Writing marble tests

A mixed combination of react exploration, creativity and assemble's inactivity, forced me to pop the hood of this blog and make some serious changes that I am happy to share with you (not that you care that much, but anyway)

Motivation

I felt inactive for quite some time now, so the idea came to update this site. Being a software engineer, I cared mostly about the things under the hood though. I do know that for a blog like this a static site generator is just fine. In fact assemble did a great job but I was stuck with Grunt while I was desperatelly wanted to move to gulp. Not that it has a big difference, but it is more fun to play with pipes rather than trying to configure plugins.

I was waiting for the new version of assemble for a couple of months (more that a year actually) as it supposed to add support for both building tools. Unfortunatelly the github repo seemed inactive the whole time so I assumed that it was just another dead javascript project. It looks like there is some activity latelly but unfortunatelly that was after I started working on the revamp of the site.

On top of the above I really wanted to explore gulp plugins. So I said, let's create a set o minimal plugins that could create my static website out of my markdown files. Ofcourse I could't stop thinking about react and typescript, so inevitably there should be something based on that.

The typescript part was easy. All I had to do is to write my gulp plugins with that. But react being a client side framework was not that straight forward to use it. Of course you could use it to create static markup but it is not it's main application. That was the reason I was forced to find a problem so that I could apply react on it. I did not have to think that much though. The new big idea in javascript is the isomorphic application. Which means that you should have the same code on the server and client. So I had pretty much what I needed.

The plan

Create a set of gulp plugins to transform the articles written in markdown to proper html files. The whole layout of that html should be a react component (making use of other react components). Once the markup is created wrap it with the appropriate html head and "serve" it. In our case, serve it, means store it for later use.

Once served, the html should bootstrap the same react component with the same input (markdown file) but intercept the links so that there will be no need for further page reload. Next page should be one ajax call away.

At the end of the day I ended up with something like the following

module.exports = function (gulp) {
  return function () {
    var plugin = require('../build/typescript/gulp');
    return gulp.src([
        'src/articles/**/*.md'
      ])
      .pipe(plugin.mdFile()) //create metadata out of the *.md files
      .pipe(plugin.toArticle()) //convert the metadata to articles
      .pipe(plugin.addIndex()) //add an index of all articles
      .pipe(plugin.addMetafiles()) //add metadata files as json
      .pipe(plugin.adaptPaths()) //make sure all files are in a folder named after the title
      .pipe(plugin.wrapHtml('src/templates/index.jade')) //wrap created html
      .pipe(gulp.dest('build')) //write the output
  };
};

which I hope is self explanatory.

Marked react

Unfortunatelly I did not feel that good with using react's dangerouslySetInnerHTML so I had the idea of making sure that markdown would directly get translated to react components. I was using marked for my markdown and luckily it allows you to provide custom renderer implementations. After some searching, I only found markdown-to-react-components. It looks like that (till now) it is the only renderer that would actually translate markdown to react components.

Due to absence of tests and the fact that the code was making use of regexp to find inline component placeholders and replace them with the actual components, forced me to try to implement my own renderer. Just like any adventure, you start with a "how hard can it be?". At the end of the day it took me more than I was expecting but managed to make this work for my cases. It is not a complete renderer though as I had to skip some abbreviations I did not use.

After that I felt I could get rid of github requests for gists stuff when showing an article and download everything on the creation of the page (having the gists as metadata).

tl;rd

All in all it was a very good excerise, with some performance benefits.

Ofcourse perfomance benefits is a little bit subjective as the initial page load costs now more but all the other requests are superfast. The question is if a site like this needs such a thing, but as I said, it was a very good excerise.

The code can be found at github.

This year's devoxx was once again an excuse to write a new blogpost.

Before 2015

I was lucky enough to attend the two previous conferences:

• An overview of devoxx 2013
• An overview of devoxx 2014

Although that can be nothing compared to what other attendees attended, it gives me something to compare.

tl;dr

Based on the talks I saw last two times during the university days, I thought it would be better to just skip them. The rest of the conference gave me a bitter sweet taste, as I either did not see anything more than I expected or my expectations were not that big because of lack of big events during this year.

The most interesting talk for me, was "Graal.JS - high-performance JavaScript on the JVM". The title is misleading here, as it has not to do with javascript on the JVM directly but with polyglotism on the JVM. Worth checking it out I believe.

A list of stuff worth watching in my opinion are the followings:

Last but not least, here are the talks I found interesting (event if I did not attend all of them) with no specific order:

• Graal.JS - high-performance JavaScript on the JVM
• The Silver Bullet Syndrome

Regarding the rest of the conference, let's break it up to smaller pieces...

University days

As I said I did not attend them as I felt that they were too "boring" the last to years. The combination of 3 hours duration and lack of actual hands on programming, makes you wanting to change the topic as soon as possible. In other way, I can only remember just one of these talks. There, the speaker managed to time box different aspects of web development and speak for each one a couple of minutes. Which technically is more than one talk made by one guy.

It can of course that I'm just a little bit strange. The truth is that I haven't heard either anything really positive regarding the university days except from this years ones. Some friend of mine told me that they found the first 2 days this year much more interesting. Maybe I am just unlucky.

Anyway, keeping someone interested for 3 hours is not an easy job, and this is why the days have been named University days. The attendant should be willing to stay and filter the information he gets.

So let's get to the actual conference days...

The bigger ever

It looks like this year's devoxx was the bigger ever. I think it was about 3500 attendees, or to put it by another way it was big enough to run out of neck wraps for the entrance tickets. Yes, this time there was no wrist wrap, but rather a simple neck wrap (if you were not late) with a printed bar code on it. Not that cool but for sure much faster as a process. At least this time I did not have to wait more than 3 minutes at the queue.

Apart from Devoxx Belgium, there were also Devoxx UK, Devoxx PL, Devoxx FR and Devoxx MA along with bunch of voxxed days, which I did not keep track of. It looks like the conference market is expanding and Stephan make use of it.

Although the bigger devoxx ever, there were no major problems with overcrowded rooms (or should I say cinema halls?).

The keynotes

From the technology point of view, these were the poorest ever keynotes I've ever seen. No big announcements, which was kind of expected as no ground breaking technology made it into this year's radar.

So it was mostly Stephan announcing the end of Parleys and immediate use of Youtube/Vimeo for publishing conference videos. It is a pity for parleys but I got the feeling that the whole conference got a huge marketing push as speakers could reference to their talks a couple of hours after it. This means that you can have now access to all of the conference talks.

Then Mark Reinhold took the step to talk about jdk9 and mostly about modulatity. Given the fact the we've been talking about it for the last 3 major Java releases, I did not see anyone extremely excited about it.

Finally theoretical physics professor Lawrence M. Krauss talked about how irrelevant what we were doing at devoxx was. I do not know if that is the best way to start a conference, but for sure it was the best talk of day one.

Lack of advertisements talks

I did tweet the following:

No advertising talk till now at #devoxx2015 either my filter works better or there is an improvement there!

— Yoryos Valotassios (@valotas) November 12, 2015

Although someone can argue that there were no marketing talks, I have to say that this time at least they were not that obvious. This is in my opinion a major improvement. I can understand the need for them but I do not see what they can bring in terms of marketing if there are only a couple of people watching them.

A very good example of such a talk in my opinion was The Twelve Factor app: Best Practices for Java Deployment where you at least got a couple of useful information regarding the 12 factor app (which in my opinion is 12 rule of thumbs for creating microservices) and the only advertisement was that you could deploy that super easy with heroku.

As I said before there were not big news this year and maybe that is why there were no interest from big names to advertise anything.

Trends

Although there were no big news, there were some obvious trends. It looks like reactive programming and microservices are still the thing this year.

Reactive programming

Working with streams has been the thing for some years now. Unfortunately Microsoft who is I think the main player behind the whole reactive idea, did not present anything in the last 3 years at devoxx. But this is another topic...

Back to reactive, as I watched some talks on this topic the previous years, I skipped these talks this time. Unfortunately, Java has only recently added basic support for that with the changes of JDK8, which means that it lacks the apis on the EE side and not only.

There are solutions available. Mainly RxJava and akka based ones from typesafe. I still ask myself though if I'm going to depend for example on an sql server, how am I going to be reactive if I have to wait anyway for a response. I have to wrap the sql api to something "reactive", but this is not the standard anymore. Not that it is bad (spring or hibernate never was the standard for example), but it lets us know that it will take a while till reactive will be widely adopted.

More about reactive can be found at reactivex.io.

Microservices

Yes, microservices are still the trend. Unfortunately, the term is so abstract, that you can give your own description. For some reason I tend to agree with what Hadi Hariri said about them, at his funny talk: The Silver Bullet Syndrome. They are what SOAP used to be but, SOAP is so bad, you do not want to name them like that :)

Never the less, if you would like to play with microservices, I haven't been to a talk on this topic without hearing once about Spring boot. It looks like it does the job just fine, but it doesn't mean that there are no alternatives. Actually there is one offering pretty much exactly what Spring boot does by embedding an application server with your app which I think deserves a serious look at. It is no other than Wildfly Swarm and you can guess which application server is used.

If you think that EE cannot be used for microservices, think again. If you don't like EE anymore, then there is what you can also check out:

• grpc for communication
• spark for rest
• dropwizard for a little bit more than spark
• vertx which is kind of I can do everything framework

After checking them all out, let me know what kind of database microservice you are going to use. Although this comment is kind of sarcastic, when you do something, you have to remember that it is all about the code. So, have a look at Microservices and Modularity or the difference between treatment and cure! and see how an EE application can be refactored to something osgi based (yes, it still lives).

What about the JVM?

Back to Graal.js...

When we are talking about Java, we are mostly referring to Java as a language. This wouldn't be possible without that beast called JVM. I did not spot any JVM specific talks to be honest, but I did find one with a title that was giving you the impression that it was yet another javascript on the jvm implementation.

Apparently the talk was mostly about how mature the JVM became and how it can nowadays be used as a platform by other languages. Given the number of languages implemented on top of that, this is not new news, but the interoperability of them is. It's worth having a look at the talk, to see how javascript code has been used from within R code.

So, you just did your first angularjs webapp and are ready for production. And then comes the requirement that before initialization you have to do an ajax request. There should be an easy way to do that right?

The problem

Well, it looks like there is no straight forward way to solve this problem. And the funny think is that it is quite common practice to actually run some initialization code before you let your angularjs app start. Now, what this has to do with routing and why it is a problem?

Till now I haven't seen a single angular webapp that is not using any kind of routing and the first out of the box solution is ngRoute. The thing is though that this module has been build with the idea that routes should be defined at the config phase. This looks like pretty logical except from the fact that you can only inject providers within the config phase.

That means that once your app starts, routing has already been defined. If you force the user to start with a specific route, the initialization of the app should not be a problem. But if your user tries to reach /route/xxx without reaching let's say the root route (/) where all the initialization should take place, you have the problem of trying to use services and controllers depending on stuff that have not been properly initialized.

So a solution to the problem would be to have a way to always run some initialization code before defining the routes so that angular can continue with the routing after that. And it looks like this is not possible in your config phase, as no services are available there and during run phase no providers are available.

A $routeProvider provider

Now let's try a naive hack and find a way to inject $routeProvider after the config phase. Then we can configure it lazily (let's say after our initialization). It looks like $provider is a service available at the config phase and we can use it to register new services/factories. So, let's try to register a $routeProvider service:

.config(function ($provide, $routeProvider) {
  $provide.factory('$routeProvider', function () {
    return $routeProvider;
  });
})

This allows us to directly use $routeProvider later on in our app. An example which uses this technique can be found here.

The lazyRoute module

As mentioned before such a solution would only be considered as a proof of concept. Having that working though can let us create a proper angularjs module which can make use of this technique.

So the requirement is to have at the end a route module that can be configured both on config and run phase. It should be able to initialize routes after a given promise is resolved. Ideally we would like something like the following:

.config(function (lazyRouteProvider) {
  lazyRouteProvider.when('/', {...})
    .when('/route', {...})
    .lazyWhen('/route/after/init', {});
})
.run(function (lazyRoute, initService) {
  lazyRoute
    .initAfter(initService.init())
    .with(function (initServiceResult) {
      lazyRoute.when('/another/route/after/init',{
        //config based on initServiceResult
      });
    });
})

So, let's try to split it up in steps.

Dummy lazyRoute that delegates to $routeProvider

As we need something that should be configurable at config time we should go with an angular provider. Let's see how it would look like:

.provider('lazyRoute', function ($routeProvider) {
  this.when = function(path, route) {
    $routeProvider.when(path, route);
    return this;
  };

  this.$get = function () {
    return {
      initAfter: function () { return this; },
      with: function () { return this; },
      when: function () { return this; }
    }
  };
})

Implement lazyWhen

So, the idea here is to just keep track of the arguments passed to the lazyWhen function for later use:

.provider('lazyRoute', function ($routeProvider) {
  var lazyRoutes = [];

  this.when = function(path, route) {
    $routeProvider.when(path, route);
    return this;
  };

  this.lazyWhen = function(path, route) {
    lazyRoutes.push({
      path: path,
      route: route
    });
    return this;
  };

  //rest of the implementation skiped
})

Implement the lazyRoute module ($get function)

Now, let's try to implement our service which is what is returned by the $get function.

First of all the when function should delegate again to $routeProvider. On top of that, we should also reload the $route service as it has already been set up. Reloading for every new route definition is not so efficient but ok for our proof of concept.

this.$get = function () {
  return {
    when: function (path, route) {
      $routeProvider.when(path, route);
      $route.reload();
      return this;
    }
  };
};

then we should also store the with function so that we can later use it within the initAfter. We wrap the with function into a promise. That way, we can make use of it directly within initAfter but it will get executed only when it is resolved:

this.$get = function ($q) {
  var deferredWith = $q.defer();

  return {
    with: function (fn) {
      deferredWith.resolve(fn);
      return this;
    }
  };
};

finally we need within the initAfter to wait for the given init promise to get resolved and then setup the lazyRoutes and call the withFn:

this.$get = function ($route) {
  var deferredWith = $q.defer();

  return {
    initAfter: function (initPromise) {
      $q.all({
        initPromise: initPromise,
        withPromise: deferredWith.promise
      })
      .then(function (result) {
          var initPromise = result.initPromise,
              withFn = result.withPromise;

          angular.forEach(lazyRoutes, function (routeDefinition) {
            $routeProvider.when(routeDefinition.path, routeDefinition.route);
          });

          withFn(initPromise);
          return initPromise;
        });
      return this;
    }
  };
};

An example with the full solution can be found at: http://jsbin.com/jokibe

tl;dr: Angular new Router

Finally, it looks like the new angular's router does not have the same problem as the injected service can be directly configured saving us from the encapsulation above. Diving into this though is a very good exercise for exploring promises, angularjs provider and the a little bit of the $route service.

This year I was lucky enough to attend the devoxx conference yet another time. Here are what I saw/felt especially in regards to my previous visit.

Java

This year the conference was a lot more about java. There were some sessions for other stuff too but it did not have these 24 sessions for angularjs.

As it was about java, it was of course about java8 and its new features. So, as you may know lambdas have been introduced to java8 and this alone is enough to start building a lot more features on top of the platform.

So along with the lambdas, we had streams and a refined collections api that allows a more functional way of programming.

Having heard so much about lambda/streams, here are my thoughts. The feature brings ease and beauty to the language mostly by reducing boilerplate code. No one likes boilerplate but remember that this boilerplate is information/documentation to the user. Therefore I would suggest to make use of it with extreme caution as it is really easy to misuse it in the name of coolness. So please have a second thought before you go ahead and refine your api.

Reactive programming

Well, there is nothing new here as Vlad said...

#Devoxx@valotas is not new. Is we use with .niet in Azamat for centuries

— Born to be Borat. (@DevoxxBorat) November 14, 2014

The thing is though that even with a delay of some years reactive programming becomes a trend to Java too (and once again in software engineering in general). There were some libraries for reactive programming before but lambdas again can make the existing apis much more fluent.

I believe a very good answer to most of the newbies questions would be netflix's article regarding the use of a reactive programming model. Interestingly, I find this post a very good abstract of Venkat Subramaniam's university session which was the most popular session within the 5 days of the conference.

Docker

I do not think that there is much to say for this topic. Docker can be described as a very lightweight linux base vm. Everybody seems to be using it and it looks like it is a matter of time before you'll be using it. I do not believe that there is any question regarding its benefits and therefore just go for it as soon as possible.

As a side note. I do not know if it is java developers disliking system programming, but I did not hear a single thing regarding the background of the tool. It is a success story for Google's golang but nobody felt like mentioning it.

Google, Microsoft and others

Speaking of google, its presence this year at the conference was really quiet. The keynote was about material design, something that was of little to no interest to the developers. I do not know though how an audience accustomed to backwards compatibility would feel about the news that one of the last years' hot topic will be a new thing with no migration path.

So regarding the web. Everybody should now switch to this new thing (angular.js v2) or use the other cool thing called polymer which is focused on Web Components. No big google logo anywhere, but still backed by it. And all that not mentioning the other new cool kid on the block, coming from facebook and called react. This fragmentation is driving me crazy!

Apart from that Microsoft's target was once more the marketing of the Azure platform and nothing more. That is again sad considering the changes that were made public during the devoxx days. I would really like to see some more programming centric stuff from Microsoft in such a conference.

Finally this years devoxx had more non programming related stuff. A bunch of startup related stuff among with a talk regarding creativity by the Chief Creativity Evangelist (cool title) named Denise Jacobs. The talk was too American for my taste, but still a good addition to the conference.

Devoxxed

This year Stephan's idea was to get the first letters of the conference name and put them at the end of the remaining. Therefore something called voxxed was born which had its first outage at the demo during the keynote. I do not know if it was a good sign or not. What I know is that it is not only a site aiming to be the new theserververside.com or a better dzone. There will be one day devoxx conferences around Europe with the name voxxed days.

I do not know if the idea of mini devoxx conferences came first or not but I do like the idea of having some decentralization. Unfortunately this seems not to be working as the conference this year was overbooked even though there are another 2 taking place in the UK and France.

There is quite a buzz nowadays (2014) for functional programming and so we all try to move our thinking in a more functional way. Still, the object oriented approach seems to be the easiest way to understand and so to program at the end. The object oriented programming can be applied to any structural schematic. One of that structures though that we tend to forget is html. So, let's try to visualize it.

What is an html object

It is not that hard if you think of it as something that you allready know which is the DOM. If you've written a little bit of javascript you've interacted with it in order to bind functionality to your presentation layer. But it is not only about javascript. To simplify things, think of every html tag as an object. Object composition allows objects to contain other objects and there for you can have your document object to contain your paragraph object (p) which contains a banch of text.

Object signatures may differ based on the type of the html element they describe but they all share some functionality. We can easily find out that the interface below describes all the known html elements:

interface Node {
  getClassNames(): List<String>
  getChilds(): List<Node>
  getParent(): Node
}

Now the way we represent markup to objects is straigh forward and is not the part of this article. What I am more interested in is this getClassNames() part which can be easily misunderstood. Every html element can have a class attribute where the user can put as many space separated classes as he wants. These classes are then accessible by this getClassNames() method.

A css example

This is the most important method for us, as developers, as it gives a binding point for styling and behaviour. This can be used in order to add css classes and define the style of the object. As an example, let's look at this markup:

<div class="main article white">
  <h1>The header of the article</h1>
  <p>lorem ipsum</p>
</div>

In our pseudocode world, the result of the mainArticleDiv.getClassNames() would be the list: ['main', 'article', 'white']. But this list is not just another attribute of our div. It describes some of the div's functionality (styling) and therefor it looks more like metadata of the mainArticleDiv's interface rather than data of the interface. Since it is metadata, a list within our class is not the right way to describe them.

Let's make a pause here and go back to our computer science world and specifically to object oriented programming. There you can find the term mixin which is actually a way to describe a part of the functionality of an object. Assuming that the browser creates a mixin for every defined css class of ours and the fact that Node itself can be a mixin implemented by the browser, we can then describe our div with a class:

class MainArticle extends Node, CssMain, CssArticle, CssWhite {
  //Nothing to implement
}

and our Node interface becomes simpler:

interface Node {
  getChilds(): List<Node>
  getParent(): Node
}

Now everything looks simpler. You have to still write the metadata in html as we just did, but at least we now can give a representation of an html element in an object oriented way.

Javascript: behaviour mixins

Now that we can style our node with css mixins, what about adding some behaviour there? This is where javascript is comming. So let's say that we have a <button/> that we would like to use in order to show an alert to the user. Assuming that our button is also styled, we can describe this button with the following pseudo class:

class AlertMessageButton extends Node, CssButton, CssMessage, AlertMessageAction {
  //Nothing to implement
}

In html that would be something like the below:

<button class="button message action-alert-message">Alert!</button>

and now all we have to do is to somehow implement this AlertMessageAction mixin. And that would be kinda easy:

function alertMessage(msg) {
  window.alert(msg);
}

Now all that left is to bind that mixin to the elements's action-alert-message class. There is no standart way doing that but with jquery we can do it pretty easy:

$('.action-alert-message').click(alertMessage);

But let's pimp our example a little bit more. As you see our mixin needs a msg to show. That means that our AlertMessageButtton should implement a getAlertMessage method or have an alertMessage field (we do not care about how we represent that on the class base). Well what about tweaking a little bit our html:

<button class="button message action-alert-message" data-alert-message="Hi!">Alert!</button>

And now we are done. We have the classname of our object, the mixins that it is implementing and also some fields with data. Sounds like a proper instance to me.

One last thing to take care would be the actuall binding of the mixins to the element. The styling is something that our browser takes care of, but the behaviour is something that we should deal with. For that we have to use the hooks provided by our browser and just add our behaviour. With jQuery that would be something like:

$(function (){ //on document load

  // bind our function to the click event of the .action-alert-message
  $(document).on('click', '.action-alert-message', function() {
    var msg = $(this).data('alert-message');
    alertMessage(msg);
  });

});

And I guess we are done! Do not try to hack stuff arround anymore. Use classes and mixins the way you should (even if they are kind of restrictive as they are in our html world).

But why not to use data atributes as mixin names

Well, I believe that this is an opiniated question and therefor I can only give an opiniated answer. Many frameworks are going that way, trying to let the user work with classes in a css central way. My answer to that is why not do the same with css? Why not use data attributes to style your elements?

As I can not find a straight answer to my question, let's say that I just like to have everything properly grouped. Therefor I use the class attribute to define the characteristics of an element and data attribute to give actual data to the element. It just looks much more logical to me, but still opiniated.

References

• DOM
• Object composition
• Object-Oriented CSS: What, How, and Why
• Mixin
• Abstract classes vs. interfaces vs. mixins

On 9th of September Gion Kunz and me ran a workshop for ch/open. The idea was to introduce the basics of angular js so that someone could start experiment with it. During the workshop we did some exercises with jsbin and hacked together a chat application.

Some notes for the workshop

For me that was the first time trying to help people on a subject outside of my colleague/friends cycle. Of course I'm always more than excited to help, but when you are running a workshop, you always have some stress as the expectations are not the same. After all someone attending the workshop should take back what he came in for as you can not be next to him everyday.

This is how I started and although the workshop has been announced a little bit late, we tried hard to come up with something that would have been fun to program and educational enough in order to get something useful back.

The workshop

We started with a very basic introduction to the angular.js framework and an example covering all the aspects that we would talk about. The last was mostly to see how advanced/familiar was the audience with the framework in order to adapt our exercises.

The basics

This is one of the most difficult things to come up with. Mostly because people tend to have different opinions on what the basics of a framework are but most importantly because there is a huge difference in what you are about to describe in order to teach something. For example someone with a very strong programming background would consider the dependency injection and the providers of angular the basic stuff that someone should know about. After all DI is a fundamental thing in the backend and nowadays is taken for granted. Someone else would want to talk about the binding capabilities of the framework as it is one of the coolest things.

The problem with either approach though is that you end up explaining theory which takes all the joy of programming away. Therefore we defined as basics some directives that would help us to create something. So, the introduction to the angular world was the ng-app, ng-init, ng-repeat directives (we considered ng-app a directive) along with expressions. Interestingly enough, all that can help you make a page more interactive. And that is what we did.

Some more interesting stuff

After that we introduced $scope, ng-controller and ng-model which are the fundamental elements to start creating something useful. To illustrate that we provided yet another todolist example: http://jsbin.com/cuqoc. Whoever is interested in, can try to do the exercises of the workshop:

• Clear the input after adding the todo to the list
• Initialize the array with some tasks
• Implement a way to clear all the tasks
• Implement a way to delete a specific task

The important thing here is to understand what is a controller, how is it used and how our model is somehow (or should I say automagically) bound to the html.

Architecture. Or where you need DI

When you are happy enough to create your own apps, you easily come to the point that your controllers are big enough to be completely unmaintainable. And this is where DI shines. Although a little bit difficult to illustrate the need, you at least can illustrate the coolness of it. That was the last exercise of the introduction to the framework. Modules and factories. We worked again on jsbin: http://jsbin.com/huguj and this time the exercises were:

• Modify the service in order to be able to add a task
• Implement a clearAll method (just like the previous example)
• Extract the version into a commons module and use it as a dependency (try it with both other modules)
• Redefine the version value again on the footer module

Here we would like to find out how we use modules as it is the basic ingredient for an angularjs application. Most of the getting started guides do not talk about it, but it is something the sooner you learn it the better. So we thought it would be better to directly write apps using modules and it is an advise I give to any newcommer to the framework.

Markdown chat app

Knowing all the above, we could dive more into an application. This time we created a markdown chat application based mostly on what we learned before. The development was splitted in steps which we could change with the use of git. In addition to that we did before we also had the chance to also work with filters and touch the surface of angular.js directives.

As this example was a little bit bigger we also had the chance to talk about where code should be, something that could only be illustrated with a graph. But again, theory and practice are only the same in theory, right?

On top of that you can also see a real use case where DI shines as we jumped from an in memory chat application to a backend based one by just changing implementation of one service. Again for someone coming from the backend world, that would be obvious, but in javascript I never felt that there was something as close to what I'm used to have on the backend as Angular.

What we did not do

One thing that I was really uncomfortable with, was the fact that we did not talk about testing. Angular has been built with testing in mind providing you everything that you may need to proceed with that. Still, we could not find enough examples that would be easy enough on their testing part so that we could spend the right time on the actual code. We are working on some ideas though and hope that we'll improve on that.

References

• The slides from the presentation
• The chat application at github
• ch/open: Unleash Your Creativity with Angular.js

It was about time to get rid of the blogger platform. It serves well up to the point that you feel like your page is overbloated with stuff that you do not need.

valotas.com

The new home of the blog is now valotas.com and all traffic from blog.valotas.com has been redirected there. The funny thing is that this new home is "hosted" at a Raspberry Pi at my living room. Of course there is a cdn in front of it just in case it goes down, but I do not think that anybody will really care that much if my blog goes down for some minutes/hours (hopefully not days).

Back to the blogger.com. It is a decent platform and a lot have been improved especially on the backend, but this wysiwyg editor is just killing me. I feel like it is poluting my text rather than transforming it to html. I would even add here proper html as it tends to wrap everything in to divs/spans with no reason. Having worked with markdown, this thing looks like an overkill. You can always write simple html but then your writting stops being a writting and becomes something else.

I tried to ignore that, which is not so easy. Having your articles in simple text files is really concice and can be transfered wherever you want. Still as I tend to forget stuff, having everything in one place as an "article repository" (blogger in this case) was really handy. Having that in mind I tried to just write my stuff in simple html to keep everything clean. After all I do not write that often. So the authoring part worked pretty ok.

Then it comes the frontend. I am by no means an html designer, but still I do like simple stuff and the blogger layouts are by no means simple. Creating custom layout looks more like a hack where you have to always conform yourself with what blogger provides you. On top of that you have that javascript stuff that goes with your text that I never felt like checking what they do. Although I did not check, I was visiting my blog regularly from a pc and/or mobile and had some interesting lag. That should not happen at a simple html page right?

Back to the "article repository". I felt ok with blogger hosting my articles up to the point that git made it's debut in to my life. Combine that with github and/or bitbucket which both provide a service for git and some how this centralized article repository I had before, stoped being an advantage. In fact the decentralized one started to be more natural to what I had. On top of that you get versioning for free!

assmeble.io

Having all the above was enough to make me change to something else, but I could not find that something else. Github pages would have been a solution, but jekyll's language (ruby) would have been yet another one that I had to deal with. In theory I wouldn't have to deal with yet another language, but somehow I always find something in a framework that I would like "improved".

Having persuaded myself that a static site generator would have been the best solution for my markdown text files (call them articles, pages or blog posts), I had to find something that would fit me. Fit me means, transforming my static text but also allow me to change layouts on demand. This on demand requirement is quite tricky as nobody wants to write plain css anymore nor to write javascript without jshint or tests. So this on demand means something that can blend in to the current css/js tool chain which is grunt or gulp. And this is where assemble.io comes in to the game.

In fact the whole site is a simple git repository that I can clone, edit and push anytime, anywhere. I have local preview and the deploy is a grunt deploy of my current preview away.

Surprisingly (or not) the site became much faster (plain html with css and only google analytics as javascript) and it is responsive something that blogger lacks for the moment.

That is all, enjoy my new web home :)

Recently I had a look at the Dart programming language and that reminded me of the getters and setters mechanism that someone has with C#. So let's have a look at them.

Where we use them

Well, the idea in OO languages is very simple. You have a class and you have a property, a field or whatever piece of data that you want to access:

class Person {
    String name = null;
}

You start using this property in other classes:

var george = new Person();
george.name = "George";
assert george.name === "George"; //true

Now you need to change your class a little bit. The name property is not enough anymore but you need to define a firstname and lastname. In order to provide backwards compatibility you do not want to just remove the name property. That is straight forward, just return something like firstname username when someone is trying to access the name field. For simplicity we also do not allow any direct updates of the name field. "Speaking" in pseudo code, it would be cool to have the above:

var george = new Person();
george.firstname = "George";
george.lastname = "Valotasios";
assert george.name === "George Valotasios" //true

Well, something like that is a no-go for java, but there are languages (Scala, C# and Dart to name a few) that allow you to do that. I do not really care about the syntax of the feature but let's take dart's syntax as an example and provide a solution to our problem:

class Person {
    String firstname
    String lastname

    Person(this.firstname, this.lastname);

    String get name => "$firstname $lastname"
}

var george = new Person("George", "Valotasios");
assert george.name === "George Valotasios"

Just another method declaration

So, what is happening now? Well the vm knows that the class Person does not have the field named name but there is a special method that can be called whenever a someone tries to access the Person's name field.

This is actually what we are forced to do with Java where no such feature exists. The normal way of dealing with data objects in Java is to have only private fields and provide methods named after the field with a get or set prefix. So in java the same class would look like:

class Person {
    private String firstname;
    private String lastname;

    public Person(String firstname, String lastname) {
        this.firstname = firstname;
        this.lastname = lastname;
    }

    public getName() {
        return firstname + " " + lastname;
    }
}

and making sure that you followed the rule of having private fields, you only end up having to always write some more characters for accessing fields: person.getName() instead of persons.name.

All in all, till now getters and setters looks like "special" methods. They either have a special name or a special syntax. That speciality allows you to make some assumptions about what is going to happen. But should it be that way?

Getters and setters should not be regular methods

Getters and setters in general should describe a property of a class. That said they should just update or return that property value. So in theory no computation should happen in them, they should be fast and they should be side-effect free. For simplicity, lets change a little bit the definition of them and apply only two rules:

• they should be fast
• they should have any side-effect other than updating the underlying property (in the case of a setter)

These two rules are a simplification of the principle of least surprise and tells you for example that you should not do there any IO stuff or spawn new threads or even update any fields other that the one that the name of the setter defines. Still these two rules has been violated even by developers who's expertise in a language is not questionable.

As I can not question their expertise, I can only assume that it easy to just abuse this feature by mistake. But if that is the case, there should at least be some precautions. Static analysis tools have been around for a while, so for sure they can help us there.

It would be also cooler if the compiler of a language would be able to do the analysis for you as part of the compilation. In that case it makes sense to help the compiler and yourself with a special definition of the feature:

• The compiler will know that it should be stricter with the content of such a method definition
• You will know when and how to use getters and setters
• Prevent newbies from misusing them

At the end, I do not see the need for a syntactic sugar for a method definitions, if that syntactic sugar only forces assumptions about the underlying execution.

Do you really need a getter or a setter?

Having said that and assuming that our code provide good implementations of getters and setters, there is still the question of their need.

A simple mistake that I used to do for example was to just let my IDE create them for me. I ended up having for every class stuff that never needed. In other words, before you use the stuff of your IDE, just make yourself a question: "does a field represent an internal state of your class?". If so, why do you need to provide a getter or a setter for this?. Even if you have a clear yes for a getter (it could be), question again yourself about the setter. Do you always need to update a property of a class? If not why to provide a setter for it?

Not all of our object should be DAOs and with that exception I hardly have to provide any setters for other object. Apparently even the use of getters is kind of limited outside the scope of a DAO.

So google choose to announce the release of the production ready dart lang at this year's devoxx. That was though mostly a rebrand of the 0.8 version that was available some weeks earlier but that is marketing.

A little bit of background

For some reason I felt like I should try the language earlier so I found a simple project to try to implement it with it. Apparently as dart is at version 1.0, I felt like I should also rebrand my latest branch of mustache4dart to v1.0, but given that I've used it for about a year now, I though that it is the right time to write some things about the language.

First impression

Unlike most of the new programming languages, this one comes with an IDE (based on Eclipse) and it makes it much more simple to get start with. That was a really good first impression.

Apart from that, given my java background, I can say that I felt really familiar with the language it self. The syntax has been kept really close to java and C# and it looks like that was on purpose. They really wanted to make it easy for these programmers to be a dart programmers and I can say that they managed to do that. From the other point of view though, I do not know why, at the year 2013, we should be using semicolons on the end of each line at our code.

Regarding the differences with Java someone could take a look at this article. I think it describes most of the key features of the language and someone can compare with his own favorite language.

Getters and setters

A thing that was new to me was the getters and setters of the language, something that you can find with C# but not with Java. To be honest I can not see the need of them as they look like methods and they act like methods, but they should not be used like ones (ex: do not try to perform an IO operation with a getter or setter). In java world the same rule applies with the exception that you must always provide a getter/setter method for each of your fields you would like to be accessible and there is no special syntax for a getter/setter but a naming convention. This thing exists in C# though.

I personally do not see the need to have yet another syntax for them, but I would like to have them without writing them (thank you project lombok). I know that I'm biased here but I'm just used to the java way of dealing with fields and the simple rule: never try to access them directly. C# and Dart gives you the ability to do so, as you can later on change the behaviour of what looks only as a field access.

Named parameters

A thing that I liked with the language is the ability to have named parameters. This is really cool and something that would make for example the builder pattern in java to go away. I do not now if such a thing exists with C# but given the fact that its progress as a language is much faster than java's I wouldn't expect not to have something like that.

Constructors

Another interesting thing is constructors. First they are much more concise which I liked (event though the syntax may look awkword at the beginning). Second each class can have more that one named constructor. I tend to think of them as java's static factory methods but with access to the object itself (there is a this available). So, if we would like to change the name of java's pattern, that would be factory methods.

On top of that, there is the so called factory constructor which is an unnamed static factory method. In any case the introductions of these features are targeting the elimiation of java's object construction patterns. While general this is a good thing (language without patterns), I've already seen the creation of yet another pattern which is the use of a factory constructor only in cooparation with another "internal" named constructor.

For sure the whole construction thing was a little bit strange at the beginning and I think both java and javascript developers will have to spend sometime until they'll learn to use them right.

Functions

One of the good parts of dart are functions. Yes, the language supports functions as first lever citizens. In fact every object can be used as a function by implementing a call method. That is a good way to introduce the idea of function to a purely object oriented thinking guys. Of course there are also normal named and anonymous functions (lambdas or closures anyone?).

After all lambdas are finally available to Java and it was for some time in C#. On top of that, Javascript was born with closures and is more functional as a language rather than object oriented. So there is no question, any language trying to replace them or just look familiar with them should support them.

Types

Another thing that the dart guys tried to advertise is that it is an optionally typed language. So you can start scripting and as soon as you grow you can start using types. This is ofcourse not true. Comming from java, I was always thinking of typed solutions. While developing with dart, I found myself only to drop back to untyped api when I did not want to introduce yet another method for something not so important.

So I think that the optionally typed part was because it targets also javascript developers. As soon as javascript does not have types it would be hard to enforce them in a new language.

Asynchronous programming

Another interesting thing with the language is its support for asynchronous programming. Yes, you can do that with any language but good luck with java's Future object. When it commes to the ability of writing asynchronous code, javascript way much better than java. What dart did was to take a programming model well known to the javascript world nowadays and standarize it with the use of it's Future implementation. Think of it as jQuery's Deferred object or Q library or as java's new CompletableFuture (yes java is catching up there).

Although I'm slowly moving to a promise/future based way of thinking for async tasks and I'm still getting used to it, the api felt like fresh air to me.

Some other's thoughts

Well, I am not a language expert and as it is a new language and it is from google, it is a little bit hard to find any serious arguments against the it. Though, there are some written at the begining of it's lifecycle:

• Why dart is not the language of the future
• Dart; or Why JavaScript has already won

Final thoughts

Just like anything new, it is a little bit hard to say if it will be successful or not. For sure google wants to push it and the whole strategy behind the language is to target as many developers as possible so that they can then push for the adaption of the VM to more browsers.

One language to rule them all

Dart is targeting the web and that for javascript. There are some blog posts about why it is cooler than javascript syntacticaly and a lot of comments stating that dart is what javascript should have been. Not surprising to me these comments are not coming from the javascript word.

The best break down of the language till now can be found at this article and I would like to copy what the author wrote:

If you like Java and can't get yourself to like JavaScript, you program Dart.

If you like Ruby and can't get yourself to like JavaScript, you program CoffeeScript.

If you like JavaScript, you program JavaScript.

We like it or not javascript will be with us for a while. All the cool new programming languages targeting the web are getting compiled down to javascript and this can not be changed just by the introduction of a new one.

One thing though that I did not mention at all is that dart can be also run on the server side which helps a lot for the elimination of duplicated code. We tend nowadays to move most of our "application" code to the browser having stupid (or should I say stateless?) backend, but still, it really hurts me when I have to write input validation code on both the server and the client side.

For the moment the only solution is, guess what, javascript with nodejs. Some could also mention clojure with clojurescript, but do not forget that it still compiles down to javascript. So, I should put Dart to the candidates of the "One language to rule them all" but can not be anything else at the moment but a candidate.

The future of Dart

Google of course has the resources and the market share (google chrome) to push for such a change. In fact during devoxx it has been said that at some point next year the dart virtual machine will ship with the official version of google chrome. Having said that, do not forget that microsoft tried some years ago to do that too with IE (more than 80% of the market share) and VBScript and just failed.

On top of that, considering languages that have been developed targeting javascript as a runtime (they compile down to javascript) such as typescript, coffeescript or clojurescript, it looks to me that dart falls in to the same category (as it compiles down to javascript) with the only advantage that it will run faster on chrome in the future.

Given though the fact that it comes with it's own vm, it looks more likely to me to see dart being a language for android or chrome os or chome apps and then pushing for being the language for the web, rather than replacing javascript on the web.

So, devoxx 2013 is over. My first java community (or not) conference is over. The thing is that devoxx is not about java. It is more a technology/programming conference which I personally find even better as you have a better overview of what is going on and what the future will be.

Java vs Google

Having said that, we must always have in mind that it has its roots in the java world, but I'm pretty sure that it can make it without it. Just like a lot of parts of google have been built with java, it is not that google cannot live without it. Oh, did I say google, yes, that is maybe because if you take half of the talks out of the conference you end up with... AngularJS and dart. And that is by considering android talks java related. So let's dive into what I got from this conference.

Java 8 is here

I mean, it has not been shipped, but finally the language gets lambda support. That was the most significant thing I took from Mark Reinhold and Brian Goetz. As the last one said, we know that the community has been expecting that for a long time but it was a hard thing to implement something in a way to maintain backwards compatibility.

Never the less, we'll soon be able to use this new idiomatic feature of java. Apart from that I saw Mark Reinhold's need to state that java is not dead. It was his closing statement, but it was kinda funny to try to convince 3500 mostly java developers that already paid to attend a java conference that java is not dead. Of course we can take this statement as Oracle's submission to the java language and this is quite exciting.

From the other point of view, he could just be feeling the pressure of the competition (Kotlin, Ceylon, Dart, Clojure, Javascript, Scala, Groovy gave their presence to the conference)

IoT

In fact one other thing that came to my attention was the IoT (Internet of things). I never really thought seriously about it, but it looks like an emerging market (for developers and not only). Java wants to be part of this market and therefore there were some talks regarding this. For sure a thing to keep an eye on.

Not only Java

As I said before, devoxx is not only java and the proof of that is that two big sponsors of the conference, RedHad and Google, have chosen Devoxx to announce the stable (v1.0) release of their new language. So Redhat announced Ceylon and Google announced Dart. Guess what, both of them are compiling to this strange bytecode called javascript 😁.

That is another interesting thing. We do not like javascript but, we write code compiling down to javascript in order to be able to run it everywhere! Any way, I did not follow the ceylon stuff, but I've been following dart for about a year now.

Dart

So, a language that looks like a hybrid of java and javascript, has its own virtual machine but also compiles down to javascript (or should I say optimized javascript). It's supposed to be a language for structured web apps that does not fragment the web browsers more. Yes this is google's description.

Of course I do not know how yet another language for the web will not fragment web browsers. For sure Google will try to push it as much as possible. I cannot think that it has been announced by coincidence at the second day's keynote with the title "Shaping the future of web development". On top of that, I heard Lars Bak saying that soon Chrome browser will get shipped with the dart virtual machine in order to run dart applications natively.

AngularJS

For sure Dart was the thing for the second day of the conference, but all the previous days google was busy giving talks about the cool javascript framework called AngularJS. I did not follow most of the talks there as I'm aware of the framework and used it, but just to give you an Idea of how good it is, google is in the process of porting it to Dart too.

Given though the fact that the framework is popular enough already, I would not expect 25 talks (that is the number the Java Posse guys counted) being given by almost the same guys.

Never the less, it is my framework of choice for some time now.

Google glasses

Last day has been dominated again by google and the google glasses presentation. For some reason I'm not interested in this so I just picked a more java related thing to attend.

Chrome Apps / HTML5 Apps

That was the most interesting for me. I did not know that chrome provides you with the tools to create a native app using html5. You can find more info at here.

It looks like google wants everything to be done in the browser and that is why dart on chrome makes sense for me. On the long run, as soon as the user gets comfortable with the browser they'll just port the browser to something else. Of course that means that everything will then be so easily installed to your machine, you'll just go to the chrome app store and install. From the other point of view, google will be in charge of all your apps so I would rather go with something in between. Node-Webkit looks like it's trying to do the same thing. Of course you do not have the ease of click and install but you can still have your html5 app running "natively" on your OS of choice.

Buzz words

A new buzzword to add to my vocabulary is Reactive programming. There were some talks about it. I attended one but Joshua Suereth spent about 20 minutes trying to explain how the java api of the play framework would have been with the new lambda support of java. For sure the worst talk of the conference so far.

Room for improvements

This is not for criticizing the event, but rather remarks to get better. Some of the talks were more like an advertisement. There is a remedy for that though, filter out the talks where there is no engineer talking.

The food was bad but again you do not go there for the food. Still, if you want to have fun, just search for #devoxx salad at twitter.

And a remark to Microsoft which surprisingly was one of the main sponsors. Next time come with something cool for developers. Typescript for example, is much more appealing to a software engineer than how Azure works.

I am always searching for the last tomcat configuration every time I need to deploy a new one, so, here is what I use.

Common tomcat configuration

Tomcat comes with some configuration tools. A command line one and a Window based one (tomcat6.exe and tomcat6w.exe respectively). Lets use the GUI. One thing to remember is that the executable should have the same basename with your service name. So we rename the tomcat6w.exe file to tomcat6020w.exe, and use it.

One other thing to mention is that it's better not to rename the tomcat6.exe file as it will be used in order for the service to start up.

Java Options

What are the most common java options in my opinion:

-Dcatalina.home=TOMCAT_HOME
-Dcatalina.base=TOMCAT_HOME
-Djava.endorsed.dirs=TOMCAT_HOME\common\endorsed
-Djava.io.tmpdir=TOMCAT_HOME\temp
-Dfile.encoding=UTF-8
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=TOMCAT_HOME\conf\logging.properties
-Dcatalina.logbase=C:\SOME\PATH\FOR\LOGS
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled
-XX:MaxPermSize=128m

I should add an explanation for each of the properties but it is not the time yet

That is a quick one. These days I was playing around with Jersey in order to create a rest web service. The thing is that I would like to test what I create. The good thing is that Jersey provides a testing framework. But...

Jersey test setup

As the project I was working on was a mavenized one, all I had to do was to add the dependency:

<dependency>
  <groupId>com.sun.jersey.jersey-test-framework</groupId>
  <artifactId>jersey-test-framework-grizzly2</artifactId>
  <version>${jersey.version}</version>
  <scope>test</scope>
</dependency>

As a quick reminder, you can not only use the grizzly web container with the jersey framework, but as I needed some a way to test the actual http api, I went that way initializing everything I would have initialize at a web container with an in memory database.

So after that you can extend the com.sun.jersey.test.framework.JerseyTest class and use it with jUnit.

The problem

The problem is that the JesreyTest initializes the webcontainer before each test and shut it down after that. That means that the procedure became real slow and I could not run parallel tests as the testing framework would try to initialize the web container twice causing an exception regarding the address that is already in use.

The solution

After reading the JerseyTest source code I ended up at the setUp method:

@Before
public void setUp() throws Exception {
  tc.start();
}

but as tc was the com.sun.jersey.test.framework.spi.container.TestContainer class that is getting initialized at the contstractor of the JerseyTest, there was not anything that I could do there!.

As I used the JerseyTest(AppDescriptor ad) you can see here:

public JerseyTest(AppDescriptor ad) throws TestContainerException {
  this.tc = getContainer(ad, getTestContainerFactory());
  this.client = getClient(tc, ad);
}

I could change two things. Either the getContainer method or the getTestContainerFactory. From these two, only the second one was protected and could let me do something like that.

One TestContainer per AppDescriptor

So what I needed was a custom TestContainerFactory that would cache the resulted TestContainer per AppDescriptor. Something like:

class OnePerAppDescriptorTestContainerFactory implements TestContainerFactory {
  private static final ConcurrentMap<AppDescriptor, TestContainer> cache = new ConcurrentHashMap<AppDescriptor, TestContainer>();

  private final TestContainerFactory tcf;

  public OnePerAppDescriptorTestContainerFactory(TestContainerFactory tcf) {
    this.tcf = tcf;
  }

  @Override
  public <T extends AppDescriptor> Class<T> supports() {
    return tcf.supports();
  }

  @Override
  public TestContainer create(URI baseUri, AppDescriptor ad) {
    if (cache.get(ad) == null) {
      cache.putIfAbsent(ad, tcf.create(baseUri, ad));
    }
    return cache.get(ad);
  }
}

Now I can wrap any other TestContainerFactory and only use it if I do not have any other TestContainer ready for a given AppDescriptor. The last problem would be a shutdown hook. As I'm using TestNG the @BeforeSuite with @AfterSuite annotations worked just fine:

public class BetterJerseyTest extends JerseyTest {

  //Fire up jersey with Guice
  private static final AppDescriptor APP_DESCRIPTOR = new WebAppDescriptor.Builder("com.some.package.name")
    .filterClass(GuiceFilter.class)
    .contextPath("jersey-ctx-path")
    .servletPath("/")
    .clientConfig(new DefaultClientConfig(JacksonJaxbJsonProvider.class))
    .build();

  public JerseyTestNG() {
    super(APP_DESCRIPTOR);
  }

  @Override
  protected TestContainerFactory getTestContainerFactory() throws TestContainerException {
    return new OnePerAppDescriptorTestContainerFactory(super.getTestContainerFactory());
  }

  @BeforeSuite
  public void initTestContainer() throws Exception {
    setUp();
  }

  @AfterSuite
  public void tearDownTestContainer() throws Exception {
    tearDown();
  }
}

The jUnit users may have a look here to check how they can implement a shutdown hook or just don't at all :-).

Well, at the first glance the answer to this problem would be to just make an http request to the jsp file and get the output!

The problem

In fact, many libraries can help you do that and you also have java's HttpUrlConnection. This solution will work in most cases. At least it worked for me but I was never really happy with it. The jsp is within my webapp, why should I make an http connection to my webapp to get the computed output of the jsp? Not to mention that I had to pass all of my parameters again as my jsp could not use the attributes within my current request scope.

Thanks God, the HttpUrlConnection solution stopped working after changing the network topology of the webapp. So instead of trying to tweak this solution I just thought of the better one as at that point I had the knowledge I needed!

The goal

So, the goal is to have the web container generate the output and instead of writing to the servlet output, write it to "something else" I had access to. Cool, how to do that? After a little reading you can find out that the container writes the output of the jsp using the writer of HttpServletResponse.getWriter() just like you do when you want to directly write to the output of you response. That happens because after all the jsp's are translated into servlets, get compiled and used as plain servlets.

The solution

Applying some OO patterns, the problem seems strait forward. First of all we need a custom HttpServletResponse where we would provide our custom writer when someone uses the getWriter() method. As for the writer, we need one that we can the access in order to get it's content. Again java provides us with this tool and it's called CharArrayWriter. Finally we can use the HttpServletRequestWrapper to decorate our response!

import java.io.CharArrayWriter;
import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletResponseWrapper;

public class CharArrayWriterResponse extends HttpServletResponseWrapper {
	
  private final CharArrayWriter charArray = new CharArrayWriter();

  public CharArrayWriterResponse(HttpServletResponse response) {
    super(response);
  }

  @Override
  public PrintWriter getWriter() throws IOException {
    return new PrintWriter(charArray);
  }

  public String getOutput() {
    return charArray.toString();
  }
}

The wiring

Ok done! Now how do I use it? Well, let's say that you want to get the output of a jsp file in order to use it for the body of an html email (that was my case). Then you should wrap the actual HttpServletResponse with your's and let the servlet container use that one. There is a servlet that does exactly that:

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@SuppressWarnings("serial")
public class ServletUsingCustomResponse extends HttpServlet {

  @Override
  protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
    String template = req.getParameter("tmpl");

    CharArrayWriterResponse customResponse  = new CharArrayWriterResponse(resp);
    req.getRequestDispatcher(template).forward(req, customResponse);

    System.out.println(String.format("The output of %s is %s", template, customResponse.getOutput()));
  }
}

Ok what the hell is the decorator pattern? Why do I need it?

Theory

I can not say that I am good explaining theory. I just read it and try to apply it where possible! After all practice and theory, theoretically are exact the same! So If you are intrested in theory, I thing that this article at Wikipedia can explain pretty well what it is all about. But if you are like me I do not know if you will remember what exactly is and where to use it if you haven't used it at least once!

A real example using the Decorator pattern

The funny thing is that I've been using this pattern a lot (even if I did not realize it from the first moment) and I'm pretty sure that many of you have. Where? Within every web application that uses the servlet directly. The scenario is that I want to make sure that when I use the getRemoteAddr() of a request I should always first check if there is an X-Forwarded-For header within the request and if so use that ip address. The solution is a custom HttpServletRequest. The servlet api gives us the tool in order not to have to write a lot of boilerplate code which is called HttpServletRequestWrapper. The class should look like the one below.

public class XFFAwareReq extends HttpServletRequestWrapper {
  
  public XFFAwareReq(HttpServletRequest request) {
    super(request);
  }

  @Override
  public String getRemoteAddr() {
    String xff = getXForwardedFor();
    return xff != null ? xff : super.getRemoteAddr();
  }

  public String getXForwardedFor() {
    String xff = getHeader("X-Forwarded-For");
    if (xff == null || "".equals(xff)) return null;
    return getIpFromXFF(xff);
  }

  protected static final String getIpFromXFF(String xff) {
   //extract and return the ip from the X-Forwarded-For header
  }
}

If you are also intrested in how to actually use it,that is with a Servlet filter:

public class HttpServletRequestDecoratorFilter implements Filter {
  @Override
  public void init(FilterConfig filterConfig) throws ServletException {
    //do nothing
  }

  @Override
  public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
    
    request = decorate(request);
    chain.doFilter(request, response);
  }

  private ServletRequest decorate(ServletRequest request) {
    try {
      HttpServletRequest req = (HttpServletRequest) request;
      return new XFFAwareReq(req);
    }
    catch (ClassCastException e) {
      // As we are not able to cast the request to an Http one, we just return the same object
      return request;
    }
  }

  @Override
  public void destroy() {
    // Do nothing
  }
}

Cool, but where is the power of decoration? We just wrapped a class providing some more logic to it. Well, the other day you have the request to prevent your site from xss attacks. A way to do that is to filter the request parameters and strip or escape bad characters or group of characters. Of cource we can just add some functionality to our existing class (XFFAwareRequest) and also rename it to reflect the new functionality added. This solution thought breaks the principle of separation of concerns. What we do then? Easy, create another decorator to provide the new functionality (and only that) overriding the getParameter method and replacing or removing the bad stuff.

public class XSSAwareReq extends HttpServletRequestWrapper {

  protected XSSAwareReq(HttpServletRequest req) {
    super(req);
  }

  @Override
  public String getParameter(String name) {
    return super.getParameter(name)
      .replaceAll("<", "&lt;")
      .replaceAll(">", "&gt;")
      .replaceAll("\\(", "&#40;")
      .replaceAll("\\)", "&#41;")
      .replaceAll("'", "&#39;")
      .replaceAll("eval\\((.*)\\)", "")
      .replaceAll("[\\\"\\\'][\\s]*javascript:(.*)[\\\"\\\']", "\"\"")
      .replaceAll("script", "");
  }
}

Then all you have to do is to decorate a little bit more the first HttpServletRequest which is as easy as adding the line req = new XSSAwaareReq(req) before the return new XFFAwareReq(req); one letting the XFFAwareRequest untouchable and not breaking for example existing tests! Now you know that you've used the decorator pattern with in your web apps!

I really wanted to experiment a little bit with nodejs.

Target

So some days ago a friend of mine told me that he used a cool python library in order to scrap from a site the information he wanted to! So that sounded to me like a very good exercise!

Solution

The idea is to use jQuery powerful selectors to extract whatever I wanted to from an html page. If I can do that, I can also extract the url of the next page and the put the mechanism to work again for the that page! The code I finally had to write was about 50 lines!

var jsdom = require( 'jsdom' ),
	//fs = require( 'fs' )
	//underscore = fs.readFileSync( './underscore.js' ).toString(),
	scrapy = function( conf, counter ) {
		var url = conf.url || null,
			getNextUrl = conf.getNextUrl || null,
			filterPage = conf.filterPage || null,
			finalDone = conf.done || null,
			counter = counter || 1,
			done = function() {
				counter--;
				if ( counter === 0 && finalDone !== null) {
					finalDone();
				}
			};

		jsdom.env({
			html: url,
			scripts: [ 'http://code.jquery.com/jquery-1.5.min.js' ],
			done: function( errors, window ) {
				if ( errors ) {
					console.log( errors );
				}
				var $ = window.$,
					nexturl;

				// First of all try to scrap the next url available
				if ( getNextUrl ) {
					nexturl = getNextUrl( $ );
					if ( nexturl ) {
						scrapy( { 
							url: nexturl, //getNextUrl( $ ), 
							getNextUrl: getNextUrl,
							filterPage: filterPage,
							done: done }, counter++	 );
					}
				}

				if ( filterPage ) {
					filterPage( $ );
				}
				else {
					console.log( 'No filterPage available' );
				}
				
				done();
			}
		});
		//console.log( 'Done with scrap of ' + url);
	};

exports.scrapy = scrapy;

The conf argument that scrapy function expects should have a url (the page we want to filter), a filterPage function that filters the page and a findNext function that filters the page to find the next url that should be scraped. If you want to you can also provide a done function which will get called only when all the scraps will finish! Here is an example:

var scrapy = require('./scrapy.js').scrapy,

	getAdFromTD = function( $td ) {
		var ad = {
				title: $td.find('a').html().replace(/\s*$/g, '')
			},
			type;

		$td = $td.next();
		ad.address = $td.find('a').html().replace( /<br>/g, ',' );

		$td = $td.next();
		type = $td.find('a').html().split( '<br>' ); //.replace(/<br>/g, ',').replace(/\n|\s{2,}/g, ''),
		ad.rooms = type[0].replace( /\sRooms[\n|\s]+/g,  '' );
		ad.floor = type[1].replace( /.\s Floor/g, '' );
		ad.space = type[2].replace( /[\n|\s]+$/g, '' );

		$td = $td.next();
		type = $td.find('a').html().split('<br>'); //.replace(/<br>/g, ',').replace(/\n|\s{2,}/g, '');
		ad.type = type[0].replace( /[\n|\s]+$/g, '' );
		ad.build = type[1].replace( /[\n|\s]+$/g, '' );
		ad.price = type[2];

		return ad;
	},
	ads = []; 

scrapy( {
		url: "http://www.homegate.ch/rent/apartment-and-house/region-zuerich/matching-list?a=default&tab=list&l=default&cid=1585974&ao=&am=Z%C3%BCrich&ep=1&ac=1.5&ad=2.0&incsubs=default&tid=1&fromItem=ctn-zh&ag=1000&ah=2000&be=",
		getNextUrl: function( $ ) {
				return $('a.forward.iconLink').attr( 'href' );
			},
		filterPage: function( $ ) {
				$('#objectList tr')
					.each( function() {
						var $td = $( this ).find( '.tdTitle' ),
							ad;
				
						if ( $td.size() == 1) {
							ad = getAdFromTD( $td ); 
							ads.push( ad );
							console.log( ad );
						}
					});
				console.log( 'Found ' + ads.length);
			},
		done: function() {
				console.log( 'Done! found ' + ads.length + ' ads in total!');
			} 
		} );

Playing with grizzly and guice. Considering that you already have a ServletModule and and a GuiceServletContextListener defined the above code should do the job.

The code

First a handler

import javax.servlet.ServletContextEvent;

import org.glassfish.grizzly.servlet.ServletHandler;

import com.google.inject.servlet.GuiceFilter;
import com.google.inject.servlet.GuiceServletContextListener;

public class GuiceHandler extends ServletHandler {
  private static final Logger logger = Logger.getLogger(GuiceHandler.class.getName());
  private final GuiceServletContextListener guiceServletContextListener;

  public GuiceHandler(GuiceServletContextListener guiceServletContextListener) {
    this.guiceServletContextListener = guiceServletContextListener;
    addFilter(new GuiceFilter(), "guiceFilter", null);
  }

  @Override
  public void start() {
    if (logger.isLoggable(Level.INFO)) logger.log(Level.INFO, String.format("Starting %s", GuiceHandler.class.getSimpleName()));
    guiceServletContextListener.contextInitialized(new ServletContextEvent(getServletCtx()));
    super.start();
  }

  @Override
  public void destroy() {
    super.destroy();
    guiceServletContextListener.contextDestroyed(new ServletContextEvent(getServletCtx()));
    if (logger.isLoggable(Level.INFO)) logger.log(Level.INFO, String.format("Destroyed %s", GuiceHandler.class.getSimpleName()));
  }
}

and the webserver:

import java.io.IOException;

import org.glassfish.grizzly.http.server.HttpServer;
import org.glassfish.grizzly.servlet.ServletHandler;

public class WebServer {

  public static void main(String[] args) {
    HttpServer server = HttpServer.createSimpleServer();

    ServletHandler handler = new GuiceHandler(new Live24WebApp());
    server.getServerConfiguration().addHttpHandler(handler, "/*");

    try {
      server.start();
      System.in.read();
      server.stop();
    }
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}

Still have to test it as I only had a simple servlet working!

For some reason allmost all the init.d scripts for starting and stoping tomcat just calls tomcat's startup.sh and shutdown.sh scripts. I am ok with that as far as they work. The startup script works just fine but when I want to restart or stop a tomcat instance, in most cases I have to just kill the process.

A solution

So what I need is a way to try stop tomcat with it's shutdown script, wait a little bit to see if tomcat shutdown and if not just kill the process. The script below does exactly that.

#!/bin/bash
#
# tomcat7     This shell script takes care of starting and stopping Tomcat
#
# chkconfig: - 80 20
#
### BEGIN INIT INFO
# Provides: tomcat7
# Required-Start: $network $syslog
# Required-Stop: $network $syslog
# Default-Start:
# Default-Stop:
# Description: Release implementation for Servlet 2.5 and JSP 2.1
# Short-Description: start and stop tomcat
### END INIT INFO

## Source function library.
#. /etc/rc.d/init.d/functions
export JAVA_HOME=/usr/java/default
export JAVA_OPTS="-Dfile.encoding=UTF-8 \
  -Dcatalina.logbase=/var/log/tomcat7 \
  -Dnet.sf.ehcache.skipUpdateCheck=true \
  -XX:+DoEscapeAnalysis \
  -XX:+UseConcMarkSweepGC \
  -XX:+CMSClassUnloadingEnabled \
  -XX:+UseParNewGC \
  -XX:MaxPermSize=128m \
  -Xms512m -Xmx512m"
export PATH=$JAVA_HOME/bin:$PATH
TOMCAT_HOME=/usr/share/tomcat7
SHUTDOWN_WAIT=20

tomcat_pid() {
  echo `ps aux | grep org.apache.catalina.startup.Bootstrap | grep -v grep | awk '{ print $2 }'`
}

start() {
  pid=$(tomcat_pid)
  if [ -n "$pid" ] 
  then
    echo "Tomcat is already running (pid: $pid)"
  else
    # Start tomcat
    echo "Starting tomcat"
    ulimit -n 100000
    umask 007
    /bin/su -p -s /bin/sh tomcat $TOMCAT_HOME/bin/startup.sh
  fi


  return 0
}

stop() {
  pid=$(tomcat_pid)
  if [ -n "$pid" ]
  then
    echo "Stoping Tomcat"
    /bin/su -p -s /bin/sh tomcat $TOMCAT_HOME/bin/shutdown.sh

    let kwait=$SHUTDOWN_WAIT
    count=0;
    until [ `ps -p $pid | grep -c $pid` = '0' ] || [ $count -gt $kwait ]
    do
      echo -n -e "\nwaiting for processes to exit";
      sleep 1
      let count=$count+1;
    done

    if [ $count -gt $kwait ]; then
      echo -n -e "\nkilling processes which didn't stop after $SHUTDOWN_WAIT seconds"
      kill -9 $pid
    fi
  else
    echo "Tomcat is not running"
  fi
 
  return 0
}

case $1 in
start)
  start
;; 
stop)   
  stop
;; 
restart)
  stop
  start
;;
status)
  pid=$(tomcat_pid)
  if [ -n "$pid" ]
  then
    echo "Tomcat is running with pid: $pid"
  else
    echo "Tomcat is not running"
  fi
;; 
esac    
exit 0

It would be much better to have a pid file at start time and use that one and not try to figure out the pid from ps. Maybe next version.

Lately I was developing a jax-rs application. What I really like with jaxrs is the way you can handle any kind of parameter values of a request using annotations. So when I need a query parameter I just annotate a field or an argument of a Resource class with @QueryParam("paramName") and the parameter value will get injected for me.

The problem

The problem arise when you want to handle a lot of parameters. This is the scenario of an html form posted by a user. In that case you can end up injecting a lot of @FormParam witch isn't the most maintainable code you can write (As Uncle Bob says, avoid methods with arguments). There is of course a solutions which is called the Builder pattern or the Facroty pattern.

Resteasy jaxrs offers you such an implementation out of the box. Just create a class that will work as a simple data transfer object, annotate their getters or fields with @*Param annotations and use it at a Resource class annotated with the @Form annotation.

Unfortunately Jersey did not provide something like that out of the box. At least some google research did not give me anything valuable. That lead me to try to find a solution. That means that I wanted a way to inject a custom class in a jaxrs resource. The class should get generated by a factory provided by me

The real solution to the problem is: InjectParam but I didn't find it before trying to do what I describe below.

A Jersey solution

After a small research found that a custom jersey @Provider will do the job just fine. So, we start with the our data transfer object class. We consider that we want to create an object holding the information:

public class ContactFormParams {
  private String name;
  private String email;
  private String comments;

  // Getters and setters...
}

Jersey provides a way of injecting custom classes to a resource class. In order to do that we should provide Jersey with that information. The way to do it is the Injectable interface.

import com.sun.jersey.api.core.HttpContext;
import javax.ws.rs.core.Context;

public class ContactFormParamsProvider implements Injectable<ContactFormParams> {
  @Context HttpContext ctx
  public ContactFormParams getValue() {
      // Create and return a ContactFormParams based on parameters
      // we can get from the ctx which Jersey will inject for us
  }
}

In order to make our life a little bit easier we are going to use com.sun.jersey.server.impl.inject.AbstractHttpContextInjectable witch is not part of the Jersey API but the Jersey implementation itself:

import com.sun.jersey.api.core.HttpContext;

public class ContactFormParamsProvider 
  extends AbstractHttpContextInjectable<ContactFormParams> {
  @Ovveride
  public ContactFormParams getValue(HttpContext ctx) {
    // Create and return a ContactFormParams based on parameters
    // we can get from the ctx which Jersey will inject for us
  }
}

Ok, we now have a factory that creates ContactFormParams for as and can get injected by Jersey. But in order be able to actually inject that class we must also provide an InjectableProvider. That tells jersey what kind of a factory should use in order to eject a value. We are going to use the same class for this:

import com.sun.jersey.api.core.HttpContext;

@Provider // Tell jersey that this class is a provider
public class ContactFormParamsProvider 
  extends AbstractHttpContextInjectable<ContactFormParams> 
  implements InjectableProvider<Context, Type> {

  @Ovveride
  public ContactFormParams getValue() {
    // Create and return a ContactFormParams based on parameters
    // we can get from the ctx which Jersey will inject for us
  }

  // InjectableProvider implementation:
  public ComponentScope getScope() {
    return ComponentScope.Singleton;
  }

  public Injectable getInjectable(ComponentContext ic, Context ctx, Type c) {
    if (ContactFormParams.class.equals(c)) return this;

    // In any other case:
    return null;
  }
}

Finally add this class to the Singleton set of your Application subclass and you can use the @Context annotation wherever you want with a ContactFormParams class and it will get created an injected for you:

@Path("/contact") // Tell jersey that this class is a provider
public class SimpleContactFormController { 
  @Post
  public String getValue(@Context ContactFormParam formParameters) {
    // Use formParameters
    return "Some answer!"
  }
}

DRY

Done! After that boilerplate code I have what resteasy gives me out of the box! I have to make it more abstract! A custom annotation (maybe @Form) that gives the same functionality with resteasy's @Form. After a research and mailing list reading came up with InjectParam annotation which seems to be for Jerset exactly what @Form annotation is for resteasy:

public class ContactFormParams {
  @FormParam("name")
  private String name;
  @FormParam("email")
  private String email;
  @FormParam("comments")
  private String comments;

  // Getters and setters...
}

//The resource class:
@Path("/contact") // Tell jersey that this class is a provider
public class SimpleContactFormController { 
  @Post
  public String getValue(@InjectParam ContactFormParam formParameters) {
    // Use formParameters
    return "Some answer!"
  }
}

Conclusion

Is it just me or Jersey really lacks documentation?

References

• http://blogs.sun.com/sandoz/entry/ejb_injection
• http://codahale.com/what-makes-jersey-interesting-injection-providers/

This is a strange combination. Not the hibernate with mssql (and any kind of sql) but the mssql server and limit clause. Yes Sql Server never supported the limit clause (and will probably never will). Of course there is workarounds for this until the 2005 version came out witch provide as the ROW_NUMBER() function. This can give solution to our problem witch again isn't as elegant as the limit clause used by MySql for example.

Hibernate dialect

Unfortunately hibernate does not make use of this function in order to get paginated data (as we can see here). Instead uses top to get the first x data and then a scrollable result to only fetch what the user needs. After downloading the classes and experimenting with them I ended up with the class bellow witch is a mix of the ones someone can find at the provided link.

import java.sql.Types;

import org.hibernate.dialect.SQLServerDialect;

/**
 * Modified version based on the work found at {@link http://opensource.atlassian.com/projects/hibernate/browse/HHH-2655}
 *
 */
public class SqlServer2008Dialect extends SQLServerDialect {

  public SqlServer2008Dialect() {
    super();
    registerColumnType(Types.VARCHAR, 1073741823, "NVARCHAR(MAX)");
    registerColumnType(Types.VARCHAR, 2147483647, "VARCHAR(MAX)");
    registerColumnType(Types.VARBINARY, 2147483647, "VARBINARY(MAX)");
  }


  /**
   * Add a LIMIT clause to the given SQL SELECT
   *
   * The LIMIT SQL will look like:
   *
   * WITH query AS (SELECT ROW_NUMBER() OVER (ORDER BY orderby) as __hibernate_row_nr__, original_query_without_orderby)
   * SELECT * FROM query WHERE __hibernate_row_nr__ BEETWIN offset AND offset + last
   * --ORDER BY __hibernate_row_nr__: Don't think that wee need this last order by clause
   *


   * @param querySqlString
   *            The SQL statement to base the limit query off of.
   * @param offset
   *            Offset of the first row to be returned by the query (zero-based)
   * @param limit
   *            Maximum number of rows to be returned by the query
   * @return A new SQL statement with the LIMIT clause applied.
   */
  @Override
  public String getLimitString(String querySqlString, int offset, int limit) {
    if (offset == 0) return super.getLimitString(querySqlString, offset, limit);

    StringBuilder sb = new StringBuilder(querySqlString.trim());

    String querySqlLowered = querySqlString.trim().toLowerCase();
    int orderByIndex = querySqlLowered.toLowerCase().indexOf("order by");
    String orderby = orderByIndex > 0 ? querySqlString.substring(orderByIndex) : "ORDER BY CURRENT_TIMESTAMP";

    // Delete the order by clause at the end of the query
    if (orderByIndex > 0) sb.delete(orderByIndex, orderByIndex + orderby.length());

    // Find the end of the select statement
    int selectIndex = querySqlLowered.trim().startsWith("select distinct") ? 15 : 6;

    // Isert after the select statement the row_number() function:
    sb.insert(selectIndex, " ROW_NUMBER() OVER (" + orderby + ") as __hibernate_row_nr__,");

    //Wrap the query within a with statement:
    sb.insert(0, "WITH query AS (").append(") SELECT * FROM query ");
    sb.append("WHERE __hibernate_row_nr__ ");
    if (offset > 0) {
      sb.append("BETWEEN ")
        .append(offset)
        .append(" AND ")
        .append(limit);
    }
    else {
      sb.append(" <= ").append(limit);
    }

    //I don't think that we really need this last order by clause
    //sb.append(" ORDER BY __hibernate_row_nr__");

    return sb.toString();
  }

  @Override
  public boolean supportsLimit() {
    return true;
  }

  @Override
  public boolean supportsLimitOffset() {
    return true;
  }

  @Override
  public boolean bindLimitParametersFirst() {
    return false;
  }

  @Override
  public boolean useMaxForLimit() {
    return true;
  }
}

I've also updated the jira issue. The class has not been heavily tested yet but it seems to work.

Another thing I didn't come up with is how to have the offset and limit parameters bindable at the query we've created. I hope to find some time working on that unless you have the solution and would like to drop me a line.

A month ago, I had to do some work on a legacy application. As there is no definition of the "legacy application" expression I refer to an old one where no actual framework has been used for its creation. So there is no any ORM technology here. In that case you have to deal with plain old JDBC.

The Model 2 Pattern

The most common way to handle database connectivity to a legacy application is the Model 2 pattern where you have a servlet doing all the connectivity with the database, creating the model, binding it to the request scope and forwarding the request to the view technology. As we are talking about a legacy application this is jsp.

More than good practice

If you have a web application with traffic you will definitely know that you have to close the connections you use. That means that the connection will get returned to the connection pool you are using or just get closed. In case you do not close them your webapp will get to the state that will have all possible connections. That means that any request that needs a database interactivity won't be able to get server (until some how the connections will get closed).

In order do that and make sure that always the connection will get closed all the database logic should be enclosed within a try-catch-finally block and close the connection, possible statements and result sets object at the finally block.

Of course when you try to close a connection you should also be aware that the close operation itself can throw a SQLException. The solution is to enclose every operation that tries to close a resource to a try-catch block again. The code will end up to something like the below snipset

Connection conn = null;
PreparedStatement st = null;
ResultSet = null;

try {
  //Assuming that we have a datasource object available
  conn = datasource.getConnection();
  st = conn.prepareStatement('fancy sql here');
  rs = st.executeQuery();
}
catch(SQLException) {
  // do whatever you want with the SQLException
}
finally {
  try {
    if (rs != null) rs.close();
  }
  catch (SQLException e) {
    //Do nothing
  }
  try {
    if (st != null) st.close();
  }
  catch (SQLException e) {
    //Do nothing
  }
  try {
    if (conn != null) conn.close();
  }
  catch (SQLException e) {
    //Do nothing
  }
}

DRY

Having three same code of blocks in every class that has connection interactivity causes a lot of boilerplate code. So lets try to avoid that creating a Utility class:

public class ConnUtil {
  public static final closeIgnoringExceptions(ResultSet rs) {
    try {
      if (rs != null) rs.close();
    }
    catch (SQLException e) {
      //do nothing
    }
  }

  public static final closeIgnoringExceptions(Statement st) {
    try {
      if (st != null) st.close();
    }
    catch (SQLException e) {
      //do nothing
    }
  }

public static final closeIgnoringExceptions(Connection conn) {
    try {
      if (conn != null) conn.close();
    }
    catch (SQLException e) {
      //do nothing
    }
  }
}

Using this utility class the final block of the previous snippet becomes:

finally {
  closeIgnoringException(rs);
  closeIgnoringException(st);
  closeIgnoringException(st); // The problem!
}

So I fixed the bug, but I wasn't completely happy as I was thinking that autocomplete should be used without having to worry for cases like that. On solution is to change the names of the methods to something like closeResultSetIgnoringExceptions(ResultSet rs), closeStatementIgnoringExceptions(Statement st) and closeConnectionIgnoringExceptions(Connection conn) and make uncle Bob happy. Looking for a much more elegant way, I come up with an additional utility method:

public static final closeIgnoringExceptions(Connection conn, Statement st, ResultSet rs) {
  closeResultSetIgnoringExceptions(rs);
  closeStatementIgnoringExceptions(st);
  closeConnectionIgnoringExceptions(conn);
}

Making that finally block one liner: closeIgnoringExceptions(conn, st, rs) and letting the IDE do its job warning you if you don't use the right arguments and helping you with the autocomplete!

Of course you will not always be able to use this method as sometimes you may need to use some of the objects and close only the ResultSet or the Statement ones. I personally try to avoid it.

It's been a while since I last bloged but here I am again with something that took me a little bit in order to figure out how exactly should someone define a mail Session at the config files of jetty and tomcat. So the problem here is to define a mail session with name mail/Session using the host my.mail.server.com as mail server with username mailserver_username and password mypassword.

Show me the code!

Actually in both cases I found documentation on how to define the mail session but didn't find out how to define the authentication part of it. Let's jump to the solution

Jetty

Jetty has the jetty-env.xml file you can use to define resources of the web application. So with in the Configure xml element we add the following

<New id="mailSessionId" class="org.eclipse.jetty.plus.jndi.Resource">
  <Arg>mail/Session</Arg>
    <Arg>
      <New class="org.eclipse.jetty.jndi.factories.MailSessionReference">
        <Set name="user">mailserver_username</Set>
        <Set name="password">mypassword</Set>
        <Set name="properties">
          <New class="java.util.Properties">
            <Put name="mail.host">my.mail.server.com</Put>
            <Put name="mail.debug">true</Put>
            <Put name="mail.smtp.auth">true</Put>
          </New>
        </Set>
      </New>
    </Arg>
</New>

Tomcat

Just like jetty, tomcat has its own file to put configuration about possible references. That is context.xml where we should put the following with in the Context xml element:

<Resource name="mail/Session" auth="Container"
  type="javax.mail.Session"
  mail.smtp.host="my.mail.server.com"
  mail.debug="false"
  mail.smtp.auth="true"
  mail.user="mailserver_username"
  password="mypassword" />

Note

In both cases I had to dig more than the usual as I din't find anywhere an as is solution. Things became a little bit clear when start reading the javadoc of its container. For jetty things was a little bit better as you only need the javadoc and then you can figure out the configuration (very nice xml based way to deal with classes but not as short as a configuration should be).

For tomcat the configuration is much more smaller but in order to see how to define the host and the username to use I had to have a look at tomcat source code. I mean why there isn't any full example at the java doc on the attributes a user can use in order for the org.apache.naming.factory.MailSessionFactory to create a javax.mail.Session using an javax.mail.Authenticator with given username and password.

Getting the session

With in the web application we get the session looking for the given name form an InitialContext. The following snipet will do the trick

Context initial = new InitialContext();
Session session = (Session) initial.lookup("java:/comp/env/mail/Session");

I really do not know why such an issue hasn't been fixed. I do not care who is responsible for the uglyness of Eclipse on Linux but unfortunately this is a fact. After some search I found a solution.

Let me know!

That is to change the .gtk2rc (or something like that) file on your home directory to something like...

style "gtkcompact" {
 font_name="Sans 8"
 GtkButton::default_border={0,0,0,0}
 GtkButton::default_outside_border={0,0,0,0}
 GtkButtonBox::child_min_width=0
 GtkButtonBox::child_min_heigth=0
 GtkButtonBox::child_internal_pad_x=0
 GtkButtonBox::child_internal_pad_y=0
 GtkMenu::vertical-padding=1
 GtkMenuBar::internal_padding=0
 GtkMenuItem::horizontal_padding=4
 GtkToolbar::internal-padding=0
 GtkToolbar::space-size=0
 GtkOptionMenu::indicator_size=0
 GtkOptionMenu::indicator_spacing=0
 GtkPaned::handle_size=4
 GtkRange::trough_border=0
 GtkRange::stepper_spacing=0
 GtkScale::value_spacing=0
 GtkScrolledWindow::scrollbar_spacing=0
 GtkExpander::expander_size=10
 GtkExpander::expander_spacing=0
 GtkTreeView::vertical-separator=0
 GtkTreeView::horizontal-separator=0
 GtkTreeView::expander-size=10
 GtkTreeView::fixed-height-mode=TRUE
 GtkWidget::focus_padding=0
}
class "GtkWidget" style "gtkcompact"
style "gtkcompactextra" {
 xthickness=0
 ythickness=0
}
class "GtkButton" style "gtkcompactextra"
class "GtkToolbar" style "gtkcompactextra"
class "GtkPaned" style "gtkcompactextra"

And yes! You will have a much pretier Eclipse and a real ungly gnome desktop! So lets rename the file to something like gtkrc4eclipse. Now instead of using the eclipse executable it self we will create a small bash script to execute it:

#!/bin/sh
export MY_ECLIPSE_GTKRC=/path/to/gtkrc4eclipse
export GTK_RC_FILES=$GTK_RC_FILES:$MY_ECLIPSE_GTKRC
export GTK2_RC_FILES=$GTK2_RC_FILES:$MY_ECLIPSE_GTKRC
export JAVA_HOME
/path/to/current/eclipse &

References

• Making Eclipse look good on Linux
• Make your eclipse look better on ubuntu

Inspired by the title of Douglas Crockford book, "Javascript, the good parts" and having to write an introduction on grails as a course for some colleagues, I thought it would be good to just blog about grails and the first steps some one should take.

Installation

Well the installation is not as easy as one double click but I can live with that and anyone that will have to deal with web application deployment should. That is because you will probably find harder things to deal with! The place to start is grails download page and the official installation instructions.

I assume that these steps are easy to follow. If not you should really reconsider your thoughts about the framework and web development. Not that you wouldn't be able to create a simple project but because you probably wouldn't understand a thing of what you were doing.

First grails project

We start with a

$ grails create-app firstgrailsproject
$ cd firstgrailsproject
$ grails run-app

Yes! this is it! You've got your first web application ready!!! You don't believe me? Just try to visit http://localhost:8080/firstgrailsproject and see it yourself.

The good parts

Ok! This is not what you had in mind but, it is a really big step. You have a servlet container with a lot of libraries and many other nice things ready waiting for you! That means that the framework not only provides you with a good development environment but also with a build system to use. Let's use it!

Before we start lets have some theory as a background. One thing you should care about is MVC. MVC is a pattern and is THE pattern when developing a web application and especially in java. We may not deal with java here directly but we do live in a java environment. What is MVC? MVC stands for Model View Control and lets have an example explaining it.

Lets say that a client needs something from the server. An xml, an html or whatever he may want. The server should be responsible for giving him what we want. This is where the developer (that should be you) have to do his job. Should check what the client needs and serve it. So you need a mechanism that will analyze what a client want retrieve the data and serve them in a form that the client asked for. This mechanism is called Controller.

We also mentioned the data. That is probably something we store in our database. Generally the data are never in a format that the client want so you give them in the format he wants. So the data is your Model and the format the View. We may have many views. For example an xml representation of the data or an html representation of the same data.

Grails is built with this pattern in mind. Ok this is another good thing! I also mention data here. Yes grails takes care of this too! It provides you with a ready to use database within the development environment. Another thing that you should not bother about until you'll reach the production phase.

The controller

As I said before, grails is build around MVC. So lets try to create a new Controller. By conversion all controllers have a name that ends with Controller and are placed inside grails-app/controllers directory. Let's create an echo controller.

$ grails create-controller com.valotas.firstgrailsproject.Echo

Don't forget that you are playing within a java environment and so, even if packages can be skipped, nobody will suggest such a thing. So lets have a look at our controller. Just open the generated groovy class/file with your favorite editor. That would be grails-app/controllers/com/valotas/firstgrailsproject/EchoController.groovy. As the name of the controller suggests, we would like to just return us a page writing the parameter phrase of our request. Lets edit our controller to do this exactly:

package com.valotas.firstgrailsproject

class EchoController {

  def index = { }

  def echophrase = {
    //if the request parameters has one named phrase use it's value
    //if not use the 'No phrase has been given' value for the phrase variable
    def phrase = params.phrase ? params.phrase : 'No phrase has been given'

    //return the model witch in our case is just a phrase
    return [message:phrase]
  }
}

This should do exactly what we want to. But before we continue we should also see how are we going to represent this data to the user

The View

So we must take care of the layout/representation or whatever you would like to call it, of the data we have. In this case we will use an html page. As you can see we returned a model within the echophrase action (that's how we call it from now on) in the EchoController. This is just a map who's key is the string message and the value of it is the value of the phrase variable

Now lets create a simple page that will display this message. All we have to do is create a new gsp file under grails-app/views/echo folder named echophrase.gsp. The above will be just fine:

<html>
  <head>
    <title>EchoController - echophrase</title>
    <meta name="layout" content="main" />
  </head>
  <body>
    <p>${message}</p>
  </body>
</html>

Now we can check if that works pointing our browser here: http://localhost:8080/firstgrailsproject/echo/echophrase?phrase=1,2,3 and here: http://localhost:8080/firstgrailsproject/echo/echophrase

Controller - View glue

Well, how did that happen? Again grails wires many things together in order to have this kind of magic without doing anything. The whole concept is that every action can return a model witch is just a map. When this happens grails looks for a gsp page having the same name with our action inside the folder witch have the same name with our controller, except the Controller part. All the views must be placed inside the views folder. That is why we've placed the echophrase.gsp inside the echo folder in the views.

This is what we should if we have to return html pages. As we've said before a model may have more than one representation. So we could just return the model as xml or as json. Of course our main concern is html pages, so in this case we will stick to the gsps. Note the returning an xml or a json is hard to do! It is easier as you don't even have to write a gsp template!

The Model: take our webapp to the next level: database

As we have seen in our example the model part is just a phrase a user passed to our controller as a parameter to it's request. This isn't the case of course in real web applications as in most cases we also have to do some interaction with the database. In order to do that we will also save the Echo phrase in the database and show the user a small history of the previous Echos.

To do that we must somehow tell grails what we would like to save to the database. We define this using the so called domain classes. For our example we would like our echos to have a date they have been submitted and the actual phrase the user has submitted. Domain classes are simple groovy classes inside the grails-app/domain folder of our application. Let's create one manually (by creating an appropriate file) or let grails do it for us: $ grails create-domain-class com.valotas.firstgrailsproject.Echo. Now just edit the newly created file:

package com.valotas.firstgrailsproject

class Echo {
  String phrase
  Date dateCreated

  static constraints = {
    phrase(size:1..100, blank:false)
  }
}

We defined what we would like our model to have. Note here that we would also like our phrases to have a size not bigger than 100 characters. We also defined that within the constraints.

Now let's add an action to also save the echos, and except from the last posted one display the previous ones ordered by the dateCreated descended.

def echoandlist = {
  //Get the list of the previous posted echos:
  def echolist = Echo.findAll([sort:'dateCreated', order:'desc'])

  def msg = 'Your echo has been saved' //The default message

  //Create a new echo and try to save it to the db
  def echo = new Echo(phrase:params.phrase)
  if (echo.hasErrors() || !echo.save()) msg = 'Could not save your echo'

  [message:msg, lastecho:echo, echolist:echolist]
}

One thing to to mention here is the hasErrors, save and findAll methods. These methods like many others are not defined to our class, but has been created thanks to groovy's AST. In other worlds grails provide us with a lot of utility methods. Three of these are the ones used here and I think that are self explained.

The last thing to do is to create our new view for this action. That will be something like the above

<html>
  <head>
    <title>EchoController - echoandlist</title>
    <meta name="layout" content="main" />
  </head>
  <body>
    <div class="${lastecho.phrase ? 'message' : 'errors'}">
      ${message}
      <ul>
        <g:eachError bean="${lastecho}">
        <li><g:message error="${it}"/></li>
        </g:eachError>
      </ul>
    </div>

    <g:if test="${lastecho.phrase}">
    <h2>${lastecho.phrase}</h2>
    </g:if>

    <h3>Previous echos:</h3>
    <ul>
      <g:each var="echo" in="${echolist}">
      <li>${echo.phrase}</li>
      </g:each>
    </ul>
  </body>
</html>

Again, lets check if that works pointing our browser to the following links: http://localhost:8080/firstgrailsproject/echo/echoandlist?phrase=1,2,3, http://localhost:8080/firstgrailsproject/echo/echoandlist?phrase=foo, and: http://localhost:8080/firstgrailsproject/echo/echoandlist

Yes it was that simple. We now save our echo to an inmemory database witch we then query to take all the Echos we previously saved and show them the user.

Productivity

As you can understand, grails not only provide us with utility methods within the domain class to validate it, but it also binds these errors to the domain class, and provide us with a custom tag to check the errors of a domain class within the view. Generally you can find a lot of utility methods and tags to use.

Except for those which make our life easier, grails provide us with a more exotic feature which pushed productivity to a next level. It is called scaffolding and it is the ability of grails to provide you with a simple yet very useful html based user interface. What exactly is this? If you are familiar with Domain Driven Development then this feature is your gateway!

Domain Driven Development

Generally when you create a new web application, the first thing to do is to write down all the dynamic features of the site. All the things that will get changed by the end users and not by the developer. All these features describes the domain of your web application. So a good domain description can get a good web application. And here when we say good, we mean a site that will be usable by the end user.

In our case we wanted to keep a track of the posted Echo phrases. In case of a Bookstore webapp, the first thing that you want is to get a track of the books available at the bookstore. Of course a book has a title but it also has an Author witch could also have written more books. So in this example you should also have an Author domain class witch could have many Books (so Book should be another domain class) each of them could have many other authors.

Scaffolding

As you can see, things are getting complicated. But not with scaffolding. Scaffolding help you get prototypes up and running by defining just the domain classes. Most of the tutorials of grails describes scaffolding. It something that shouldn't covered one more time but it is something the user should start playing only after knowing the basics witch I tried to describe here.

I've seen people who just get stack on how are they going to change something like the label of an input field of a scaffolded view. That's why I would suggest not to get experiment with scaffolding until you've understand the basic flow used to handle user's requests. So once you've done something the echo example you can start playing with scaffolding.

The idea behind scaffolding is that having domain described by your domain classes you will have to do some basic operations with them. These operations are Create Read Update Delete operations which are also known as CRUD operations. Things like that with todays tools are in most cases extremely easy to implement. But you still have to implement them and you have to do it for all your domain classes. A good programmer will consider having a helper class for this case but when you come to html form that things are getting more difficult. So why not we just have some kind of templates for such things that could easily get overwritten.

In that case all we need is the implementation of simple actions and its views. These actions are list, show, delete, edit, update, create, save and their views: create.gsp, edit.gsp, list.gsp, show.gsp. The views are not equal with the actions as some actions actually redirect the user to someother action. That is what scaffolding do. Generate boilerplate code that otherwise you should write. In my opinion this generated code is not written the best possible way and that is why I would suggest to use it only when prototyping a project.

Well, when I'm prototyping and generally developing with grails I do not want to use generate-all and just edit the generated controllers and gsps. I try to find my way overwriting maybe some actions but not edit them and some or more changes to the scaffold templates.

Setup

So after adding the basic domain classes I always end up doing a grails install-templates in order to start playing with the templates. In most cases this works just fine except from one case. File uploading is a little bit tricky. Most times (actually all the times) I wan't the uploaded files to be stored somewhere (not in the database) and just keep a record of the path of the file to the database. Consider something like the domain class below:

class UploadedFile {
  String filepath
}

The point here is to just use this class for the prototyping and then handle more "exotic" stuff like fileupload. Lets say that the time to handle this exotic stuff has came...

beforeInterceptor to the rescue

Grails has the ability to intercept calls to actions. All we want to do is before an action (save, and update in this case) just do the fileupload and replace the posted parameter with the path in order to be used by the action.

class UploadedFileController {
  def scaffold = true
  def beforeInterceptor = [action:this.&fileupload, only: ['save', 'update']]
 
  def fileupload() {
    def filepath //the path of the uploaded file
    // handle normal file upload as per grails docs
    params.filepath = filepath
  }
}

This is it! Isn't it? Well, it isn't! For some reason data Binding does not work witch means that we have to also edit the Controller template in order exclude the filepath and put it manually. So a the save action we replace the def ${propertyName} = new ${className}(params) with:

def ${propertyName} = new ${className}()
bindData(${propertyName}, params, [exclude: 'filepath'])
if (params.filepath) ${propertyName}.filepath = params.filepath

And in the update action replace ${propertyName}.properties = params with

bindData(${propertyName}, params, [exclude: 'filepath'])
if (params.filepath) ${propertyName}.filepath = params.filepath

And if you don't care about deleting the files when a UploadFile domain gets deleted you are done.

After having static resources get served right with Jetty within an OSGI environment you may want to be able to use groovy and more specific some groovlets

More or less the problem is the one that has been described at a previous post. Once again all we have to do is give some more "space" to the groovy.servlet.GroovyServlet to look for our groovlets

The code

package com.valotas.osgi.jetty.servlet;


import java.net.URLConnection;
import java.util.logging.Level;
import java.util.logging.Logger;

import groovy.servlet.GroovyServlet;
import groovy.util.ResourceException;


public class OsgiGroovyServlet extends GroovyServlet {
  private static final long serialVersionUID = 4017944232557220667L;
  private static final Logger logger = Logger.getLogger(OsgiGroovyServlet.class.getName());
  private static final String resourcesPrefix = "/webapp";


  @Override
  public URLConnection getResourceConnection(String path) throws ResourceException {
    if (logger.isLoggable(Level.FINER)) logger.log(Level.FINER, String.format("Trying to serve resource %s", path));
    try {
      return super.getResourceConnection(path);
    }
    catch (ResourceException e) {
      return getResourceConnectionFromOSGI(path);
    }
  }


  private URLConnection getResourceConnectionFromOSGI(String path) throws ResourceException {
    String actualpath = path;
    if (!actualpath.startsWith(resourcesPrefix)) {
      actualpath = resourcesPrefix + path;
    }

    if (logger.isLoggable(Level.FINER)) logger.log(Level.FINER, String.format("Trying to serve resourse %s (%s) from our osgi container", actualpath, path));

    try {
      return this.getClass().getResource(actualpath).openConnection();
    }
    catch (Exception e) {
      throw new ResourceException(String.format("Script %s couldn't be found", path), e);
    }
  }
}