Estimate your skills!

Every developer wants to estimate his skills. One way to do it is to ask a test task for applying some job, even if you are not actually going to apply. We suggest you a simple way. The following is our test task. You can try to solve it and understand: are you ready for working at Codeborne - even if you are not going to. We typically use Java or Scala, but you can choose any language this time. This task is neither too complex nor simple, it's just an example of our everyday job. So, the task is to compute electricity day- and night- consumption of a client.

• Given client's per-hour electricity consumption for some month.
• Goal is to compute total day- and night- consumption.
• Hours are enumerated from 1 starting from beginning of month. Day hours are 7-23 at working days (8-24 at daylight saving time or summer). All other hours are night hours.

Example for January 2012:

• 1 - 2kW (hour #1 is January 1, Sunday, 00:00 - 01:00 - NIGHT)
• 2 - 3kW (hour #2 is January 1, Sunday, 01:00 - 02:00 - NIGHT)
• 26 - 3kW (hour #26 is January 2, Monday, 01:00 - 02:00 - NIGHT)
• 32 - 6kW (hour #32 is January 2, Monday, 07:00 - 08:00 - DAY)
• ...
• 744 - 0kW (hour #744 is January 31, 23:00 - 24:00)

In this case, night consumption is 8 = 2+3+3 and day consumption is 6.

POST YOUR SOLUTION and get feedback in English, Estonian or Russian.
It's anonymous. It's free. It should be fun.

PS. Veiko and Tauno, we have lost your email addresses. Please repost them to get feedback.

SoapUI Tests with Ant+Ivy

Do you like quest games? Everybody likes!
Today we had to play one. It's name was "run SoapUI tests with ANT script".


SoapUI is de-facto number 1 tool for testing SOA services. It's yound enough to have Maven 2 plugin, and it's old enough to have Maven 1 plugin. But we are not lucky enough: SoapUI doesn't have Ant task.

This article shows how to run SoapUI tests from Ant script.

We suppose you are already using Ivy.
So, let's begin.

Ant script

Ant task is simple enough:

<target name="test">
    <mkdir dir="test-results"/>
    <java classname="com.eviware.soapui.tools.SoapUITestCaseRunner" errorproperty="tests-failed" fork="yes" dir="test-results">
      <arg line="-j -f${basedir}/test-results"/>
      <arg value="-t${basedir}/soapui-settings.xml"/>
      <arg value="${basedir}/MY-SMART-soapui-project.xml"/>
      <classpath>
        <fileset dir="lib" includes="*.jar"/>
      </classpath>
    </java>

    <junitreport todir="test-results">
      <fileset dir="test-results">
        <include name="TEST-*.xml"/>
      </fileset>
      <report format="frames" todir="reports/html"/>
    </junitreport>

    <fail if="tests-failed"/>
  </target>

Ivy configuration

Next, you need to add corresponding dependencies to ivy.xml:

<dependencies defaultconf="default->default">
    <dependency org="junit" name="junit" rev="4.10+" />
    <dependency org="eviware" name="maven-soapui-plugin" rev="4.0.1" />
    <dependency org="net.sf.jtidy" name="jtidy" rev="r938+"/>
    <exclude org="jtidy" module="jtidy"/>
  </dependencies>

And finally, you need to add eviware.com repository to your ivysettings.xml:

<ivysettings>
  <settings defaultResolver="default"/>

  <resolvers>
    <ibiblio name="public" m2compatible="true"/>
    <ibiblio name="eviware" m2compatible="true" root="http://www.eviware.com/repository/maven2/"/>
    <chain name="default" returnFirst="true">
      <resolver ref="eviware"/>
      <resolver ref="public"/>
    </chain>
  </resolvers>
</ivysettings>

Yes, eviware should come first, because it overrides some artifacts from the central repository, e.g. javax.jms:jms.

SoapUI settings

Typically all tests have some configuration parameters. It's common to declare them in SoapUI's global properties. Fortunately, these can be stored in VCS in file soapui-settings.xml:

<?xml version="1.0" encoding="UTF-8"?>
<con:soapui-settings xmlns:con="http://eviware.com/soapui/config">
  <con:setting id="GlobalPropertySettings@properties"><![CDATA[<xml-fragment xmlns:con="http://eviware.com/soapui/config">
  <property xmlns="http://eviware.com/soapui/config">
    <name>MY_serverUrl
    <value>http://my-dev-server.chucknorris.com:8080
  </property>
  <con:property>
    <con:name>MY_dbConnectionString
    <con:value>jdbc:oracle:thin:chuck/donteventry@dev.chucknorris.com:1521:dev
  </con:property>
</xml-fragment>]]></con:setting>
</con:soapui-settings>

Running

Execute ant test and be prepared for download ~60 MB of jars. I have no idea why SoapUI needs so much (for example, why it needs javax.jms?), but that's Java, guys. After running, you will find JUnit-style html reports in folder reports/html. And several log files in the test-results folder.

Troubleshooting

Missing dependencies

Since eviware repo overrides some artifacts from the central repository, you can get in trouble if some artifacts are already cached in your local repository. For example, we have get this problem with javax.jms:jms artifact.

[ivy:retrieve] :::: WARNINGS
[ivy:retrieve]   [NOT FOUND  ] javax.jms#jms;1.1!jms.jar (0ms)
[ivy:retrieve]  ==== public: tried
[ivy:retrieve]    http://repo1.maven.org/maven2/javax/jms/jms/1.1/jms-1.1.jar
[ivy:retrieve]   ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve]   ::              FAILED DOWNLOADS            ::
[ivy:retrieve]   :: ^ see resolution messages for details  ^ ::
[ivy:retrieve]   ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve]   :: javax.jms#jms;1.1!jms.jar
[ivy:retrieve]   ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve] 
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

Workaround is simple:

rm -fr ~/.ivy2/cache/javax.jms/jms


Conclusion

Though SOA concept is offspring of the devil, SoapUI is a great tool for testing this, and automated testing is a great practice. Putting them together is a good level for programmer's quest game. Maven repository is inevitable beast of satan at the end of level. Be patient and kill this boss.

MobileId and Selenide demo in Devclub

Codeborne has presented its open-source projects MobileID and Selenide in Devclub.

We did a demo application showing how to use both libraries:

mobi.codeborne.com/

Project source is available in github.


Video (in Estonian) is available ar devclub.ee:

devclub.ee/lugu/video/open-source-codeborne-andrei-solntsev-ja-anton-keks

Feel free to try both libraries and give your feedback!

Why IDEA is better than Eclipse

At Codeborne we use Intellij IDEA as a Java IDE. Yes, we pay money for IDEA Ultimate Edition, and we beleave it's worth it.

This article describes why we think IDEA is better than Eclipse. Shortly, it's because IDEA is a really intelligent tool that helps us focus on a problem. This is explained in details below with screenshots and code snippets.

Enjoy reading.

Holy war

There are "eternal" questions that have no single correct answer. E.g. what is better: Windows or Linux, Java or C#; who is stronger, Chuck Norris or Van Damme.
One of such holywars is selection of the best Java IDE:

VS

There is a lot of disputes on this topic discussing which of them has more plugins, shortkeys and so on. There is so many differences that it's difficult to decide what of them are the most important. As a result, people typically claim that both IDEs are equal in their capabilities, and choosing of one of them is a matter of taste.

I claim that it is not just a matter of taste. There are objective reasons why

Intellij IDEA (as a Java IDE) is definitely better than Eclipse.

I am not going to compare lot of tiny differences like plugins, shortcuts etc - there is a plenty of blog posts on this topic, like this one: [part 1] [part 2] [part 3]. I am going to show just one major difference between IDEA and Eclipse. Typically, neither IDEA nor Eclipse fans know it. The first ones used to it and cannot imagine that IDE can miss it. The second ones used to live without it and cannot imagine that IDE can do such things. Moreover, Eclipse users typically don't notice this difference when trying IDEA for fun, just because they used to work "in Eclipse mode".

Why am I so confident?

Before I start, let me explain who I am and why is my opinion worth listening to.

I've been working on Eclipse for 5 years, I knew it very well, I wrote plugins for it and sincerely loved Eclipse. Then I switched to another company, and was forced to move to IDEA. After surviving a serious break-up, I finally realized why is IDEA steeper. And now I have been working on IDEA for a couple of years. So I know both IDEs well and can compare them.

The main difference between IDEA and Eclipse

The main reason why IDEA is steeper is: IDEA feels context. That's what JetBrains employees mean when they name IDEA intelligent. What it really means? IDEA indexes the whole your project, analyses everything it has, and even builds the syntax tree. Thanks to it, at any time, wherever you put the cursor, IDEA knows where you are and what can be done there.

Still not clear? Do not worry, it will become clear in the following examples.


This ability to understand the context is expressed in many, many ways, here are just a few.

1. Debugging

As a part of debugging process, we often want to evaluate some expression to see its value. In Eclipse you need to select this expression. It's important to exactly select the whole expression, otherwise Eclipse cannot evaluate it. Now you press Ctrl+Shift+I and see the value of expression.



With IDEA you don't need to select anything. You just put cursor at any place inside your expression (at method hasAttribute in given case) and press Alt+F8. IDEA understands which expression you probably need and shows a dialog window suggesting several possible variants for your expression. You can also edit and immediatelly evaluate the expression in this dialog. Very convenient! After trying this feature, you just cannot debug in Eclipse anymore.

It turns out that both IDEs, in principle, allow you to do the same thing. But with IDEA it's much easier and faster. I'm serious, the difference is huge - it's just Heaven and Earth. In this small dialog window IDEA will provide autofilling, syntax highlighting and everything you need.

2. Autocomplete

Autocomplete is what distinguishes any IDE from the notepad. In this area feeling the context gives IDEA a qualitative advantage. For example, we started writing a line of code:

assertElement(By.id("errorMessage"), vi

and now we want to find what options do we have: what can start with letters "vi".

What IDEA does? Not waiting for any keystrokes, it immediatelly understands that method assertElement wants an Condition class instance as a second parameter, and there is a static variable in class Condition with name visible. And IDEA immediatelly suggests the only valid option:


And what Eclipse does? Alas, it does not understand the context. It does not know that the cursor is located where the second parameter of the method assertElement should place. So when you press the sacred Ctrl + Space, Eclipse simply shows everything in Universe that begins with the letters "vi":


In a beautiful pop-up window we see a lot of pretty-highlighted well-documented useless information. :(

3. Refactoring

Professional programmers are able to be productive using refactorings provided by their IDE. All modern IDE offer a very impressive set of refactorings. But then again, IDEA refactorings are intelligent. They realize what you want, and offer different options that are suitable for most situations.

For example, suppose we have a method assertErrorMessageIsHidden:

public void assertErrorMessageIsHidden() {
    assertElement(By.id("errorMessage"), Condition.visible);
  }

And we want the String "errorMessage" to come as a parameter to this method.

Let's start from IDEA. Put the cursor to any place inside the string "errorMessage", press Ctrl+Alt+P (meaning "parameter"), and IDEA suggests what expression we probably could extract to a parameter:


As soon as the expression "errorMessage" is selected, IDEA suggests several possible names for this parameter.

Wow every month

You will be surprised many, many times at how smart behaves IDEA, offering options for the name of something. It takes into account method name, variable type and even value, and names of such variables in other places, and those names that you gave such a variable before - except that your zodiac sign is ignored. Believe me, you will say: "Wow, IDEA is able to do such thing too?" at least once every month.

And now let's look that Eclipse suggests.

Do not forget: select the entire expression "errorMessage" (always with quotes, or get a wild message "An expression must be select to activate this refactoring"), choose a refactoring "Introduce parameter" (from the menu, there is no hot key) and get about the same result. However, Eclipse doesn't suggest options for a parameter name, but thanks for that.

Conclusion

If we are choosing a Java IDE then Intellij IDEA is definitely better than Eclipse. It's not just a matter of taste. IDEA is objectively better. It lets you to quickly and easily write and change the code, suggests appropriate names, finds the appropriate methods. It does not require you to exactly select the expressions, but guesses what you want to do and how you wanted to name it. IDEA anticipates and suggests.

P.S. Remarks

I claim that IDEA is better Eclipse just as a Java IDE. If you are considering them in some other capacity - such as the IDE for other languages ​​(C++, Python, Scala), or as a platform for building desktop application, Eclipse may very well win.

Actually it follows from their definition. Eclipse has positioned itself as an abstract platform for building anything with plug-ins, and IDEA is positioning itself as a "intelligent IDE for Java". So it is.

For the sake of interest I will try to name some aspects in which the Eclipse is probably better than IDEA:

• Eclipse looks better (thought, this is a matter of taste). All this fuss about SWT and native controls was worth it. Eclipse looks like a solid well thought-out application with pretty fonts and icons, while the IDEA at first glance looks like a stunt toy with a lame hack icons and awkward Swing interface.
• Eclipse has a richer project structure support. In IDEA you have a project consisting of modules. In Eclipse you have a workspace consisting of projects which can be closed/opened, grouped and hidden. But do you actually need it?
• It seems to be easier to write plugins for Eclipse.
• IDEA uses more resources (memory) than Eclipse. This actually makes sense - because he knows so much.
• And finally, Eclipse is free. However, IDEA Community Edition is just enough for me.

My final recommendation

If you need pretty icons, platform for creating desktop applications, IDE for C++, or you work on an old weak laptop, then Eclipse is problably better choice for you. But if you are a serious Java developer, and you need a fast and convenient tool that helps you focus on the problem instead of distracting, then the IDEA this is exactly what you need.

I hope one more holywar is over today.

Develop with pleasure!

Codeborne enters the open-source world

Recently Codeborne started two open-source projects. Today we finally uploaded binaries to the central Maven repository, so now everyone can use them just by declaring dependency:

Mobile-ID

Mobile-ID is a client library for Java and Scala - you just need to write few rows of code to use Estonian Mobile-Id authentication.

<dependency org="com.codeborne" name="mobileid" rev="0.9.1"/>


Selenide

Selenide is a library for easy using of Selenium WebDriver for automated tests in Java.

<dependency org="com.codeborne" name="selenide" rev="1.0" conf="test->default"/>


Follow github links for more documentation, examples and source code.

Both libraries are already used in real projects, so be sure that they work.
We are sincerely waiting for your critics, suggestions and bug reports.

Enjoy!

Reducing memory usage of Scala/Scalate web application

We have a couple of web applications written in Scala using the Scalatra web framework from LinkedIn. A good option with Scalatra is to use Scalate as a template engine. We use SCAML syntax, which is a scala-based HAML.

Anyway, the problem is that Scalate uses scala compiler to compile templates on-the-fly. This is perfect for development (you change the file and just push reload in the browser), but is very memory-hungry on production: scala compiler runs several threads and caches lots of stuff in the heap as well as all reloaded templates consume permgen - the memory that is never freed by garbage collector.

To reduce memory usage and increase first request performance, it makes sense to precompile your templates. Unfortunately, Scalate user guide only provides some hints on how to do it with Maven (there is a plugin) and SBT (there is also a plugin). We prefer to use Ant for our builds, so there is no a ready-made solution.

Actually, as part of the research process I tried to switch to SBT, but found it a bit too difficult to customize - it uses the same horrible directory layout as Maven by default, but we are very happy with out much simpler approach. I actually got SBT working with out project, but the precompile plugin mentioned in the user guide is built for an older version of SBT and doesn't seem to be supported by the author anymore.

The solution was to figure out a lightweight way of precompiling the templates. After examining sources of Scalate and experimenting, I came out with this:

object TemplatePrecompiler {
  val workdir = new File(System.getProperty("scalate.workdir", "tmp"))

  def main(args: Array[String]) {
    val precompiler = new org.fusesource.scalate.support.Precompiler
    precompiler.sources = Array(new File("webapp/WEB-INF/"))
    precompiler.workingDirectory = workdir
    // precompiler.targetDirectory - set this to your WEB-INF/classes 

    // path if you are packaging a war file
    precompiler.execute()
  }
}

Fortunately, the Precompiler class is already included in the scalatra-core jar.
The biggest trick here is to make sure the precompiled templates get into the correct place where classloader will look for them. Note the use of scalate.workdir system property. Do not leave this unset - or it will generate a new directory each time under the /tmp. Set this to some writeable directory that doesn't change for this application. We just use "tmp" under the application directory, just like Play Framework does.

Your precompiled classes will be put under tmp/classes and generated scala sources to tmp/src. You can ignore the sources - they are not needed when Scalate tries to load precompiled templates. It looks for them first in scalate.workdir/classes (tmp/classes in our case) and then your normal Webapp classloader, so you can also put them to WEB-INF/classes.

We then run this precompiler class from Ant at the end of the build with java ant task.


The result - decreased memory usage by about 300 Mb, from 400-500 to about 130 Mb.

How to use Mobile-ID in Python

Mobile-ID (Mobiil-ID) is a personal mobile identity in Estonia and Lithuania, provided by an additional application on a SIM card. The good thing is that it is backed by government and provides the same level of security for authentication and digital signatures as a national ID card without the need of having a smart card reader.

So, while thinking on adding Mobile-ID authentication to our era.ee free domain service, I have came up with this Python code, which is incredibly simple.

Replace +372xxxxxx with your own phone number for testing and register it at openxades.org before running the code.

from sys import exit
from suds.client import Client
import time
import logging
logging.basicConfig(level=logging.INFO)

url = 'https://www.openxades.org:8443/?wsdl'  # test env
client = Client(url)

mid = client.service.MobileAuthenticate('', '', '+372xxxxxx', 'EST', 'Testimine', 'era.ee login', '12345678901234567890', 'asynchClientServer', 0, False, False)
if mid.Status != 'OK':
    print mid.Status
    exit(1)

print 'Challenge: ' + mid.ChallengeID

status = 'OUTSTANDING_TRANSACTION'
while status == 'OUTSTANDING_TRANSACTION':
    time.sleep(2)
    status = client.service.GetMobileAuthenticateStatus(mid.Sesscode, False).Status

if status != 'USER_AUTHENTICATED':
    print status
    exit(2)

print 'Authenticated: ' + mid.UserGivenname + ' ' + mid.UserSurname + ', ' + mid.UserIDCode

Don't forget to install the SUDS Python library that does the magic of creating SOAP requests (apt-get install python-suds on Debian & Ubuntu)