What an eye sore!! (102,796)

Well, I finally managed to download all the components for Visual Studio 2008. It was long a process because of the size of the archives involved. Regardless, I have to say that the process was smooth.

Now to the substance, comparing DevRiot and the test tools of VS 2008.

I started with the basic: take an int and return it.

Nothing else fancier where DevRiot has advantage over VS: gui testing (on PC and devices, on and off screen), dynamic array/data structure manipulation, dynamic build test tress, built-in load and stress testing, etc :-}

It is an eye sore: test projects, test types, test attributes, test contexts, asserts, clicks, clacks to get to the point of to modify the test source to add the only thing that is of interest to the user: desired input and expected output. That is all a user needs. If the user needs to add a second test: repeat the process :-}

The thing is dog slow. It took (on average) 102,796 times more ticks than DevRiot. Granting that I was using Virtual PC but the times I am using for DevRiot are from 4 years ago on much slower machine than the current one I own.

If we assume a 50 times handicap (meaning that a 5000% improvement) DevRiot is still around 2000 times faster.

No wonder the execution time report is not the default one.

I tried to mess MS tool with changing return types, it choked like a child. Then I tried exceptions or even setting scenarios. It would have required getting into coding, api, etc..

I do not think they have clear concept of what automation means.

So, bottom line: why spend more time using MS QA infrastructure if it is not going to yield faster development cycles?

The pricing structure and road map discussion, I will leave for other post.

Beta FAQ 2:

For those research units interested in what is coming down the pipe:

*) The .Net beta will include Gui-Off: that is the ability of running the gui execution engine off the screen to increase throughput.

*) The Test Writing Engine will not be included in yet. We need field-proven execution engines before design on the writing engine continues.

*) The eclipse/java version will be overhauled after the .Net beta is available: it will become an Eclipse feaure, etc.

DevRiot .Net gets going, Beta for Eclipse almost available!!

3 People got their hands on DevRiot for Eclipse 3.X (Windows version) meanwhile I decided to get the .Net version a jolt.

Here is a snapshot of the GUI in the .Net environment. Pretty much the same as in Eclipse/Java. That's the idea: no new tricks to learn, no new tests to write. They are portable right off the start.

The Proof is in the pudding...

This is a snapshot of DevRiot running on Eclipse 3.3M3 and Java 1.5. Note that the engine executed 100 tests and all of them failed. Despite the extra effort of reporting the failures, the little engine still needs a good set of brakes:-}

We are getting closer to the point where we can release this eclipse/java beta. We do want to move on to the VS/.Net beta because the performance gains here a orders of magnitude bigger than in Java.

Stay tuned...

A shorter update...

Still wiring the gui, unit, and activation logic. The activation logic will work like this, or close enough:

When a user clicks on the DevRiot button if the currently selected file is a source file then it continues, if the cursor is NOT inside a method then the tests are run. If the cursor is INSIDE a method, the form (that is on the screenshots) reconfigures itself and lets the user add new tests. When the popup is closed then the tests are run...

Back to coding :-}

Short Update...

Still rewiring. Decided, for the Eclipse plug-in beta, to support Eclipse 3.2 (for Windows & Linux) and 3.3 M3 (for Mac OS X if their workaround for a bug I opened is good)

It may be time to get the VS .Net add-in up to speed :-}

Stay tuned...