Recompile The Move...

Ok it's hard to move across the ocean again and fix bugs but we have managed to fix some areas that were choking while recompiling: abstract classes + interfaces.

Still no departure date.

Params, Threads, Generics This is another progress demo. It involves: dynamically allocating params, genereics, threads, and switching between classes. Everything is handled transparently and effortlessly to the user. With such a small learning curve, ease of use, and blasting speeds the testing "Total Costs of Ownership" should be extremely low while attaining effectiveness never seen before. Enjoy your Sunday morning...

Smooth...

After cleaning up some bugs, a task that is still going, here is another sampler: a 500 thread session, with generics included, ran twice under 3 minutes from scratch.

That is generics type, tests + threads creation, and run. Everything completely synchronized.

General Dynamic Fixtures

This is the first work-in-progress report in a while. It shows how DevRiot dynamically generates, stores, and retrieves tests between sessions.

The "fixtures" are handled dynamically. We can mix and match the generics parameter types on-the-fly, keeping the engineers effort easy, flexible, and quick. Extremely quick.

Very few tools on the market can offer such a low Cost of Ownership, in the short or long run :-}

Enjoy...

Summer Blues Report...

I must admit that moving suites/tests from storage to binary form has been a challenge. It has become more difficult than DevRiot for Java because it has more features to take care of.

Generics Generation has posed its own set of binarization challenges.

But still snailing away...

Still working out the bugs...

...inside the generics engine to recompile the tests from storage. Not being able to ask questions makes the work slow down a lot.

Will have another demo soon.

The Referencer

Is still working on fixing bugs. The logic to implement new/old method matching is uncovering a lot of bugs. Not as close to beta as I thought or would like...

Generating Generics...

Stuck here. Trying to combine serialization/de-serialization of suites with generics and not generics is proving to have several traps...

Less than two minutes...

We have been tweaking around the serializer (getting rid of duplicate IO work) and writing, storing 100,000 tests takes less than 2 minutes. The development costs aren't why they used to be...

A more realistic approach to this feature can be: there is a method that takes a matrix and returns a modified version of it. Given the nature of matrices, a user could create a "base" test batch with a baseline matrix then modify the individual matrices with minimum effort.

Going academic!!!

It is always encouraging to see that not only businesses find the white paper useful. A lot of universities, institutes of technology from all over the world have been downloading the white paper (even for course work)

Even a national police from South America :-?

Well, the serialization server is moving forward nicely, although the weekend was lost with a clogged up laptop. It has been deleting *.tst file for the last 48 hours...

Storage...

I must have to say that developing code to serialize large number of objects can be quite time consuming.

My laptop seems to handle 100,000 or less files quite well. When the number jumps to 1,000,000, it becomes messy molasses to continue working.

Will keep you posted...

Gluttony...

Being able to handle large number of tests (5,000,000 in my dev machine, but it could be higher) is that serialization of suites becomes a little bit harder.

Well, that is what I am working on now...

5,000,000

After tweaking some generation logic to economize the memory consumption, DevRiot reached the 5 million tests mark in less than 34 seconds.

Sounds fast, isn't it? Now, really going back to suite serialization...

I tripled THAT...

1,500,000 in less than ten seconds. Back to Suite serialization...

500,000 and counting...

The Suite serialization is slowly starting. I am polishing some performance features.

But I wanted share with the latest readers some of the brute-force finesse of the tool:

500,000 tests, no coding, no wasted time. Enjoy.

It does not register

Well after having a question/post mysteriously disappear from the VSX forum, which is a first, I decided not to wait more on that.

Two registry entries will allow the user to gauge according their HW configuration what the max number of user/threads DevRiot can handle and the emergency TTL (when all current users/threads are harvested to avoid hosing down the box)

To be a product that has a really small exposure, there a lot of big copycats :-}

Time To Change...

Well after fiddling around with the load/stress engine + 1,600 threads/users for a while, I think it makes more sense to let users set the TTL per test.

It is easier for the user to set it according to their knowledge of the code than use any predefined formula.

It also becomes part of the verification data of the test which should be useful.

Thread Quartet in K reverse...

The following demo shows a Load/Stress session on a object using four of its methods X 400 threads/users (1600 total).

In addition, there is the Assertion, Out-Of-Scope, etc, in a codeless, easy-to-use, low-skills-needed sort of way.

It is a little bit longer than the previous ones but worth watching (it has been edited to avoid the initial setup of the threadpool) :-}

1,500 Threads in action

Well , we wanted to try a higher number on our load/stress engine. So, we tried 1,500. Watch and enjoy the execution times after the initial run (when all the setup occurs).

58 Seconds...

That is how long it takes to create, launch, execute, synchronize, and verify a load/stress session with DevRiot. Check it out for yourself...

Why wouldn't you want that level of efficiency in your SDLC? :-}

Visual Threading!!

This is another progress demo with assertions, out-of-scope verifications, multiple threads were involved in a series of tests. Not a single line of code was written.

Try to achieve the same with other technology: Microsoft's, etc. See how much time it takes you to achieve the same. Then check how much more time it takes to get the results back.

How do you want to spend your time and budget? :-}

VUT Threads...

This is almost identical to the previous demo but with the difference that the Load/Stress testing engine is hooked up.

Note that the video follows the same script as the previous one, so no PrivateObject API or coding or scripting. Even the thread creation, execution, and harvest.

The other difference is that we intentionally tried to add a test without pre-compiling a parameter so the tool could complain :-] Three threads are created and used against the method under test.

Automation?

We are currently working on the ContextMenu manager. It should be a breeze since the Generics support actually meant a major addition of functionality.

GUI support should come soon too, its major piece of functionality is ready to wire.

Which lead me to look at the Automation library that MS has on .Net 3.5-VS 2008. It is not a surprise , and not a particularly interesting competitor.

First is stuck in PC land. None of the major classes are supported by the Compact Framework.

Second, it is another hard to digest class/method behemoth.

I guess they are not competing on usability and product definition/differentiation...

Compare me if you can...

These two are a couple of progress snapshots from the Generics Engine.

To show the flexibility and ease-of-use of DevRiot's Visual Unit Testing(tm) technology here we have a generic method that takes 3 generic parameters. Each one of them is a bound generic type A made of an array of bound generic type B. Type B is made of an array int.

Just a few clicks away. No Source code...

That ugly description turns into an easy to use tree widget:

General Complexity

The Generics Generator is pretty smooth now:

This is an snapshot of a that involves: generic method, out parameter, multidimensional arrays, out of scope data (PrivateObject in VS parlance).

And it is the same as if it a method that takes and returns an integer. The user just spends time on what it matters most: substance and not boiler plate noise.

None of the competitors can achieve such a level of usability and ease-of-use. Let alone the execution speeds. We are pretty close to wire up the GUI testing engine...

GG:

Well, shortcuts do not always cut or are short. We need to frame a Generics Generator that makes the trick in the long run and not a longcut that comes to bit us later when it is hard to touch the innards

This is where we are now...

Anything VUT...

Visual Unit Testing, this is the acronym that best describes the visual nature of DevRiot's user experience. It also applies to the GUI testing engine but there are more salient features that describe DevRiot's GUI testing engine: device independence and ditching the capture-and-replay paradigm.

Back to ironing out some bad bugs in the generics generator logic...

Params Row...

The thingy handling the params modifier is pretty stable now. There is a scenario where it NPE's but I am about to fix it. It is a little bit convoluted because it involves also the dynamic-scoping of parameters.

Next will be databases (offering a clean way for the user to test databases). Collections/Data structures will be partially supported for the beta. The GUI testing engine work will begin soon, hello .Net Compact we are going your way!!

DevRiot for Eclipse NOT available

Since last saturday, we have seen several failed attempts to dowload the beta version of DevRiot for Eclipse. It is not available until further notice. I guess the links do not have the same note as the html page. We apologize even though is beyond our control.

We suggest waiting until the improvements and features developed in the .Net version spill over to the Java version. You can register for both here.

Thanks for your patience.

For Completeness' sake...

I just compared DevRiot for .Net against Nunit 2.4.3. DevRiot is 2,400+ times faster than Nunit.

And it includes GUI and Load/Stress testing, try to find those inside Visual Studio or Nunit :-}

The waves are just getting started. I'll try a two session day tomorrow :-}

The Fog of Fall...

Well after the dry and brown fall we have had so far (Galicia is like Seattle/Washington, rainy and green all year long) Some fog and rain has arrived last week, and with them some nice fall swells which I plan to take advantage.

As I mentioned in an earlier post, I managed to compare DevRiot .Net with Visual Studio 2008 Tools.

The performance of DevRiot .Net is really good, given we do not have access to any of the internals: on the low end DevRiot .Net is between 800-900 times faster than Visual Studio tools.

The time it takes Visual Studio to process the first test, DevRiot is close to 1,000. But if we reach the 100,000 tests, DevRiot outperforms Visual Studio tools nearly twice that figure.

As any sensible management guy would tell you: why waste time writing code to get slower?

I will be working on the GUI testing engine, particularly support for the .Net Compact Framework.

And, I will try to catch some waves. Enjoy...

Tick, Tick, Tick, Tick, Tick...

This has been a really productive week. I accomplished to run DevRiot for .Net under the new architecture for the first time.

Let's say I will hold off until early next week to announce the performance improvements...

Simple Update...

Well, just working in the guts of the engine now. Where the fireworks occur. No snapshots to show but really entertaining to fiddle with.

Still undecided of how to cache things for the compact framework devices but making great progress.

Context Adjustments...

Some times it is amazing where pointers to adjust features can come from.

And I have to say that MS seems to be on the ball supporting the extensibility community (not related to previous line)

Anyways, DevRiot's UI has changed a little bit to make things easier on inherited methods,fine tunning runs and having multiple instances of VS up.

We wanted to get this stable before get into the gui testing engine for .Net compact and device integration. It is going to be fun :-}

P.S. The second "Run Current Tests" really runs the whole suite.

Tell me what posts one is reading...

And I will tell you what one's marketing plan missed :-}

Well, Leopard came out. It may have solved some of the SWT/AWT problems that DevRiot was experiencing on OS X. I need to get back to it later. Some people got really upset that Java 6 is not included or better java support on OS X is demanded.

It is understandable but things should get less emotional.

After all, not long ago Apple was in the doldrums. It certainly came back but it is behind every big player in the Java world. And it is not going to let IBM or Sun have a piece of the Apple pie out of goodwill.

There are a lot more Java/IBM developers than OS X developers. That in itself is a major threat.

Well, I am going back to .Net and make sure DevRiot does not choke when several instances of Visual Studio are up and running...

Context it!!

We have done a little backtracking with regards of the original way we planned to handle assertions. Interfaces and abstract classes will be handled the same way: context strip.

Here is goes another snap