The Referencer

Is still working on fixing bugs. The logic to implement new/old method matching is uncovering a lot of bugs. Not as close to beta as I thought or would like...

Plugging the plug-in

We are paying the error of working while sick on a key piece of code regeneration. Bad code, or at least worse than usual, that needs rewriting.

But it also has given us the opportunity of thinking about deployment.

We are toying with the idea of adopting "perpetual delivery" mode. Since the engine is not larger than 150K, downloading it can part of the add-in setup.

Generating Generics...

Stuck here. Trying to combine serialization/de-serialization of suites with generics and not generics is proving to have several traps...

Deploying the report,

Still making a lot of progress with suite recompilation and integrity.

Also some time has been spent on figuring out how to deploy the tool while minimizing the IP exposure.

There is VS Gallery and other channels. Obfuscation is a good deterrent but it needs help from other angles...

The S Option

Given the enormous amount of tests can be generated with DevRiot, our old serialization architecture is proving not up-to-par.

SQL seems to be most logical way to handle this.

Well keep you posted. Sorry for the inconvenience!

Less than two minutes...

We have been tweaking around the serializer (getting rid of duplicate IO work) and writing, storing 100,000 tests takes less than 2 minutes. The development costs aren't why they used to be...

A more realistic approach to this feature can be: there is a method that takes a matrix and returns a modified version of it. Given the nature of matrices, a user could create a "base" test batch with a baseline matrix then modify the individual matrices with minimum effort.

Going academic!!!

It is always encouraging to see that not only businesses find the white paper useful. A lot of universities, institutes of technology from all over the world have been downloading the white paper (even for course work)

Even a national police from South America :-?

Well, the serialization server is moving forward nicely, although the weekend was lost with a clogged up laptop. It has been deleting *.tst file for the last 48 hours...

Gluttony...

Being able to handle large number of tests (5,000,000 in my dev machine, but it could be higher) is that serialization of suites becomes a little bit harder.

Well, that is what I am working on now...

It does not register

Well after having a question/post mysteriously disappear from the VSX forum, which is a first, I decided not to wait more on that.

Two registry entries will allow the user to gauge according their HW configuration what the max number of user/threads DevRiot can handle and the emergency TTL (when all current users/threads are harvested to avoid hosing down the box)

To be a product that has a really small exposure, there a lot of big copycats :-}

Thread Quartet in K reverse...

The following demo shows a Load/Stress session on a object using four of its methods X 400 threads/users (1600 total).

In addition, there is the Assertion, Out-Of-Scope, etc, in a codeless, easy-to-use, low-skills-needed sort of way.

It is a little bit longer than the previous ones but worth watching (it has been edited to avoid the initial setup of the threadpool) :-}

Almost Context Full...

The context manager is 60% done. 4 snapshots of the context manager at work: assertions, cleaning up, etc.

Short...

Now the generics engine can handle pretty much everything it is thrown at it.

The first ever Visual Unit Testing tools is coming your way...

General Complexity

The Generics Generator is pretty smooth now:

This is an snapshot of a that involves: generic method, out parameter, multidimensional arrays, out of scope data (PrivateObject in VS parlance).

And it is the same as if it a method that takes and returns an integer. The user just spends time on what it matters most: substance and not boiler plate noise.

None of the competitors can achieve such a level of usability and ease-of-use. Let alone the execution speeds. We are pretty close to wire up the GUI testing engine...

Anything VUT...

Visual Unit Testing, this is the acronym that best describes the visual nature of DevRiot's user experience. It also applies to the GUI testing engine but there are more salient features that describe DevRiot's GUI testing engine: device independence and ditching the capture-and-replay paradigm.

Back to ironing out some bad bugs in the generics generator logic...

Params Row...

The thingy handling the params modifier is pretty stable now. There is a scenario where it NPE's but I am about to fix it. It is a little bit convoluted because it involves also the dynamic-scoping of parameters.

Next will be databases (offering a clean way for the user to test databases). Collections/Data structures will be partially supported for the beta. The GUI testing engine work will begin soon, hello .Net Compact we are going your way!!

Back on line!!

It arrived! After several weeks of waiting the snail mail to deliver the laptop converter. It arrived.

It was kind of an experience trying to order the spare part from here having bought the laptop in USA.

It would have been great if the surf had been less chaotic but winter storms can be messy around here.

Anyway, it is time to get back on track with the .Net beta :-}

Time Report...

Still working on how out/ref parameters are handled. We have been able to reduce the size of the execution engine by a good chunk. The goal was to make it small enough for the Compact Framework/J2ME.

But this means that the parameter processing needs some massaging. Out, ref, params can lead to some more spaghetti code than really needed.

Stuck here before fleshing out the widget handling logic (GUI testing engine)...

For Completeness' sake...

I just compared DevRiot for .Net against Nunit 2.4.3. DevRiot is 2,400+ times faster than Nunit.

And it includes GUI and Load/Stress testing, try to find those inside Visual Studio or Nunit :-}

The waves are just getting started. I'll try a two session day tomorrow :-}

The Fog of Fall...

Well after the dry and brown fall we have had so far (Galicia is like Seattle/Washington, rainy and green all year long) Some fog and rain has arrived last week, and with them some nice fall swells which I plan to take advantage.

As I mentioned in an earlier post, I managed to compare DevRiot .Net with Visual Studio 2008 Tools.

The performance of DevRiot .Net is really good, given we do not have access to any of the internals: on the low end DevRiot .Net is between 800-900 times faster than Visual Studio tools.

The time it takes Visual Studio to process the first test, DevRiot is close to 1,000. But if we reach the 100,000 tests, DevRiot outperforms Visual Studio tools nearly twice that figure.

As any sensible management guy would tell you: why waste time writing code to get slower?

I will be working on the GUI testing engine, particularly support for the .Net Compact Framework.

And, I will try to catch some waves. Enjoy...

Tick, Tick, Tick, Tick, Tick II

Well, I just tried VS 2008 testing tools.

Ahh, what a way to get it wrong from marketing to development.

I will continue working on the GUI testing engine and device support for the Compact Framework. I will keep the performance figures for later...

Tick, Tick, Tick, Tick, Tick...

This has been a really productive week. I accomplished to run DevRiot for .Net under the new architecture for the first time.

Let's say I will hold off until early next week to announce the performance improvements...

Simple Update...

Well, just working in the guts of the engine now. Where the fireworks occur. No snapshots to show but really entertaining to fiddle with.

Still undecided of how to cache things for the compact framework devices but making great progress.

Context Adjustments...

Some times it is amazing where pointers to adjust features can come from.

And I have to say that MS seems to be on the ball supporting the extensibility community (not related to previous line)

Anyways, DevRiot's UI has changed a little bit to make things easier on inherited methods,fine tunning runs and having multiple instances of VS up.

We wanted to get this stable before get into the gui testing engine for .Net compact and device integration. It is going to be fun :-}

P.S. The second "Run Current Tests" really runs the whole suite.

Slow drip...

Sometimes things happen in a way that makes you wonder.

I have not worked on DevRiot's eclipse plug-in in ages. Obviously, I am concentrating on .Net since Visual Studio 2008 is going to be released soon.

But the funny thing is that the demo of DevRiot running the GUI testing engine on J2ME has become pretty popular this month.

How information flows in the business world is amazing!

Needless to say, it is going to be a lot easier to implement the GUI testing engine for .Net than for J2ME. Everyhting is already in the IDE (VS 200x) On the other hand getting the same device support on Eclipse/J2ME is like secretively impossible. Back to work...

Context it!!

We have done a little backtracking with regards of the original way we planned to handle assertions. Interfaces and abstract classes will be handled the same way: context strip.

Here is goes another snapshot of the work-in-progress:

What an eye sore!! (102,796)

Well, I finally managed to download all the components for Visual Studio 2008. It was long a process because of the size of the archives involved. Regardless, I have to say that the process was smooth.

Now to the substance, comparing DevRiot and the test tools of VS 2008.

I started with the basic: take an int and return it.

Nothing else fancier where DevRiot has advantage over VS: gui testing (on PC and devices, on and off screen), dynamic array/data structure manipulation, dynamic build test tress, built-in load and stress testing, etc :-}

It is an eye sore: test projects, test types, test attributes, test contexts, asserts, clicks, clacks to get to the point of to modify the test source to add the only thing that is of interest to the user: desired input and expected output. That is all a user needs. If the user needs to add a second test: repeat the process :-}

The thing is dog slow. It took (on average) 102,796 times more ticks than DevRiot. Granting that I was using Virtual PC but the times I am using for DevRiot are from 4 years ago on much slower machine than the current one I own.

If we assume a 50 times handicap (meaning that a 5000% improvement) DevRiot is still around 2000 times faster.

No wonder the execution time report is not the default one.

I tried to mess MS tool with changing return types, it choked like a child. Then I tried exceptions or even setting scenarios. It would have required getting into coding, api, etc..

I do not think they have clear concept of what automation means.

So, bottom line: why spend more time using MS QA infrastructure if it is not going to yield faster development cycles?

The pricing structure and road map discussion, I will leave for other post.

Progress in General:

Here we have a couple of the .Net tool working with generics:

It is funny how monitoring works. A big portion of the traffic drops, and then hits on the whitepapers and J2ME topics surge. And then a big portion of the traffic is back.

Off-topic:

Well, one thing led to the next. It started with using .Net 3.0 and the GUI testing engine, and ended up in: "we need to make sure that the off-screen engine works".

The off-screen engine is a feature of the GUI testing engine that allows these type of tests to run at speeds close to that of the unit test engine. The main goal is to increase the throughput of the process without losing accuracy.

BTW, it works...

Generics 3.0...

Adding support for generics is well underway but trying the GUI testing engine with WPF?

Help needs to be local, msdn needs to be installed...

Ohh well!

Autumn Nesting...

As my workload with Phurnace diminishes, DevRiot for .Net picks up.

Multi-dimensional array support is alive and well as the snapshot shows. Support for generics will take some time. The desktop and compact frameworks behave differently and the behavior of the engine needs to reflect this.

Well, the snapshot:

DynArray's P.S.

The one-click assert has been moved to a context menu where we will be able to provide a better way to handle assert conditions: greater than, etc, to return value, out parameters and post-test states...

One-click assert!

This is another of the progress snapshots: one click assert.

This feature allows the user to easily set the expected value, or range of values that an object should have. In this context a null or not null return value.

It does not matter if the object is private to a private field: just click away and forget about typing Asserts, PrivateObjects, or any of those rudimentary approaches!

Hit first, hit twice

Apparently USA's congress has changed the patent rules for a "first to the clerk" type of deal. Not good or bad, just a change in the playground.

DevRiot for .Net is coming along well, variable number of parameters and arrays are cleanly and dynamically handled.

Comparing usability with competitors...

Still doing interesting stuff with my buddies @ phurnace.

Prototipo; fundamentals, what fundamentals

The VS tool is coming along smoothly but a little bit slow. Phurnace's project has been quite interesting: started with a prototype with an rcp front-end and has turned into a WebSphere process server, web-services thing.

I have to admit that is highly interesting.

In this astonishing summer of news, there was this particular one about onshoring on cnn: an indian company was offshoring to Ohio some call center operations because the lower wages were not enough.

If the process and where economic value is created are not understood, there is little chance to success by trying anything else...

DevRiot for .Net is moving:

Forget about PrivateObjects, TestTypes, or writing code to create tests. Here is a snapshot of DevRiot for .Net:

Copyright (c) 2003-2007, efeKctive, L.L.C.

Hibernahhh...

Well, we are on our way to the airport. The "vacation" is over. For those with questions about De vRiot architecture:

It is totally homegrown. It was built to not depend on Xunit (as opposed to Agitar's) to gui testing, let alone hibernate.

Happy flights!!!

.Net Progress, back to Bay Area, and other great escapes chapter...

Well, this RCP work does take its time but it is fun and it helps us. I have started to get the new architecture on Visual Studio together. This a little snapshot of the toolbar. (Not particularly different from DevRiot for Eclipse. And that's is the idea: lowering the learning curves...)

I also have been in touch with my friend to put DevRiot through the grinder of DataMining development. Progress but it will take time to show.

On another front, I have been meeting with some researchers from Santiago to start looking at the "writing engine" (so users will do nothing but look at test results and develop)

Finally, after 18 months I will be back to the States. The rest of the crew has been back several times but not me.