One down, two more to go...

The first install image of DevRiot is ready, some more sanity checks while I work on the other two and the documentation before it is ready for downloading!

DevRiot for Eclipse Beta 0 = 55.5 Kb

It is not bad, isn't it? To hold the plugin logic and the engine logic of a Unit/Gui testing tool, that is. Much of the footprint is the plugin/user interface logic. The engine is a reduced version.

A full fledged engine should not be much more than. It should fit in some devices, or it could be customized :-}

Well, next in line are the installers and documentation...

Finally, Dynamic building update!!!

After a lot of trying on video formats and embedding, we have a another video update, we are pretty close to work on the installers. Things have smoothed over a lot.

We will create a couple of tests: one for the text field with a simple verification, and another for the button that is a little bit more complex. Here we will try to verify that after a click the field "AbstractMe"'s "ReturnMe" field has an arbitrary value.

Then, we will change projects to delete "ReturnMe" from Abstract.java and change back to run the suite again and witness that the tests were rebuilt automatically.

So if it was already an effortless task to create those tests, combined with the widget hound technology and the trimming/dynamic building, the net result is an extremely effective way to let people concentrate their time on more value added task.

I hope you enjoy it :-}

So far so good...

We are in the final test-debug stages. The process is going well with no surprises.

As mentioned before we are going to leave array, Virtual Hardware, and Swing support out of the beta.

We wanted to concentrate on the depth of the functionality rather than the breadth of the functionality, which will come later.

It will also be the common subset of native types and AWT widgets that exists between CDC and J2SE. The CDC testing has been pretty much brute force when more finesse is needed.

The "test-trimmer" and "widget-hound" features will be there and working.

Right now we are debating between letting the Apple version out without the GUI engine support or wait until the problem is fixed and include the GUI engine support...

The Widget Hound!!

The progress has not stopped around here. We have another little video to share meanwhile other features steady a little bit more.

This time we have a video of a feature called "Widget Hound". It is in WMV format and is less than 400K. The feature shows one of our approaches to test problematic GUIS.

In the video we will show a button-clicked method that every time is invoked, it changes position. In a capture-and-replay model this would break a lot of things, but we can rest assured that DevRiot can handle it.

The story line of the video is as follows, with the cursor already placed on the target method.

1) Launch debug session an click 3 times the button to show changing positions.

2) Launch DevRiot and a test case that generates the same 3 button clicks.

As you will be able to see, DevRiot can chase the running widget down and honor the clicks required by the test case. The maintenance cost of test suites drop dramatically with this approach.

The link to the video is here...

Working like a GPS! (Update)

The internal building process is stable, almost ready to face the public.

On the side I had been wiring/connecting a feature that makes the gui engine work like a GPS. I do not think I mentioned before.

It knows at all times where the widget-under-test is. So what's the benefit? The benefit is that the engine knows where to position the cursor at all times. It does not miss a click.

The benefits of skipping capture-and-replay ;-}

Back In Barcelona...

I had to come to Barcelona for 4 days. I am still in Barcelona, flying back tomorrow afternoon.

It had been almost 13 years since the last time I came. A lot of construction, and new luxury stores, etc. But still a pleasure to visit.

The Gothic district with its thousands of antiquaries, vintage stores, churches is worth roaming around. There is a small restaurant where Gaudí, Picasso, Dalí, Miró used to hang out. It is still open for business.

Casa Batlló, my favorite Gaudí building, is open to the public, there is a fee to pay though.

The Eclipse plug-in is almost there. The trimmer is a lot more stable. The internal building process plays well with the trimmer.

Just an simple and easy test retrieval interface is missing...

Code Generation + the perils of Political Marketing...

Well, apparently the Spanish government is starting the contacts with other political forces to create an anti-terrorism policy that has wide support. The funny thing is that they said it as if they were talking about where to buy bread. They are running a country.

So, what were they doing before? No back-up plan? After the 3rd broken cease-fire? Thinking that they were smarter, more illuminated than the rest of human kind?

The funny thing is that they take the same, or worse, attitudes of the foes they most criticized to get into office. Does it sound familiar?

And no, I do not like early elections. These are matters that transcend a poll.

Well, like a marketeer friend of mine, who has worked in political campaigns, said: there is so much you can do with the product you got.

I am currently working the IO/Instrumentation Units compiling/Trimmer. It is coming along fine. It was not as stable as I thought it was.

A small video sample ...

This is a small video of DevRiot Gui Engine running on Windows. The action is:

1. Clicking 6 times on a Button and verify that t == 7. This verification fails because the tool clicked 6 times only
2. Type 3 letters on a TextField and verify that t == 9. This verification succeeds.
3. Delete the logic handling the clicking on the button, and work on the Choice widget.
4. The "trimmer" kicks in, finds out that the first test is not useful anymore, and deletes it. The tool points out the obsolete is deleted
5. Select item 5 in the Choice widget and force a failure to show that the event actually happened.

The interesting things are: there is no need to capture-and-replay anything. And when the underlying code changes, the tool updates the suites automatically so there is no maintenance.

This is the link. It is in Windows Media format and a little bit over 1.5 Mb

Happy New Year!!

Still going...

Working on AWT menus. These were kind of tough cookies but finally got them under control...

It works!!

Well, the "constant gardener" works (the feature that keeps the test-base in sync with the source) but most importantly the GUI engine works like silk on Windows/Linux.

No capture-and-replace, no missed clicks just a clean way to trace the behavior of your logic: be it through the GUI, the unit tests, or both. No dependence on Windows, or Linux, or Mac.

I will try to have a little demo of the gardener running on Mac tomorrow. For a GUI testing demo I will wait a couple of days so more functionality is up-and-running.

Pre-Christmas gift!!

Well, the GUI testing engine (inside the eclipse plug-in) still does not work in Mac OS X. Despite upgrading to DP1. I guess the SWT/AWT compatibility has some more tweaking ahead.

On the brighter side, I am finishing wiring what I call "the gardener": code changes, method signatures change, methods are deleted. The gardener takes care of this by keeping the tests up to date with the real code.

I am still shooting to get a GUI engine up and running for Christmas :-}

Not so fast son!!

Well, the swt/awt/apple problems are not solved with the compatibility library released by apple. Eclipse keeps on crashing and when it does not crash the AWT/Swing widgets-under-test miss events or get them too late.

I know the problem is not the gui-engine logic because it delivers the events orderly and timely on other widget types (SWT, ...)

So DevRiot will not support GUI automation on Apple for now. It will support the unit testing automation which runs good.

Who wants to get in the middle of the fight of this two? Not us :-}

1001 Tests...

Just to see where the engine started to show signs of effort, I decided to wire up some stress/load logic. I used a simple method that takes an int and returns it.

The rest is on the progress snapshots:

Enjoy :-}

The Proof is in the pudding...

This is a snapshot of DevRiot running on Eclipse 3.3M3 and Java 1.5. Note that the engine executed 100 tests and all of them failed. Despite the extra effort of reporting the failures, the little engine still needs a good set of brakes:-}

We are getting closer to the point where we can release this eclipse/java beta. We do want to move on to the VS/.Net beta because the performance gains here a orders of magnitude bigger than in Java.

Stay tuned...

Still rocking on java 5.0!!

We just got the first taste of DevRiot running on java 5.0 and we like it. The speed (ours) is still blazing fast.

There are a couple of bugs in java 5.0 that were affecting DevRiot and needed to address so they didn't hurt performance wise. That was done.

We will try to show how fast it goes before the weekend is over... :-}


Sorry for the unusual formatting. Blogger is not showing the formatting toolbar at this moment!

A shorter update...

Still wiring the gui, unit, and activation logic. The activation logic will work like this, or close enough:

When a user clicks on the DevRiot button if the currently selected file is a source file then it continues, if the cursor is NOT inside a method then the tests are run. If the cursor is INSIDE a method, the form (that is on the screenshots) reconfigures itself and lets the user add new tests. When the popup is closed then the tests are run...

Back to coding :-}

Short Update...

Still rewiring. Decided, for the Eclipse plug-in beta, to support Eclipse 3.2 (for Windows & Linux) and 3.3 M3 (for Mac OS X if their workaround for a bug I opened is good)

It may be time to get the VS .Net add-in up to speed :-}

Stay tuned...

Busy Signal!

So I was working rewiring the GUI engine while it occured to me that the cell phone providers could have improved their support for the CDC platform and there were more emulators which support it (and java.awt.Robot which we do not need to run inside phones/pdas but it would nice to have)

The surprise is that they have improved their support of CDC but not so deeply. And from the answers, it seemed to be a weird question.

This is the type of things that I do not understand: with everything converging into everything, PDAs as powerful as older computers. Why are they waiting? Do they want to fall into the same trap the PC software fell?

I know of big blue companies that have teams of people tapping on PDAs to see if the app works as expected. After a while who is going to change that process and miss a deadline?

Ahh, the return of the keystone cops...

Rewiring in progress!

This is a snapshot of the gui testing logic partially rewired. As mentioned, it does not rely on capture-and-replay, and shares the same user interface as the unit test logic.



BTW: What is going on in the open source world today?

Promising news!

I was trying to solve a problem that happens with SWT and Mac OSX, (well the eclipse guys were solving it. I was telling them what the problem was) I found that Apple released a compatibility library that addresses a problem between SWT and AWT running on Mac.

It had something to do with thread ownership. SWT expects to own thread XX while AWT on Mac also expects to own it. You get the picture.

This caused big problems for apps that used both, SWT and AWT/Swing, which are a lot. We were not directly afected our engine but it was going to force us to use extra processes and hack a solution.

Hopefully the three platforms will have the same foundation :-}

Except me, Mac!

These are a couple of snapshots of DevRiot Exception handling. It allows to click on one the exceptions thrown, and to verify that the exception gives the expected message. I have seen released product display pretty dummy messages :-}

I will jump to the gui testing engine as soon as some problems between Eclipse and Mac OS X I am experiencing are solved...

Selecting an Exception

Setting the expected message

Parameters more, parameters less:

This a "show-me-the-progress" couple of snapshots. The second one shows a slight change from the initial approach to parameters. Now everyhting, except Exceptions and Load/Stress, will have the same tree approach.

Now off to wire the GUI testing engine with the rest...

One Click-Interfacing

The following 3 snapshots (all from Windows, it was Sunday night, what can I say?) show the flexibility of DevRiot when handling interfaces, abstract classes, and Object types.

Snapshots 1 & 2 show how easily is to nest these types while describing the pre-state, post-state of an object. Even parameter generation works this way.

Ideally, the tool should generate these types on the fly if the user does not have them handy. But this would future feature :-}

Snapshot #1

Snapshot #2

Snapshot #3

One Click away - Too much noise!

The following screenshots show how DevRiot takes away almost of the work from the user. Everything is just a click away. Well except for interfaces and abstract classes where the tool needs some guidance from the user.

On Mac OSX, about to set pre-test state of the private field:



On Windows, showing how DevRiot deals with compiling errors:



On Linux, finally a success!



Well, given how many hits the Amber postings have taken in the last few days and the people reading them, it makes me wonder if it should hibernate at all :-}

Amber's hibernation, DevRiot update

There are a couple of progress snapshots of DevRiot's new UI: precompiling code and related tasks. Although, sometimes is discouraging put too much effort on the UI since QA tools should generate tests automatically, data and plumbing work :-} but...

Amber is in hibernation, I do not know how deep. The Mono people started a direct implementation of Indigo/WCF. Amber's goal was to avoid the IP restriction imposed on WCF/Indigo but since there is an open-source version of them on its way...

Their work is part of Mono Olive, they seem to be cruising on that front. There is also a google group for Mono Olive.

Well that's all for now...

On Mac OS!!

Although, given the problems that exist between Mac, AWT/Swing, and SWT, it will be a while before we could support this configuration.

Improved UI on Linux and Windows!!

Well, the snapshots managed to have some text to go along with them :-}

During this time we have had some useful feedback that allowed us to change the UI of the Eclipse plugin for the better, outside and inside. It looks more integrated with the Eclipse IDE

We had been able to delete a bunch of code that was more fat than substance.

The Linux snapshots show how devriot deals with pre-test conditions and post-test conditions when invoking a constructor.

The pre-test conditions are not allowed while the post-test conditions are. Not big surprise there. Still the state of the object can be accessed/set through the UI without access restrictions.

The Windows snapshots show how methods and its invoking parameters are dealt with. A series of tabs and useful data about the type at hand will be available to the users.

The Eclipse versions that will be available for the beta are 3.0x, 3.1X, and 3.2. The operating systems will be Windows and Linux/GTK (I have not run the GUI logic in a Motif box, so the behavior there is unknown), After this the .Net version will come.

Where was I?

"Quality is in the process not in the product" This tidbit of info from Deming is truly valuable. How many software shops use most of their budgets in hiring the best and brightest developers but have a much lighter quality control group? Like the 30 developers, 1 qa shop mentioned the other day.

Just from MMM, we know that there is a 30% chance that a change in one line of code will break something else downstream. Let's think about all the changes that occur in a day with 10 developers. It does not matter how good the developers are, things happen. Ask Nokia or MS.

So, it seems that the quality process should be a prerequisite to a good development team, otherwise the effort of that talented team is less than effective: how many companies with truly competent developers release products late that require patches?

The process, I think is what is missing: not a checklist of things that when in place they make a process. Saying that we need a rocket to reach the moon does not make anyone a rocket scientist.

Who needs a cheaper, faster way of creating software?

Last week, I asked people from different groups on the web to give me their thoughts/opinions on the white paper. There were favorable opinions, not so favorables ones, and suggestions.

But the thing that struck me the most was when it was argued that the difference between software and the Ford example is that Ford and the assembly line tried to produce goods with least amount of difference between them at a maximum rate and at a minimum cost. I do not understand how this does not apply to software.

The argument went on to saying that software is different because it is an intellectual effort. The logical continuation of that argument should be to figure out which parts of software development are intellectual: design, writing code, code reviews, all yes. QA? just the test creation.

How much intellect is needed to: pick up the latest build, install it, run the tests, and report the results to the appropiate personnel as fast as possible so problems can be addressed promptly? Not much. It is know-how easily transferable to machines.

Furthermore, even the creation of the vast majority of the tests can be automated with the technology available today. It would be a matter of some unorthodox thinking but in theory is possible.

The way I see it is: machines are really cheap today, if we are able to push the bulk of the less valued added tasks to them, software companies will be able to free resources to high value added tasks. But, hey these are just my ramblings.

I need to get back to the code and the hardware emulation layer...

Bibliographic Sources and other tidbits...

Thanks all for the feedback so far. It has been useful and it will be reflected on the paper.

One person asked us to let people know the sources we used in our research. Well, they are listed at the end of the posting.

The Marketing plan is coming along. The branding chapter of TAOTS has been quite useful lately. That together with the refresher course I am taking at IGAPE (which is the local governement economic development arm, more on Spain's daunting-for-the-non-initiated political and geographic map later)

It is an interesting group of people: the woman giving the seminar is from Vigo, lived and worked in Madrid and London and came back to Galicia. There are a couple of guys from the Mediterranean coast who are on their way to open an B&B/pilgrims refuge in one of the most beautiful places of Galicia (in Lugo Province just on the French Route of "Camino de Santiago")

When I have better details of the B&B, I will forward them.

Perspectives on Business Innovation, Issue 1 "Five Myths that Slow Down Software Development" by Cap Gemini Ernst & Young

Valuation: Measuring and Managing the Value of Companies, by McKinsey & Company Inc., Tom Copeland, et all (I would recommend just the first 2, maybe 3, chapters, the rest is kind of repetitive)

What high tech can learn from slow-growth industries, The McKinsey Quarterly.

"A Better Bug Trap" in Technology Quarterly, The Economist.

More Evidence...

Earlier last week, before MS announced delaying the initial release of Vista, there was this exchange in one of the Google groups about ratios, developers to QA and others.

One of the postings stated that MS currently uses a 1:1.1 developer to QA ratio. I knew that in the past they used a 3:2. I replied that it seemed like an aberration. It was the equivalent of Henry Ford packing the plants with workers and ditching the assembly line, I stated.

And voliá! two days later Vista is delayed for the consumer market. Who knows the state in which the corporate clients will get the other version of Vista.

This event reinforces the main points of our white paper: it's not a matter of how much money or people you throw at this problem. It is a matter of reformulating the problem to solve.

Anyways, there are a couple of screenshots (towards the end of this posting) of the Eclipse plug-in that I wanted to share with you, the intention is to compare DevRiot with Junit in terms of speed. This plug-in also complies with J2ME CDC.

So imagine that you could automate all the GUI/unit testing of your logic on any of the cool Nokia gadgets, or a PC running Windows or Linux?

In addition, my wife gave me as a present "The Art of The Start".

Hopefully, I will learn something :-}



Copyright José Cornado (c) 2005-2006