Tell me what posts one is reading...

And I will tell you what one's marketing plan missed :-}

Well, Leopard came out. It may have solved some of the SWT/AWT problems that DevRiot was experiencing on OS X. I need to get back to it later. Some people got really upset that Java 6 is not included or better java support on OS X is demanded.

It is understandable but things should get less emotional.

After all, not long ago Apple was in the doldrums. It certainly came back but it is behind every big player in the Java world. And it is not going to let IBM or Sun have a piece of the Apple pie out of goodwill.

There are a lot more Java/IBM developers than OS X developers. That in itself is a major threat.

Well, I am going back to .Net and make sure DevRiot does not choke when several instances of Visual Studio are up and running...

Beta Available for Download

DevRiot for Eclipse Beta is now available for download.

Some notes on the product:

• DevRiot is a testing engine that combines unit testing and GUI testing (without the problems that capture-and-replay cause).
  
• It does not require coding or scripting. Tests are automatically created from minimal user input.
  
• It allows the user to reach any member without regard to their protection level. It also has an internal maintenance process that keeps the tests continuously updated.
• It is small: 56Kb (plugin + engine). This makes it possible to run on both J2SE and J2ME devices.
• It is fast: approximately 200 times faster than Junit.
  
• Features planned for future development: support for arrays and generics, EJB(with dynamic type generation), WS, and JDBC.
  
• Both unit testing and GUI testing interfaces are available for Linux and Windows. Currently, only the unit testing interface is available for Mac users.

Your feedback is essential to us, so we are asking those that download the DevRiot Beta to fill out a short registration form. We will not share your information with anyone - promise!

Depending on the version you are evaluating, you can contact us with feedback or questions at the following addresses:

• beta_linux AT efekctive.com
• beta_mac AT efekctive.com
• beta_windows AT efekctive.com

Please state in the subject line if you have a problem to report [Bug], or a feature request [Feature].

Any general questions can be sent to: beta AT efekctive.com.

Thanks for your interest in DevRiot!

Current beta is DevRiot for Visual Studio 200X

DevRiot .Net gets going, Beta for Eclipse almost available!!

3 People got their hands on DevRiot for Eclipse 3.X (Windows version) meanwhile I decided to get the .Net version a jolt.

Here is a snapshot of the GUI in the .Net environment. Pretty much the same as in Eclipse/Java. That's the idea: no new tricks to learn, no new tests to write. They are portable right off the start.

One down, two more to go...

The first install image of DevRiot is ready, some more sanity checks while I work on the other two and the documentation before it is ready for downloading!

DevRiot for Eclipse Beta 0 = 55.5 Kb

It is not bad, isn't it? To hold the plugin logic and the engine logic of a Unit/Gui testing tool, that is. Much of the footprint is the plugin/user interface logic. The engine is a reduced version.

A full fledged engine should not be much more than. It should fit in some devices, or it could be customized :-}

Well, next in line are the installers and documentation...

Micro pudding!!

As we get closer to packaging the beta, features become more usable. This sample video shows DevRiot running under J2ME CDC.

It is a GUI testing only video but the unit testing engine works fine as well. We previously mentioned that our approach to J2ME was coarse or brute force. In fact, we just run the Ide under J9 to force our plug-in to use.

As you will be able to see: it works and it is stable. It would be nice to have access to some emulators with Robot-like support but the main point is there.

Finally, Dynamic building update!!!

After a lot of trying on video formats and embedding, we have a another video update, we are pretty close to work on the installers. Things have smoothed over a lot.

We will create a couple of tests: one for the text field with a simple verification, and another for the button that is a little bit more complex. Here we will try to verify that after a click the field "AbstractMe"'s "ReturnMe" field has an arbitrary value.

Then, we will change projects to delete "ReturnMe" from Abstract.java and change back to run the suite again and witness that the tests were rebuilt automatically.

So if it was already an effortless task to create those tests, combined with the widget hound technology and the trimming/dynamic building, the net result is an extremely effective way to let people concentrate their time on more value added task.

I hope you enjoy it :-}

Please bear with me...

The internal build chain is pretty stable right now.

If a test case TC for type A in project P refers for some reason type B in project P1, any change in B is communicated to DevRiot and its internal build process (the trimmer to be exact checks any) updates TC with those changes.

We need to add a couple of buttons to the plugin so adding, running, and loading tests cases are separate tasks.

We are also changing the layout of the blog a little bit :-}

So far so good...

We are in the final test-debug stages. The process is going well with no surprises.

As mentioned before we are going to leave array, Virtual Hardware, and Swing support out of the beta.

We wanted to concentrate on the depth of the functionality rather than the breadth of the functionality, which will come later.

It will also be the common subset of native types and AWT widgets that exists between CDC and J2SE. The CDC testing has been pretty much brute force when more finesse is needed.

The "test-trimmer" and "widget-hound" features will be there and working.

Right now we are debating between letting the Apple version out without the GUI engine support or wait until the problem is fixed and include the GUI engine support...

The Widget Hound!!

The progress has not stopped around here. We have another little video to share meanwhile other features steady a little bit more.

This time we have a video of a feature called "Widget Hound". It is in WMV format and is less than 400K. The feature shows one of our approaches to test problematic GUIS.

In the video we will show a button-clicked method that every time is invoked, it changes position. In a capture-and-replay model this would break a lot of things, but we can rest assured that DevRiot can handle it.

The story line of the video is as follows, with the cursor already placed on the target method.

1) Launch debug session an click 3 times the button to show changing positions.

2) Launch DevRiot and a test case that generates the same 3 button clicks.

As you will be able to see, DevRiot can chase the running widget down and honor the clicks required by the test case. The maintenance cost of test suites drop dramatically with this approach.

The link to the video is here...

Working like a GPS! (Update)

The internal building process is stable, almost ready to face the public.

On the side I had been wiring/connecting a feature that makes the gui engine work like a GPS. I do not think I mentioned before.

It knows at all times where the widget-under-test is. So what's the benefit? The benefit is that the engine knows where to position the cursor at all times. It does not miss a click.

The benefits of skipping capture-and-replay ;-}

Back In Barcelona...

I had to come to Barcelona for 4 days. I am still in Barcelona, flying back tomorrow afternoon.

It had been almost 13 years since the last time I came. A lot of construction, and new luxury stores, etc. But still a pleasure to visit.

The Gothic district with its thousands of antiquaries, vintage stores, churches is worth roaming around. There is a small restaurant where Gaudí, Picasso, Dalí, Miró used to hang out. It is still open for business.

Casa Batlló, my favorite Gaudí building, is open to the public, there is a fee to pay though.

The Eclipse plug-in is almost there. The trimmer is a lot more stable. The internal building process plays well with the trimmer.

Just an simple and easy test retrieval interface is missing...

Code Generation + the perils of Political Marketing...

Well, apparently the Spanish government is starting the contacts with other political forces to create an anti-terrorism policy that has wide support. The funny thing is that they said it as if they were talking about where to buy bread. They are running a country.

So, what were they doing before? No back-up plan? After the 3rd broken cease-fire? Thinking that they were smarter, more illuminated than the rest of human kind?

The funny thing is that they take the same, or worse, attitudes of the foes they most criticized to get into office. Does it sound familiar?

And no, I do not like early elections. These are matters that transcend a poll.

Well, like a marketeer friend of mine, who has worked in political campaigns, said: there is so much you can do with the product you got.

I am currently working the IO/Instrumentation Units compiling/Trimmer. It is coming along fine. It was not as stable as I thought it was.

Running natively on Linux...

The following link will take you to another video of DevRiot running on Linux. Notice that by getting around the limitations of capture-and-replay, things are a lot simpler. For instance, there is no need to use Windows to test a Gui in Linux, or use JavaBeans for that matter.

The same applies to other devices like PDA, etc.

Well, here is the link

Still going...

Working on AWT menus. These were kind of tough cookies but finally got them under control...

It works!!

Well, the "constant gardener" works (the feature that keeps the test-base in sync with the source) but most importantly the GUI engine works like silk on Windows/Linux.

No capture-and-replace, no missed clicks just a clean way to trace the behavior of your logic: be it through the GUI, the unit tests, or both. No dependence on Windows, or Linux, or Mac.

I will try to have a little demo of the gardener running on Mac tomorrow. For a GUI testing demo I will wait a couple of days so more functionality is up-and-running.

Pre-Christmas gift!!

Well, the GUI testing engine (inside the eclipse plug-in) still does not work in Mac OS X. Despite upgrading to DP1. I guess the SWT/AWT compatibility has some more tweaking ahead.

On the brighter side, I am finishing wiring what I call "the gardener": code changes, method signatures change, methods are deleted. The gardener takes care of this by keeping the tests up to date with the real code.

I am still shooting to get a GUI engine up and running for Christmas :-}

Movie time!!

This is a little demo of DevRiot - Eclipse running on a Mac. The movie is in QuickTime format and about 4.5Mb in size.

It shows DevRiot at work:

1. finds where the cursor is
2. picks the method to test
3. generates a test form
4. lets the user fill the form with ease
5. add and run

Hopefully for Christmas we will have a similar demo running the GUI testing engine. click on the image below to launch the demo:

Not so fast son!!

Well, the swt/awt/apple problems are not solved with the compatibility library released by apple. Eclipse keeps on crashing and when it does not crash the AWT/Swing widgets-under-test miss events or get them too late.

I know the problem is not the gui-engine logic because it delivers the events orderly and timely on other widget types (SWT, ...)

So DevRiot will not support GUI automation on Apple for now. It will support the unit testing automation which runs good.

Who wants to get in the middle of the fight of this two? Not us :-}

1001 Tests...

Just to see where the engine started to show signs of effort, I decided to wire up some stress/load logic. I used a simple method that takes an int and returns it.

The rest is on the progress snapshots:

Enjoy :-}

The Proof is in the pudding...

This is a snapshot of DevRiot running on Eclipse 3.3M3 and Java 1.5. Note that the engine executed 100 tests and all of them failed. Despite the extra effort of reporting the failures, the little engine still needs a good set of brakes:-}

We are getting closer to the point where we can release this eclipse/java beta. We do want to move on to the VS/.Net beta because the performance gains here a orders of magnitude bigger than in Java.

Stay tuned...

Still rocking on java 5.0!!

We just got the first taste of DevRiot running on java 5.0 and we like it. The speed (ours) is still blazing fast.

There are a couple of bugs in java 5.0 that were affecting DevRiot and needed to address so they didn't hurt performance wise. That was done.

We will try to show how fast it goes before the weekend is over... :-}


Sorry for the unusual formatting. Blogger is not showing the formatting toolbar at this moment!

A shorter update...

Still wiring the gui, unit, and activation logic. The activation logic will work like this, or close enough:

When a user clicks on the DevRiot button if the currently selected file is a source file then it continues, if the cursor is NOT inside a method then the tests are run. If the cursor is INSIDE a method, the form (that is on the screenshots) reconfigures itself and lets the user add new tests. When the popup is closed then the tests are run...

Back to coding :-}

Short Update...

Still rewiring. Decided, for the Eclipse plug-in beta, to support Eclipse 3.2 (for Windows & Linux) and 3.3 M3 (for Mac OS X if their workaround for a bug I opened is good)

It may be time to get the VS .Net add-in up to speed :-}

Stay tuned...

Rewiring in progress!

This is a snapshot of the gui testing logic partially rewired. As mentioned, it does not rely on capture-and-replay, and shares the same user interface as the unit test logic.



BTW: What is going on in the open source world today?

Promising news!

I was trying to solve a problem that happens with SWT and Mac OSX, (well the eclipse guys were solving it. I was telling them what the problem was) I found that Apple released a compatibility library that addresses a problem between SWT and AWT running on Mac.

It had something to do with thread ownership. SWT expects to own thread XX while AWT on Mac also expects to own it. You get the picture.

This caused big problems for apps that used both, SWT and AWT/Swing, which are a lot. We were not directly afected our engine but it was going to force us to use extra processes and hack a solution.

Hopefully the three platforms will have the same foundation :-}

Except me, Mac!

These are a couple of snapshots of DevRiot Exception handling. It allows to click on one the exceptions thrown, and to verify that the exception gives the expected message. I have seen released product display pretty dummy messages :-}

I will jump to the gui testing engine as soon as some problems between Eclipse and Mac OS X I am experiencing are solved...

Selecting an Exception

Setting the expected message

Parameters more, parameters less:

This a "show-me-the-progress" couple of snapshots. The second one shows a slight change from the initial approach to parameters. Now everyhting, except Exceptions and Load/Stress, will have the same tree approach.

Now off to wire the GUI testing engine with the rest...

One Click-Interfacing

The following 3 snapshots (all from Windows, it was Sunday night, what can I say?) show the flexibility of DevRiot when handling interfaces, abstract classes, and Object types.

Snapshots 1 & 2 show how easily is to nest these types while describing the pre-state, post-state of an object. Even parameter generation works this way.

Ideally, the tool should generate these types on the fly if the user does not have them handy. But this would future feature :-}

Snapshot #1

Snapshot #2

Snapshot #3

One Click away - Too much noise!

The following screenshots show how DevRiot takes away almost of the work from the user. Everything is just a click away. Well except for interfaces and abstract classes where the tool needs some guidance from the user.

On Mac OSX, about to set pre-test state of the private field:



On Windows, showing how DevRiot deals with compiling errors:



On Linux, finally a success!



Well, given how many hits the Amber postings have taken in the last few days and the people reading them, it makes me wonder if it should hibernate at all :-}

Amber's hibernation, DevRiot update

There are a couple of progress snapshots of DevRiot's new UI: precompiling code and related tasks. Although, sometimes is discouraging put too much effort on the UI since QA tools should generate tests automatically, data and plumbing work :-} but...

Amber is in hibernation, I do not know how deep. The Mono people started a direct implementation of Indigo/WCF. Amber's goal was to avoid the IP restriction imposed on WCF/Indigo but since there is an open-source version of them on its way...

Their work is part of Mono Olive, they seem to be cruising on that front. There is also a google group for Mono Olive.

Well that's all for now...

On Mac OS!!

Although, given the problems that exist between Mac, AWT/Swing, and SWT, it will be a while before we could support this configuration.

Improved UI on Linux and Windows!!

Well, the snapshots managed to have some text to go along with them :-}

During this time we have had some useful feedback that allowed us to change the UI of the Eclipse plugin for the better, outside and inside. It looks more integrated with the Eclipse IDE

We had been able to delete a bunch of code that was more fat than substance.

The Linux snapshots show how devriot deals with pre-test conditions and post-test conditions when invoking a constructor.

The pre-test conditions are not allowed while the post-test conditions are. Not big surprise there. Still the state of the object can be accessed/set through the UI without access restrictions.

The Windows snapshots show how methods and its invoking parameters are dealt with. A series of tabs and useful data about the type at hand will be available to the users.

The Eclipse versions that will be available for the beta are 3.0x, 3.1X, and 3.2. The operating systems will be Windows and Linux/GTK (I have not run the GUI logic in a Motif box, so the behavior there is unknown), After this the .Net version will come.

Where was I?

"Quality is in the process not in the product" This tidbit of info from Deming is truly valuable. How many software shops use most of their budgets in hiring the best and brightest developers but have a much lighter quality control group? Like the 30 developers, 1 qa shop mentioned the other day.

Just from MMM, we know that there is a 30% chance that a change in one line of code will break something else downstream. Let's think about all the changes that occur in a day with 10 developers. It does not matter how good the developers are, things happen. Ask Nokia or MS.

So, it seems that the quality process should be a prerequisite to a good development team, otherwise the effort of that talented team is less than effective: how many companies with truly competent developers release products late that require patches?

The process, I think is what is missing: not a checklist of things that when in place they make a process. Saying that we need a rocket to reach the moon does not make anyone a rocket scientist.

Less beta is more beta (we hope)

For the beta, we decided that the most important things we needed to convey were:

• simplicity,
• speed,
• ease of use

So, we are taking out the dynamic array logic, the load/stress/performance features, and are including a limited set of AWT and Swing widgets to demonstrate how our approach to GUI quality control is more effective than capture-and-replay. The beta will also include the "GUI accelerated mode": which is bassically a feature that bypasses a lot of the friction caused by screen repaints thus increasing the effectiveness of the GUI tests.

How to trim down the noise out of a beta (with help deluxe)

Between diaper changes, writing marketing plans in spanish while thinking about them in english, tracking the surf forecast (it has been flat for the last 2 weeks or so until friday) we decided to put the beta together.

I was spinning my wheels until frustration lead me to send GK an e-mail. What the heck? what is the worst that could happen: getting a response?

Well, we did - twice!

His suggestions were on target. But we also found a wealth of useful pointers/articles on his blog. The one about DEMOgod seems pertinent even for betas from companies with limited resources, like us.

There was another one entitled something like "open letter to CXO" that took me back to the moment I quit my last job. I wish I had read it in 2002.

Well enough for now, it is time to start shaping up the code for beta and enjoying the crazy spring storm that is raging outside right now.

It is sunny, it is raining, and the winds are so strong that birds do not dare to fly...

Who needs a cheaper, faster way of creating software?

Last week, I asked people from different groups on the web to give me their thoughts/opinions on the white paper. There were favorable opinions, not so favorables ones, and suggestions.

But the thing that struck me the most was when it was argued that the difference between software and the Ford example is that Ford and the assembly line tried to produce goods with least amount of difference between them at a maximum rate and at a minimum cost. I do not understand how this does not apply to software.

The argument went on to saying that software is different because it is an intellectual effort. The logical continuation of that argument should be to figure out which parts of software development are intellectual: design, writing code, code reviews, all yes. QA? just the test creation.

How much intellect is needed to: pick up the latest build, install it, run the tests, and report the results to the appropiate personnel as fast as possible so problems can be addressed promptly? Not much. It is know-how easily transferable to machines.

Furthermore, even the creation of the vast majority of the tests can be automated with the technology available today. It would be a matter of some unorthodox thinking but in theory is possible.

The way I see it is: machines are really cheap today, if we are able to push the bulk of the less valued added tasks to them, software companies will be able to free resources to high value added tasks. But, hey these are just my ramblings.

I need to get back to the code and the hardware emulation layer...

Bibliographic Sources and other tidbits...

Thanks all for the feedback so far. It has been useful and it will be reflected on the paper.

One person asked us to let people know the sources we used in our research. Well, they are listed at the end of the posting.

The Marketing plan is coming along. The branding chapter of TAOTS has been quite useful lately. That together with the refresher course I am taking at IGAPE (which is the local governement economic development arm, more on Spain's daunting-for-the-non-initiated political and geographic map later)

It is an interesting group of people: the woman giving the seminar is from Vigo, lived and worked in Madrid and London and came back to Galicia. There are a couple of guys from the Mediterranean coast who are on their way to open an B&B/pilgrims refuge in one of the most beautiful places of Galicia (in Lugo Province just on the French Route of "Camino de Santiago")

When I have better details of the B&B, I will forward them.

Perspectives on Business Innovation, Issue 1 "Five Myths that Slow Down Software Development" by Cap Gemini Ernst & Young

Valuation: Measuring and Managing the Value of Companies, by McKinsey & Company Inc., Tom Copeland, et all (I would recommend just the first 2, maybe 3, chapters, the rest is kind of repetitive)

What high tech can learn from slow-growth industries, The McKinsey Quarterly.

"A Better Bug Trap" in Technology Quarterly, The Economist.

Dynamic Array Manipulation Snapshot!!

Slowly but surely the dynamic array manipulation through DevRiot's gui is coming together.

This will allow the user just concentrate on the most important things, in this case what the array contents should be. The for loops, etc, etc, are on the house :-]

This one is short because I am about to go for an afternoon surfing session.



Copyright José Cornado (c) 2005-2006

More Evidence...

Earlier last week, before MS announced delaying the initial release of Vista, there was this exchange in one of the Google groups about ratios, developers to QA and others.

One of the postings stated that MS currently uses a 1:1.1 developer to QA ratio. I knew that in the past they used a 3:2. I replied that it seemed like an aberration. It was the equivalent of Henry Ford packing the plants with workers and ditching the assembly line, I stated.

And voliá! two days later Vista is delayed for the consumer market. Who knows the state in which the corporate clients will get the other version of Vista.

This event reinforces the main points of our white paper: it's not a matter of how much money or people you throw at this problem. It is a matter of reformulating the problem to solve.

Anyways, there are a couple of screenshots (towards the end of this posting) of the Eclipse plug-in that I wanted to share with you, the intention is to compare DevRiot with Junit in terms of speed. This plug-in also complies with J2ME CDC.

So imagine that you could automate all the GUI/unit testing of your logic on any of the cool Nokia gadgets, or a PC running Windows or Linux?

In addition, my wife gave me as a present "The Art of The Start".

Hopefully, I will learn something :-}



Copyright José Cornado (c) 2005-2006