The S option seems to be a viable option!

Good ol' SQL improved the serialization of tests by leaps and bounds.

Now it takes less than minute and a half to create and store 5,000,000 tests. Before this much time was needed for 100,000.

Slow drip...

Sometimes things happen in a way that makes you wonder.

I have not worked on DevRiot's eclipse plug-in in ages. Obviously, I am concentrating on .Net since Visual Studio 2008 is going to be released soon.

But the funny thing is that the demo of DevRiot running the GUI testing engine on J2ME has become pretty popular this month.

How information flows in the business world is amazing!

Needless to say, it is going to be a lot easier to implement the GUI testing engine for .Net than for J2ME. Everyhting is already in the IDE (VS 200x) On the other hand getting the same device support on Eclipse/J2ME is like secretively impossible. Back to work...

Beta Available for Download

DevRiot for Eclipse Beta is now available for download.

Some notes on the product:

• DevRiot is a testing engine that combines unit testing and GUI testing (without the problems that capture-and-replay cause).
  
• It does not require coding or scripting. Tests are automatically created from minimal user input.
  
• It allows the user to reach any member without regard to their protection level. It also has an internal maintenance process that keeps the tests continuously updated.
• It is small: 56Kb (plugin + engine). This makes it possible to run on both J2SE and J2ME devices.
• It is fast: approximately 200 times faster than Junit.
  
• Features planned for future development: support for arrays and generics, EJB(with dynamic type generation), WS, and JDBC.
  
• Both unit testing and GUI testing interfaces are available for Linux and Windows. Currently, only the unit testing interface is available for Mac users.

Your feedback is essential to us, so we are asking those that download the DevRiot Beta to fill out a short registration form. We will not share your information with anyone - promise!

Depending on the version you are evaluating, you can contact us with feedback or questions at the following addresses:

• beta_linux AT efekctive.com
• beta_mac AT efekctive.com
• beta_windows AT efekctive.com

Please state in the subject line if you have a problem to report [Bug], or a feature request [Feature].

Any general questions can be sent to: beta AT efekctive.com.

Thanks for your interest in DevRiot!

Current beta is DevRiot for Visual Studio 200X

DevRiot .Net gets going, Beta for Eclipse almost available!!

3 People got their hands on DevRiot for Eclipse 3.X (Windows version) meanwhile I decided to get the .Net version a jolt.

Here is a snapshot of the GUI in the .Net environment. Pretty much the same as in Eclipse/Java. That's the idea: no new tricks to learn, no new tests to write. They are portable right off the start.

One down, two more to go...

The first install image of DevRiot is ready, some more sanity checks while I work on the other two and the documentation before it is ready for downloading!

DevRiot for Eclipse Beta 0 = 55.5 Kb

It is not bad, isn't it? To hold the plugin logic and the engine logic of a Unit/Gui testing tool, that is. Much of the footprint is the plugin/user interface logic. The engine is a reduced version.

A full fledged engine should not be much more than. It should fit in some devices, or it could be customized :-}

Well, next in line are the installers and documentation...

Micro pudding!!

As we get closer to packaging the beta, features become more usable. This sample video shows DevRiot running under J2ME CDC.

It is a GUI testing only video but the unit testing engine works fine as well. We previously mentioned that our approach to J2ME was coarse or brute force. In fact, we just run the Ide under J9 to force our plug-in to use.

As you will be able to see: it works and it is stable. It would be nice to have access to some emulators with Robot-like support but the main point is there.

Finally, Dynamic building update!!!

After a lot of trying on video formats and embedding, we have a another video update, we are pretty close to work on the installers. Things have smoothed over a lot.

We will create a couple of tests: one for the text field with a simple verification, and another for the button that is a little bit more complex. Here we will try to verify that after a click the field "AbstractMe"'s "ReturnMe" field has an arbitrary value.

Then, we will change projects to delete "ReturnMe" from Abstract.java and change back to run the suite again and witness that the tests were rebuilt automatically.

So if it was already an effortless task to create those tests, combined with the widget hound technology and the trimming/dynamic building, the net result is an extremely effective way to let people concentrate their time on more value added task.

I hope you enjoy it :-}

Please bear with me...

The internal build chain is pretty stable right now.

If a test case TC for type A in project P refers for some reason type B in project P1, any change in B is communicated to DevRiot and its internal build process (the trimmer to be exact checks any) updates TC with those changes.

We need to add a couple of buttons to the plugin so adding, running, and loading tests cases are separate tasks.

We are also changing the layout of the blog a little bit :-}

So far so good...

We are in the final test-debug stages. The process is going well with no surprises.

As mentioned before we are going to leave array, Virtual Hardware, and Swing support out of the beta.

We wanted to concentrate on the depth of the functionality rather than the breadth of the functionality, which will come later.

It will also be the common subset of native types and AWT widgets that exists between CDC and J2SE. The CDC testing has been pretty much brute force when more finesse is needed.

The "test-trimmer" and "widget-hound" features will be there and working.

Right now we are debating between letting the Apple version out without the GUI engine support or wait until the problem is fixed and include the GUI engine support...

The Widget Hound!!

The progress has not stopped around here. We have another little video to share meanwhile other features steady a little bit more.

This time we have a video of a feature called "Widget Hound". It is in WMV format and is less than 400K. The feature shows one of our approaches to test problematic GUIS.

In the video we will show a button-clicked method that every time is invoked, it changes position. In a capture-and-replay model this would break a lot of things, but we can rest assured that DevRiot can handle it.

The story line of the video is as follows, with the cursor already placed on the target method.

1) Launch debug session an click 3 times the button to show changing positions.

2) Launch DevRiot and a test case that generates the same 3 button clicks.

As you will be able to see, DevRiot can chase the running widget down and honor the clicks required by the test case. The maintenance cost of test suites drop dramatically with this approach.

The link to the video is here...

Working like a GPS! (Update)

The internal building process is stable, almost ready to face the public.

On the side I had been wiring/connecting a feature that makes the gui engine work like a GPS. I do not think I mentioned before.

It knows at all times where the widget-under-test is. So what's the benefit? The benefit is that the engine knows where to position the cursor at all times. It does not miss a click.

The benefits of skipping capture-and-replay ;-}

Code Generation + the perils of Political Marketing...

Well, apparently the Spanish government is starting the contacts with other political forces to create an anti-terrorism policy that has wide support. The funny thing is that they said it as if they were talking about where to buy bread. They are running a country.

So, what were they doing before? No back-up plan? After the 3rd broken cease-fire? Thinking that they were smarter, more illuminated than the rest of human kind?

The funny thing is that they take the same, or worse, attitudes of the foes they most criticized to get into office. Does it sound familiar?

And no, I do not like early elections. These are matters that transcend a poll.

Well, like a marketeer friend of mine, who has worked in political campaigns, said: there is so much you can do with the product you got.

I am currently working the IO/Instrumentation Units compiling/Trimmer. It is coming along fine. It was not as stable as I thought it was.

Running natively on Linux...

The following link will take you to another video of DevRiot running on Linux. Notice that by getting around the limitations of capture-and-replay, things are a lot simpler. For instance, there is no need to use Windows to test a Gui in Linux, or use JavaBeans for that matter.

The same applies to other devices like PDA, etc.

Well, here is the link

A small video sample ...

This is a small video of DevRiot Gui Engine running on Windows. The action is:

1. Clicking 6 times on a Button and verify that t == 7. This verification fails because the tool clicked 6 times only
2. Type 3 letters on a TextField and verify that t == 9. This verification succeeds.
3. Delete the logic handling the clicking on the button, and work on the Choice widget.
4. The "trimmer" kicks in, finds out that the first test is not useful anymore, and deletes it. The tool points out the obsolete is deleted
5. Select item 5 in the Choice widget and force a failure to show that the event actually happened.

The interesting things are: there is no need to capture-and-replay anything. And when the underlying code changes, the tool updates the suites automatically so there is no maintenance.

This is the link. It is in Windows Media format and a little bit over 1.5 Mb

Happy New Year!!

It works!!

Well, the "constant gardener" works (the feature that keeps the test-base in sync with the source) but most importantly the GUI engine works like silk on Windows/Linux.

No capture-and-replace, no missed clicks just a clean way to trace the behavior of your logic: be it through the GUI, the unit tests, or both. No dependence on Windows, or Linux, or Mac.

I will try to have a little demo of the gardener running on Mac tomorrow. For a GUI testing demo I will wait a couple of days so more functionality is up-and-running.

Pre-Christmas gift!!

Well, the GUI testing engine (inside the eclipse plug-in) still does not work in Mac OS X. Despite upgrading to DP1. I guess the SWT/AWT compatibility has some more tweaking ahead.

On the brighter side, I am finishing wiring what I call "the gardener": code changes, method signatures change, methods are deleted. The gardener takes care of this by keeping the tests up to date with the real code.

I am still shooting to get a GUI engine up and running for Christmas :-}

Movie time!!

This is a little demo of DevRiot - Eclipse running on a Mac. The movie is in QuickTime format and about 4.5Mb in size.

It shows DevRiot at work:

1. finds where the cursor is
2. picks the method to test
3. generates a test form
4. lets the user fill the form with ease
5. add and run

Hopefully for Christmas we will have a similar demo running the GUI testing engine. click on the image below to launch the demo:

Not so fast son!!

Well, the swt/awt/apple problems are not solved with the compatibility library released by apple. Eclipse keeps on crashing and when it does not crash the AWT/Swing widgets-under-test miss events or get them too late.

I know the problem is not the gui-engine logic because it delivers the events orderly and timely on other widget types (SWT, ...)

So DevRiot will not support GUI automation on Apple for now. It will support the unit testing automation which runs good.

Who wants to get in the middle of the fight of this two? Not us :-}

1001 Tests...

Just to see where the engine started to show signs of effort, I decided to wire up some stress/load logic. I used a simple method that takes an int and returns it.

The rest is on the progress snapshots:

Enjoy :-}

The Proof is in the pudding...

This is a snapshot of DevRiot running on Eclipse 3.3M3 and Java 1.5. Note that the engine executed 100 tests and all of them failed. Despite the extra effort of reporting the failures, the little engine still needs a good set of brakes:-}

We are getting closer to the point where we can release this eclipse/java beta. We do want to move on to the VS/.Net beta because the performance gains here a orders of magnitude bigger than in Java.

Stay tuned...

Still rocking on java 5.0!!

We just got the first taste of DevRiot running on java 5.0 and we like it. The speed (ours) is still blazing fast.

There are a couple of bugs in java 5.0 that were affecting DevRiot and needed to address so they didn't hurt performance wise. That was done.

We will try to show how fast it goes before the weekend is over... :-}


Sorry for the unusual formatting. Blogger is not showing the formatting toolbar at this moment!

A shorter update...

Still wiring the gui, unit, and activation logic. The activation logic will work like this, or close enough:

When a user clicks on the DevRiot button if the currently selected file is a source file then it continues, if the cursor is NOT inside a method then the tests are run. If the cursor is INSIDE a method, the form (that is on the screenshots) reconfigures itself and lets the user add new tests. When the popup is closed then the tests are run...

Back to coding :-}

Short Update...

Still rewiring. Decided, for the Eclipse plug-in beta, to support Eclipse 3.2 (for Windows & Linux) and 3.3 M3 (for Mac OS X if their workaround for a bug I opened is good)

It may be time to get the VS .Net add-in up to speed :-}

Stay tuned...

Busy Signal!

So I was working rewiring the GUI engine while it occured to me that the cell phone providers could have improved their support for the CDC platform and there were more emulators which support it (and java.awt.Robot which we do not need to run inside phones/pdas but it would nice to have)

The surprise is that they have improved their support of CDC but not so deeply. And from the answers, it seemed to be a weird question.

This is the type of things that I do not understand: with everything converging into everything, PDAs as powerful as older computers. Why are they waiting? Do they want to fall into the same trap the PC software fell?

I know of big blue companies that have teams of people tapping on PDAs to see if the app works as expected. After a while who is going to change that process and miss a deadline?

Ahh, the return of the keystone cops...

Rewiring in progress!

This is a snapshot of the gui testing logic partially rewired. As mentioned, it does not rely on capture-and-replay, and shares the same user interface as the unit test logic.



BTW: What is going on in the open source world today?

Except me, Mac!

These are a couple of snapshots of DevRiot Exception handling. It allows to click on one the exceptions thrown, and to verify that the exception gives the expected message. I have seen released product display pretty dummy messages :-}

I will jump to the gui testing engine as soon as some problems between Eclipse and Mac OS X I am experiencing are solved...

Selecting an Exception

Setting the expected message

Parameters more, parameters less:

This a "show-me-the-progress" couple of snapshots. The second one shows a slight change from the initial approach to parameters. Now everyhting, except Exceptions and Load/Stress, will have the same tree approach.

Now off to wire the GUI testing engine with the rest...

One Click-Interfacing

The following 3 snapshots (all from Windows, it was Sunday night, what can I say?) show the flexibility of DevRiot when handling interfaces, abstract classes, and Object types.

Snapshots 1 & 2 show how easily is to nest these types while describing the pre-state, post-state of an object. Even parameter generation works this way.

Ideally, the tool should generate these types on the fly if the user does not have them handy. But this would future feature :-}

Snapshot #1

Snapshot #2

Snapshot #3

One Click away - Too much noise!

The following screenshots show how DevRiot takes away almost of the work from the user. Everything is just a click away. Well except for interfaces and abstract classes where the tool needs some guidance from the user.

On Mac OSX, about to set pre-test state of the private field:



On Windows, showing how DevRiot deals with compiling errors:



On Linux, finally a success!



Well, given how many hits the Amber postings have taken in the last few days and the people reading them, it makes me wonder if it should hibernate at all :-}

Amber's hibernation, DevRiot update

There are a couple of progress snapshots of DevRiot's new UI: precompiling code and related tasks. Although, sometimes is discouraging put too much effort on the UI since QA tools should generate tests automatically, data and plumbing work :-} but...

Amber is in hibernation, I do not know how deep. The Mono people started a direct implementation of Indigo/WCF. Amber's goal was to avoid the IP restriction imposed on WCF/Indigo but since there is an open-source version of them on its way...

Their work is part of Mono Olive, they seem to be cruising on that front. There is also a google group for Mono Olive.

Well that's all for now...

On Mac OS!!

Although, given the problems that exist between Mac, AWT/Swing, and SWT, it will be a while before we could support this configuration.

Improved UI on Linux and Windows!!

Well, the snapshots managed to have some text to go along with them :-}

During this time we have had some useful feedback that allowed us to change the UI of the Eclipse plugin for the better, outside and inside. It looks more integrated with the Eclipse IDE

We had been able to delete a bunch of code that was more fat than substance.

The Linux snapshots show how devriot deals with pre-test conditions and post-test conditions when invoking a constructor.

The pre-test conditions are not allowed while the post-test conditions are. Not big surprise there. Still the state of the object can be accessed/set through the UI without access restrictions.

The Windows snapshots show how methods and its invoking parameters are dealt with. A series of tabs and useful data about the type at hand will be available to the users.

The Eclipse versions that will be available for the beta are 3.0x, 3.1X, and 3.2. The operating systems will be Windows and Linux/GTK (I have not run the GUI logic in a Motif box, so the behavior there is unknown), After this the .Net version will come.