Visual Threading!!

This is another progress demo with assertions, out-of-scope verifications, multiple threads were involved in a series of tests. Not a single line of code was written.

Try to achieve the same with other technology: Microsoft's, etc. See how much time it takes you to achieve the same. Then check how much more time it takes to get the results back.

How do you want to spend your time and budget? :-}

Context it!!

We have done a little backtracking with regards of the original way we planned to handle assertions. Interfaces and abstract classes will be handled the same way: context strip.

Here is goes another snapshot of the work-in-progress:

Beta FAQ 2:

For those research units interested in what is coming down the pipe:

*) The .Net beta will include Gui-Off: that is the ability of running the gui execution engine off the screen to increase throughput.

*) The Test Writing Engine will not be included in yet. We need field-proven execution engines before design on the writing engine continues.

*) The eclipse/java version will be overhauled after the .Net beta is available: it will become an Eclipse feaure, etc.

Generics 3.0...

Adding support for generics is well underway but trying the GUI testing engine with WPF?

Help needs to be local, msdn needs to be installed...

Ohh well!

DevRiot for .Net is moving:

Forget about PrivateObjects, TestTypes, or writing code to create tests. Here is a snapshot of DevRiot for .Net:

Copyright (c) 2003-2007, efeKctive, L.L.C.

Beta Available for Download

DevRiot for Eclipse Beta is now available for download.

Some notes on the product:

• DevRiot is a testing engine that combines unit testing and GUI testing (without the problems that capture-and-replay cause).
  
• It does not require coding or scripting. Tests are automatically created from minimal user input.
  
• It allows the user to reach any member without regard to their protection level. It also has an internal maintenance process that keeps the tests continuously updated.
• It is small: 56Kb (plugin + engine). This makes it possible to run on both J2SE and J2ME devices.
• It is fast: approximately 200 times faster than Junit.
  
• Features planned for future development: support for arrays and generics, EJB(with dynamic type generation), WS, and JDBC.
  
• Both unit testing and GUI testing interfaces are available for Linux and Windows. Currently, only the unit testing interface is available for Mac users.

Your feedback is essential to us, so we are asking those that download the DevRiot Beta to fill out a short registration form. We will not share your information with anyone - promise!

Depending on the version you are evaluating, you can contact us with feedback or questions at the following addresses:

• beta_linux AT efekctive.com
• beta_mac AT efekctive.com
• beta_windows AT efekctive.com

Please state in the subject line if you have a problem to report [Bug], or a feature request [Feature].

Any general questions can be sent to: beta AT efekctive.com.

Thanks for your interest in DevRiot!

Current beta is DevRiot for Visual Studio 200X

DevRiot .Net gets going, Beta for Eclipse almost available!!

3 People got their hands on DevRiot for Eclipse 3.X (Windows version) meanwhile I decided to get the .Net version a jolt.

Here is a snapshot of the GUI in the .Net environment. Pretty much the same as in Eclipse/Java. That's the idea: no new tricks to learn, no new tests to write. They are portable right off the start.

DevRiot for Eclipse Beta 0 = 55.5 Kb

It is not bad, isn't it? To hold the plugin logic and the engine logic of a Unit/Gui testing tool, that is. Much of the footprint is the plugin/user interface logic. The engine is a reduced version.

A full fledged engine should not be much more than. It should fit in some devices, or it could be customized :-}

Well, next in line are the installers and documentation...

Finally, Dynamic building update!!!

After a lot of trying on video formats and embedding, we have a another video update, we are pretty close to work on the installers. Things have smoothed over a lot.

We will create a couple of tests: one for the text field with a simple verification, and another for the button that is a little bit more complex. Here we will try to verify that after a click the field "AbstractMe"'s "ReturnMe" field has an arbitrary value.

Then, we will change projects to delete "ReturnMe" from Abstract.java and change back to run the suite again and witness that the tests were rebuilt automatically.

So if it was already an effortless task to create those tests, combined with the widget hound technology and the trimming/dynamic building, the net result is an extremely effective way to let people concentrate their time on more value added task.

I hope you enjoy it :-}

Code Generation + the perils of Political Marketing...

Well, apparently the Spanish government is starting the contacts with other political forces to create an anti-terrorism policy that has wide support. The funny thing is that they said it as if they were talking about where to buy bread. They are running a country.

So, what were they doing before? No back-up plan? After the 3rd broken cease-fire? Thinking that they were smarter, more illuminated than the rest of human kind?

The funny thing is that they take the same, or worse, attitudes of the foes they most criticized to get into office. Does it sound familiar?

And no, I do not like early elections. These are matters that transcend a poll.

Well, like a marketeer friend of mine, who has worked in political campaigns, said: there is so much you can do with the product you got.

I am currently working the IO/Instrumentation Units compiling/Trimmer. It is coming along fine. It was not as stable as I thought it was.

A small video sample ...

This is a small video of DevRiot Gui Engine running on Windows. The action is:

1. Clicking 6 times on a Button and verify that t == 7. This verification fails because the tool clicked 6 times only
2. Type 3 letters on a TextField and verify that t == 9. This verification succeeds.
3. Delete the logic handling the clicking on the button, and work on the Choice widget.
4. The "trimmer" kicks in, finds out that the first test is not useful anymore, and deletes it. The tool points out the obsolete is deleted
5. Select item 5 in the Choice widget and force a failure to show that the event actually happened.

The interesting things are: there is no need to capture-and-replay anything. And when the underlying code changes, the tool updates the suites automatically so there is no maintenance.

This is the link. It is in Windows Media format and a little bit over 1.5 Mb

Happy New Year!!

It works!!

Well, the "constant gardener" works (the feature that keeps the test-base in sync with the source) but most importantly the GUI engine works like silk on Windows/Linux.

No capture-and-replace, no missed clicks just a clean way to trace the behavior of your logic: be it through the GUI, the unit tests, or both. No dependence on Windows, or Linux, or Mac.

I will try to have a little demo of the gardener running on Mac tomorrow. For a GUI testing demo I will wait a couple of days so more functionality is up-and-running.

Busy Signal!

So I was working rewiring the GUI engine while it occured to me that the cell phone providers could have improved their support for the CDC platform and there were more emulators which support it (and java.awt.Robot which we do not need to run inside phones/pdas but it would nice to have)

The surprise is that they have improved their support of CDC but not so deeply. And from the answers, it seemed to be a weird question.

This is the type of things that I do not understand: with everything converging into everything, PDAs as powerful as older computers. Why are they waiting? Do they want to fall into the same trap the PC software fell?

I know of big blue companies that have teams of people tapping on PDAs to see if the app works as expected. After a while who is going to change that process and miss a deadline?

Ahh, the return of the keystone cops...

Rewiring in progress!

This is a snapshot of the gui testing logic partially rewired. As mentioned, it does not rely on capture-and-replay, and shares the same user interface as the unit test logic.



BTW: What is going on in the open source world today?

Except me, Mac!

These are a couple of snapshots of DevRiot Exception handling. It allows to click on one the exceptions thrown, and to verify that the exception gives the expected message. I have seen released product display pretty dummy messages :-}

I will jump to the gui testing engine as soon as some problems between Eclipse and Mac OS X I am experiencing are solved...

Selecting an Exception

Setting the expected message

Parameters more, parameters less:

This a "show-me-the-progress" couple of snapshots. The second one shows a slight change from the initial approach to parameters. Now everyhting, except Exceptions and Load/Stress, will have the same tree approach.

Now off to wire the GUI testing engine with the rest...

Insuficient People V6...

Last week, I think, there was this piece of news on the NYT which stated that India's pool of tech people may reach critical levels in the near future.

And then this weekend Money/CNN published an article where, among other things, it was clearly presented the enormous potential that IP V6 has in terms of addresses, connectivity, etc.

Enormous potential = Need more people.

Taking with a grain of salt the NYT article, what will happen then? Are we going to dig out developers under the rocks?

It seems that a better/more rational way of using the development resources is in order...

One Click-Interfacing

The following 3 snapshots (all from Windows, it was Sunday night, what can I say?) show the flexibility of DevRiot when handling interfaces, abstract classes, and Object types.

Snapshots 1 & 2 show how easily is to nest these types while describing the pre-state, post-state of an object. Even parameter generation works this way.

Ideally, the tool should generate these types on the fly if the user does not have them handy. But this would future feature :-}

Snapshot #1

Snapshot #2

Snapshot #3

One Click away - Too much noise!

The following screenshots show how DevRiot takes away almost of the work from the user. Everything is just a click away. Well except for interfaces and abstract classes where the tool needs some guidance from the user.

On Mac OSX, about to set pre-test state of the private field:



On Windows, showing how DevRiot deals with compiling errors:



On Linux, finally a success!



Well, given how many hits the Amber postings have taken in the last few days and the people reading them, it makes me wonder if it should hibernate at all :-}

On Mac OS!!

Although, given the problems that exist between Mac, AWT/Swing, and SWT, it will be a while before we could support this configuration.

Improved UI on Linux and Windows!!

Well, the snapshots managed to have some text to go along with them :-}

During this time we have had some useful feedback that allowed us to change the UI of the Eclipse plugin for the better, outside and inside. It looks more integrated with the Eclipse IDE

We had been able to delete a bunch of code that was more fat than substance.

The Linux snapshots show how devriot deals with pre-test conditions and post-test conditions when invoking a constructor.

The pre-test conditions are not allowed while the post-test conditions are. Not big surprise there. Still the state of the object can be accessed/set through the UI without access restrictions.

The Windows snapshots show how methods and its invoking parameters are dealt with. A series of tabs and useful data about the type at hand will be available to the users.

The Eclipse versions that will be available for the beta are 3.0x, 3.1X, and 3.2. The operating systems will be Windows and Linux/GTK (I have not run the GUI logic in a Motif box, so the behavior there is unknown), After this the .Net version will come.

Thanks!!

Looking at the wide variety of organizations downloading the whitpapers and postings, specially the more technical ones, it is encouraging after a relatively "light & silent" period where some dead weights had to be let go.

Sometimes less is more :-}

The beta of our java QA engine is coming together: the footprint is still around 50Kb (full, stripped down less than that) this should be enough to get inside a good number of handleds, PC, etc.

The advantage of avoiding the capture & replay model is that there is a lot of room for becoming lean & mean :-}

Typ-o:

the actual spelling of the path is:

http://efekctive.com/blogging/wp/quality_advantage_V2.pdf

Web Advange!!

The downloads of the whitepaper keep on coming which is nice to see. After all I have been in stealth mode lately.

I do not know if it was because word of mouth or a typo but there have been a bunch of attempts to download

http://efekctive.com/blogging/wp/quality_advange_V2.pdf (which does not exist)

the actual spelling of the path is:

http://efekctive.com/blogging/wp/quality_advantage_V2.pdf


Hope this helps...

Gates meets Gabo + Sólo Pruebas

I have been off the blog a few weeks but alert. The MS developments, Gates announcing retirement plus some defections to Google, reminds me of "Crónica de Una Muerte Anunciada" not because anybody died but because of the fatefulness.

I remember that right around the time MS started its war with Netscape, somebody from the WSJ, or the financial world for sure, wrote an article which stated that MS had already lost the www train. This was almost 8 or 9 years ago.

The PR machine launched "The Road Ahead", some "changes" happened at MS that aligned the company with the new reality, Netscape was blown off the earth.

But these days, it is the same story: somebody writing about MS not getting the gist of the www, etc. It's a little bit like deja vu...

Last week, I was in Madrid for an industry event called "Sólo Pruebas". It was a good opportunity to see what people is doing around Spain, look for prospective partners, and check the competition, etc.

It was funny to see how Acme calls B its new version of tool A. It is just tool A bundled with something else. Instead of making things simpler and more effective, the prevailing theme still seems to be keep on selling bloatware.

There were a couple of announcements made by Emca that makes this company look like a acquisition target by Acme. But this is just my impressions.

It is interesting to see how the big established players use the QA metrics. There was this talk on the first day where one of the slides pointed out that yes testing and debugging takes up to 80% of any project.

What they failed to mention was this figures was a lot smaller 20 years ago. I guess if they had mentioned that they would have to admit that their product has helped to grow that ugly stat...

Where was I?

"Quality is in the process not in the product" This tidbit of info from Deming is truly valuable. How many software shops use most of their budgets in hiring the best and brightest developers but have a much lighter quality control group? Like the 30 developers, 1 qa shop mentioned the other day.

Just from MMM, we know that there is a 30% chance that a change in one line of code will break something else downstream. Let's think about all the changes that occur in a day with 10 developers. It does not matter how good the developers are, things happen. Ask Nokia or MS.

So, it seems that the quality process should be a prerequisite to a good development team, otherwise the effort of that talented team is less than effective: how many companies with truly competent developers release products late that require patches?

The process, I think is what is missing: not a checklist of things that when in place they make a process. Saying that we need a rocket to reach the moon does not make anyone a rocket scientist.

The Keystone cops strike back! (Addedum):

The other tidbit of wisdom that Deming left around to be used was the fact that quality is in the process not in the product.

The Keystone Cops strike back!!

A couple of weeks ago, there was this exchange on one of the usenet forums where the original poster was at a loss asking for help. He was the only and new QA/QC in a software software shop that had 30+ developers.

Tall order. A lot of people chipped in, myself included. But there was one posting that struck me because it was odd. The post argument was that because QA/QC is a "support activity" (that encompases with debugging around 80% of any software project) the solution was for the guy to start his own quality movement in the hope that help would jump in after a while. Apparently, this approach was suggested in a tech book.

If one were to contrast that approach with Deming's TQM (something hard to argue against: Toyota, etc...) is a failed Quality 101. Deming's work proved that quality has to come from the top of the company to be really successful.

So what is a "support" guy to do?

Then the answer came through this posting on GK blog: the fad du jour. It is not a secret that a lot of companies manage by fads. These ones do come from the top.

So until quality becomes the fad du jour (FDJ) and stays that way long enough the "support" guy has to go with the FDJ, or risk being fired, or quit.

And top management will not support Quality until...

The cycle repeats itself: Vista is nowhere to be seen and this is the most visible one. What about the ones that do not get as much press coverage?

Real-Politik plus 150

Reading The Economist's article about MS and the beating that its stock price took, I couldn't stop thinking about the realpolitik that should be going on inside.

Traditionally, within an organization, groups become important and gain budgetary weight if their size increases. It is a matter of visibility.

This is important because the leader of that group, say the VP of engineering in a software company, will get stock options or bonuses depending on that visibility.

People are rewarded by the responsibility and visibility they seem to have, not by the improvements in their groups. So, for a Director of Quality Assurance of Acme Software, having lots of people is the way up.

And for this Director, becoming more efficient by testing more with less people is not a particularly enticing proposition. Especially if the credit for that improvement goes to someone else.

If on top of that there is a chance that in the past, some releases were less than acceptable and the CTO or VP needs to do some cyb'ing, what we have is the makings of a royal mess: The VP/CTO trying to cover her/his back, the Director of QA protecting the turf, and a product's quality going down hill.

It woudn't surprise me that something like that is going on inside MS.

_________________

Well, we have reached 150 downloads of the white paper.

This number is more than we expected, and we appreciate the feedback we've gotten.

We would love to see more feedback. Please feel free to email me directly with anything, we mean anything, you have to say about the paper.

http://efekctive.com/blogging/wp/quality_advantage_V2.pdf

Who needs a cheaper, faster way of creating software?

Last week, I asked people from different groups on the web to give me their thoughts/opinions on the white paper. There were favorable opinions, not so favorables ones, and suggestions.

But the thing that struck me the most was when it was argued that the difference between software and the Ford example is that Ford and the assembly line tried to produce goods with least amount of difference between them at a maximum rate and at a minimum cost. I do not understand how this does not apply to software.

The argument went on to saying that software is different because it is an intellectual effort. The logical continuation of that argument should be to figure out which parts of software development are intellectual: design, writing code, code reviews, all yes. QA? just the test creation.

How much intellect is needed to: pick up the latest build, install it, run the tests, and report the results to the appropiate personnel as fast as possible so problems can be addressed promptly? Not much. It is know-how easily transferable to machines.

Furthermore, even the creation of the vast majority of the tests can be automated with the technology available today. It would be a matter of some unorthodox thinking but in theory is possible.

The way I see it is: machines are really cheap today, if we are able to push the bulk of the less valued added tasks to them, software companies will be able to free resources to high value added tasks. But, hey these are just my ramblings.

I need to get back to the code and the hardware emulation layer...

Bibliographic Sources and other tidbits...

Thanks all for the feedback so far. It has been useful and it will be reflected on the paper.

One person asked us to let people know the sources we used in our research. Well, they are listed at the end of the posting.

The Marketing plan is coming along. The branding chapter of TAOTS has been quite useful lately. That together with the refresher course I am taking at IGAPE (which is the local governement economic development arm, more on Spain's daunting-for-the-non-initiated political and geographic map later)

It is an interesting group of people: the woman giving the seminar is from Vigo, lived and worked in Madrid and London and came back to Galicia. There are a couple of guys from the Mediterranean coast who are on their way to open an B&B/pilgrims refuge in one of the most beautiful places of Galicia (in Lugo Province just on the French Route of "Camino de Santiago")

When I have better details of the B&B, I will forward them.

Perspectives on Business Innovation, Issue 1 "Five Myths that Slow Down Software Development" by Cap Gemini Ernst & Young

Valuation: Measuring and Managing the Value of Companies, by McKinsey & Company Inc., Tom Copeland, et all (I would recommend just the first 2, maybe 3, chapters, the rest is kind of repetitive)

What high tech can learn from slow-growth industries, The McKinsey Quarterly.

"A Better Bug Trap" in Technology Quarterly, The Economist.

Good Ol' Bean Counter, where art thou:

There is this study published few years ago by Cap Gemini Ernst & Young which states the following: 65% of the in-memory image of a software program belongs to software developed by a third party.

Put this figure together with the NIST stats (80% of a project is testing and debugging) and the magical number is 52%. Any software project could use up to 52% of its resources (time, people, etc) executing somebody else's software.

And doing the marketing research update for the BP, I ran into Mercury's QTP 8.X.

This tool allows users to continue using capture & replay through another layer of indirection (their ActiveScreen feature)

So a capture & replay user just interested in:

button1_clicked(because is the only logic that she/he can modify)

has to wait for the tool to execute who knows how many screen repaints, OS and specific tool calls (all of which are third party) to get to the real test target:

the fabled "button1_clicked".

How many more tests could be executed instead? Aren't we always short of time for quality? Are we using the time we have wisely?

On the same token, if we could get rid of the extra layers of indirection. Could we test everything all the time?

The Mythical Man Month considered this not economically/physically feasible. But back then the computing power was in its infancy and most of the production of software was done manually. Is it possible now?

Software Development & Europe, a mirror

I am about to hit the BP again but could not stop thinking an exchange that I had with someone on usenet. This person who also teaches proposed that automated testing is not relieable and the only solution is manual testing (not exactly with these words but that's the only conclusion that could be drawn)

But when confronted with the economic reality of the fixed physical throuput capacity of humans got into trouble: what to do with a growing codebase? was he going to force people to move their hands and eyes faster? So his only solution is to use more inputs to produce the same good?

This is bad lesson in economics 101. It should take less inputs to create a good over time. That is in essence efficiency.

This is the same approach as "pair programming": adding more people to see if they make it work. If recent ship date delays are a guide, adding people to a flawed process just does not work.

So here we are several years after the tech bust we are, in short, running to stand still:

Manual testing advocates think they can get it right after 25 years.

Traditional testing tool companies have "somewhat fixed" the problems they created (quoting a Forrester report) But what about non PC gadgets? Can they run inside a Pocket PC?

An XP advocates think that the solution is to add more people to:

1. Ask developers to write their tests, as if they had time to write their own code.
2. Ask developers to add special code so they can write their own tests.
3. Ask testers to write code that is going to become useless in the next round of refactoring.

And then came France and its PCE (first employment contract, or close enough) the students and the unions kept the PM from turning PCE into law. The PCE was an attempt make France's laber market more fluid and create jobs.

PCE is not a law which may have been a bad law but France and Europe in general need something along the changes that are needed in the software industry:

The population is growing old, so more immigration or more natality is needed.

In 2004 just Spain had positive demographic growth.

According to the OCDE, just 2 european universities are among the top 20/top 50 in the world. The rest are in USA, Japan, India, etc.

These two are english. According to a survey published recently when Europeans emmigrate, they mostly choose England because of the jobs and pay.

With China and India growing into world powers, what is Europe going to bring in the future to the table of big decision makers?

Education? Job opportunities? a healthy work force?

Anyways that business plan is waiting...

About the Keystone Cops, The Cart and The Horse:

The vast majority of the software projects that I have seen, in one way or another, always made me think of the Keystone Cops (the early 20 century comedians) trying to compete in F1 with a car with square tires.

It doesn’t matter how hard the Keystone’s try to push, or how many of them are pushing, the car will not run at all or fast.

If the tires were round, the requirement of Keystones will be less and there will be a chance of winning the race.

Now let’s go back to software. What happened during the last 20 odd years?

People trying to reach first mover advantage set up companies that could not maintain that advantage. In other words they set up themselves for failure in the future: near, medium, or long.

Everything needed to be fast: quality? Effieciency? Next release. But once the ball got rolling, who was the one to improve the process when there were two more deadlines in the coming months?

So the sound marketing principle of “It is cheaper to keep an existing customer happy than acquiring a new one” went down the drain by consistently having quality problems.

The MS case is interesting because they are the ones who tried to use the 1:1 ratio or something close to it. But these inefficiency problems are everywhere.

I guess this event it is going to put a new twist in to outsourcing. Why look for cheaper labor, in the hope to increase the head-count, if increasing the head count does not solve the problem?

I do not have anything against outsourcing. But does it make sense to outsource a flawed process? Do we need more people or a more rational use of all the resources at hand?

The economic impact of delayed schedules and bad quality software (lost sales and extra expenses) should be measured not only for MS but for the whole industry. I guess the figure would be mind-boggling.

On to other things: I just found a beta version of VS .Net 2005 (a year to use after installing) Hopefully it would work still, so I can go back to our .Net engine or Amber.

I was just checking the new Junit release (4.0) It’s looking like Nunit but in Java.

But the same flaws remain in both: why would a user care about anything else but the input and the output of a test? Why expend so many resources executing the inner workings of these two when so much logic under test remains unexecuted in gthe overall process?

More Evidence...

Earlier last week, before MS announced delaying the initial release of Vista, there was this exchange in one of the Google groups about ratios, developers to QA and others.

One of the postings stated that MS currently uses a 1:1.1 developer to QA ratio. I knew that in the past they used a 3:2. I replied that it seemed like an aberration. It was the equivalent of Henry Ford packing the plants with workers and ditching the assembly line, I stated.

And voliá! two days later Vista is delayed for the consumer market. Who knows the state in which the corporate clients will get the other version of Vista.

This event reinforces the main points of our white paper: it's not a matter of how much money or people you throw at this problem. It is a matter of reformulating the problem to solve.

Anyways, there are a couple of screenshots (towards the end of this posting) of the Eclipse plug-in that I wanted to share with you, the intention is to compare DevRiot with Junit in terms of speed. This plug-in also complies with J2ME CDC.

So imagine that you could automate all the GUI/unit testing of your logic on any of the cool Nokia gadgets, or a PC running Windows or Linux?

In addition, my wife gave me as a present "The Art of The Start".

Hopefully, I will learn something :-}



Copyright José Cornado (c) 2005-2006

Correction: the new engine is faster (25-200 times)

We have been updating the performance comparisons of our new architecture against the competition. As mentioned before, we currently have numbers for the java/eclipse product only.

The performance comparison was to measure how much time would it take to execute a suite that exercises 19 methods of the form:

public XX ReturnXX(XX param){
return param;
}

Where XX is an native type or its equivalent in the java.lang package. We ran one set where every test was a failed one and another set where all were successes.

The idea was to measure how efficient the engines were at handling different amounts of work. A failure should take more work than a success because the engine has to report where the failure is.

The numbers are better than we thought. In the all-failures scenario our engine is, consistently, 25 times faster than Junit. In the all-successes scenario our engine is 200 times faster than Junit.

With the advantage that we can handle all types of access to methods and fields, and strict gui testing.

We will be updating the white paper with these new figures and snapshots of the product, soon.

Well, that's all for now...

A Snapshot of the Eclipse plug-in!!

NOTE: The GUI capture here is obsolete for a newer version please visit:

http://efekctive.com/picture_library/blogging/devriot-ui-new-linux1.gif

Copyright José Cornado (c) 2005-2006

There are a couple of things to notice, we think:

The user just needs to provide the parameter and expected return values. The rest is optional information: an ID to execute the test in a particular order, pre-conditions and post conditions of the object under examination, number of times to execute, and load/stress level.

The method shown is private, but our technology can still reach it. It is really poweful and dynamic. One of the advantages of this feature is that your QA investment is immune to Refactoring.

As several people have noted over different lists and forums, Refactoring can (and does) make public or protected methods private.

In this case the QA efforts are lost because Junit, or Nunit for that matter, just can not handle the change. What to do? wait and let the code base grow without testing? risk to lose several weeks of QA development? or just do not use refactoring?

Another is the ability of testing even the exceptions (type and contents) thrown by the logic under scrutiny.

Also, this is the same interface that will be used to handle GUI testing. Nothing to capture :-}

The new architecture/engine is fast

The new architecture that I mentioned in the previous post refers to our automated testing engine. Please note that:

• It is a testing engine, not a GUI only testing engine or an unit-testing only engine. It's both, so users have to deal with one test tool only, we hope.
• It works as an IDE plug/add-in or as a automation server.
• It does not force Capture & Replay, writing code or scripts on the user. It uses what we call Reduce Input/Output Testing.
• It is platform and OS independent. For example, the gui tests created under Linux can be executed in a Windows environment.
• It is language independent.

Now the numbers: the average execution time per test (unit or gui) of our Java product is less than 1 millisecond. Junit throws in the same environment a lower limit of 20 milliseconds per test.

This is a performance increase in excess if 2,000%, so imagine that your QA technology performed 2000% faster?

To put in a nutshell, our goal is to provide a technology that minimizes the users' effort (input), minimizes the post-test creation maintenance (input), and executes faster than any competitor (output) thus making the user more productive.

The .Net version, even though is still under the old architecture, offers similar performance numbers when compared against Nunit.

Well, that's all for now. Hopefully, I will be able to get back to Amber and .Net pretty soon...

BTW, our most recent white paper is this (password: AUSTIN).
(The images are old though. We will update them shortly)