Wednesday, February 03, 2010

Available!

We are happy to announce that DevRiot for Visual Studio 200X(Beta) is available for downloading at:


The documentation can be found at:


Don't forget you will need a working email address, a working internet connection, and a SQL Server instance available to install and use DevRiot (the email address is required during setup only)

We hope you find it useful and worth the time to try it! If you have any questions please contact us at this address: beta at efekctive dot com.

Labels: , , , , , , , , , , , , , ,

Monday, February 01, 2010

Just waiting...

Everything is just waiting for a final delivery hurdle to clear and the beta will be available....

Labels: , , , , , , , ,

Saturday, January 23, 2010

Almost there...

The obfuscator has been selected. Building + obfuscating + packaging + installing + monkey testing. In short, making sure it works before actually buying it...

Labels: , , , , , , , ,

Wednesday, January 13, 2010

Documented

The documentation is "done". It needs editing to polish my written "spanglish". The setup needs changing to avoid a random identification failure while installing a final round of "monkey testing" and beta should be ready to go!

Labels: , , , , , , , ,

Tuesday, January 12, 2010

Wiking!!

For the last few days we have been putting together a simple documentation for this beta. It can be found at: http://www dot efekctive dot com/docs.

Labels: , , , , , , , , , ,

Tuesday, January 05, 2010

Mild and off-target...

After a harmless hack attempt at the wrong site, it was suggested by my isp to harden the site logic. We did so it is robust enough to avoid the easy gotchas, we believe.

Now off to put a little documentation together with the previous images and posts and it should be ready for the public to break...

Labels: , , , , , , , ,

Monday, January 04, 2010

Deployment is steady...

Well deployment of the engine is steady, as well as registering and unregistering. A few runs of these three together and some poking around the engine to make sure one last time and it should be ready to be downloaded! Happy new year!!

Labels: , , , , , , , ,

Monday, December 28, 2009

Registration is there...

The first part of deployment is ready, or almost ready. Tomorrow we will start on deployment of the engine itself. Happy Holidays!

Labels: , , , , , , ,

Sunday, December 13, 2009

Shallow waters

Finally we packaged a shallow/dummy version of DevRiot for .Net. The goal is to make sure the installers work correctly and package the resources needed by the tool to work. No functionality yet.

We are working on the full deployment logic.

They can be found here

Feel free to try it and let us know what you think at beta [at] efeKctive dot com

Labels: , , , , , , ,

Monday, December 07, 2009

Done.

The generics engine, and the engine in general, is stable enough for somebody else to break it. The install/deploy phase begins. A video with the full shebang of features will come in a few...

Labels: , , , , , , , ,

Wednesday, December 02, 2009

One Bug Down...

DevRiot's generics compiler has a couple of bugs that make it behave different from .Net's standard behavior. One is already down. The other should come down soon and deployment start...

Labels: , , , , , , , ,

Monday, November 30, 2009

Random thoughts

The GUI acts now as it should be (more stable). Just a couple of scenarios/bugs in the GG engine and the deployment phase should start in earnest.

The federal catch 22: labour market is in the doldrums. Stimulus vs. outsourcing? Spend more tax payer money that could be sent overseas in 2/3 years? Black Friday purchases shrunk almost 10% in compared to 2008. Not a seat for the faint of heart.


Labels: , , , , , ,

Monday, November 23, 2009

FormMinder!

Some messy code on our behalf renders DevRiot's GUI unusable. And it looks random so far. Working on it...

Labels: , , , , , ,

Thursday, October 22, 2009

Generics Generator getting a life...

We have hit a can of worms, actually a couple of weeks back, that made the automatic recompilation of tests algorithm working for generic types only or non-generic types only. Obviously this was not acceptable.

This algorithm was rewritten and it looks cleaner and working...

Labels: , , , , , , ,

Saturday, September 26, 2009

Crawling back to deployment

The recompilation process has hit a couple of obstacles while dealing with generic methods but we are working on them. But we have found a clean, simple way of deployment that is worthwhile trying...

Labels: , , , , , , ,

Wednesday, September 02, 2009

DevRiot Beta is wating for deployment...

We fixed the nested generics problems. It looks good enough to let people really find its problems.

This leaves DevRiot beta ready to be used once the deployer method is ready. Will be off the grid on a 2-week friends & family & surfing trip. A break is called for.

Labels: , , , , , , , ,

Friday, August 28, 2009

Nested Generics...

We are experiencing a little bit of trouble with the way nested generic types are being stored. In the meantime a couple of buttons/shortcuts were added so the engine can be disengaged when a series of builds trigger continuously DevRiot's automatic build.

Next week will be a simple install and finish the engine deployment method. Beta is just a short steps away...

Labels: , , , , , , ,

Sunday, August 09, 2009

1,600 T

I have been trying to squeeze more than 1,600 concurrent threads out the engine without changing any settings. No luck. Obviously, it will need a closer, slower look and better tools. Still 1,600 is a number that can be useful, specially when one invests 30 seconds or less to create, use, and harvest.



Labels: , , , , , , ,

Friday, August 07, 2009

Pit Stop!!

From time to time things deadlock, etc. Well, for those occasions DevRiot lets you deal with them seamlessly...

Labels: , , , , , ,

Thursday, August 06, 2009

Measuring up...

The L&S engine is working just fine. Some changes were made to allow users to measure on a per-thread basis the execution times. Like the Time-To-Live concept: if a method takes more than X ticks, a failure is reported.



Labels: , , , , , ,

Tuesday, August 04, 2009

Progress one thread at a time...

Arrays and Dynamic fixtures seem to be stable. A couple of bugs that we'll need to get back to but nothing to keep us from moving on to threading/Load & Stress...


Labels: , , , , , , , ,

Thursday, July 16, 2009

Objects

This is an snapshot of DevRiot latest build. It shows a run with a large amount of dynamic fixtures. Mainly System.Object in various shapes and forms (arrays, block arrays, etc) Depsite the number of failures, the execution times still are minimal. Enjoy!



Labels: , , , , ,

Monday, July 06, 2009

Short Flight update

The block arrays logic is smooth now. There is a little glitch when mixing and nesting different type of arrays like TypeA[,,][][,,][]. But we are working on it.

Labels: , , , , ,

Wednesday, June 24, 2009

Short Stop.

Block arrays have become stable. Cut and Paste of tree nodes will wait.

Labels: , , , , , , ,

Saturday, June 20, 2009

Tweeterish update

Making great progress. Generics is stable. Block arrays (of the form [,,,]) not so much.

Labels: , , , , , ,

Monday, June 01, 2009

3,400,003 and counting...

We spent the weekend fixing an old disconnection problem when running large amount of tests. We fixed it. It works like a champ, it executed some really ineffient code for 24 hours straight without a bleep.

We also managed to squeeze 3.4 million tests with some complexity: runtime generics, different levels of validation and the engine ran like a champ. The pudding:



 

Labels: , , , , , ,

Sunday, May 31, 2009

DevRiot for .Net Beta Update...

Having a little bit of trouble with nested generic types and bulk creation (25,000 or more) but other than stable an smooth.

BTW properties will not be supported for this beta. 

Labels: , , , , , ,

Friday, May 29, 2009

Another Short Update

DevRiot for .Net is coming along well. We have spent extra time ironing out the code that deals with System.Object, interfaces, and abstract types because it is adding more dynamic input to an already dynamic engine. 

Since the user never writes code, tests are generated/executed on the fly, the former types add more complexity to the logic so the ironing of bugs turns more complex too.

Hit a unexpected behavior between the recompiling process of DevRiot and VS debugger. Specially annoying when handling 10,000 tests or more.

Labels: , , , , , ,

Friday, February 13, 2009

Feature complete!!

Finally, the feature set that is to be included in DevRiot for .Net is complete and stable. Next in line comes the installer and the delivery logic.

We are considering delivering a double plug-in . The users will install a Visual Studio add-in that will download DevRiot's engine. This way fixes will delivered easily.

This beta will support only Desktop Framework and unit testing. GUI testing, since it depends more on the running device, will be delivered at a later time when support for Compact and Micro Frameworks is polished.

A demo video will follow soon.

Labels: , , , , , , , , ,

Wednesday, February 04, 2009

Another glitchy update...

We have been dealing, for the last several days, with an annoying timing issue that messes up all the symbols  displayed in the UI.


Labels: , , , , , , ,

Tuesday, January 27, 2009

Cause & Effect:

One would have to think there is a correlation: a tech giant profit falls, they download our whitepaper. 

It seems that our wp has a reputation.

The beta is coming along great. I have had to devote time to help family, but I fixed bugs and DevRiot, I hope, it is getting close to be as useful as the whitepaper.

Labels: , , , , , ,

Sunday, January 11, 2009

Me too!!

There will be no public beta for DevRiot .Net. Just access to the people who kindly signed up to try it out.

So for those me-too with a laundry list of me-too features, stop poking around!


Labels: , , , , , , ,

Tuesday, January 06, 2009

Great Progress!!

The last couple of weeks have seen a lot of bug fixing, a more transparent GAC manipulation. Th engine is a lot more stable. It needs to be because it will also handle the GUI tests. Back to work...

Labels: , , , , , , , ,

Thursday, December 25, 2008

The octopuss

Recompiling load & stress tests is taking some bug-fixing effort. Working on it :-}

Labels: , , , , , , , ,

Friday, November 14, 2008

Recompile The Move...

Ok it's hard to move across the ocean again and fix bugs but we have managed to fix some areas that were choking while recompiling: abstract classes + interfaces.

Still no departure date.

Labels: , , , , , , ,

Sunday, October 12, 2008

Params, Threads, Generics This is another progress demo. It involves: dynamically allocating params, genereics, threads, and switching between classes. Everything is handled transparently and effortlessly to the user. With such a small learning curve, ease of use, and blasting speeds the testing "Total Costs of Ownership" should be extremely low while attaining effectiveness never seen before. Enjoy your Sunday morning...











Labels: , , , , , , , ,

Sunday, October 05, 2008

Smooth...

After cleaning up some bugs, a task that is still going, here is another sampler: a 500 thread session, with generics included, ran twice under 3 minutes from scratch.

That is generics type, tests + threads creation, and run. Everything completely synchronized.











Labels: , , , , , , , ,

Wednesday, October 01, 2008

General Dynamic Fixtures

This is the first work-in-progress report in a while. It shows how DevRiot dynamically generates, stores, and retrieves tests between sessions.

The "fixtures" are handled dynamically. We can mix and match the generics parameter types on-the-fly, keeping the engineers effort easy, flexible, and quick. Extremely quick.

Very few tools on the market can offer such a low Cost of Ownership, in the short or long run :-}

Enjoy...













Labels: , , , , , , , ,

Wednesday, July 30, 2008

Summer Blues Report...

I must admit that moving suites/tests from storage to binary form has been a challenge. It has become more difficult than DevRiot for Java because it has more features to take care of.

Generics Generation has posed its own set of binarization challenges.

But still snailing away...

Labels: , , , , , , , ,

Monday, July 14, 2008

Still working out the bugs...

...inside the generics engine to recompile the tests from storage. Not being able to ask questions makes the work slow down a lot.

Will have another demo soon.

Labels: , , , , , , , , ,

Wednesday, July 02, 2008

The Referencer

Is still working on fixing bugs. The logic to implement new/old method matching is uncovering a lot of bugs. Not as close to beta as I thought or would like...

Labels: , , , , , , , , , ,

Friday, June 20, 2008

Plugging the plug-in

We are paying the error of working while sick on a key piece of code regeneration. Bad code, or at least worse than usual, that needs rewriting.

But it also has given us the opportunity of thinking about deployment.

We are toying with the idea of adopting "perpetual delivery" mode. Since the engine is not larger than 150K, downloading it can part of the add-in setup.

Labels: , , , , , , , , , ,

Wednesday, June 11, 2008

Deploying the report,

Still making a lot of progress with suite recompilation and integrity.

Also some time has been spent on figuring out how to deploy the tool while minimizing the IP exposure.

There is VS Gallery and other channels. Obfuscation is a good deterrent but it needs help from other angles...

Labels: , , , , , , , ,

Tuesday, June 03, 2008

From the lonely tree to the red sound (Progress report)

The recompilation process is moving slowly but solidly. As anyone might have guessed bugs have been uncovered and are about to disappear...

Labels: , , , , , , , ,

Thursday, May 15, 2008

The S option seems to be a viable option!

Good ol' SQL improved the serialization of tests by leaps and bounds.

Now it takes less than minute and a half to create and store 5,000,000 tests. Before this much time was needed for 100,000.

Labels: , , , , , , , ,

Tuesday, May 13, 2008

The S Option

Given the enormous amount of tests can be generated with DevRiot, our old serialization architecture is proving not up-to-par.

SQL seems to be most logical way to handle this.

Well keep you posted. Sorry for the inconvenience!

Labels: , , , , , , , ,

Wednesday, May 07, 2008

Less than two minutes...

We have been tweaking around the serializer (getting rid of duplicate IO work) and writing, storing 100,000 tests takes less than 2 minutes. The development costs aren't why they used to be...

A more realistic approach to this feature can be: there is a method that takes a matrix and returns a modified version of it. Given the nature of matrices, a user could create a "base" test batch with a baseline matrix then modify the individual matrices with minimum effort.

Labels: , , , , , , ,

Monday, May 05, 2008

Going academic!!!

It is always encouraging to see that not only businesses find the white paper useful. A lot of universities, institutes of technology from all over the world have been downloading the white paper (even for course work)

Even a national police from South America :-?

Well, the serialization server is moving forward nicely, although the weekend was lost with a clogged up laptop. It has been deleting *.tst file for the last 48 hours...

Labels: , , , , , , , , ,

Friday, May 02, 2008

Storage...

I must have to say that developing code to serialize large number of objects can be quite time consuming.

My laptop seems to handle 100,000 or less files quite well. When the number jumps to 1,000,000, it becomes messy molasses to continue working.

Will keep you posted...

Labels: , , , , , , , ,

Tuesday, April 29, 2008

Gluttony...

Being able to handle large number of tests (5,000,000 in my dev machine, but it could be higher) is that serialization of suites becomes a little bit harder.

Well, that is what I am working on now...

Labels: , , , , , , , , ,

Friday, April 25, 2008

5,000,000

After tweaking some generation logic to economize the memory consumption, DevRiot reached the 5 million tests mark in less than 34 seconds.

Sounds fast, isn't it? Now, really going back to suite serialization...

Labels: , , , , , , , , ,

Wednesday, April 23, 2008

I tripled THAT...



1,500,000 in less than ten seconds. Back to Suite serialization...

Labels: , , , , , , , ,

Tuesday, April 22, 2008

500,000 and counting...

The Suite serialization is slowly starting. I am polishing some performance features.

But I wanted share with the latest readers some of the brute-force finesse of the tool:




500,000 tests, no coding, no wasted time. Enjoy.

Labels: , , , , , , , , , ,

Monday, April 21, 2008

It does not register

Well after having a question/post mysteriously disappear from the VSX forum, which is a first, I decided not to wait more on that.

Two registry entries will allow the user to gauge according their HW configuration what the max number of user/threads DevRiot can handle and the emergency TTL (when all current users/threads are harvested to avoid hosing down the box)

To be a product that has a really small exposure, there a lot of big copycats :-}

Labels: , , , , , , , , ,

Wednesday, April 16, 2008

Time To Change...

Well after fiddling around with the load/stress engine + 1,600 threads/users for a while, I think it makes more sense to let users set the TTL per test.

It is easier for the user to set it according to their knowledge of the code than use any predefined formula.

It also becomes part of the verification data of the test which should be useful.

Labels: , , , , , , , , , ,

Monday, April 07, 2008

Thread Quartet in K reverse...

The following demo shows a Load/Stress session on a object using four of its methods X 400 threads/users (1600 total).

In addition, there is the Assertion, Out-Of-Scope, etc, in a codeless, easy-to-use, low-skills-needed sort of way.

It is a little bit longer than the previous ones but worth watching (it has been edited to avoid the initial setup of the threadpool) :-}










Labels: , , , , , , , , ,

Sunday, April 06, 2008

1,500 Threads in action

Well , we wanted to try a higher number on our load/stress engine. So, we tried 1,500. Watch and enjoy the execution times after the initial run (when all the setup occurs).










Labels: , , , , , , , , ,

Wednesday, April 02, 2008

58 Seconds...

That is how long it takes to create, launch, execute, synchronize, and verify a load/stress session with DevRiot. Check it out for yourself...

Why wouldn't you want that level of efficiency in your SDLC?
:-}










Labels: , , , , , , ,

Saturday, March 29, 2008

Visual Threading!!

This is another progress demo with assertions, out-of-scope verifications, multiple threads were involved in a series of tests. Not a single line of code was written.

Try to achieve the same with other technology: Microsoft's, etc. See how much time it takes you to achieve the same. Then check how much more time it takes to get the results back.

How do you want to spend your time and budget? :-}










Labels: , , , , , , ,

Monday, March 24, 2008

VUT Threads...

This is almost identical to the previous demo but with the difference that the Load/Stress testing engine is hooked up.


Note that the video follows the same script as the previous one, so no PrivateObject API or coding or scripting. Even the thread creation, execution, and harvest.


The other difference is that we intentionally tried to add a test without pre-compiling a parameter so the tool could complain :-] Three threads are created and used against the method under test.










Labels: , , , , , , ,

Friday, February 15, 2008

Almost Context Full...

The context manager is 60% done. 4 snapshots of the context manager at work: assertions, cleaning up, etc.








Labels: , , , , , , ,

Saturday, February 09, 2008

Automation?

We are currently working on the ContextMenu manager. It should be a breeze since the Generics support actually meant a major addition of functionality.

GUI support should come soon too, its major piece of functionality is ready to wire.

Which lead me to look at the Automation library that MS has on .Net 3.5-VS 2008. It is not a surprise , and not a particularly interesting competitor.

First is stuck in PC land. None of the major classes are supported by the Compact Framework.


Second, it is another hard to digest class/method behemoth.

I guess they are not competing on usability and product definition/differentiation...

Labels: , , , , , , , ,

Friday, February 08, 2008

Compare me if you can...

These two are a couple of progress snapshots from the Generics Engine.

To show the flexibility and ease-of-use of DevRiot's Visual Unit Testing(tm) technology here we have a generic method that takes 3 generic parameters. Each one of them is a bound generic type A made of an array of bound generic type B. Type B is made of an array int.

Just a few clicks away. No Source code...




That ugly description turns into an easy to use tree widget:



Labels: , , , , , , , ,

Thursday, February 07, 2008

Short...

Now the generics engine can handle pretty much everything it is thrown at it.

The first ever Visual Unit Testing tools is coming your way...

Labels: , , , , , , , ,

Thursday, January 31, 2008

General Complexity

The Generics Generator is pretty smooth now:

This is an snapshot of a that involves: generic method, out parameter, multidimensional arrays, out of scope data (PrivateObject in VS parlance).

And it is the same as if it a method that takes and returns an integer. The user just spends time on what it matters most: substance and not boiler plate noise.

None of the competitors can achieve such a level of usability and ease-of-use. Let alone the execution speeds. We are pretty close to wire up the GUI testing engine...


Labels: , , , , , , , ,

Monday, January 14, 2008

Anything VUT...

Visual Unit Testing, this is the acronym that best describes the visual nature of DevRiot's user experience. It also applies to the GUI testing engine but there are more salient features that describe DevRiot's GUI testing engine: device independence and ditching the capture-and-replay paradigm.

Back to ironing out some bad bugs in the generics generator logic...

Labels: , , , , , , , ,

Wednesday, January 09, 2008

Params Row...

The thingy handling the params modifier is pretty stable now. There is a scenario where it NPE's but I am about to fix it. It is a little bit convoluted because it involves also the dynamic-scoping of parameters.

Next will be databases (offering a clean way for the user to test databases). Collections/Data structures will be partially supported for the beta. The GUI testing engine work will begin soon, hello .Net Compact we are going your way!!

Labels: , , , , , , ,

Thursday, January 03, 2008

Back on line!!

It arrived! After several weeks of waiting the snail mail to deliver the laptop converter. It arrived.

It was kind of an experience trying to order the spare part from here having bought the laptop in USA.

It would have been great if the surf had been less chaotic but winter storms can be messy around here.

Anyway, it is time to get back on track with the .Net beta :-}

Labels: , , , , , , , ,

Thursday, December 13, 2007

.Nest My Private Arrays

Despite the hardware troubles we have been able to make some progress.

It is still a little bit unstable but we can dynamically define a series of nested arrays (arrays of arrays) of type A, and set the values of any private member.

And everything takes 3 clicks and 4 keystrokes. Try that with other solutions :-}

Labels: , , , , ,

Monday, December 10, 2007

Stop!!

This is a not so good progress report. The development laptop batteries are not charding and the converter is not working :-}

I guess Dell will get another support call from around here.

Other than that, adding support for collections and modifying generics compilation is coming along well.

Time to wait...

Labels: , , , , , ,

Monday, December 03, 2007

Time Report...

Still working on how out/ref parameters are handled. We have been able to reduce the size of the execution engine by a good chunk. The goal was to make it small enough for the Compact Framework/J2ME.

But this means that the parameter processing needs some massaging. Out, ref, params can lead to some more spaghetti code than really needed.

Stuck here before fleshing out the widget handling logic (GUI testing engine)...

Labels: , , , , , ,

Thursday, November 29, 2007

For Completeness' sake...

I just compared DevRiot for .Net against Nunit 2.4.3. DevRiot is 2,400+ times faster than Nunit.

And it includes GUI and Load/Stress testing, try to find those inside Visual Studio or Nunit :-}

The waves are just getting started. I'll try a two session day tomorrow :-}

Labels: , , , , , , ,

Wednesday, November 28, 2007

The Fog of Fall...

Well after the dry and brown fall we have had so far (Galicia is like Seattle/Washington, rainy and green all year long) Some fog and rain has arrived last week, and with them some nice fall swells which I plan to take advantage.

As I mentioned in an earlier post, I managed to compare DevRiot .Net with Visual Studio 2008 Tools.

The performance of DevRiot .Net is really good, given we do not have access to any of the internals: on the low end DevRiot .Net is between 800-900 times faster than Visual Studio tools.



The time it takes Visual Studio to process the first test, DevRiot is close to 1,000. But if we reach the 100,000 tests, DevRiot outperforms Visual Studio tools nearly twice that figure.



As any sensible management guy would tell you: why waste time writing code to get slower?

I will be working on the GUI testing engine, particularly support for the .Net Compact Framework.


And, I will try to catch some waves. Enjoy...

Labels: , , , , , ,

Tuesday, November 27, 2007

Tick, Tick, Tick, Tick, Tick II

Well, I just tried VS 2008 testing tools.

Ahh, what a way to get it wrong from marketing to development.

I will continue working on the GUI testing engine and device support for the Compact Framework. I will keep the performance figures for later...

Labels: , , , , , , ,

Sunday, November 25, 2007

Tick, Tick, Tick, Tick, Tick...

This has been a really productive week. I accomplished to run DevRiot for .Net under the new architecture for the first time.

Let's say I will hold off until early next week to announce the performance improvements...

Labels: , , , , , , , , ,

Saturday, November 24, 2007

Simple Update...

Well, just working in the guts of the engine now. Where the fireworks occur. No snapshots to show but really entertaining to fiddle with.

Still undecided of how to cache things for the compact framework devices but making great progress.

Labels: , , , , , , ,

Wednesday, October 31, 2007

Context Adjustments...

Some times it is amazing where pointers to adjust features can come from.

And I have to say that MS seems to be on the ball supporting the extensibility community (not related to previous line)

Anyways, DevRiot's UI has changed a little bit to make things easier on inherited methods,fine tunning runs and having multiple instances of VS up.

We wanted to get this stable before get into the gui testing engine for .Net compact and device integration. It is going to be fun :-}



P.S. The second "Run Current Tests" really runs the whole suite.

Labels: , , , , , , , , ,

Sunday, October 28, 2007

Tell me what posts one is reading...

And I will tell you what one's marketing plan missed :-}

Well, Leopard came out. It may have solved some of the SWT/AWT problems that DevRiot was experiencing on OS X. I need to get back to it later. Some people got really upset that Java 6 is not included or better java support on OS X is demanded.

It is understandable but things should get less emotional.

After all, not long ago Apple was in the doldrums. It certainly came back but it is behind every big player in the Java world. And it is not going to let IBM or Sun have a piece of the Apple pie out of goodwill.

There are a lot more Java/IBM developers than OS X developers. That in itself is a major threat.

Well, I am going back to .Net and make sure DevRiot does not choke when several instances of Visual Studio are up and running...

Labels: , , , , , , , , , ,

Thursday, October 25, 2007

Context it!!

We have done a little backtracking with regards of the original way we planned to handle assertions. Interfaces and abstract classes will be handled the same way: context strip.

Here is goes another snapshot of the work-in-progress:


Labels: , , , , , , , , , , ,

Wednesday, October 24, 2007

What an eye sore!! (102,796)

Well, I finally managed to download all the components for Visual Studio 2008. It was long a process because of the size of the archives involved. Regardless, I have to say that the process was smooth.

Now to the substance, comparing DevRiot and the test tools of VS 2008.

I started with the basic: take an int and return it.

Nothing else fancier where DevRiot has advantage over VS: gui testing (on PC and devices, on and off screen), dynamic array/data structure manipulation, dynamic build test tress, built-in load and stress testing, etc :-}

It is an eye sore: test projects, test types, test attributes, test contexts, asserts, clicks, clacks to get to the point of to modify the test source to add the only thing that is of interest to the user: desired input and expected output. That is all a user needs. If the user needs to add a second test: repeat the process :-}

The thing is dog slow. It took (on average) 102,796 times more ticks than DevRiot. Granting that I was using Virtual PC but the times I am using for DevRiot are from 4 years ago on much slower machine than the current one I own.

If we assume a 50 times handicap (meaning that a 5000% improvement) DevRiot is still around 2000 times faster.

No wonder the execution time report is not the default one.

I tried to mess MS tool with changing return types, it choked like a child. Then I tried exceptions or even setting scenarios. It would have required getting into coding, api, etc..

I do not think they have clear concept of what automation means.

So, bottom line: why spend more time using MS QA infrastructure if it is not going to yield faster development cycles?

The pricing structure and road map discussion, I will leave for other post.

Labels: , , , , , , , , , ,

Wednesday, October 17, 2007

Progress in General:

Here we have a couple of the .Net tool working with generics:




It is funny how monitoring works. A big portion of the traffic drops, and then hits on the whitepapers and J2ME topics surge. And then a big portion of the traffic is back.

Labels: , , , , , , , , , , ,

Thursday, October 04, 2007

Off-topic:

Well, one thing led to the next. It started with using .Net 3.0 and the GUI testing engine, and ended up in: "we need to make sure that the off-screen engine works".

The off-screen engine is a feature of the GUI testing engine that allows these type of tests to run at speeds close to that of the unit test engine. The main goal is to increase the throughput of the process without losing accuracy.

BTW, it works...

Labels: , , , , , , , , , , ,

Wednesday, October 03, 2007

Generics 3.0...

Adding support for generics is well underway but trying the GUI testing engine with WPF?

Help needs to be local, msdn needs to be installed...

Ohh well!

Labels: , , , , , , , , , ,

Thursday, September 27, 2007

Autumn Nesting...

As my workload with Phurnace diminishes, DevRiot for .Net picks up.

Multi-dimensional array support is alive and well as the snapshot shows. Support for generics will take some time. The desktop and compact frameworks behave differently and the behavior of the engine needs to reflect this.

Well, the snapshot:

Labels: , , , , , , , ,

Wednesday, September 26, 2007

DynArray's P.S.

The one-click assert has been moved to a context menu where we will be able to provide a better way to handle assert conditions: greater than, etc, to return value, out parameters and post-test states...

Labels: , , , , , , , ,

DynArray...

The dynamic array, or data structures, logic is almost ready. We just need to add the dynamic shrinking to the nodes' context menu. Here we have a series of snapshots of how it's working:

1) Double-click the node to set or change the array length.



2) Type in the new desired length.



3) The tree readjusts itself.

Labels: , , , , , , , ,

Wednesday, September 12, 2007

One-click assert!

This is another of the progress snapshots: one click assert.

This feature allows the user to easily set the expected value, or range of values that an object should have. In this context a null or not null return value.

It does not matter if the object is private to a private field: just click away and forget about typing Asserts, PrivateObjects, or any of those rudimentary approaches!

Labels: , , , , , , , ,

Saturday, September 08, 2007

Hit first, hit twice

Apparently USA's congress has changed the patent rules for a "first to the clerk" type of deal. Not good or bad, just a change in the playground.

DevRiot for .Net is coming along well, variable number of parameters and arrays are cleanly and dynamically handled.

Comparing usability with competitors...

Still doing interesting stuff with my buddies @ phurnace.

Labels: , , , , , , , ,

Thursday, August 23, 2007

Prototipo; fundamentals, what fundamentals

The VS tool is coming along smoothly but a little bit slow. Phurnace's project has been quite interesting: started with a prototype with an rcp front-end and has turned into a WebSphere process server, web-services thing.

I have to admit that is highly interesting.

In this astonishing summer of news, there was this particular one about onshoring on cnn: an indian company was offshoring to Ohio some call center operations because the lower wages were not enough.

If the process and where economic value is created are not understood, there is little chance to success by trying anything else...

Labels: , , , , , , , , , , ,

Friday, August 10, 2007

DevRiot for .Net is moving:

Forget about PrivateObjects, TestTypes, or writing code to create tests. Here is a snapshot of DevRiot for .Net:



Copyright (c) 2003-2007, efeKctive, L.L.C.

Labels: , , , , , , , , ,

Saturday, May 05, 2007

.Net Progress, back to Bay Area, and other great escapes chapter...

Well, this RCP work does take its time but it is fun and it helps us. I have started to get the new architecture on Visual Studio together. This a little snapshot of the toolbar. (Not particularly different from DevRiot for Eclipse. And that's is the idea: lowering the learning curves...)



I also have been in touch with my friend to put DevRiot through the grinder of DataMining development. Progress but it will take time to show.

On another front, I have been meeting with some researchers from Santiago to start looking at the "writing engine" (so users will do nothing but look at test results and develop)

Finally, after 18 months I will be back to the States. The rest of the crew has been back several times but not me.

We will be visiting San Carlos/Bay Area. A new nephew is on the way :-}

I will post some pictures of an early spring trip to Costa da Morte which was quite pleasant: great waves and great seafood :-}

Labels: , , , , , , , ,

Saturday, March 24, 2007

Variable Load!

Interesting visitors to our blog lately. Some have registered in the beta program, some have not.

We are looking at ways to cleanly allow the user invoke methods with a variable number of parameters. So far we are leaning towards double-clicking the parameters tab to insert a new parameter state tab.

The design on Load & Stress is going too. Before we try to measure memory and all that, we should make sure the user agrees with the state the object is after the beating. It requires some thought.

Labels: , , , , , , , , , , , , ,

Thursday, March 22, 2007

Why all that bloating?

As I move along with the VS add-in of DevRiot, I started to look again at the built-in features of VS.

What a mess! To end up asking the user the same information that DevRiot directly asks, there is an endless sequence of menus, options!

Orcas is following the same path. Why does the user need to deal with code? Because it executes faster? Obviously not.

That automatic code generation just works with native types. After that is back to set/get methods to set objects ready to tests. Or play PrivateObject nesting game.

Does it provide GUI testing for PC or devices? Nope.

Does it analyze build trees or AST? Nope.

Again, it is a step forward but short. Otherwise people would not still be asking "why white-box testing is ignored"

Labels: , , , , , , , , , ,

Wednesday, March 21, 2007

Beta FAQ:

A few questions have come up:

Do you Support Swing?

Yes. Actually, we are far along in that regard. We just wanted to release something that worked in J2SE and J2ME uniformly.

Swing is not an official J2ME package so we decided to remove it from the release.

How does the Private members access work?

Think of it as Visual Studio's PrivateObject but on steriods. And for Eclipse, too.

It is always there working for the user. But the user does not know it. The feature is transparent and does not require coding. It is also immune to refactoring changes.

It also works for all the types involved in a test (gui or unit): parameters, fields.

It also works like a drill: the user can click into an object state as deep as needed.

Is it only the plug-in/add-in?

No. The plug-in/add-in has two things in it: the plug-in/add-in logic and the reduced version of the engine. The plan would be to provide an automation server where the output of the plug-ins/add-ins are consumed and value added stats are gathered.

This could be provided as a service or as an attachement to SCM.

Labels: , , , , , , , , , , , , ,

Tuesday, March 20, 2007

Virtual Solution 2209

Well, the actual performance increase over Nunit is 2209 times. Our average cost per test is around 8 ticks, while for Nunit 2.4 is 1700+ ticks per test.

So, I decided to look at the Test tools offered by VS 2005. Although a step in the right direction, not even close to the streamlining that most software operations need.

It is still a solution based on writing code (as if figuring out how to make code fail does not take time), they have to learn a never-ending list of attributes.

No wonder, people google for "why+white-box+testing+is+ignored" :-}

It allows to access private members (fields and methods) but their approach seems a babushka doll. It works fine if those fields are native types.

But if those fields are something more complex? Can the users apply the PrivateObject trick to types not-under-test?

After all, the type-under-test methods are affected by the internal state of any of its internal state components. Ah, the babushka doll!

For load testing, Visual Studio does not "require" to write code. It just puts the all the tests the user already wrote in a bundle.

What about daisy-chaining tests from the data layer or business logic layer to the gui to verify behavior across the layers? Even some tools, like code coverage, are there with the wrong crowd.

Hey, I am just trying to differentiate our product! ;-}

Labels: , , , , , , , , ,

600+ Faster

As we polish the new architecture in .Net, we decided to compare the execution speeds of Nunit 2.4 and DevRiot.

DevRiot is still cookin!! Just considering the overall times, not the average, DevRiot is in the order of 600+ times faster. Plus it is also a GUI testing engine that runs in Windows as well as PocketPC.

Why do we consider this important? Testing and quality represents a huge percentage, according to NIST it could reach 80%, of a given project costs. Vista is another example of this efficiency problem.

The startling thing is very few, if any, high-tech strategy courses or implementations really address this. So which kind of strategy ignores altogether 80% of the problem?

A former, I think, SAP CEO mentioned the software industry needs to copy the auto industry. McKinsey & Co has written articles about it too. Microsoft Research aims in that direction also. But the facts seem to say something else. Just a look at the different internet forums will confirm this.

So if we want to improve the situation we need to use the scarce human time in more efficient ways, and let the computers use their cycles in something useful.

Anyways enough ramblings, here is the snapshot of DevRiot .Net

Labels: , , , , , , , , , , ,

Sunday, March 18, 2007

Weird Downloads!

Well, After announcing the beta availability really low key, we have had a great download rate. Although some weird downloads:

Somebody using msn downloaded the Mac version of DevRiot for Eclipse and some wip snapshots.

Switching to Visual Studio after some months of serious Eclipse plug-in development is a true shift in the way of developing tools. Enjoyable but different :-}

Labels: , , , , , , , , , ,