Recent Posts

Business Geeks: Automated Software Testing as Competitive Advantage

By on September 13, 2006

This blog’s audience can be simplistically divided into two types of people: 

1.  technology geeks (folks with a technology background, and more specifically a software development background) that have an interest in the business issues because they’ve founded or are thinking of kicking off a startup.

2.  business geeks (folks with a business/sales/strategy background) that have an interest in technology because they’ve founded a software startup.  For more on my thoughts on business geeks, read “Business Geek:  Not An Oxymoron

A number of my articles address one group or the other (like my “Presentation Tips for the Technically Gifted”.

This one looks at the value of automated software testing from the perspective of the business-side.  The reason for the focus is that most programmers I know and respect already understand the upside to automated testing and know way more than I do.  If this is you, feel free to stop reading.  I won’t be offended.

Business Thoughts On Automated Software Testing
Automated software testing is a large and relatively complex area that takes a while to understand.  But, let’s work with a simple definition:  It is the process of using computers (instead of humans) to run repeated tests to determine whether the software does what it is supposed to do.  It is important to note that most automated software testing still involves humans in the beginning (to design and develop the tests), but it’s the repeatability that makes it so powerful.  Once the tests are developed, the payback is continuous because the costs of running the tests are near zero.

In order to better illustrate my points, I’ll use Pyramid Digital Solutions (the first software company I started).  Pyramid ran successfully for 10+ years and was recently sold, but I like to use it as an example because I actually lived a lot of these lessons and I find it helpful to have a real-world example to talk about.
  1. Build Better Software:  This one is obvious, but is at the core of the value so needs to be said.  By building a library of automated tests, you are generally going to ship better software that at least, at a minimum works when used in certain, predictable, preconceived ways (the use cases that have been accounted for in the tests).  This is a good thing.

  1. Test Continuously:  As noted, once you have tests automated, there is very little cost to running the test.  As such, once you’ve made the investment in building automated test scripts, there is no good reason not to run them frequently (and lots of good reasons to do so).  In my prior startup, we eventually got to over 20,000+ test scripts that run for several hours.  We ran them every night.  Each night a process would fire off that would retrieve the latest source code the programmers had checked in, build our product (automated builds) and then run our test scripts.  Every morning, the results of the test scripts got emailed to management and the development team.  

  1. Cheaper To Fix Bugs:  Most software has bugs.  From the business perspective, the questions are:  which bugs do you know about, when do you “find” them and how much does it cost to fix them?  As it turns out, when you find them and how much it costs to fix them are highly correlated.  Lets take an example.  From my prior (real-world) example, lets say a programmer inadvertently makes a code change and checks it in.  The code has a bug.  In the old way we used to operate, it often be days, weeks or months before that big got caught (based on what part of the product the code was in, whether it was caught internally, or made it out into the “wild” to be found by customers).  The more time that elapsed from the when the code actually changed, to when the bug was actually found, the more expensive the bug became to find and fix.  We’re talking major (orders of magnitude) increase in costs.  Now, in the new world (where we had automated tests running every night), this bug may be caught by the automated test scripts.  If so, the very next morning we would know there was a problem and we could go fix it. The reason it was so must cheaper to find and fix the bug was because the “surface area” of change was so small.  A limited number of things got changed in the prior 24 hours (since the last test), so the bug could more easily be discovered.  I cannot emphasize enough how much money you can save by catching bugs within hours (instead of days) of the bug being introduced.

  1. Freedom To Change:  As software systems get bigger, it becomes harder and harder to make changes without breaking things.  Development teams do what they can to refactor the ugly bits of code as they can (and as time allows), but even then, a sufficiently large-scale codebase that has been around for a while will almost always have “corners” of it that nobody wants to touch (but are important).  The business risk to this situation is that you may find yourself in a situation where customers are asking for things or the market shifts in some way that causes the need for change (this should not come as a surprise).  If the programmers are fearful of changing core parts of the system for fear that they’ll break something, you’ve got a problem.  If you’ve got a large battery of automated test scripts, it frees the programmers to do lots of cool things.  They can refactor the code (the automated testing is a “safety net”), they can add/change features, etc. with a lot less loss of sleep.  What you will find, by investing in automated testing is that your organization actually moves faster than it did before.  You can respond to market change quicker, you roll out features quicker and you have a stronger  company.

  1. Clients Are Happier:  At Pyramid, we had quarterly meetings with our clients (and an annual conference where a bunch of clients got together).  At each of these, one of the key metrics we shared was how large our automated tests were.  This gave clients some comfort.  This comfort translated into a higher likelihood that they would install newer versions of the software (when the became available).  Since we were in the high-end, enterprise software space, this was a big deal.  If we could get 20% more of our customers to move to Version 5 of our software (instead of staying stuck on Version 4), we had advantage.  Less support costs and higher retention rates.

I like to think of technology strategy in the form of technology debt (take short cuts now, but payback comes later – with interest).  Read “Short-Cuts Are Not Free” if you’re curious about this.  Similar to financial debt, this is often necessary (as is technology debt) but it has a cost.  The reverse of this is Technology Investment (in the classic sense).  This too has an interest rate – the rate you get “paid” (i.e. ROI) on that investment.  I think investment in automated testing is one of the best interest rates you can find.  The payback period is a little longer, but it is worth it.  If you have competition (which you likely will), you will find that having a strong investment in automated testing will give you advantage.  You’ll add features quicker, fix bugs cheaper and ship better software.

Of course, as is always the case, situations vary.  Pyramid was in a different market than my current startup HubSpot – but I’m still passionate about automated testing.  Will continue to share experiences as I progress.
Continue Reading

Web 2.0: Soft Landings vs. Hard Crashes

By on September 11, 2006

I’ve gotten a bit of criticism for a couple of my prior articles about the Web 2.0 startup crashes of  Kiko and HuckABuck, both of which listed their companies for sale on eBay.  [Note:  Kiko sold for $258,000 whereas HuckABuck was not able to reach its $20,000 reserve price.]

Reflecting on this a bit more, I think the criticism around my use of the word “crash” was well placed.  Indeed, I think the term “crash” was not particularly accurate of what actually happened (such as in the case of Kiko).  If the companies had raised tons of VC and gone down in burning flames, the term crash may have been appropriate.  But, given that they did manage a graceful exit (well, at least Kiko did) and didn’t burn a bunch of capital in the process, they had more of a soft landing than a hard crash.

In Support of Soft Landings
  1. Capital Efficiency:  Startups are raising less money.  Gone are the days of tens of millions of dollars being shoved into many early stage startups.  Without anxious investors involved, a real crash is less likely, as nobody is pushing for the “grow fast” strategy. 

  1. Lower Price Requirements:  This is related to #1, but since startups are raising less capital, the price acquirers need to pay is lower.  As such, there are a larger pool of possible acquirers. Had Kiko raised $4MM in VC (like many VC-backed startups), they would have had a harder time finding an exit.

  1. Features vs. Companies:  Many of today’s startups are focused on building features instead of full companies.  As such, they are more attractive to specific types of buyers that are looking to enhance their product offering.  This seemed to be the case with Kiko’s acquirer Tucows.

  1. Quicker Market Results:  A majority of Web 2.0 startups are focused on traffic/user aggregation.  As such, they are usually closely watching the “uptake” in the market.  If they don’t hit the numbers they want, they may lose hope and decide to move on to something else.  Basically, they’re playing the Web 2.0 lottery.  Once the results are in, and they know they likely don’t have a winning ticket, founders may decide to just try something different.  In this case, they just generate what cash they can and move on.

  1. Efficient Channels:  It seems that there are now more channels out there for startups to find an exit path.  One striking example is eBay (the path chosen by both startups I recently wrote about).  By using more “efficient” markets online to sell their startups, founders have the option to not get up in the quagmire of locating a buyer and negotiating a price.  This makes it easier to “land softly”.

So, I think we’ll probably see more of these “soft landings” in the future.  One point I’ll still hold firm on (despite the criticism) is that no matter how you slice it, these outcomes are not really “successes”.  Sure, the founders learned a lot, had fun building something cool, were able to serve some number of users and have a great story for their friends and family (all good things).  But, it’s still a landing and not a take-off.  The good news is that these founders (like those from Kiko) will likely take another shot at it.  And that’s always a good thing.  We need more people taking risks, trying out new things and defying the odds. 
Continue Reading

Recommended For You


Let's Connect

And, you can find me on Google+, Twitter, and Linkedin.

Recent Posts

Chat with GrowthBot

It's a bot to help you with your marketing and grow. You can research your competitors, improve your SEO and a lot more. http:/