You may be asking “Is it really still winter?”  Well, here in chilly New England the answer is YES. The Gomez team has been locked inside by our virtual fire places, building major enhancements to the entire Gomez Platform. Over the last few weeks, I’ve had the pleasure of providing sneak previews to many of our customers.

One thing rings true for all our customers, they share a common struggle:  managing Websites and transactions has become ridiculously complex. Our own Gomez data tells us that the average Gomez customer transaction uses over eight hosts, or unique domains — CDNs, cloud services and other third-party providers like ad networks, Web analytics and much more.  And when an issue occurs that impacts their end-users’ experience, finding the source of the problem across the entire Web application delivery chain can be like looking for a needle in a haystack. Monitoring behind the firewall is necessary but not sufficient.

The reality is that Internet is the new data center, and we now share this data center with a million of our friends.  The primary goal of the Gomez winter release is to help sort out the complexity of this new data center. We do this by providing an end-to-end view of monitoring—from the deepest corner of the data center through the firewall, across the Internet and over the river to Grandma’s browser we go.

For this firewall to browser view, check out the Gomez “Integrated Health View”—in one place on the Gomez Portal you have access to alerts, specifying whether a problem is just yours or you are sharing the misery with others across the Internet.  And if it’s your problem, you can quickly drill into Compuware Vantage monitoring for a look behind the firewall.

What about Grandma’s browser? Did you know that her Web experience may be completely different depending on the browser she’s using? With our winter release we’ve introduced a new Firefox agent and a new version of our Gomez Recorder, enabling you to measure how your site performs in both Internet Explorer and Firefox from around the globe or from your desktop.

So, if monitoring and problem determination is difficult enough in the Internet “data center”, how can you predict your application’s performance under load? What if Grandma is using a mobile phone, trying to order your birthday present during peak shopping hours using a retailer’s smartphone app? Gomez applies the same data center to browser view for load testing, enabling our customers to precisely measure how their applications will perform under load, from their customers’ point of view. With our winter release, we are extending our load testing solution to include mobile load testing, supporting over 5000 different mobile devices. We’re the first in the market to provide this capability. And we are also enabling you to monitor the performance of smartphone apps from iPhone, Blackberry and Windows Mobile devices.  Rest easy, your birthday present is easily ordered and on its way!!

Want more details?  Please read our FAQ.  Visit the dedicated Winter release Web page , check out today’s press release and read the great feedback we’ve received from industry analysts.

And, of course, we hope you’ll share your feedback too.

For starters, we wondered if shoppers have the same expectations in the online world as they do in brick and mortar stores during the holiday shopping season. Most people expect and tolerate (reluctantly!) the hassles of finding a parking spot and the longer lines at the shopping mall the week before Christmas.  They know there will be lots of other shoppers, and they give themselves a little extra time.  But what about in the online world?  Do web shoppers tolerate sites that may be slower because of peak traffic during the holidays?  Or do they expect sites to perform equally well as during normal traffic periods?

We found that the majority of online shoppers are not at all tolerant of poor performance.  In fact, 67% expect web sites to work well regardless of how many visitors the site gets during peak traffic times.  This was especially significant in light of the fact that 51% of the respondents spend a significant percentage of their annual $1,050 retail budget during peak traffic times.

So how many people experienced slow online performance? 72% experienced slower web sites more frequently during peak traffic times than at other times.  So almost three out four shoppers experienced delays during the peak periods when the majority of online dollars are spent. Ouch!

So now the really important part – what did people do when they experienced slow performance?  Well 78% realized that other shopping options are only a click away and went to a competitor’s site.  88% said that they are less likely to return to the poor performing web site.  47% left with a poor perception of the brand.  And finally, 42% said that they talked about the bad experience with either friends or online.  All really bad stuff if you are an ecommerce manager that invested a lot of time and money in attracting shoppers to the site in the first place.

What about consumers’ experiences with financial services and travel sites?  Are they equally as intolerant to bad performance during peak times?  You betcha.  53% would abandon a travel web site at peak traffic times and book somewhere else after only one or two bad experiences.  57% of online stock traders would switch to a competitor if dissatisfied with their financial provider’s web site.  The financial services finding was especially interesting, because it means opening up a new account at another financial institution – a much bigger deal than simply going to another retail shopping site.

So what’s a web business manager to do? For starters it is critically important to ensure your site can handle traffic during peak traffic periods.  For more details, you can get a copy of the full report at http://bit.ly/a2EU8Q, or view it here.

Outside and Inside

There has been so much feedback already from customers, analysts and the press on the unprecedented solution we putting into the market.  One of the greatest things is how easy it is to explain and have our customers understand quickly how we can help them. It goes something like this:

1) The Internet

We see more and more applications and infrastructure accessed across the Internet. Used to be mostly shopping type apps but now everyone expects to access everything across the Internet from home or from their smart phone.

2) The Most Critical

These are some of the most critical applications which are usually visible to customers, dealers, employees.  When there’s a performance problem, it’s visible and there is impact on the enterprise reputation, brand, revenue etc.

3) Solving the Whole Problem

So Gomez’s best in class solution for the Internet can identify the problem be it ISP, CDN or datacenter. Combined with the Vantage solution for the datacenter customers really can quickly solve the problem. It’s the “holy grail” – being able to trace that externally visible problem to the exact piece of offending infrastructure so it can be quickly resolved – wow.  NO ONE COMES CLOSE TO SOLVING THIS LIKE US.

4) Integrated

One of the biggest complexities to solving IT issues is everyone has a different point of view on the problem because they use different tools to see it.  With the combined solution there is only one viewpoint allowing the right people to be working on the right stuff, rather than the hunt & pack method which can be frustrating and time consuming.

5) Managing

Rolling that all up into a dashboard that makes it all visible has very high value to our customers. They can see what the biggest issues are, look at trends to identify systemic problems, and talk to their internal customers in a much more meaningful way.

To the Gomez team

Welcome! We are so glad to have you join us. We have been rebuilding Compuware over the past year or two. We are looking to learn a lot from your team on different ways to address the market and customers.  I am amazed in every meeting how so many talented, motivated and focused people came together to create the remarkable success you’ve had to date.  Of course lots more hard work ahead for all but sure looks like a bunch of fun ahead too.

There hasn’t been this kind of excitement in Compuware in years. Inspired by what can be our opportunity to “Take the Hill!”

Mark

Over the years, though, even a novice sawdust-maker like me has come to appreciate the smart woodworker’s creed: “Measure twice, cut once”. It’s critically important to make sure that everything is in order before you put saw to wood. Many times, you only get one shot at it. (A too-short board, as it turns out, will always be too short no matter how many times you cut it.)

Last week, I was reminded of that sage advice despite the fact that I was miles away from my woodshop. I had the privilege of delivering a webinar with co-presenter Amy Schade from the Nielsen Norman Group. Amy and I were asked to give a presentation on “10 Mistakes Your Website is Making (And How to Fix Them)” .

Amy kicked things off by providing a few examples of how things turn out badly without proper planning and insight. She went on to provide examples of mistakes such as “Believing people read what you write” and “Blocking progress”. Each example pertained to an important design or content consideration, and each was illustrated with a real world example.

Soon, it was my turn. I managed to unglue my hand from the conference room table and advance to my first slide. The first mistake I described was “Not taking a walk in your users’ shoes”. Over the past several years, web applications have become more powerful, more distributed, and more complex. Unfortunately, while an application’s end-users are commonly strewn across the globe, operational IT metrics are often myopically focused on the infrastructure – an increasingly poor proxy for true end-user experience.

Next, I described the mistake of conducting functional and acceptance testing using only a subset of the web browsers that the application’s actual audience chooses. The fact is that web browser choice is increasing and an even treatment of “standards” by these browsers is decreasing. The assumption that “if it’s good in one major browser, it’s probably good in the others” is costing a lot of businesses money right now. These losses stem from abandoned shopping carts, escalating call center and operational costs, and brand damage.

Finally, I described the mistake of “Not preparing for success”. This happens when the owners of a web application fail to measure the likely experience of end-users when the application is under load. Just because the database doesn’t fall over, it doesn’t mean that real end-users are going to wait around for an extra 30 seconds to read a news article or search for a hot product.

It was great fun to present with Amy, and the webinar was very well attended. I think that there were two things that made this presentation particularly engaging and effective:

1. I didn’t share any of my best practices in shop safety.
2. The tips covered both non-technical and technical aspects of web applications.

The simple fact is that both the design and technical aspects of a website have to work. To me, “measure twice” relates to the need to measure both of these critical dimensions of user experience. Who hasn’t experienced the frustration of a poorly designed website or one that is chock full o’ broken links and errors?

Go ahead and check out the webinar and see if you agree. Whether you do or whether you don’t, you’re always welcome to drop by my shop and tell me your thoughts – just follow the smoke. Or you could just drop in a comment below. It’s probably safer that way.

Over the last decade, companies have worked hard to create Web applications that quickly, effectively, and efficiently serve the needs of the high bandwidth, desktop/laptop, wired /WiFi world. It wouldn’t be a stretch to say that the traditional Web has become addicted to fat, fast pipes (or tubes, for some folks in Alaska).

Now these same companies have to turn their world around and spin off smaller (in screen size) and lighter (in byte count) versions of their Web applications without sacrificing functionality or compromising performance.

When it comes to the mobile Web, companies clearly see the gold in them thar hills. But in the rush to the hills, are companies leaving behind the Web performance lessons learned over the last decade? A new survey conducted by Equation Research for Gomez found that 60% of respondents have experienced a mobile Web performance issue in the last year. Most folks reported cases where mobile sites were perceived as being too slow. Others found that mobile Web sites didn’t render well on the devices they used, or didn’t function as expected.

Two out of three mobile users have had problems with a mobile website

It’s as though as they race to chase the gold in the mobile Web world, companies are leaving the mule with all their Web performance tools behind at the saloon.

Does performance matter in the mobile Web world? The same study found that 58% of respondents expect mobile Web application performance to be equal to or faster than traditional Web performance. And when the performance doesn’t meet their expectations, they won’t visit the site again, and may use a competitive mobile site the next time.

Is it realistic for mobile Web users to demand equal Web performance on connections that at any minute could experience performance issues or disappear altogether?

No, it’s not realistic. Does that matter? No, not really. People don’t see problems with the network when a site is slow; they see issues with the site. The mobile Web is being held to a higher standard than the traditional Web was at the same point in its evolution as people have come to expect consistent and fast online experience wherever they are. This expectation of performance will only increase as the mobile Web becomes even more embedded in our daily lives.

Creating a mobile Web application or site is only the first step in entering this new territory. Remembering to bring the mule with all your critical Web performance tools and methodologies with you should make the trip more profitable.

I’d like to use the rest of this post to share some of the key takeaways that Imad, myself, and contributors from our audience shared during the discussion.  These are actions you can take today to make your site perform and handle better in today’s multi-browser world:

1. Always know which browsers are visiting your site. Before you can create a cross-browser performance strategy, you need to know what browsers you should focus on. The browsers that hit your servers can vary significantly depending on your site’s audience. You can find this out using your marketing analytics tools or a performance analytics tool. Once you know this, you can tailor your performance strategies to have the maximum impact on your users.
2. Perform functional testing on all browsers. You may miss critical defects in your own code or in third party components if you only test using one browser. Multi-browser functional testing can create a difficult scaling problem for your test bed because you have to run each test multiple times, but there are tools to help with this.
3. Re-evaluate your old performance optimizations. Traditional performance optimizations may not be relevant to your audience anymore. Techniques like domain sharding that were clear wins in the IE6 era may not have the same effect now that modern browsers use more connections per hostname.
4. Focus on client side execution time. Much of the innovation from the newest crop of browsers is in the areas of parsing and script execution speed. There can be significant differences in the performance of Javascript and CSS engines across browsers, so you need to check your performance on all of them.
5. Look at differences in cache behavior. There are subtle differences in how the popular browsers cache objects, so it’s important to test cache hit ratios across all browsers to make sure you are getting good cache performance across the board. During our session at TAE we shared a case of a site experiencing extremely good caching behaviors in IE, but very poor performance in Firefox and Safari.
6. Look at object download order on all browsers. This goes to perceived performance – the idea that given two pages with the same download time, the page that loads the most important objects first “feels” faster. While object download orders tend to be similar across browsers, subtle differences may have a significant impact on the perceived performance of your site for a critical set of users. We showed an example of this in our session where the most prominent object on a page was loaded last by a popular browser.

The ongoing challenge we all face is that we have to constantly adapt our performance techniques to stay in sync with changes in user expectations and browser capabilities. I hope these six suggestions provide a start for getting your site to perform well in all the browsers your users use.

I’d love to hear what cross-browser performance tips others have found – please share using the comments box below.

By the way, if you’re interested in checking out the slides Imad and I presented at The Ajax Experience, they are available via Slideshare at http://www.slideshare.net/Gomez_Inc/web-compatibility-and-performance-testing-in-a-multibrowser-world

I was conducting a bit of performance testing and optimization the other day, when my mind turned to KISSES. I decided to take a break and read up on the world-famous candy over at my favorite website, Wikipedia.

It turns out that KISSES were introduced in the United States in 1907. The signature “plume” that emanates from the tip of a KISS is inserted as a part of an automated wrapping process at one of the Hershey’s factories. Good thing, too, considering that the company produces 80 million HERSHEY’S KISSES brand chocolates every day.

Now that’s a lot of chocolate.

With that many KISSES being produced every day, I’m certain that the good folks over at Hershey have worked hard to squeeze every delicious ounce of efficiency out of the manufacturing process. A cost overrun of even $0.001 per KISS would eventually add up to the GDP of a small city-state. (No offense, Monaco.)

This is what got me thinking about the KISS in the first place. Almost every website that I analyze has failed to take a Hershey’s manufacturing-style approach in its construction. I regularly find bloated and unnecessary web code, spurious server connections, and poorly performing 3^rd party contributors.

Sure, taken at face value, a small website inefficiency looks about as sizable as a delectable drop of chocolate. But as Lucille Ball knows all too well, chocolates rolling off the assembly line can really add up quickly! Each inefficiency is added to the next until the entire end-user experience is put at risk. Multiply this across a high volume of web visits and it’s bye bye completed online orders, so long advertising dollars, hello escalating call center costs, buon giorno escalating infrastructure costs … oh, you get the point.

Check this out:

When I went back to my research, I figured I should visit the web home of HERSHEY’S KISSES. I was greeted by a very cool interactive Flash application depicting – what else – an animated assembly line. I decided to analyze one simple component of the website, its Cascading Style Sheet (CSS).

As of this writing, the site’s CSS stylesheet is an external reference located at:

http://www.hersheys.com/kisses/lib/css/styles1.css

I popped that URL into a free online optimization tool called Code Beautifier that analyzes stylesheets for any excessive notation or characters.

Within seconds, the tool pointed out that there may be an opportunity to slim down the chocolately stylesheet:

Input: 20.495KB, Output: 13.61KB, Compression Ratio: 33.6%(-6885 Bytes)

Using an even more aggressive compression algorithm, further gains appear to be possible:

Input: 20.495KB, Output: 12.531KB, Compression Ratio: 38.9%(-7964 Bytes)

Now ~8KB doesn’t sound like a lot, but across thousands or even millions of hits, every little bit can add up. Plus, most websites would benefit from multiple optimization techniques that collectively could speed things up.

There’s an undeniable link between web performance and user experience/behavior. If we want to wring every bit of potential out of each user visit, we need to wring every bit of performance out of our web applications. If you’re a web developer, use a few of the myriad of available tools to diagnose and optimize web performance. If you’re a marketer or businessperson, ensure that your techie counterparts are doing everything they can to keep things moving. Sadly, many website owners and the businesspeople who count on these critical web apps don’t even realize there’s a better way.

If you’re just not sure where to begin, let me know. I realize this topic can be confusing and a bit overwhelming at first. If you’re really desperate, lay out a fresh bowl of HERSHEY’S KISSES and I’ll be right over.

I’m honored to have witnessed quite a few of those moments lately.  The vast majority of them have involved a discussion about load testing.  These typically have fallen into two categories:  conversations with people who are familiar with internal load testing and are being exposed to outside-in load testing for the first time, and conversations with  people who are familiar with outside-in load testing but haven’t thought it through to its logical conclusion:  real-world load-testing.

Given that most of you reading this blog undoubtedly understand and appreciate the need to test and monitor Web applications from beyond your firewall, I’ll talk mostly about the second category of conversations that I’ve had.

One of the surprising trends that I uncovered is that, while these customers understood the need to monitor the entire application from the end-users’ perspective, they didn’t always translate and transfer that belief to the load testing world. I found that their load tests, although conducted from beyond their firewalls, would often target only the components of the Web application that lived within the firewall.  Our conversation would then go something like this:

So, you’re telling me that you only load test the components that come from behind your firewall?

Yes, of course.

What about the other components that make up the Web application? For example, your CDN, your ad vendors, your analytics, your other third parties?

Well, I’m certainly not going to load test my CDN! That would cost too much, and there’s no need! It’s supposed to handle any load I throw at it! Same with the other third parties. That’s what the SLAs cover.

Aren’t you interested in how all these various components interact, and how they impact the end-user response time when they finally come together?

That’s a silly question. Of course we are. We do some spot testing of the complete application in our QA and staging environments.

And isn’t one of your requirements getting to a test scenario that is as close to the real world as possible before you go live?

It absolutely is!

So, when’s the first time that you get to put all these components together and see how the entire application works under a real-world load?
Well, that would be…uh… when we go into production…oh….OH!

At this point, I typically do get the pleasure of seeing that twinkle, that expression. After a suitably long moment of reflection, we carry on and discuss the finer points of understanding how end-user response times vary as a function of the load put on the entire Web application delivery chain. We turn the focus on how that load is generated. Does the Web application use any JavaScript? If so, is your load generation running a full browser or merely spitting out canned, pre-recorded HTTP requests? If it’s the latter, will it truly be able to deal with the dynamic nature of your Web application? Do you know that the order in which components are called is governed by many parameters and might change with each execution of the JavaScript engine, based on the browser, the performance of individual third party objects, and countless other subtleties that can greatly change the end-to-end response time? If your load generator consists of running a large number of browsers on the same machine or cloud instance, do you know that you’ll be invalidating any response times that the system will collect, simply because the browsers will be putting an unrealistically heavy load on the machine or cloud instance they’re running on, affecting the end-to-end response time?

This is typically when the next “oh!” can be expected.

Then another moment of reflection.

Then: “so, tell me again, what’s the right way of doing this?”

Well, the right way is to test the entire Web application, from the end-users’ perspective, from wherever they are going to be. Yes, large machines or cloud computing locations can certainly be invoked to help generate high load volumes to put stress on the entire infrastructure, third parties and all. However, response times should be collected from a true end-user network,  where each testing location is running a full browser, and is only running one test at a time (just like in real life!), so as to provide an accurate end-user response time. That means that the network has to be large enough and diverse enough to reflect the real world, YOUR real world.

“Aha!”

How are you injecting reality into your load testing?

1. Availability
2. Response Time
3. Consistency

This week, I decided to review our July Gomez/dotMobi Mobile Web Experience Banking Benchmark results and compare them to our longstanding US Retail Banking Benchmark results  from the traditional web.

Initially, I was struck (figuratively) by the similarities between the two benchmarks – mobile and fixed web – when comparing the averages for Availability, Response Time, and Consistency. For example, the banks in both benchmarks regularly provided more than 99.5% Availability for the month. Even in terms of Response Time average, the fixed web and mobile web came in at 3.38 seconds and 4.30 seconds, respectively.

For some, these comparable results may be more surprising than a Patrick Duffy dream sequence.

Before we rush to judge the results from the fixed web and mobile web to be “essentially the same”, let’s make another quick comparison. The range of fixed web Response Times is substantially greater than that found on the mobile web. Said differently, Gomez found that in July the slowest mobile banking website was 50% slower than the fastest while the difference between the tortoise and the hare was 1000% for the fixed web.

I offer up three potential reasons for this difference in the observed ranges of response times:

1. Mobile web applications are generally lighter than their fixed web counterparts. More homogeneity in mobile page sizes may be influencing the response time results.
2. Most of our mobile banking benchmark participants are top performers in our fixed web benchmark. These top performing banks “get” web performance (fixed and mobile) and have made investments to get it right.
3. Mobile web optimization techniques aren’t as prevalent as those in the fixed web. This might have the impact of keeping mobile website experiences more similar than different – a phenomenon that is definitely not in force on the fixed web where differences between the haves and have nots are both dangerous and common.

It’s critically important that we keep our eye on mobile banking web experiences. For their part, IDC Financial Insights is predicting  “imminent” and substantial growth for mobile banking. In a rush to launch mobile web offerings, let’s hope that banks can keep up with end-user experience expectations. This includes the full gamut of expectations – from Availability to Response Time to Consistency.

Let me know if you have any questions about the current and future state of performance on the mobile web, from banking to search to social networking to whatever. I’d also encourage you to pop on over to visit our friends at dotMobi.

Every day, more and more people are using their smartphones and other mobile web-based devices in new and powerful ways. Let’s work together to make sure that we stand ready to serve them.

I’ll be writing about a range of topics, including web performance, business performance, user experience, application design and testing, and industry trends. I’ll also be writing about the “people side of performance” including communication, organizational change, goal-setting, and education.

In case you didn’t pick up on my subtle Heathers reference in the opening line, I should probably mention that I like to keep things a bit less formal. I might make a feeble attempt at humor or share my own experiences from around the web. Who knows? You may even see an A-Team reference (because that show was so awesome).

I do have a favor to ask, though. If you see something you like, please weigh in. If you see something you don’t like or don’t agree with, definitely weigh in. You can use the comments feature below, follow me on Twitter using @mattpoepsel, or email me at mpoepsel@gomez.com. I look forward to hearing from you.

And now onto the real meat of the matter …..

About a year ago, I finally jumped on the HDTV bandwagon. High Definition Television is one of those things that’s difficult to describe until you experience it for yourself. You’ve either seen it or you haven’t.

When I first turned on my shiny new television, I’d watch anything in HD. Sports, movies, home improvement shows, Kathy Griffin – you name it. For weeks, I’d stare at the screen, marveling at the sharpness and clarity of the picture.

Over time, the newness wore off. I got used to seeing the individual blades of grass on the Fenway Park infield and all of the other incredible visual details. Now, when I’m forced to watch a low-def television (or Kathy Griffin), I want to throw myself out the nearest window.

What happened? Easy – my expectations around what constitutes an “acceptable television watching experience” evolved. What’s really amazing is that I find the visual experience even impacts my interpretation of the actual content of the show.

A similar increase in expectations has happened online. Twelve or fifteen years ago, it was cool if a company had a website at all. (If you want to take a trip down memory lane, check out the Internet Archive). In 1996, even brochure-ware was welcome. Today, web visitors expect fully-featured websites and web applications that are zippy to load and to use.

In my consulting work, I frequently reference this evolution. I encourage clients to baseline the technical experiences they’re delivering along several dimensions that mirror visitors’ expectations, including:

• Availability – Can users successfully load pages and complete their desired tasks?
• Responsiveness – Do pages load quickly?
• Consistency – Do pages load in a uniform manner for every user, every time?

But, while most business’ IT groups regularly track these metrics, they’re not always collected from – and therefore are not reflective of – the end-user’s perspective. After all your customers don’t live in data centers, do they? A better approach is to monitor your web performance from the same geographies, connection speeds, and browsers that real users choose. (And, judging from the wide range of results in our benchmarks, I’d say there’s a lot of work to be done.)

Unlike A-Team reruns, web users’ expectations have changed over the years. They have come to expect more, and they will punish companies who don’t meet those expectations. Experience tells us that they will click over to a competitor, call your call center, or complain to friends and co-workers.

Companies who aren’t regularly making sure they’re meeting these heightened expectations are taking unnecessary chances that may be impacting both their brands and their bottom lines.

My advice: measure, analyze, and improve the experiences you’re delivering. Your users will appreciate it, and it’s a lot cheaper than the alternative.