PHISSUG

I have users telling me that 3 to 5 seconds OLTP wait state is bordering on not acceptable.

I have users telling me 15 seconds is too long for a report to be generated.

I have users giving me a sharp look for a batch proc running for 20 minutes.

I am curious, when do you say to yourself that what you did was slow?

In general for me anything that is (too) slow to be acceptable, especially when you know that this process shouldn't really this be slow, I investigate. The users are impatient, that's a given, a little information about the process alleviates, although not all the times, that anxiousness or impatientness.

I was supposed to blog about this topic but I want an engaging discussion on this topic.

i agree. generally, speed is relative. however, the moment you give people speed, they will crave for more. And we as developers cant say to users, 'hey, that is fast enough', when in fact, for them is it pathetically slow. I have users say that 3 seconds is too long for anyone to stare at the PCs waiting for something to finish whle at the back of my mind 3 seconds is the best that i can achieve.

the reason why i posted such questions is for me to guage how slow really is slow given similar applications. For example, given you have an OLTP application, perhaps like a POS system all over a gigantic mall, is a 10 seconds wait state after posting acceptable? with batting an eyelash, 10 seconds could be a breeze, but is it? how about 3 seconds?

with the absence of a performance requirement from a customer, how do you treat speed when designing databases and applications. is it one factor that influence our design or is it an afterthought?

on a related observation, I also have noticed that most web developers dont give high regard to speed, owing perhaps to the very nature of web apps where it is normal to see users wait while the browsers display in all its glory varying degress of progress indicators. I also noticed users tend to be patient when their apps are run via the browser, than when you see them use the same app running as a console or a windows forms. They easily see "speed" issues when they are not engaged with a browser.

So when we design our databases and apps, how do we set speed criteria? I believe we have to.

ggsubsubscribe:

i agree. generally, speed is relative. however, the moment you give people speed, they will crave for more. And we as developers cant say to users, 'hey, that is fast enough', when in fact, for them is it pathetically slow. I have users say that 3 seconds is too long for anyone to stare at the PCs waiting for something to finish whle at the back of my mind 3 seconds is the best that i can achieve.

This usually happens on a from the ground-up application implentation, the reason they're getting a quick response is because the data isn't that big enough. Usually designs/scalability issues surfaces when the data is already huge, performance problem rears its ugly head especially when the app wasn't just designed to handle that kind data. We experienced this one when our data baloon exponentially, the application w/c was made for us, we discovered wasn't up to the task(databound controls littered all over and processes w/c takes several trip to the server etc. yikes!!) Temporary solution was to scale out the server, then we dug ourselves dirty with their code to at least minimize the problem.

I agree that most of the times users are unreasonable, the barometer being our deprecated DOS/Clipper apps, especially browsing and printing, we end up hand-coding our reports(we abandoned crystal reports) and utilize asynchronous data fetch for a more "faster" load. 3 seconds is too slow if you're just retrieving Patient information, this should be below that time, however sometimes it is way beyond your control as developer. Network reliability, hardware faults and most of the time users are at fault(they forgot to confirm the printing hence they waited staring at the printer) are some of the reasons why process is slow. We have a WAN configuration, however we're using Laser to connect instead of digging a fiber on the ground, when sandstorm hits the users couldn't even connect.

ggsubsubscribe:

the reason why i posted such questions is for me to guage how slow really is slow given similar applications. For example, given you have an OLTP application, perhaps like a POS system all over a gigantic mall, is a 10 seconds wait state after posting acceptable? with batting an eyelash, 10 seconds could be a breeze, but is it? how about 3 seconds?

10 seconds is slow for a POS, what's even slower is the swipe cards w/c connects thru a modem, I experienced waiting by almost half a minute then timed-out. Now those are things that is out of hand already for a developer, nomatter how carefully designed his/her system is.
 

ggsubsubscribe:

with the absence of a performance requirement from a customer, how do you treat speed when designing databases and applications. is it one factor that influence our design or is it an afterthought?

I guess this is almost all time missing from the specs, I guess on our part this is good idea, if the client specifies a speed threshhold we might be spending alot of time finetuning that we might miss the deadline or concentrate too much on that portion that we might be underminning other portion of the architecture. Having said that however, if the application is time sensitive(eg. airline booking), then the threshhold shouldn't be an afterthought. I guess in the end we'd have to weigh the trade offs especially if we're still in system designs, we might be prematurely optimizing.

ggsubsubscribe:

on a related observation, I also have noticed that most web developers dont give high regard to speed, owing perhaps to the very nature of web apps where it is normal to see users wait while the browsers display in all its glory varying degress of progress indicators. I also noticed users tend to be patient when their apps are run via the browser, than when you see them use the same app running as a console or a windows forms. They easily see "speed" issues when they are not engaged with a browser.

I don't think that web dev should just relax and think that way, I think Windows Live Mail and Yahoo Web Mail is too slow(with all its ajaxy effects), I couldn't live with that. Web dev should have a UI that is "Fast Enough' in the user's perspective(e.g. GMail is still the best here, although it's technique hidden IFRAME is ancient in todays fancy Ajax techniques)

ggsubsubscribe:

So when we design our databases and apps, how do we set speed criteria? I believe we have to.

ditto

bonskijr:

This usually happens on a from the ground-up application implentation, the reason they're getting a quick response is because the data isn't that big enough. Usually designs/scalability issues surfaces when the data is already huge, performance problem rears its ugly head especially when the app wasn't just designed to handle that kind data. We experienced this one when our data baloon exponentially, the application w/c was made for us, we discovered wasn't up to the task(databound controls littered all over and processes w/c takes several trip to the server etc. yikes!!) Temporary solution was to scale out the server, then we dug ourselves dirty with their code to at least minimize the problem.

This could be mitigated by having load testings prior to production. Without load testing, you run the risk of experiencing this possibility as you wont have an idea how your database would perform given 5 years worth of data. This is one kind of testing that is difficult to perform though but is highly doable. For a fairly predictable data growth rate, you can craft load testing a bit easily. It would be more of a challenge if you cant predict data growth rate. In this case you need to really max out your estimates and do a worst case scenario load testing.

I had experience one load testing before in a big project and it was really expensive to conduct one.

bonskijr:

I agree that most of the times users are unreasonable, the barometer being our deprecated DOS/Clipper apps, especially browsing and printing, we end up hand-coding our reports(we abandoned crystal reports) and utilize asynchronous data fetch for a more "faster" load. 3 seconds is too slow if you're just retrieving Patient information, this should be below that time, however sometimes it is way beyond your control as developer. Network reliability, hardware faults and most of the time users are at fault(they forgot to confirm the printing hence they waited staring at the printer) are some of the reasons why process is slow. We have a WAN configuration, however we're using Laser to connect instead of digging a fiber on the ground, when sandstorm hits the users couldn't even connect.

I always consider that other factors outside of the DB that affects performance are of a different thing. For this thread, let us just assume that those are working well and we just have to focus on the DB para nde tayo ma off-topic. But yeah, that is also an interesting topic to discuss.

 

bonskijr:

ggsubsubscribe:

with the absence of a performance requirement from a customer, how do you treat speed when designing databases and applications. is it one factor that influence our design or is it an afterthought?

I guess this is almost all time missing from the specs, I guess on our part this is good idea, if the client specifies a speed threshhold we might be spending alot of time finetuning that we might miss the deadline or concentrate too much on that portion that we might be underminning other portion of the architecture. Having said that however, if the application is time sensitive(eg. airline booking), then the threshhold shouldn't be an afterthought. I guess in the end we'd have to weigh the trade offs especially if we're still in system designs, we might be prematurely optimizing.

Di naman. most of the projects i have been involved with have performance requirements. In fact, I prefer that perf numbers be required and defined if possible so that a point of reference be made prior to the development. Then the design should be based on those numbers. It has to be part of everything. We cant even say that architecture and speed can be both treated separately. Speed has to be part of the criteria for any architecture related decisions or selection. There are times that you have to do away with sound architectural decisions to favor speed.

Worst, speed can even dictate on the computer language that you will use.

I wont be comfortable with projects not clear on speed as speed expectations are ever present and felt. Better be safe than drown in relativity later on. I happen to be involved in one project before that stated in the contract that the database design should be able to crunch 3500 transaction per second before the first payment is ever made. It forces the project to use Oracle, C++, and several servers and a load-balancing design to nail down the number.

i hate it when users are being unresonable. when i'm irked, i normally ask them "and how long do you think it would take you to do it manually?" end of discussion.

hahaha. nice newbie_dba.

newbie_dba:

i hate it when users are being unresonable. when i'm irked, i normally ask them "and how long do you think it would take you to do it manually?" end of discussion.

hehehe. wont work all the time sir. we can also be users and sometimes we become unreasonable. but in most cases, we are aware when we become reasonable and when we become unreasonable. for example, we would be very irked if a POS systems takes like 3 seconds to process a single item or if a box office counter takes a couple of minutes to reserve you a seat. I'd begin to criticize an airline if it would take me 10 minutes to book a ticket. i'd be forced to take a look at the system if a report takes like a couple of days to generate. I actually hate when Vista takes a few seconds more to show me that it is determining the files to be deleted and takes a few more to actually delete the files :P

ggsubscribe:

I actually hate when Vista takes a few seconds more to show me that it is determining the files to be deleted and takes a few more to actually delete the files :P

 You know what G, I'm beginning to think there's nothing really goin on behind the so called "computing blah blah time". MS just wants to flaunt their new animation and graphics even for this very mundane tasks. It reminds me of an amateur technique of executing a very long for () loop, even if it's not doing anything, just to show beautiful splash screen showcasing photoshop talents.

 On to the serious side. I'm not sure about you guys but I can always put my feet in the customer shows whenever I verify reports about performance. Like what GG said, some of the things that constitute slowness can be judged through common sense we gathered from everyday experience. What's difficult are those "uncommon" transactions which we are not really accustomed to in a daily basis. We can't just equate excution time with the number of data(rows). Like in the recent work I got involved with, the client's primary request is to somehow cut the loading time of the bulk updates by 50% so a 4-hr update should take around 2 hours. I couldnt judge rightaway if the script was indeed screwed up because I have no idea what data are involved and how many of them. We've chopped the load time to below 1 hour, but again someone would still think it's not fast enough and there's still room for improvement. There's no stopping client's greed. As my boss said, if we were an airline company, a client may even request to bring the plane home with him.

twistur:

ggsubscribe:

I actually hate when Vista takes a few seconds more to show me that it is determining the files to be deleted and takes a few more to actually delete the files :P

 You know what G, I'm beginning to think there's nothing really goin on behind the so called "computing blah blah time". MS just wants to flaunt their new animation and graphics even for this very mundane tasks. It reminds me of an amateur technique of executing a very long for () loop, even if it's not doing anything, just to show beautiful splash screen showcasing photoshop talents.

 On to the serious side. I'm not sure about you guys but I can always put my feet in the customer shows whenever I verify reports about performance. Like what GG said, some of the things that constitute slowness can be judged through common sense we gathered from everyday experience. What's difficult are those "uncommon" transactions which we are not really accustomed to in a daily basis. We can't just equate excution time with the number of data(rows). Like in the recent work I got involved with, the client's primary request is to somehow cut the loading time of the bulk updates by 50% so a 4-hr update should take around 2 hours. I couldnt judge rightaway if the script was indeed screwed up because I have no idea what data are involved and how many of them. We've chopped the load time to below 1 hour, but again someone would still think it's not fast enough and there's still room for improvement. There's no stopping client's greed. As my boss said, if we were an airline company, a client may even request to bring the plane home with him.

i agree. i think this is where SLA's come in and are very important. when drafting an SLA and being discussed between client and provider, it is very important that both parties come to terms with what is poor, acceptable and good performance. that way, no one becomes unreasonable (as far as users/clients) and smart-ass (as far as providers). this agreement should be made even before system design and development commences so that everyone involved in the project knows what the client's expectations are and what they need to deliver, by hook or by crook.

without SLA's, users begin to think that they always have the right to complain while providers have the right to reason out.

on another note, based on experience, some users are ALWAYS complaining and are never happy (especially females [no offense]). later on you'd realize that they are only trying to get your attention. nagpapa-cute lang pala. aysus!

newbie_dba:

twistur:

ggsubscribe:

I actually hate when Vista takes a few seconds more to show me that it is determining the files to be deleted and takes a few more to actually delete the files :P

 You know what G, I'm beginning to think there's nothing really goin on behind the so called "computing blah blah time". MS just wants to flaunt their new animation and graphics even for this very mundane tasks. It reminds me of an amateur technique of executing a very long for () loop, even if it's not doing anything, just to show beautiful splash screen showcasing photoshop talents.

 On to the serious side. I'm not sure about you guys but I can always put my feet in the customer shows whenever I verify reports about performance. Like what GG said, some of the things that constitute slowness can be judged through common sense we gathered from everyday experience. What's difficult are those "uncommon" transactions which we are not really accustomed to in a daily basis. We can't just equate excution time with the number of data(rows). Like in the recent work I got involved with, the client's primary request is to somehow cut the loading time of the bulk updates by 50% so a 4-hr update should take around 2 hours. I couldnt judge rightaway if the script was indeed screwed up because I have no idea what data are involved and how many of them. We've chopped the load time to below 1 hour, but again someone would still think it's not fast enough and there's still room for improvement. There's no stopping client's greed. As my boss said, if we were an airline company, a client may even request to bring the plane home with him.

i agree. i think this is where SLA's come in and are very important. when drafting an SLA and being discussed between client and provider, it is very important that both parties come to terms with what is poor, acceptable and good performance. that way, no one becomes unreasonable (as far as users/clients) and smart-ass (as far as providers). this agreement should be made even before system design and development commences so that everyone involved in the project knows what the client's expectations are and what they need to deliver, by hook or by crook.

without SLA's, users begin to think that they always have the right to complain while providers have the right to reason out.

on another note, based on experience, some users are ALWAYS complaining and are never happy (especially females [no offense]). later on you'd realize that they are only trying to get your attention. nagpapa-cute lang pala. aysus!

mukhang madaming cute cguro dyan sa inyo newbie_dba. hehehe.

yep. SLAs should set the baseline for this. Performance requirements should be obvious right from the start (like the POS system should be able to process a single item with a maximum response time of 1 second under normal circumstances).Though in most SLAs i have seen and been involved with, only the major/critical processes, reports are identified and included prior to development. The rest are governed by who has the best argument when performance becomes an issue. In most cases, SLAs get upgraded over time.

Here is the catch though. By experience, usually SLAs are drafted not by users but by upper management types which seldom get their hands dirty with the application once things are rolled out. It the users who complain without the benefit of knowing (in most cases) that an SLA do exist. And before they get to know, users have already profiled in their minds that your app is at best crappy and slow. This is what I am trying to determine with this thread ... that knowing when your process is slow spares you from your users' wrath. No matter how unreasonable they can become most of the time, it is still prudent to reassure them that things can be improved. They just need to how things are done and they become understanding enough and probably become cute too hheheh.

As an ISV, I dread each time I get a negative user feedback concerning my product or system than being strong-armed by one of their bosses or executives. It's the users that I need to pamper. It is them that should be made happy. Once i get happy users, I dont have to worry about their bosses. In most cases, actual users are the ones to influence whether your system stays or goes. In my case, if the system stays, I stay, else I get booted out.

Bottomline is that ... we can't be (in my case) hostile to our users. They are our bread and butter. :)