Friday, July 18, 2008

LoadRunner Internal Architecture

LoadRunner Internal Architecture
This Visio 2002 file is based on LR8.0





application servers under test are placed under stress by
driver processes mdrv.exe (the Multi-threaded Driver Process) and r3vuser.exe which emulate application clients such as Internet Explorer web brower. It performs 3 main actions:
Kli> cpp (C language pre-processor)
cci (C pre-compiling) which creaes a file with ci file, and
execute using the driver for the protocol technology being tested.

Runs can be invoked to run "silently" by invoking Mdrv.exe from a Windows batch script.

Mdrv can automatically stop loading Vusers because they communicate with Vusers and monitor CPU usage on Windows Load Generator machines.

A separate JVM is instantiated by each Java-based Vuser on Windows-based machines. Java Vusers are not supported on Unix platforms.


virtual Vusers are invoked as groups (logical collection of virtual users running the same script on a specific load generator machine)
by agents (3,900K magentproc.exe) running as a service or as a process
on load generator client machines.


Each machine hosting agents maintains an Execution Log in a .qtp file.
When logging is enabled, the agent also creates within the results folder a sequential log file for each Vuser (segregated by Vuser group).
During execution, this file is displayed in the view > Show Output window on the LoadRunner Controller machine.


Upon a pre-set delay, the Scheduler running on a Controller machine instructs agents (via Windows port 54345 or dynamic Unix port) to initiate test session scenarios. The Controller (wlrun.exe) sends a copy of scenario files along with the request.
Agents are launched by the Remote Agent Dispatcher process (formerly called Remote Command Launcher (RCL)) on each load generator machine.
Each agent refer to scenario (.lrs) definition files to determine which Vuser groups and scripts to run on host machines.

This means the Controller can be started from a DOS batch (.bat) file (preferrably with a short name on a root drive):

REM Start Controller:
SET M_ROOT=C:\Program Files\Mercury Interactive\LoadRunner\bin
cd %M_ROOT%
wlrun.exe -TestPath D:\Dev\Dev1.lrs -port 8080 -Run -DontClose
pause Press Ctrl-Z to keep this window or


Including the -Run parameter is the same as manually pressing the "Start Scenario" automatically upon invocation. This is not a good idea because you may have to decide about collating the file from a previous run or want to change the output folder.
This assumes that the system's environment PATH variable was updated to include where LoadRunner is installed.





The Controller is invoked using parameter values within files in the Windows OS folder (WINNT for Windows 2000 and WINDOWS for Windows XP). The Windows folder is used because LoadRunner is designed to have only one instance of Controller running at a time on a machine.

To quickly switch among several applications, save a copy of LoadRunner's ini files after working on it within the Controller, then use Notepad to craft a batch file to copy application-specific versions of ini files before executing wlrun. An example of file copy actions for application XXX:


copy %LRDir%/config/wlrun7-XXX.ini %LRDir%/wlrun7.ini
copy %LRDir%/config/wlrun7-XXX.dft %LRDir%/wlrun7.dft

Prior to v9.0:


copy %WinDir%/wlrun7-XXX.ini %WinDir%/wlrun7.ini
copy %WinDir%/wlrun7-XXX.dft %WinDir%/wlrun7.dft

Some defaults you might want to change:

In the wlrun7.ini file file [output] section, MaxNumberOfOutputMessages= from 10000 to 100000 for long runs. This limits the number of output messages stored in the database.
MaxOutputUIRowsToShow limits the amount of messages/errors (lines) displayed in the Controller's Output window.
In the QTWeb.lrp file within the LoadRunner Program Files dat\protocols folder section [Vugen], add entry MaxThreadPerProcess=5 to limit the number of threads managed by each load generator mdrv.exe process.

Values for DefaultScenarioDir, DefaultScriptDir, DefaultResultDir, and [Recent File List] stored in the wlrun5.ini and wlrun7.dft files updated whenever values are changed within the Controller.



The blocks of actions taken by each Vuser are
defined in Vu scripts created using Loadrunner's VuGen.exe. When this program is invoked, it stores in the Windows folder a comparamui.INI file to save under "[LastTablesUsed]" a history of files and [ParamDialogDates] specified using menu option Insert > New Parameter > Dates.

VuGen stores and retrieves a vugen.ini file in the Windows folder. When using Java, enable additional debug options:

[DynaDlg]
JavaLevel=3

When using 8.0 scripts within VuGen 8.1, add to Vugen.ini:

[Editor]
OLDEDITOR = 1

VuGen opens in LR folder template/qtweb default.cfg and script files.

Vu scripts can be coded to use variable values obtained from parameter files external to the script.

I have a lot more on VuGen here


During a run, execution results are stored to a results folder.
I prefer to set Results Settings to "Automatically create a results directory for each scenario execution." which means that LR will increment the name of the Results Name when I start a scenario runs. For example, a value of "Res11" will be automatically incremented to "Res12" or sometimes "Res11-1".
Errors are written to the output.mdb MS Access database. See the ASP page I have written to access this database


Within each results folder, a "Log" folder is automatically created to contain a log file for each group. After a run, to view a log file from within the Controller, click then right-click on a group to select "Show Vuser Log".

As a scenario is run, monitors maintain counters locally on each host.

After a run, the "collate" process takes .eve and .lrr result files and creates in the results folder a temporary .mdb (MS-Access) database.
To prevent errors when processing large result files, use MSDE (Microsoft SQL Desktop Engine). Don't install it from the Add-in folder on the LoadRunner 7.8 CD, which is obsolete SQL7. Download MSDE 2000 Release A which includes MSDE 2000 Service Pack 3a and MDAC 2.7 SP1a for use by Analysis on any Windows machine. Extract the file and share that folder. Open a command window to run a command such as:


setup SAPWD="StrongPassword" INSTANCENAME="LR" SECURITYMODE=SQL DISABLENETWORKPROTOCOLS=0 /L*v path to log file
Using Windows Explorer, share the Data folder.
Then in Analysis Options > Database tab, use 8.3 names without spaces (indentified with DOS command DIR /X):

Input the SAPWD password specified above.
Logical Storage location: \\loadclient02\Data (the folder you shared)
Physical Storage Location: C:\PROGRA~1\MICROS~1\MSSQL\Data (not C:\Program Files\Microsoft SQL Server\MSSQL\Data)
Click "Test parameters". (This takes a few seconds)

The Analysis Module (8,320K analysisu.exe)
generates analysis graphs and reports using data from the .mdb database.
The LoadRunner Results file results_name.lrr from each scenario run -- also called an Analysis document file -- is read by the Analysis program to display Percentile graphs.
By default, the LRReport folder is created in the test analyst's local machine My Documents folder to store Analysis Session files.
They can optionally be formated in HTML.
Their format are controlled by a .tem template file.

Optionally, Mercury's Remote Performance Monitoring (RPM) MS-IIS/ASP web server for LoadRunner 7.8 can be installed on a Windows 2000 server ( but not on a Windows 2003 server) so that
load test results to be viewed using a web browser.

Monday, July 14, 2008

LoadRunner Interview Questions

Here are LoadRunner Interview Questions:

1. What is load testing?

Load testing is to test that if the application works fine with the loads that result from large number of simultaneous users, transactions and to determine weather it can handle peak usage periods.

2. What is Performance testing?

Timing for both read and update transactions should be gathered to determine whether system functions are being performed in an acceptable timeframe. This should be done standalone and then in a multi user environment to determine the effect of multiple transactions on the timing of a single transaction.

3. What are the components of LoadRunner?

The components of LoadRunner are The Virtual User Generator, Controller, and the Agent process, LoadRunner Analysis and Monitoring, LoadRunner Books Online.

4. What is Virtual Users?

Unlike a WinRunner workstation which emulates a single user's use of a client, LoadRunner can emulate thousands of Virtual Users.
Load generators are controlled by VuGen scripts which issue non-GUI API calls using the same protocols as the client under test. But WinRunner GUI Vusers emulate keystrokes, mouse clicks, and other User Interface actions on the client being tested.
Only one GUI user can run from a machine unless LoadRunner Terminal Services Manager manages remote machines with Terminal Server Agent enabled and logged into a Terminal Services Client session.
During run-time, threadedvusers share a common memory pool.
So threading supports more Vusers per load generator.
The Status of Vusers on all load generators start from "Running", then go to "Ready" after going through the init section of the script. Vusers are "Finished" in passed or failed end status. Vusers are automatically "Stopped" when the Load Generator is overloaded.

5. Explain the Load testing process? Version 7.2

Step 1: Planning the test. Here, we develop a clearly defined test plan to ensure the test scenarios we develop will accomplish load-testing objectives.
Step 2: Creating Vusers. Here, we create Vuser scripts that contain tasks performed by each Vuser, tasks performed by Vusers as a whole, and tasks measured as transactions.
Step 3: Creating the scenario. A scenario describes the events that occur during a testing session. It includes a list of machines, scripts, and Vusers that run during the scenario. We create scenarios using LoadRunner Controller. We can create manual scenarios as well as goal-oriented scenarios. In manual scenarios, we define the number of Vusers, the load generator machines, and percentage of Vusers to be assigned to each script. For web tests, we may create a goal-oriented scenario where we define the goal that our test has to achieve. LoadRunner automatically builds a scenario for us.
Step 4: Running the scenario. We emulate load on the server by instructing multiple Vusers to perform tasks simultaneously. Before the testing, we set the scenario configuration and scheduling. We can run the entire scenario, Vuser groups, or individual Vusers.
Step 5: Monitoring the scenario. We monitor scenario execution using the LoadRunner online runtime, transaction, system resource, Web resource, Web server resource, Web application server resource, database server resource, network delay, streaming media resource, firewall server resource, ERP server resource, and Java performance monitors.
Step 6: Analyzing test results. During scenario execution, LoadRunner records the performance of the application under different loads. We use LoadRunner’s graphs and reports to analyze the application’s performance.

6. When do you do load and performance Testing?

We perform load testing once we are done with interface (GUI) testing. Modern system architectures are large and complex. Whereas single user testing primarily on functionality and user interface of a system component, application testing focuses on performance and reliability of an entire system. For example, a typical application-testing scenario might depict 1000 users logging in simultaneously to a system. This gives rise to issues such as what is the response time of the system, does it crash, will it go with different software applications and platforms, can it hold so many hundreds and thousands of users, etc. This is when we set do load and performance testing.

7. What Component of LoadRunner would you use to record a Script?

The Virtual User Generator (VuGen) component is used to record a script. It enables you to develop Vuser scripts for a variety of application types and communication protocols.

8. What is a rendezvous point?

You insert rendezvous points into Vuser scripts to emulate heavy user load on the server. Rendezvous points instruct Vusers to wait during test execution for multiple Vusers to arrive at a certain point, in order that they may simultaneously perform a task. For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100 Vusers to deposit cash into their accounts at the same time.

9. Explain the recording mode for web Vuser script?

We use VuGen to develop a Vuser script by recording a user performing typical business processes on a client application. VuGen creates the script by recording the activity between the client and the server. For example, in web based applications, VuGen monitors the client end of the database and traces all the requests sent to, and received from, the database server. We use VuGen to: Monitor the communication between the application and the server; Generate the required function calls; and Insert the generated function calls into a Vuser script.

10. Why do you create parameters?

Parameters are like script variables. They are used to vary input to the server and to emulate real users. Different sets of data are sent to the server each time the script is run. Better simulate the usage model for more accurate testing from the Controller; one script can emulate many different users on the system.

11. What is correlation? Explain the difference between automatic correlation and manual correlation?

Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific. Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate.

12. How do you find out where correlation is required?

Two ways: First we can scan for correlations, and see the list of values which can be correlated. From this we can pick a value to be correlated. Secondly, we can record two scripts and compare them. We can look up the difference file to see for the values which needed to be correlated.

13. Where do you set automatic correlation options?

Automatic correlation from web point of view can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation. Automatic correlation for database can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created.

14. What is a function to capture dynamic values in the web Vuser script?

Web_reg_save_param function saves dynamic data information to a parameter.

15. What are Scenarios ?

Scenarios encapsulate the Vuser Groups and scripts to be executed on load generators at run-time.
Manual scenarios can distribute the total number of Vusers among scripts based on the analyst-specified percentage (evenly among load generators).
Goal Oriented scenarios are automatically created based on a specified transaction response time or number of hits/transactions-per-second (TPS). Test analysts specify the % of Target among scripts.

16. How do you debug a LoadRunner script?

VuGen contains two options to help debug Vuser scripts-the Run Step by Step command and breakpoints. The Debug settings in the Options dialog box allow us to determine the extent of the trace to be performed during scenario execution. The debug information is written to the Output window. We can manually set the message class within your script using the lr_set_debug_message function. This is useful if we want to receive debug information about a small section of the script only.

17. How do you write user defined functions in LR?

Before we create the User Defined functions we need to create the external library (DLL) with the function. We add this library to VuGen bin directory. Once the library is added then we assign user defined function as a parameter. The function should have the following format: __declspec (dllexport) char* (char*, char*)

18. What are the changes you can make in run-time settings?

The Run Time Settings that we make are:

1. Pacing - It has iteration count.
2. Log - Under this we have Disable Logging Standard Log and
3. Extended Think Time - In think time we have two options like Ignore think time and Replay think time.
4. General - Under general tab we can set the vusers as process or as multithreading and whether each step as a transaction.

19. Where do you set Iteration for Vuser testing?

We set Iterations in the Run Time Settings of the VuGen. The navigation for this is Run time settings, Pacing tab, set number of iterations.

20. What is Ramp up? How do you set this?

This option is used to gradually increase the amount of Vusers/load on the server. An initial value is set and a value to wait between intervals can be specified. To set Ramp Up, go to ‘Scenario Scheduling Options’

21. If you want to stop the execution of your script on error, how do you do that?

The lr_abort function aborts the execution of a Vuser script. It instructs the Vuser to stop executing the Actions section, execute the vuser_end section and end the execution. This function is useful when you need to manually abort a script execution as a result of a specific error condition. When you end a script using this function, the Vuser is assigned the status "Stopped". For this to take effect, we have to first uncheck the Continue on error option in Run-Time Settings.

22. What is the relation between Response Time and Throughput?

The Throughput graph shows the amount of data in bytes that the Vusers received from the server in a second. When we compare this with the transaction response time, we will notice that as throughput decreased, the response time also decreased. Similarly, the peak throughput and highest response time would occur approximately at the same time.

23. How do you identify the performance bottlenecks?

Performance Bottlenecks can be detected by using monitors. These monitors might be application server monitors, web server monitors, database server monitors and network monitors. They help in finding out the troubled area in our scenario which causes increased response time. The measurements made are usually performance response time, throughput, hits/sec, network delay graphs, etc.

24. If web server, database and Network are all fine where could be the problem?

The problem could be in the system itself or in the application server or in the code written for the application.

25. How did you find web server related issues?

Using Web resource monitors we can find the performance of web servers. Using these monitors we can analyze throughput on the web server, number of hits per second that occurred during scenario, the number of http responses per second, the number of downloaded pages per second.

26. How did you find database related issues?

By running Database monitor and help of Data Resource Graph we can find database related issues. E.g. You can specify the resource you want to measure on before running the controller and than you can see database related issues .

27. What is the difference between Overlay graph and Correlate graph?

Overlay Graph: It overlay the content of two graphs that shares a common x-axis. Left Y-axis on the merged graph show’s the current graph’s value & Right Y-axis show the value of Y-axis of the graph that was merged. Correlate Graph: Plot the Y-axis of two graphs against each other. The active graph’s Y-axis becomes X-axis of merged graph. Y-axis of the graph that was merged becomes merged graph’s Y-axis.

28. What does vuser_init action contain?

Vuser_init action contains procedures to login to a server

29. What does vuser_end action contain?

Vuser_end section contains log off procedures

30. What is think time? How do you change the threshold?

Think time is the time that a real user waits between actions. Example: When a user receives data from a server, the user may wait several seconds to review the data before responding. This delay is known as the think time. Changing the Threshold: Threshold level is the level below which the recorded think time will be ignored. The default value is five (5) seconds. We can change the think time threshold in the Recording options of the Vugen.

31. What is the difference between standard log and extended log?

The standard log sends a subset of functions and messages sent during script execution to a log. The subset depends on the Vuser type Extended log sends a detailed script execution messages to the output log. This is mainly used during debugging when we want information about: Parameter substitution. Data returned by the server. Advanced trace.

32. What is lr_debug_message ?

The lr_debug_message function sends a debug message to the output log when the specified message class is set.

33. What is lrd_fetch?

The lrd_fetch function fetches the next row from the result set.

34. What is lrd_stmt?

The lrd_stmt function associates a character string (usually a SQL statement) with a cursor. This function sets a SQL statement to be processed.

Some Important Topics in One Sentence:


How many types of scenario in loadrunner?

LoadRunner provides two types of scenario-

* Manual Scenario
* Goal-Oriented Scenario

What is the Manual Scenario?

Manual Scenario manages the load test by specifying the number of vuser to run.

What is the Goal-Oriented Scenario?

Goal-Oriented scenario allows loadrunner controller to create scenarion based on goal which you specified.

Which Correlation function you have used?

web_reg_save_param

How can we view the script?

We can view the script through Tree View and Script View.

What is the Tree View?

Tree view is icon based view.

What’s the Script View?

It’s text based view.

What does Result window contain?

Result Window open with two panes-

* Tree Pane
* Summary Pane

How many types of checkpoints are available in LoadRunner?

LoadRunner provides two types of checkpoints-

* Text Checkpoints
* Image Checkpoints

What’s the Correlation?

Correlation is used to obtained the data which are unique in every time of scrpit run and which are generated by nested queries.

How many types of Correlation?

Correlation are two types-

* Manual Correlation
* Automatic Correlation

What’s the Automatic Correlation?

Automatic Corrlation is set some rules for correlation.

What’s the syntex for think time?

lr_think_time

What’s the Elasped Time?

It’s total time take since request is set and obtained the result.

What type of functions are generated and inserted by LoadRunner?

LoadRunner generated two types of functions-

* LR Function
* Protocol Function

What’s the LR function?

LR function is obtained the information about the running Vuser.

What’s the Protocol Function?

Protocol function obtained the information about the type of Vuser.

What’s the Scenario Wizard?

We can create new scenario through Scenario Wizard.

What’s the Pause command?

It will change the status of running vuser to pause.

What’s the running Vuser graph?

It displays the number of Vusers that execute the Vuser script each seconds of scenario run.

What’s the percentile graph?

It displays the percentage of transaction which were performed with in given time.

What’s the Rendezvous graph?

It displays that when Vusers released from rendezvous point and how many Vusers released from each point.

How many types of running mode in LoadRunner?

LoadRunner provides two types of recording level-

* HTML based script
* URL based script

What is the HTML based script?

HTML based script generates seprated steps for each user actions.

What is the URL based script?

It records all requests and resources from server.

What is extension of LoadRunner result file?

.lrr

Which language LoadRunner uses?

ANSI C

What’s the Vuser Init Section?

In Vuser Init section, LoadRunner records log in the server.

What’s the Action Section?

In Action section, LoadRunner records end user activities.

What’s the Vuser end Section?

In Vuser end section, LoadRunner records log off into the server.

How do you edit the script?

We can edit the script by inserting the Transaction Points and Rendezvous Points.

What’s the LoadRunner Start-Transaction and it’s syntax?

It start the Transaction on the script and it’s syntax is that “lr-start-transaction”.

What’s the LoadRunner End-Transaction and it’s syntax?

It end the Transaction on the script and it’s syntax is that “lr-end-transaction”

What’s .lrs?

It’s extension of LoadRunner Scenario files.

Which component you are using for recording the vuser script?

We are using Vugen component for recording the Vuser script.

Which component you are using for playback the script?

We are using Controller component for playback the script.

What’s the Parameterization?

Parameterization is script variables which are used to pass many values to server. It emulates the actions of human user.

What are changes we can make in Run Time Settings?

We can set the Run Logic, Pacing, Log, Think Time.

What’s the Run Logic?

In Run Logic we can set the number of repititions which are performing by Vuser.

What’s the Pacing?

In Pacing we can set the time to wait between each repititions.

What’s the Log?

Log is level of information which we want to gather during run time.

Where we can set the number of repititions?

We can set the number of repititions in Run Time Settings.

What’s the Ramp-Up and where we can set this?

Ramp-Up option used to increases the number of Vuser in server and we can set the this option through scenario schedule.

What’s the Host?

Host is machine which execute the Vuser script.

What are process for developing a Vuser script?

There are five steps for developing a Vuser Script-

* Record Vuser script
* Edit Vuser script
* Change Run Time Setting
* Run Vuser script in stand alone machine
* Incorporate the Vuser script into loadrunner scenario

What’s the Remote Command Luncher?

RCL enables the controller to start the application on the host machine.

What’s the LoadRunner Agent?

Agent is interface between controller and host machine.

How to load LoadRunner Agent?

Controller insturct the RCl to lunch the LoadRunner Agent.

How many types of Vuser are available in LoadRunner?

There are many types of Vuser like DB Vuser, SAP Vuser, Web Vuser, RTE, GUI Vuser.