Section 1: Tune WAS JVM Heap TFIM Tuning Guide
Open the PDF directly: View PDF .
Page Count: 44
|Open PDF In Browser||View PDF|
Tivoli Federated Identity Manager Performance Tuning Guide Version 1.0 This document is intended to be used by FIM customers for general FIM performance tuning and best practices. The guide assumes the audience has certain level of knowledge about WebSphere Application Server and Tivoli Federated Identity Manager. Document: Luobin Wang/Singapore/IBM (firstname.lastname@example.org) Table of content Part 1: FIM General Tuning Chapter 1: Tune WAS JVM heap for FIM Chapter 2: Tune WAS Transport channel services for FIM Chapter 3: Tune WAS Session Management for FIM Chapter 4: Tune WAS Cluster Cache Replication for FIM Chapter 5: Tune IHS and WebSEAL with FIM Chapter 6: Tune Alias Service and backed DB connections Part 2: Protocol Specific Tuning Chapter 7: FIM Tuning for SAML2.0 Chapter 8: FIM Tuning for OAuth Chapter 9: FIM Tuning for User Self Care Chapter 10: FIM Tuning for STS Services Chapter 1: Tune WAS JVM heap for FIM As an application running on WAS (WebSphere Application Server), FIM (Federated Identity Manager) will need a proper WAS JVM (Java Virtual Machines) to perform well. In this section, the discussion will go over WAS Heap size determination and garbage collection settings to suit FIM performance. 1.1 WAS JVM heap setting place To modify WAS JVM heap settings: a. In the administrative console, click Servers > Server Types > WebSphere application servers > server_name. b. In the Server Infrastructure section, click Java and process management > Process definition > Java virtual machine. c. Specify a new value in either the Initial heap size or the Maximum heap size field. 1.2 Recommend setting for JVM heap The recommended JVM heap size for FIM to run is 1 GB. However, it may need to change based on incoming request rate and session settings. This will be discussed further in later sections. 1.3 Understanding the implications of heap size Setting the heap as larger as possible may not be a good idea for several reasons: a. The start up of the JVM will take longer time. b. When garbage collection happens, it will take longer time. Setting the heap small will also have some disadvantages. The most important one is that it may cause the garbage collection process to happen every frequently and cause the application performance throughput to drop. 1.4 Find the suitable heap size setting The JVM size should always be lower then the physical memory that the machine has. If it exceeds the limit, the swapping will cause the performances of JVM degrades heavily. The following process can be used to determine the suitable heap size: a. Set the JVM start up and maximum heap size to the same starting value. b. Enable Verbose garbage collection setting from the administrative console for garbage collection monitoring In the administrative console, click Servers > Server Types > WebSphere application servers > server_name. In the Server Infrastructure section, click Java and process management > Process definition > Java virtual machine. Ensure that the check box next to Verbose garbage collection is selected, and then restart the server. c. Load the application with the expected user load and user scenarios. d. Collect native_stderr.log file and analyze the GC (Garbage Collection) log entries. The optimal result for the average time between garbage collections is at least 5-6 times the average duration of a single garbage collection. If it is not achieved the application is spending more than 15 percent of its time in garbage collection. Then the rational step is to increase the heap size. In the log entries, it will specify how long it takes for each GC process to complete. If the time is too long (normally should be within 2 seconds) for the GC, it usually means the heap size is too large. Sometimes, because of the concurrent users in FIM, the heap is set to a high value and the GC will take every long or happen very frequently. In this case, tuning FIM and WAS session settings is needed to make it perform well. This will be discussed in later sections. 1.5 Recommendations The recommended JVM heap size for FIM to run is 1 GB and it may need to change based on incoming request rate and session settings. This will be discussed further in later sections. For more information on WAS JVM tuning, please see: http://publib.boulder.ibm.com/ infocenter/wasinfo/v7r0/topic/com.ibm.websphere.nd.multiplatform.doc/info/ae/ae/ container_tune_jvm.html For technical guide on tune Java GC, please see: http://www.ibm.com/developerworks/library/i-gctroub/ Chapter 2: Tune WAS Transport channel services for FIM The transport channel services manage client connections and I/O processing for HTTP and JMS (Java Management Services) requests. By applying suitable settings to each channel, the ability for FIM to handle concurrent requests can be improved and the usability also benefit from those tuning. During installation, WAS will assign default values to each transport channel services. Changing the default values for settings on one or more of the transport channels associated with a transport chain can improve the performance of that chain. Here is a figure from WAS showing the Transport Channel Service layout. From FIM performance perspective, some tuning is recommended. 2.1 Adjust Default TCP channel setting The major concern here is the Maximum open connection property. The value is set to 20000 by default, which is the maximum number of connections allowed. This should be changed to a lower value depending on the level of concurrency and application response time. High concurrency or long response time application will need a higher value of open connections. The recommended setting from Web Sphere tuning guide is 500. In addition, long response time will also require tuning on the connection Inactive timeout setting. It needs to be changed to an enough long value to waiting for the application response and write it back to the client. The default in WAS is 60 seconds. Unless FIM is under unexpected extreme load, this is adequate for most cases. To modify those two values: a. In the administration console, click Servers > Application servers > server_name > Ports. b. Then click View associated transports for the appropriate port. c. Select the transport chain whose properties you are changing. Click on the TCP transport channel defined for that chain. d. Click Apply and then Save to save these changes. 2.2 Adjust Default HTTP channel setting The default HTTP transport channel setting in WAS is adequate for FIM to perform well. One option that is potential for performance boost is the HTTP keep-alive setting. The persistent (keep-alive) connection setting controls that whether connections are left open between requests. Leaving the connections open can save setup and teardown costs of sockets if your workload has clients that send multiple requests. The default value is true. When FIM is set up for STS (security token services) or some case in SPS (security and privacy services), the client only send single requests over a considerable long period of time. In this case, disable the option and close the connections right away is a better choice, rather than to have the HTTP transport channel setup the timeout to close the connection at some later time. However, it should be designed and tested carefully before doing so to avoid the false positive case. To modify the setting: a. In the administration console, click Servers > Application servers >server_name > Ports. Then click View associated transports for the appropriate port. b. Select the transport chain whose properties you are changing. c. Click on the HTTP transport channel defined for that chain. d. Click Apply and then Save to save these changes. 2.3 Adjust Default Web container transport channel settings The major purpose is to tune the Write buffer size parameter to a value such that most requests to be written out in a single write. The default value for the write buffer is 32768 bytes. This is sufficient for FIM in most cases. However, if FIM is configured such that the size is bigger then that, such as numerous attributes in SAML (security assertion markup language) messages, changing the value could introduce a better performance. To modify the setting: a. In the administration console, click Servers > Application servers >server_name > Ports. Then click View associated transports for the appropriate port. b. Select the transport chain whose properties you are changing. c. Click on the Web container transport channel for that chain. d. Click Apply and then Save to save these changes. 2.4 Adjust thread pool settings Each TCP transport channel is assigned to a particular thread pool. Thread pools can be shared between one or more TCP transport channels as well as with other components. The default settings for a TCP transport channel is to have all HTTP based traffic assigned to the WebContainer thread pool and all other traffic assigned to the Default thread pool. In certain environment, assigning a thread pool to http port can help isolate the network traffic and improve dealing with high concurrency requests. Typical applications usually do not need more than 10 threads per processor. Default setting in FIM is adequate in the default setting. One exception is if there is some of server condition, such as a very slow backend request, that causes a server thread to wait for the backend request to complete. In such a case, CPU usage is usually low and increasing the workload does not increase CPU throughput. Thread dumps show nearly all threads in a call out to the backend resource. If this condition exists, and the backend is tuned correctly, try increasing the minimum number of threads in the pool until you see improvements in throughput and thread dumps show threads in other areas of the runtime besides the backend call. Another scenario that needs to tune the thread setting is when FIM is deployed in a WAS cluster environment. Additional thread setting will need to be tuned for dynamic cache synchronization and others. This will be discussed in detail in later sections about cluster environment tuning for FIM. Minimum thread pool setting does not mean that those threads are initialized righter after the application server starts. Threads are added to the thread pool as the workload assigned to the application server requires them, until the number of threads in the pool equals the number specified in the Minimum size field. After this point in time, additional threads are added and removed as the workload changes. However the number of threads in the pool never decreases below the number specified in the Minimum size field, even if some of the threads are idle. When facing the need to change the thread pool size associated with certain TCP channel, modify as following: a. In the administration console, click Servers > Application servers > server_name > Ports. Then click View associated transports for the appropriate port. b. Select the transport chain whose properties you are changing. c. Click on Thread pools > threadpool_name and adjust the values specified for the Minimum Size and Maximum Size parameters for that thread pool. d. Click Apply and then Save to save these changes. For more information on WAS TCP setting, please see: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=% 2Fcom.ibm.websphere.nd.multiplatform.doc%2Finfo%2Fae%2Fae% 2Furun_chain_typetcp.html http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=% 2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftprf_tunechain.html For more information on WAS Thread settings, please see: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=% 2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftprf_tunetcpip.html http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=% 2Fcom.ibm.websphere.nd.multiplatform.doc%2Finfo%2Fae%2Fae% 2Fuejb_thrdpool.html Chapter 3: Tune WAS Session Management for FIM In some configuration, FIM will need to store certain session information in WAS. A typical example is FIM is configured as SAML2.0 SP (service provider). In those situations, it is important to adjust the session management settings in WAS to avoid performance bottlenecks. The http session is store in WAS application server JVM. The resource, mainly heap memory, is shared by the applications in the server. The session information will take larger memory as more information is captured in the session data and more sessions come in to the server. If the sessions are not timed out or destroyed at log out, the memory available for use will decrease and limit the later operations on application objects. It will lead to frequently GC, which causes performance degradation, and eventually out of memory errors. In lab testing environment, it has been indicated that a SAML2.0 session data in FIM will occupy a memory size of 70k-100k, which is quite large. 3.1 Adjust default application server session settings Two parameters in the session setting are particularly important from performance perspective, Maximum in-memory count and Session timeout. The default value for maximum in-memory count is 1000 and Allow overflow is checked. The default value for Session timeout is 30 minutes. Maximum in-memory count has different meanings in different context. The meaning differs depending on whether you are using in-memory or distributed sessions. For inmemory sessions, this value specifies the number of sessions in the base session table for each web module. For distributed sessions, this value specifies the size of the memory cache for each web module's sessions. In the discussion of heap consumption, it refers to in-memory sessions. Use the Allow overflow property to specify whether to limit sessions to this number for the entire session management facility or to allow additional sessions to be stored in secondary tables. It is used by default. There are several concerns to use this option. First, the session data is optimized for searching in primary table. Second, it will post the risk of having out of memory errors in the application server. If the allow overflow is not checked, When the session cache has reached its maximum size and a new session is requested, the session management facility removes the least recently used session from the cache to make room for the new one. It is recommended that the maximum concurrent user in the server is carefully estimated and set the session count slightly higher then that value. With that, the Allow overflow can be unchecked to avoid further risks. However, the maximum concurrent user sessions in the environment is closely related to the Session timeout value. The Session timeout value can be over write by the value in web modules, and it is use as default if the value is not specified in the modules. The value is not accurate to seconds. The SessionManager invalidation process thread runs every x seconds to invalidate any invalid sessions, where x is determined based on the session timeout interval specified in the session manager properties. For the default value of 30 minutes, x is around 300 seconds. In this case, it could take up to 5 minutes (300 seconds) beyond the timeout threshold of 30 minutes for a particular session to become invalidated. When you are estimating the maximum number of concurrent user sessions, this factor should be taken in to consideration. To modify these settings to suit the environment, follow the following steps: a. In the administration console, click Servers > Application servers >server_name. Then click Session management for settings. b. Specify a value for the Maximum in-memory session count. c. Choose to check or uncheck Allow overflow option. d. Specify a value for your Session timeout. e. Click Apply and then Save to save these change 3.2 Using Application level for Session Management As stated in above section, session management in application server can be over written by Application level settings or Web Module settings. This is more appropriate when FIM is sharing the application server environment with some other applications which makes changing default application server settings not possible. The parameters for the application level session management are the same as the Application server level. To use the enterprise applications level settings to override the setting, following these steps: a. In the administration console, click Applications > Application Types >WebSphere enterprise applications. Then click ITFIMRuntime. b. Under Web Module Properties, click Session management. c. Check Override session management box. d. Change the values in the page according to the environment needs. e. Click Apply and then Save to save these change 3.3 Using Web Module level session management In some occasions, customized session management for different web modules may need to be configured. One example would be differentiated session behaviors in STS and SPS services in FIM. From performance perspective, configure just right settings for different components in FIM production environment may help. However, this will involved slightly complex management effort. To implement the Web Module level session management, following these steps: a. In the administration console, click Applications > Application Types >WebSphere enterprise applications. Then click ITFIMRuntime. b. Under Modules, click Manage Modules. c. Click on the module need to be changed. d. Under Additional Properties click Session Management. e. Click Override session management box. f. Change the values in the page according to the environment needs. g. Click Apply and then Save to save these change 3.4 Using advanced session management tools When the session in-memory implementation becomes the bottleneck of performance after all the tuning possible, WAS provides alternative choices. The Distributed environment setting in Session management allows use distributed session to databases. An even more advanced integration for session management is available using WebSphere eXtreme Scale with DataPower. For more information in WAS session settings, please see: http://publib.boulder.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=% 2Fcom.ibm.websphere.nd.iseries.doc%2Finfo%2Fiseriesnd%2Fae% 2Fuprs_rsession_manager.html&resultof=%22%73%65%73%73%69%6f%6e%22%20% 22%6d%61%6e%61%67%65%6d%65%6e%74%22%20%22%6d%61%6e%61%67% 22%20 For more information of using eXtreme Scale with DataPower, please see: http://publib.boulder.ibm.com/infocenter/ieduasst/v1r1m0/index.jsp?topic=/ com.ibm.iea.wxs/wxs/7.0/Administration/WXS70_HTTP_Session_Management/ player.html http://publib.boulder.ibm.com/infocenter/ieduasst/v1r1m0/index.jsp?topic=/ com.ibm.iea.wdatapower/wdatapower/1.0/xc10/Administration/ XC10_SessiondatagridConfiguration/player.html For more information in WAS session trouble shooting, please see: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=% 2Fcom.ibm.websphere.base.doc%2Finfo%2Faes%2Fae%2Frtrb_httpsessprobs.html Chapter 4: Tune WAS Cluster Cache Replication for FIM For high availability, load distribution and many other reasons, FIM could be deployed into WebSphere Network Deployment environment. In a cluster environment, cache replication is important when IHS (IBM HTTP Server), or other load balancing points, routs the request randomly among cluster members. This section will go over how to enable the caching and change the default settings in order for FIM to perform well. 4.1 Enable Dynamic Caching in WAS In early versions of WAS (6.1 & 7), when the WAS cluster is created, a replication domain is created and assigned to the cluster by default. If it is not created because of version or additional replication domains are needed, follow the steps to create the replication domain first: a. In the administration console, click Environment> Replication domains. Then click New…. b. Specify a Name for the replication domain. c. Set the Request timeout. d. Chose Entire Domain for Number of replicas. e. Click Apply and then Save to save these change To enable the replication, do the same steps for all the cluster members: a. In the administration console, click Servers > Server Types >WebSphere application servers. Then click server_name. b. Expand Containers Services. Then click on Dynamic cache service. c. Check the Enable cache replication. d. Use the replication domain just now created or the default as Full group replication domain. e. Choose Both push and pull as Replication type. f. Specify a value for Push frequency. g. Click Apply and then Save to save these change 4.2 Dynamic Caching tuning from Application server level In previous step, the procedure to enable the dynamic caching already suggested a few parameters can be tuned for performance purposes. This chapter will discuss the Cache size and Push frequency settings. The cache size setting in application server level limits the overall cache instances. If it is too low, the program may spend unexpected time to organize and coordinate the instances. However, if it is too high, the memory consumption is quite considerable. In a large scale environment, disk offload is required to make is sustainable. The small cache size ensures the synchronization is fast and neat. The large size ensures the capability for large scale and the speed, response time from user experience, may scarify a bit. The push frequency is the same as the cache size. The more frequent you push, the faster the information is made available across the cluster, and the faster the response time can be. However, every time it pushes, some CPU is wasted if there is no much update or the update is not required. If the push frequency is set to 0 seconds, the application server converts the property value to the default value, which is 1. Suggestion for the Cache size is that, after GC happened, there should be at least 40% of the JVM heap still available. If it cannot reach the number, try to increase the heap size, or make the cache object invalidation faster. Suggestion for the Push frequency is that, it should be 1 second when the CPU load has not exceeded designed figure. If the setting is increased to higher value, FIM should be set to cater the possibility of needed objects not replicated yet. This is covered in following chapter in this section. To change these two settings: a. In the administration console, click Servers > Server Types >WebSphere application servers. Then click server_name. b. Expand Containers Services. Then click on Dynamic cache service. c. Specify a value for Cache size.. d. Specify a value for Push frequency. e. Click Apply and then Save to save these change 4.3 Change specific Cache instance size When FIM is deployed, it will create a set of default cache instance with default size (1000). The size can be changed depending on the services type and scale. To do so, following the steps: a. In the administration console, click Resources > Cache instances >Object cache instances. Then click on the cache instance name need to be changed. b. Specify a value for Cache size. c. Click Apply and then Save to save these change 4.4 Thread pool tuning for Cache replication When the replication is enabled, the thread pool that the replication service is using should be set to suite the need. For recent 6.1 and 7.0 fix pack levels DRS (data replication services) moved to its own thread pool and the size is already set. For early fix pack and version, the DRS is using the default thread pool. In this case, the minimum and maximum size for the thread poll should be at least 40. To do so, follow the steps: a. In the administration console, click Servers > Application servers > server_name b. Under Additional properties Click on Thread pools. c. Click on Default, and adjust the values specified for the Minimum Size and Maximum Size parameters. d. Click Apply and then Save to save these changes. 4.5 Change Retry setting when using caching replication As discussed in chapter 4.2, when the push frequency is set, there will be a delay for other cluster members in the cluster when a cache object is created. From performance perspective, the frequency may need to change to suite the different resource constrains. In this case, additional FIM parameters should be tuned to suit the need. There are two which is particularly important, DistributedMap.GetRetryLimit and DistributedMap.GetRetryDelay. The default value for the retry limits is 2, and retry delay is 2 seconds. The retry control how many trial the wrapper will query the distributed map before returning that the data is not in the map. The delay controls the interval between each trial. These two settings will influence the potential CPU usage or waste and the response time to the request. Generally, the retry limits should not go beyond 5 and the delay is no to go beyond then the push frequency in the replication setting. The recommended value for the delay is 1 second or less. To change the default setting for those two parameters, variable will be added in manually in FIM runtime properties. Following the steps: a. In the administration console, click Tivoli Federated Identity Manager > Domain Management > Runtime Node Management. b. In the Runtime Nodes portlet, click Runtime Custom Properties. c. The Runtime Custom Properties panel is displayed. Click Create. d. Enter a string in the Name field. Do not insert the space character in this field. e. Enter a string in the Value field. Spaces are allowed in this field. f. Choose one of the following actions: g. Click Apply to apply the changes that you have made without exiting from the panel. h. Click OK and load the configuration to Runtime. For more information on WAS dynamic cache, please see: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=% 2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Fudyn_rcachesettings.html http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=% 2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftdyn_dynamiccache.html For more information in WAS performance tuning in Cache and Data Replication Service, please see: http://www-01.ibm.com/support/docview.wss?uid=swg27006431 For more information on FIM customer properties, please see: http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=% 2Fcom.ibm.tivoli.fim.doc%2Ftfim611_cfg126.htm Chapter 5: Tune IHS and WebSEAL with FIM In FIM environment, IHS is commonly used to load balancing the communication with WAS cluster environment. IHS, or another HTTP server, export a central point for FIM cluster members and distributed the workload to those members. WebSEAL, which is a closely integrated authentication and authorization product with FIM, is commonly used as a Point of Contact Server for FIM. Using those two components can easily bring in a number of advantages, and, sometime, it is essential to use them. However, it may also introduce some performance issues to the environment, such as longer response time. In this section, a few check-up steps will be discussed to make sure the integration will not bring in any performance bottleneck. 5.1 Identify and Tune IHS Performance Issues When IHS is used in the environment, there may be a few problems introduced, such as unresponsive behavior, slow response time and failed requests, etc. Those problems, in most cases, happen because of the concurrency settings and IHS server working with WebSphere IHS plug-in. The first thing is to identify the simultaneous connections the IHS will need to support. The typical way to determine the maximum number of simultaneous connections is to use mod_status reports. To enable the report, follow the steps: a. Locate http.conf file for the HTTP server, and open with text editor. b. Locate the Loadmodule staus_module modules/mod_status.so configuration stanza in the configuration file. c. In Allow from, add the domain that will be used to view the status. # This example is for IBM HTTP Server 2.0 and above # Similar directives are in older default configuration files. Loadmodule status_module modules/mod_status.so
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.5 Linearized : Yes Author : Luobin Create Date : 2012:01:05 18:09:06+08:00 Modify Date : 2013:12:19 10:24:49-06:00 Language : en-US Tagged PDF : Yes XMP Toolkit : Adobe XMP Core 5.2-c001 63.139439, 2010/09/27-13:37:26 Format : application/pdf Creator : Luobin Title : Section 1: Tune WAS JVM heap Creator Tool : Lotus Symphony Documents Metadata Date : 2013:12:19 10:24:49-06:00 Producer : Lotus Symphony Document ID : uuid:ee1d5853-d1e6-4faf-8973-43e669f80717 Instance ID : uuid:40f1a182-4ed0-42c2-921c-b1f7d2412ed9 Page Layout : SinglePage Page Mode : UseNone Page Count : 44EXIF Metadata provided by EXIF.tools