Service Control Explorer Manual
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 30
| Download | |
| Open PDF In Browser | View PDF |
ServiceControl Explorer User Manual Last Updated: 12/15/2018 Overview ServiceControl Explorer (SCE) is a WinForms application that allows you to make sense of messages stored in NServiceBus’ ServiceControl. You will be able to efficiently and effectively search for messages in meaningful ways that will allow you to extract answers about “how”, “what”, and “when” something happened inside your NServiceBus deployment. Think of SCE as your swiss-army knife to understanding how your NServiceBus system works. Software download SCE releases are available for download here: https://github.com/bly2k/ServiceControlExplorer/releases You only need to download the zip containing the binaries and extract it to anywhere on your local machine. There is no need to download and/or compile the source code. For example, below is the binary download for SCE 0.5.3: Prerequisites - You will need to first have Elasticsearch and Java installed. This is a one-time installation. After you install Elasticsearch, you will likely not need to install it again unless for any reason you need to swap drives or upgrade Elasticsearch in the future. Note: It is possible to run and use SCE without installing Elasticsearch. When this happens, you can only connect and search messages on a remote ServiceControl instance. This is less powerful compared to enabling local indexing of the messages into Elasticsearch but is nonetheless a viable option. For more details on this, see the section labeled “Disabling Local Indexing”. Installation steps Check/Install Java: 1. Open command prompt and run: java -version -d64 2. Check output from above and make sure there are no errors. Also Java version *must be 1.8 or higher* (for Elasticsearch) 3. Run: echo %JAVA_HOME% 4. Check that above JAVA_HOME environment variable is set and pointed to Java 1.8 (or higher) installation folder 5. If both above are good, skip the next steps and proceed below to installing Elasticsearch 6. Download and install Java/JDK for Windows x64 from here: https://www.oracle.com/technetwork/java/javase/downloads/jdk11-downloads-5066655.html 7. During installation, note the Java/JDK path you are installing to 8. Setup your Windows environment *System* variables as follows: Note: In Windows there are two sets of variables, one for User and another for System. We want to change the System variables, and NOT the User variables. Add the Java/JDK installation *bin* path to the Path *System* variable (make sure it is first in the Path list) Example: On my system, I installed Java/JDK to c:\java\jdk-10.0.2, so I will add c:\java\jdk10.0.2\bin as the *first* item in my Path system environment variable Define (or change) the *System* variable named JAVA_HOME and set it to the *root* JDK installation path Example: On my system, I set JAVA_HOME=c:\java\jdk-10.0.2 9. Close and *reopen* command prompt and rerun: java -version -d64, and make sure Java now as expected (version 1.8 or higher) 10. Run: echo %JAVA_HOME%, and make sure this is pointed correctly as configured above (pointing to Java/JDK root installation path) Install Elasticsearch 6.4.2 1. Download Windows Zip from here: https://www.elastic.co/downloads/pastreleases/elasticsearch-6-4-2 2. Unzip to a local folder (note where the Elasticsearch bin folder is) 3. Open *Admin* command prompt and run (from the Elasticsearch bin folder): elasticsearch.bat 4. Make sure there are no errors and that Elasticsearch does not terminate early (i.e. command window will remain open) 5. Open browser and verify that the Elasticsearch node is responding: http://localhost:9200/ 6. Terminate the running Elasticsearch instance (Ctrl+C) and make sure it shuts down cleanly (i.e. no Elasticsearch node/instance should be running before you proceed) 7. Install Elasticsearch as a service. Run: elasticsearch-service.bat install 8. Open windows services and find and then start the Elasticsearch service 9. Open browser and verify that the Elasticsearch node is responding: http://localhost:9200/ ServiceControl Explorer Architecture The SCE architecture is very simple and can be summarized in the below diagram: The above is explained as follows: 1) From the top left, an NSB deployment sends all messages to a destination MSMQ audit/error queue 2) From the bottom left, ServiceControl monitors the audit/error queue and pulls down the messages into its internal storage 3) From bottom right, SCE calls the ServiceControl APIs to download the (JSON) messages from its internal storage. Important Note: When SCE downloads the messages, this operation is non-blocking (i.e. think of this as a background task operation). In addition, the download is only as fast/good as your network infrastructure. SCE can download the messages as fast your SC/network infrastructure allows but your experience here will ultimately be constrained by how good your infrastructure is. Downloads can also be throttled if needed, which will be described elsewhere in this guide. 4) From the top right, SCE stores the downloaded JSON messages into Elasticsearch 5) Once a message is download and indexed into Elasticsearch, it is immediately (within ~1 second) available for search and analysis. Note: It is possible to do simple keyword searches against the ServiceControl data from SCE without downloading the messages down into Elasticsearch. However, the search that can be performed this way is less powerful than if you download all the data locally first. For more details on this, see the section labeled “Remote Messages” Starting ServiceControl Explorer After you download and unzip the SCE binaries to your local machine, run ServiceControlExplorer.exe. You will initially get this screen: This means that: 1. You do not have any messages yet downloaded from any ServiceControl instances into your local Elasticsearch node. 2. On the right side, a default a localhost ServiceControl instance is preconfigured (pre-defined) for you. This localhost instance has an auto-download capability, meaning it will download messages from that ServiceControl instance often without user-intervention. This is a convenience for local development. If you do not have a local ServiceControl instance on your machine, you will see an error when it auto-downloads. You can just safely ignore the error in this case – this is OK and will not interfere with the functionality of SCE. Downloading messages Downloading messages from an instance is easy – just right click an instance and select “Download messages” as follows: SCE will then start downloading the recent most messages (audit and errors) from the selected ServiceControl instance. Again, this operation is non-blocking and will not stop you from doing search/analysis on the download messages. As soon as a single downloaded message is indexed into Elasticsearch, you can search and analyze it immediately. Again, note that the localhost instance has auto-download capability so unless you want to download localhost messages in real time, localhost messages should always be fairly up-to-date to within the last minute or so. A few points to note: 1. If for any reason the download fails, you will see an error icon/status. To see the error, just right click on the instance and select “Show error” for more details of what has gone wrong. 2. The first download will pull down all messages from the instance. Depending on your infrastructure and message volume, this can take a while. 3. Succeeding downloads (after the first full download completes successfully) will only download new messages that have not yet been downloaded into SCE. 4. A download in progress can be stopped at any time (just right click on the instance and select “Stop download”. A stopped (or failed) download is resumable. What this is means is if you stop (or fail) at the 50% mark for example, you can resume the download from that point instead of starting from the beginning again. 5. If for any reason you want to redownload everything again, you can “Delete all downloaded messages” for an instance, and then start from the beginning again. Partial Downloads By default, SCE will down all messages (audit and errors) from a ServiceControl instance. If this is not desirable, it is possible to selectively download only specific messages. The purpose of this is: 1. Your instance might contain a lot of messages but you are only interested in looking at a small subset of messages (for example: a specific conversation(s), or messages with a specific date value, a series of correlated conversations, etc.) 2. You want to be able to pull the messages down selectively first and then analyze them locally offline using the rich capabilities of SCE. Partial downloads can be done in any of the following ways: 1. Download only failed messages Highlight any instance from the SC Instances list, right click, and select “Download failed messages”: 2. Download messages by query You will first need to run a search query against a remote instance (see section labeled “Remote Messages” for more details). Once you get the desired search results from the remote search, highlight the currently selected instance, right click, and select “Download messages by query”: 3. Download messages in a single conversation You will first need to run a search query against a remote instance (see section labeled “Remote Messages” for more details). Once you get the desired search results from the remote search, highlight any single message in the results, right click, and select “Download this conversation”: Important Note: For a single instance, only 1 active download can happen at any given time. If a download is currently in progress for an instance, you will not be able to start another download until either the active download is stopped, fails, or successfully completes. Throttling Downloads If you find that SCE is downloading “too fast”, you can throttle the download. This functionality is only available for non-localhost ServiceControl instances. The steps below will describe how to accomplish this: 1. If you want to throttle an existing/in-progress download, first stop the download. 2. Check the “Rate Limit Downloads” option and supply a reasonable value for the rps (HTTP requests per second) textbox: 3. Start or resume your download. Rate limited downloads apply to all downloads that you start after changing the rate limit. For example, if you download from more than one instance after changing the rate limit, it will then be applied simultaneously to all active downloads across the instances. In addition, changing the rate limit while an existing download is in progress has no effect on the existing download in progress. If you want the rate limit to apply to an existing download, you must first stop the download, change the rate limit, and resume the download. Adding a New Service Control Instance You can connect/download to as many ServiceControl instances as you like. To add a new instance, just right click anywhere on the SC Instances tab and select “Add new instance”. You will then be prompted for the instance API URL and then SCE will quickly ping it and if successful, the instance will be added. Once added, an instance is persistent across SCE restarts. Removing an Existing Service Control Instance You can remove an existing ServiceControl instance if you no longer want to work with it. To remove an instance, just right click on it and select “Remove instance”: Note that removing an instance will simply remove it from the SC Instances list. The downloaded messages for that instance (if any) will remain intact inside Elasticsearch. Searching Messages Note: Unless otherwise noted, all dates/times shown in SCE are local to the machine that runs SCE. There are 2 sets of messages available for search: 1. Audit Messages – these will all be shown in the Messages tab 2. Error Messages – these will all be shown in the Failed Messages tab Depending on your workflow, you will likely be working mostly with one of the above set of messages. Message Search (Audit Messages) You have the ability to search messages by 1. ServiceControl instance 2. Date/Range (against the message time_sent property) Note: When SCE runs, the “After” date filter is initialized to 7 days prior to the current day, meaning, by default, you will only see messages within the past 7 days. It is important to keep this filter on all the time for performance reasons when dealing with a lot of messages. 3. Time range window surrounding any single message: 4. Single conversation 5. Message property filters (can also filter by instance, or by endpoint) 6. Conversation type (more on this later) 7. Endpoint 8. Any arbitrary boolean/full-text query (for advanced users) In addition, high-frequency/”noise” messages are excluded by default. These generally are messages that recur very often and do not necessarily provide real value for most daily searches. However, there are some things you can do with high-frequency messages: 1. You can enable/include these messages in your search 2. You can customize this list if absolutely needed. For daily usage, this is should not be necessary. Message Search Results Message search results are automatically grouped into NServiceBus conversations. This is where the power of SCE begins. A conversation typically equates to an event that your system did (or more specifically, an event that is either currently still in progress or that has already completed). Conversations are easy to understand, discuss, and describe to people who new or are not very familiar with NServiceBus. Examples of conversations are: 1) A subject was randomized 2) A scheduled visit was completed 3) A drug order/transfer requisition was initiated If we narrow down messages first by date/time (when) and then by conversation (what), then we can efficiently find out about any event that has occurred in our system. From here, we can start to investigate other things such as what other conversations are related to the current conversation at hand and intelligently piece together a story/timeline of what has happened, when it happened, and how it happened. This is the essence of SCE. In the below example, you can see multiple conversations: Site Creation (9 messages), Seeding (1 message), SaveAutomatedSupplyOrderingStatus (2 messages), etc.: At a high-level without looking at any specific individual message, you can clearly see what events are happening chronologically inside your system. In addition, scanning from the top of the results tab you can see: 1. There are 32 pages of messages (that match the current search criteria). By default, each page consists of 1,000 messages. 2. You are currently on page 1 (of 32) 3. There are 31,238 messages that match the current search criteria (thus the 32 pages) 4. There are a total of 37,876 messages indexed for this instance. Note: Keep in mind that by default, high-frequency messages are excluded from your search unless explicitly enabled as described above. Note: The conversation groupings displayed are automatically derived from the NServiceBus messages. The actual label of the conversations are predefined in SCE and is user-configurable. If a conversation is predefined in SCE, then the predefined conversation label will be displayed. Otherwise SCE will pick the first message in the conversation and use its message type name as the label for the conversation. Note: If a conversation spans more than 1 day (for example, a conversation that triggers a saga timeout at a later date), it is broken out separately by days. This is to avoid confusion of when a specific message in a conversation was actually sent out. Expanding the first conversation (Site Creation) produces this: From the above: 1. You can see the 9 individual messages that comprise the conversation 2. Highlighting the first chronological message in the conversation (PublishSiteAdded), you can see that it is processed by a handler that in turn sends out an outgoing message (SiteAdded). The SiteAdded message is then handled in 2 endpoints (ITOps and Services.Developer) Highlighting the SiteAdded message/handler in Services.Developer, we then get this: From the above: 1. You can see that SiteAdded (in Services.Developer) is processed by a handler that then sends out 2 more messages (DetermineIfSiteHasASupplier and DetermineIfSiteHasDoNotShipIntervals). 2. You can see that SiteAdd (in Services.Developer) originated from a handler that processed PublishSiteAdded. You can repeat the above process by highlighting any message in the conversation and you can tell where you came from (originating message) and where you are going next (outgoing messages). This allows you to “single-step” and correlate messages individually by causation. When you are highlighting a single message, you will be able to see the message body/payload on the bottom right section: You can also quickly see the payloads of any unselected message by hovering over the message quickly in the messages list. SCE will show quick tooltip with a snippet of the message payload. Conversation Flow The conversation messages list gives you a “flat” chronological display of messages. It is also possible to display a higher-level “flow” display of messages. When a single conversation is highlighted, you can switch to the Conversation Flow tab to see the “flow” level visualization: Similar to the messages list, you can see the message-level causation chain (originating and outgoing messages), but this time, you see the flow in a hierarchical manner with more context. The flow is useful to be able to: 1. Quickly see from a high level how the entire conversation structure looks like 2. Quickly determine what endpoints are involved/touched in the entire conversation 3. Quickly understand how messages are sent out in terms of overall sequential/parallel timing. This is very useful to evaluate and/or debug potential timing issues when handlers are expecting messages to be sent/completed in a specific “logical” order. 4. Quickly see/evaluate critical payload properties of messages Note: A very important thing to note here is that the conversation flow will always show all the messages in the conversation. For example, although you typically search/filter and find only a subset of messages per conversation, highlighting the conversation/message in the message search results will always pull up the entire conversation on the Conversation Flow tab. This is a very powerful feature that will allow you to quickly understand and diagnose context surrounding any individual message. Conversations Aggregation SCE aggregates the different predefined conversation types and provides a count of each type. You can see this from the Conversations tab: From the above, you can see: 1. Multiple conversations were found from the matching search results. From the top, they are: Subject End of Product, Subject Kit Assignment, Subject Kit Replacement, etc. 2. For each conversation type, the aggregated number of conversations is displayed next to it. So for example, from above, there are at least 35 Subject Randomizations (in the message search results) 3. For each aggregation, SCE picks the top 25 conversations. If you expand the conversation type, you can see up to 25 individual conversations. Checking a conversation will then allow you to filter and only show messages that are about that conversation. For example, clicking Subject Randomization produces this: From above, you can see: 1. We are now narrowed down to only Subject Randomization conversations 2. In the messages results, you can see that only messages relating to Subject Randomizations are shown. Note: It is possible to predefine conversations that contain messages that overlap. In the above example, you can see that Enrollment Failure is another predefined conversation. More importantly, you can tell that of the 119 Subject Randomizations (in the current search results), 2 of those conversations are also about Enrollment Failures. Note: If there are multiple conversation types selected/checked, it will match all conversations that are either any of the selected conversation types (i.e. this is an OR filter condition). If you select a specific conversation (i.e. one of the children of a conversation type), it will narrow down only to that one conversation – in this case, the single selected conversation takes precedence and no other conversations will be shown in the search results. Property Filtering Property filtering is another powerful feature in SCE. Given any message, you can quickly narrow down your search criteria using any of the property values of the selected message. In the below example, I am currently highlighted on a RandomizeSubject message: On the right is the Properties tab showing all the header and message/payload properties of this specific message. If I want to find all other messages (regardless of message type) that contain the exact same value as the above SubjectId – meaning my intention is to find what else has been happened to this specific subject, I can right click and select Apply global filter: Which then narrows down my results to only messages/conversations about this specific subject: From above, you can see: 1. Message search results are now returning only messages that contain the SubjectId that was filtered from the above action. 2. The SubjectId filter that is currently in-effect is shown on the top of the message search results. 3. Any time you wish to remove the current SubjectId filter in-effect, you can just click the [x] button next to it and the message search will automatically re-execute without that filter. Using the above technique, you can also filter on any message header property. For example the below filter will allow you to narrow down your search to only to messages that originated from a specific UserId: In other words, you can use the above filter to ask SCE to show you everything that has been done in the system by this specific user. Note: Multiple filters can be applied. As you apply them, they will all appear in the filter section above the messages. All filters are ANDed when executed. This is by design. The purpose of filters is to quickly narrow down your search based on information about a specific message/conversation that you are currently looking at. Exceptions Failed messages in the message search results are explicitly and visibility highlighted as follows: From the above, you can see: 1. On this page, 3 of them are failed messages and you can see their corresponding conversations in context. Highlighting one of the failed messages and switching to the Exception tab, you can quickly see the details about the failed message (as well as the relevant source code lines automatically extracted for you): Note: It is very important that you look into failed messages and find the underlying cause and resolve them as soon as possible. Ignoring failed messages can possibly mean that they will go to production and eventually cause bigger problems down the line. You do not want this to happen. Ignoring failed messages will result in poor quality software. Failed Message Search (Error Message) You have the ability to search failed messages by 1. ServiceControl instance 2. Date/Range (against the message time_sent property) Note: When SCE runs, the “After” date filter is initialized to 7 days prior to the current day, meaning, by default, you will only see messages within the past 7 days. It is important to keep this filter on all the time for performance reasons when dealing with a lot of messages. 3. Time range window surrounding any single message: 4. Message property filters (can also filter by instance, or by endpoint) 5. Endpoint 6. Any arbitrary boolean/full-text query (for advanced users) Failed Message Search Results Failed message search results are much simpler and more straightforward than the messages: Each failed message is shown with a corresponding status visualization (unresolved, retried, or resolved). In addition, highlighting a single message will show the raw payload as well as the detailed exception information as follows: Similar to messages, you can apply property filters to failed messages like so: Remote Messages SCE allows you to work with the ServiceControl messages without downloading any of it down to Elasticsearch. If you do this, the functionality/experience that you will get will be limited and not as expressive compared to downloading all the messages up front. In order to do this: 1. First select/check only a single instance from the SC Instances list: 2. Then switch to the “Remote Messages” view: From here, you will be able to: 1. Perform simple keyword searches against the messages 2. Perform paging as well as jump directly to any page 3. Contextually view the conversation flow of any selected conversion Since the messages are queried remotely ad-hoc and for performance reasons, this feature will not be as rich compared to having all the messages downloaded up front. In other words, search is limited to keywords only and the results that will be shown in SCE are limited to small pages at a time (default of 100 messages per page). In addition, the conversation flow will only show the limited messages that are downloaded within a page at a time. Disabling Local Indexing If you do not wish to download/index any ServiceControl messages into Elasticsearch, you can completely disable local indexing. To do this, shutdown SCE, edit the ServiceControlExplorer.json file and change the property localIndexingEnabled=false, and restart SCE. From this point on, no local message searches/indexing is allowed. The only thing you can do is run searches against a remote ServiceControl instance.
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.5 Linearized : No Page Count : 30 Language : en-US Author : Ly, Binh Creator : Microsoft® Word 2013 Create Date : 2018:12:15 08:20:26-05:00 Modify Date : 2018:12:15 08:20:26-05:00 Producer : Microsoft® Word 2013EXIF Metadata provided by EXIF.tools