Building Application Stacks
How are App Stacks Built?
The RISC Networks Platform joins servers into groups based on the collected connectivity and process information. There are different group types the platform will create automatically whose definitions can be found on our Glossary page under the subsection "Group Definitions"
How to View Connectivity
All of the processes outlined below are completed using the following two pages of the portal:
- Add Intelligence >> All Applications
- Add Intelligence >> View Individual Application Stacks
Connectivity can be visualized by working in and expanding the right side frame on both of these pages. This includes providing the ability to visualize critical or all connectivity as well as viewing connectivity internal to the application stack, external to the application stack, or both (all). When viewing the Add Intelligence >> View Individual Application Stacks page you also have two additional options for viewing and navigating connectivity information:
- View All Connections (top left) – Provides a table summary of all connectivity and includes three levels of drill down
- Dependent Stacks (from visualization control) – Provides a visualization of all stack-to-stack connectivity relative to the individual stack you are currently viewing.
TIP: The View All Connections can be an important way to view connectivity because it is NOT limited to connectivity between licensed nodes. The visualizations are limited to connectivity between licensed nodes with the exception of the RSG-Out Of Scope Services group.
TIP: In any visualization you can click on a line segment to view the details specific to that relationship whether it be workload to workload, or application stack to application stack.
Part One: Build Application Stacks
Once a sufficient amount of connectivity data has been collected in the environment the select the Build App Stacks button in the top left of the Add Intelligence >> All Applications page. This will kick-off the RISC Networks Intelligent Application Grouping Algorithm. Which will cause servers to be joined into groups based on their connectivity.
Any group that has not been saved or confirmed when selecting Build App Stacks will be deleted and rebuilt based on its current set of connectivity and process information.
Only licensed devices will be placed into groups automatically.
Part Two: Verifying Scope
Verifying Scope with RSG-Out Of Scope Services
The initial and primary mechanism of setting the scope for is the discovery phase. During this phase you will select the subnets to include in discovery and ensure you have access to the workloads contained within the boundaries of those subnets. However, additional scope validation can be performed post-licensing through a review of a special auto-generated application stack called RSG-Out Of Scope Services. The following includes the steps needed for a detailed review of the RSG-Out Of Scope Services group. The level to which you will need to perform this is dependent on the need for true “discovery” of the environment. In other words, if you are leveraging the discovery aspects of the platform to find systems you may not know about or have documentation on, then you would perform this validation in detail. If you are running the RISC Networks Platform against a known subset of the environment, then a more cursory review of RSG-Out Of Scope Services may be sufficient.
Iterate through the RSG-Out Of Scope Services group and verify whether or not the hosts in this group should be brought into licensing/scope. Our default advice is that everything in the environment should be licensed during the burst, so working through this group can be an important first step in ensure a complete scope. Once you license the devices in the RSG-Out Of Scope Services Out of Scope Services group (or decide that they are to remain out of scope), you should re-run auto-grouping to make sure that the group is empty – or that all hosts can be safely ignored.
You should routinely re-run auto-grouping and check this group as new services may come online at any time.
Make sure that there are not any CONSUMERS of services that are not licensed. This means going through the individual application stacks and checking their connectivity to “locations”. For specific definitions of locations please refer to our Glossary under the subsection Connectivity
To check consumption of important services, click “View All Connections” when viewing the RSG-Out Of Scope Services stack. This will bring up a table of all group- to-group and group-to-location connectivity. Sort this table by the destination group such that the RSG-Out Of Scope Services group is the destination. Once sorted, on the far right of the table you will see a “View Details” link, you should click this link for the first location listed in the source column. This will bring up another table that you should then click the button in the top left of the table to show a protocol view. Review the listed protocols to understand connectivity and if there are important services being consumed outside of this group. You want to verify that the traffic is not on a well-known port or protocol name, such as MSSQL, oracle, MySQL, etc… To understand particular IPs that are communicating on a protocol select the “View Details” link on the far right of the table, this will bring up another table that lists out the specific IP addresses. You should verify this list to understand if those items currently not licensed should be brought into scope. Repeat these steps for every location group within the first group to group/location table.
TIP: If you have a large environment where going through all the AutoApp stacks would be too time consuming, then the best way to determine consumers of services out of scope is to run the auto-grouping with the 95th group and view this information against the RSG-95th Highly Connected Stack. The RSG-95th Highly Connected Stack is the top 5% of hosts by connectivity in the network. In other words, it is the list of servers that talk the MOST. The “hosts with the most” we like to say.
Verify whether or not the out of scope IPs that are CONSUMERs or are OFFERING important services described above should be licensed. The group of the service host (the one offering the service) is inconsequential. The important thing is to determine if all of those hosts should be brought into scope and licensed as they may be front end servers or other applications that are accessing servers you are focused on. Once this process has been repeated, and there are not major connections in or out of the 3 major groups (Isolated Devices, RSG-95th Highly Connected Stack, and RSG-Out Of Scope Services), you can move on to application review.
Part Three: Application Stack Review and Refinement
First, it is important to understand what your starting or initial reference point will be. For most organizations, this will be the set of Application Stacks that are automatically generated by the platform using the Build App Stack function (found in Add Intelligence >> All Applications). Some organizations may prefer to use internally documented Application-to-Server mappings as the baseline to begin this process (e.g. CMDB). In this case the Application Stacks must be manually created and workloads are assigned to them as documented. The RISC Networks Platform has several UI features that allow for bulk manipulation of server workloads and their assignment to specific application stacks. This includes the ability to filter server lists based on naming convention, shift select, and right click to assign to an existing or new Application Stack. No matter which methodology you use as your initial reference point, you will likely use a combination of connectivity data, internal documentation (CMDB), and “tribal knowledge” during the Application Review and Refinement process.
TIP: Use the Graph in Add Intelligence >> All Applications to switch the view from Application Stacks to servers. You can sort and filter on any column in order to facilitate bulk movement of workloads to Application Stacks. Since Application Stack is one of the available columns this can also help to locate systems in the platform.
There are two techniques that can be used to review and refine automatically generated Application Stacks: 95th Resolution and Visualization with Movement. We recommend having the application owners participate in either approach as they can more rapidly iterate over known hosts based on their knowledge of the environment.
TIP: The easy way to determine which approach would work best for you is to first Build App Stacks without the 95th group. This will, generally, generate several application stack groups (AutoApp-xxx). Review these groups by selecting them and viewing the number of hosts in a group. If a group appears to have over 50 hosts, then visualize the group. This will visualize connectivity for that group, if the visualization is unworkable from either a portal performance perspective or visually because there are too many connections then you should proceed with the 95th Resolution approach, otherwise the initial auto-grouping was sufficient and you can continue with the Visualization with Movement approach.
TIP: There is a filter by number of connections slider within the left slide out on the visualization that can be used to determine highly connected servers easily within large visualizations
TIP: Before you begin creating your own application stacks it is important to come up with a good prefix and suffix structure to "tag" application. This will be very useful when looking at subsequent reports and visualizations. (e.g. ITM-XXXX-PRD would be an IT management app in my production environment vs. BUS-XXXX-DEV would be a business application in my development environment.
95th Resolution Approach
This approach utilizes the RSG-95th Highly Connected Stack. By running the auto-generation of application stacks with the 95th Percentile option selected, you will generate a group called RSG-95th Highly Connected Stack – which will be a small subset of heavily connected nodes.
Review and disposition each host in the 95th Group…”What is this host used for?”. Typically, you will have knowledge of these hosts since they are so heavily connected. You should disposition each host within the group into one of two categories: Services: These would be hosts that are offering a service to the rest of your environment. We have attempted to capture some of these in our RISC Service Group(s) such as exchange, Citrix, NFS, etc, but we have not scripted for every service that can be offered. Something like antivirus, authentication, logging, etc. would be examples of services. To disposition a Service, create a new application stack (antivirus for example). Once this is complete move that host by right clicking the host and selecting Edit Stack Membership. Application Stacks: These would be hosts that are a part of an application stack. These could be things like a shared DB. There are two ways to disposition a member of an application stack. For environments with heavily shared architectures (think large jboss layers or large shared DBs) proceed to Step Two, otherwise disposition Application Stacks by simply leaving them in the 95th Group and proceeding to step two.
This is only for environments with heavily shared architectures as noted above, otherwise skip this step. Create a new application stack (Shared DB for example). Move the server(s) that are part of that cluster (or it might just be a single db server) into the new application stack you just created. You can immediately view the connections of that application stack and see where that server or cluster is communicating. You would likely see other AutoApp-xxx stacks. If this communication is important (such as DB traffic), it is an indication that these application stacks are likely front ends running off of the DB. If the DB is shared, you can leave it there, and rename the front end groups (e.g. AutoApp-xxx becomes Main SAP FrontEnd) or you can move the servers from AutoApp-xxx into the new group you built for the server(s) you pulled from the 95th group. You repeat this process until the 95th group is empty. If you want to do a second round, you can re-run the auto-grouping and select the 95th group again. Since the original top 5% were grouped into Application Stacks, a new set of top 5% hosts will be identified and included in the 95th group, and can now be rationalized and grouped using the same methods as above. This is a very effective approach in environments with heavily shared architectures (think large jboss layers or shared DBs)
Re-run the Auto Generate Groups algorithm without the 95th group. This will place any remaining workloads, not in a saved or confirmed Application Stack, into an AutoApp_XXX stack based on connectivity. Proceed to the Visualization with Movement approach. Note: Any application stacks not saved or confirmed upon re-execution of the algorithm will be deleted and rebuilt. Only those Application Stacks that have been saved or confirmed will remain intact.
Visualization and Movement Approach
This approach works well in environments that are not heavily shared (no large DB farms or jboss/tomcat layers). We recommend having the application owners participate in this process as they can more rapidly disposition hosts based on their knowledge of the environment. Run auto-grouping without the 95th group. This can sometimes result in large vertical stacks that may join together multiple stacks if they use a shared resource or an unmapped RSG (an infrastructure service that we have not identified yet). Once you begin to go through AutoApp-xxx stacks, you can visualize them. In many cases you will see a single server or a small group of servers that link all of the hosts together. You can use the table below the visualization to see the actual application processes in use on the connections (if netstat data was collected). Our auto generated groupings may group single applications or multiple if they share DBs for example. Your objective is to decide how to and if application stacks should be broken into smaller ones. You can then make a decision as to whether that host (or small group of hosts) should be allowed to link those groups together. (We have already decided YES, but you can overrule us!) If not, right click on the host, select Edit Stack Membership and create a new application stack to move the host into. You can then immediately close the visualization and re-open it, to see the result of your move. The visualization may now show multiple clusters of servers. What you are seeing is the result that you will get the NEXT time you run Build App Stacks. You must re-run the auto-grouping to have the system break that stack apart, into two or more application stacks. Once this has been completed you should disposition that group by giving it a friendly name that is meaningful to you and the business (e.g. AutoApp-xxx should be renamed to ApplicationName). Repeat this process for each AutoApp_XXX Application Stack. All of this should be done with the application owners input. So while you may iterate a few times before you engage the application owner(s), they are critical to making the ultimate decisions on validating groups.
TIP: Remember to view connectivity data to check the inter-stack relationships and make sure that there are not any critical dependencies that would reuire further investigation or stack membership changes. This is also where you can check and verify that you don’t have unlicensed hosts using a service on a licensed node.
Review for Critical Dependencies
Once you have refined and confirmed an Application Stack, it is important to review the connectivity data for critical dependencies. These are dependencies that would dictate a change in Application Stack membership (e.g. a workload that needs to be moved into the stack you are reviewing). For instances where there is connectivity that is critical and noteworthy but does not dictate a change in stack membership, you may choose to document this information for planning purposes. This can include Reporting, BI, or Data Transformation services that leverage database connectivity into these Application Stacks. There are several options available to view connectivity for this purpose.
- Workload based visualizations
- Application Stack based visualizations
- Using View All Connections
Workload Based Visualizations
When you select an Application Stack in Add Intelligence >> All Applications or when you are in the View Individual Application Stack view, the bottom right hand section of the page provides a workload based visualization. Expand the visualization to provide more working area and use the visualization controls to show All connectivity that is Critical. This provides an easy way to see workload to workload connectivity. You can click on any segment to view the details specific to that relationship. Make sure to use the column chooser to make sure you include application and process information. Review the connectivity to determine if any of it requires a change in Application Stack membership or if something that is noteworthy and must be somehow accounted for in your migration plan.
TIP: You are typically looking for “critical” connectivity such as Oracle, MySQL, MSSQL or known application traffic such as SAP.
Application Stack Based Visualizations
When viewing the Add Intelligence >> View Individual Application Stacks page you have the option to view Dependent Stacks (from visualization control). This provides a visualization of all stack-to-stack connectivity relative to the individual stack you are currently viewing. Use the visualization controls to filter out infrastructure services such as Active Directory, Antivirus, etc.. This allows you to focus on true application to application connectivity. From here you can select any Application Stack to Application Stack segment that is interesting or that requires further investigation. By default, this will return a listing of the host to host connectivity that exists between these two stacks. You can switch this to a Protocol view to review from this basis. Again you are typically looking for “critical” connectivity such as Oracle, MySQL, MSSQL or known application traffic such as SAP. If you do see a protocol that is critical or interesting, you can View Details to drill into this and see the specific host to host connectivity for this protocol. This will also list the application and processes associated with the connectivity adding additional context to your review. Continue reviewing these Application Stack to Application Stack relationships until you are satisfied that there is nothing that would dictate a change in Application Stack membership.
View All Connections
When viewing the Add Intelligence >> View Individual Application Stacks page you have the option to View All Connections (top left). This provides a table summary of all connectivity specific to that Application Stack and includes three levels of drill down. The View All Connections option can be an important way to view connectivity because it is NOT limited to connectivity between licensed nodes. The visualizations are limited to connectivity between licensed nodes with the exception of the RSG-Out Of Scope Services group. Use this option to view connectivity coming into or out of the licensed environment. For example, you can use this dialog to determine if an Application Stack is internet facing. In this case you will see connectivity coming in from the RISC-unknown-internet location.