Inception

Requirements Definition

Cost Benefit Analysis

Scoping - Elaboration

Impact Analysis

Project Planning

Project planning is the art of breaking down the entire project into distinct logical units of work that are tangible in terms of labor and cost and being able to assign resources based on capacity and capability in order to be able to execute these logical units of work with a feasible estimate. At this stage in the project, when the Requirements have been defined, Cost Benefit Analysis and Scoping has been completed, Project Planning can begin.

Though a perfect estimate is almost, always impossible to predict, estimation is the art that defines effective Project Management. The ability to evaluate individual resources, their capability, and the task at hand combined with the materials needed by those resources in order to achieve the tasks, often evolves over time. Planning alone will lay the entire project out for the team, making it easier to visualize the execution and see where failures could happen and contingency plans can be laid.

Usually, this activity is done in multiple stages using a tool like Microsoft Project, or the open source alternative - Open WorkBench (openworkbench.org).

1. Break the project down into a hierarchical organization of tasks, subtasks, sub-subtasks and so on to achieve a linear representation of each logical unit of work
2. Lay the dependency between each logical unit of work in order to arrive at a sequence. Some tasks may happen independent of the other, enabling you to parallelize the execution, while some cannot begin until some others end, indicating dependency.
3. Identify resources on a project-wide basis along with resource-availability information - schedule, work hours, etc.
4. Tie each resource to a task based on the effort required to complete that logical unit of work and the resource's availability and even capability.
5. Arrive at a logical end date for the project to complete execution.
6. Dependency is important - if one logical unit of work takes more time, or does not occur, the tasks and subtasks that depend on that task will take that much longer to complete, or may not complete.
7. Parallelism is important. It is crucial to not have resources waiting to complete their individual tasks because it has a dependency on another task. Therefore, it is important to break down the project's execution into as many parallel streams of execution as possible so that resources can get work done independent of each other.
8. Visibility - if the Project Planning is done effectively, and a good plan is in place, any abnormal activity or delay will affect the final deliverable date, thus providing management an optimized view of the deliverables. This is key.

Use Case Definition

Defining every atomic process-actor scenario, the permutation and combination sequence of which define the new / modified process.

A Use Case defines the “Who” does “What” to the System. The perspective is one of a third person viewing the Actor (user) and System (application / process) as two separate entities. The “what” is the activity that a Use Case Diagram is designed to illustrate. Modeling each use case around each requirement will ensure that all the requirements have been accounted for and greatly reduce chances of error or situations where requirements are missed.

NOTE: A Traceability Matrix plays a very prominent role in Software Development. A Traceability Matrix is defined as a matrix where a straight line can be drawn between the below elements from inception to implementation:

• Requirement/s
• Use Case/s
• Unit Test Case
• Integration Test Case
• Implemented Feature in Application or Module Version (Release)

This matrix will provide visibility to the business, as to what the implementation contains – specific to each requirement, and will enable them to understand the application implementation.

Functional Requirements

Defining how each use case and process will translate into application behavior – usually an algorithmic definition of the process flow.

For example, in Software Development, a Process Flow Diagram (PFD), Data Flow Diagram (DFD) and an Entity-Relationship (E/R) Diagram will serve to elucidate the changes both at the processing (Application) layer and at the information persistence layer (Database). These are best documented in a Functional Requirements Document that expands on the requirements document. The following sections are advisable in a Functional Requirements Document (FRD)

• Requirements
• Application / Module Details
• Process Flows
• Data Flows
• Entity Relationships
• Use Cases

You may have to repeat the above sections for each Requirement / Application and Module in question.

Conceptual Design

A definition of the design changes – with integration into an existing application infrastructure if needed, and high-level class diagrams and sequence diagrams - need not specify methods and properties at this level.

Design is by no means something that is covered in a one-page SDLC Document. You will have to incorporate standard design practices, as dictated by your platform, development methodology and environment. The purpose of conceptual design nevertheless is the most important aspect to keep in mind. Your goal is to depict and document exactly how the different modules in the code will (at a high level) address the requirements, or the problem that you are trying to solve. Pictures say a thousand words. Use livid illustrations to depict how your implementation will solve the problem. Needless to say, Conceptual design is for both programmers and managers - therefore it needs to be friendly to people who do not understand the intricacies of Remoting or CORBA for instance, but know the objective and purpose and understand the technology enough to understand your plan.

Here are a few of the artifacts that you can include as part of High Level (Conceptual) Design

• Process Flow Diagrams
• Data Flow Diagrams
• Entity Relationship Diagrams
• UML Use Case Diagrams
• UML Deployment Diagrams
• UML Class Diagrams - Object Level Modeling
• UML Sequence Diagrams - Object Behavior at a high level

Detailed (Functional) Design

Definition of each change and elucidation of the flow of code. Pictures say a thousand words. Do not type paragraphs of text in a document. 90 percent of documents are never read just because the amount of effort to convey the information is insane. Pictures do speak a thousand words. Compress - its is quality over quantity.

Class diagrams depicting the exact signature of the class indicating all accessors/mutators, methods, members, along with input and output specifications will ensure that a glance will suffice to depict the work that needs to be done. Of course, subtleties like including descriptive (Hungarian Naming Convention) method and member names will make the class diagrams themselves more readable than having to insert notes into your diagrams, which frankly take us back to the problem of wordy documents.

Behavioral modeling is a very complex aspect. From a developer / architect's perspective, process flow diagrams, use case diagrams and flowcharts are no match for what a sequence diagram can do. A sequence diagram depicts Object lifetimes, Order of instantiation and order of invocation, which without any words, depicts how your program will work. Again, this document will not teach you UML. There are a lot of web sites that attempt to teach UML. I learnt from some of the best folks back at Rational, but standards and specifications are subjective. If you and I follow the same standard, we can communicate. So the goal is to ensure that you have a standard, and that all the members on your team adhere to that standard. And if you are going to take the effort to come up with a standard, using the existing UML 2.0 standard should make your task much easier.

Here is a list of artifacts that would greatly benefit design:

• Code Documentation - Tools like Rational SODA or nDoc or Doxygen can actually generate comments right from your code comments into CHM or HTML format. Generate the documentation and put it out on a link on your company's intranet server or development portal. If you dont have one, set one up. The include the link to the documentation in your design document. Dont paste pages of class library definitions in a word document. That is painfully redundant and then when you start emailing stuff around, it thins the line between a bad practice and a crime.
• Class Diagrams and Sequence Diagrams. Both go hand in hand
• E-R diagrams for database artifacts is critical
• Also, make sure that you sufficiently document your error handling and any special enumerations and constants that are specific to your process flow.

Prototyping

A Prototype illustrates a working model of the architecture behind an implementation.

In many scenarios, when an implementation utilizes an architecture or topology that has not been implemented before, or presents performance risk, it is required to prototype that implementation. In major implementations prototyping can greatly reduce the risk of running into performance bottlenecks raised due to architectural flaws in production. Keep in mind that the volume-demands and throughput in production will be a multiple by several factors of what you see in the test environment.Prototyping serves its purpose to evaluate the underlying architecture than the business process flow, which is half the battle.

Design Review

The Design review process will involve a complete review of all artifacts ranging from Requirements documentation to the Design Document to ensure that the methodology and approach to implementing the required solution is correct and necessary.

The design review will be performed by peer developers and architects depending on if the peers are comfortable and capable of validating the design and the fact that the reviewers are not the ones who came up with the design in the first place. In smaller teams, if a 'peer' is difficult to locate, it does not hurt to go out, have a cup of coffee and come back and look at your design with an open mind. Coffee, cigarettes, whatever it takes to detach yourself from the perspective that you had while you came up with the design. Alcohol has proved to be detrimental to this effort though, almost in every case.

In some cases, as deemed necessary by the design reviewers, prototyping may be required and hence requested to further validate the design. If flaws are detected, it does mean rework, but saves effort early on, rather than in the implementation phase.

Threat Modeling - Security

Security is the most important aspect of the infrastructure. Every single change needs to be designed against vulnerabilities not only related to performance and business process errors, but also to security. Threat Modeling is the most overlooked, yet most critical part of application design.

It is always possible to design an application without security in mind and then build security into the infrastructure, but almost always too late. Businesses and people cannot afford to get two passes at the security implementation. It has to be done once. It has to be done right.

• Some common practices to keep in mind:
• Ensure that the application uses encrypted connection strings to the database
• Ensure that sensitive data fields in the database are encrypted.
  • Oracle VPD, or Application Encryption is very easy to implement and worth the effort.
  • This includes patient sensitive HIPAA (Health Information Portability and Accountability Act) information (if you are working with healthcare data) and
  • PCI (Payment Card Industry – Data Security Standard (DSS)) information (if you are dealing with payment data).
  • Even if it is just customer data, it makes sense to encrypt or implement a role based security layer to hide personally identifiable data (email, social security numbers, account numbers, zip codes, phone numbers, etc) from unauthorized access.
• Ensure that every screen, module and functionality in the application utilizes role-based security.
• Refrain from creating User / Password tables for individual applications.
  • Use Active Directory or LDAP Integration into active directory to consolidate the security credential repository within the enterprise.
• If your applications perform data exchange over TCP/IP protocols, irrespective of whether the data is sensitive or not, utilize encryption mechanisms like HTTPS, SSL, SSH, SCP, etc.
• Refrain from using weak passwords for applications
• If your application uses a generic account for logging into a network resource or a database, refrain from using that account for personal access.
• Do not hard code the password in code, or even a configuration file. Use the “Log on as” model so that the password can be controlled and changed by system administrators without risking breaking the code.
• Refrain from using generic accounts when you are accessing network or database resources manually (re-iterating on purpose)
• If your application delivers files over the Internet, use SSH. Require the use of SSH for all document exchange.
• Refrain from sending documents as email attachments – reports, etc.
• Place the document in a network / portal folder and send the path in the document as an email.
  • This will not only save network bandwidth and mail server storage space, but also prevent the unauthorized delivery of data in the event that your email gets routed incorrectly or into the wrong hands.
  • By using a UNC path, the person will still have to authenticate into the network to access the document.
• Check Web Applications for SQL Injection checks. If your web server does not comply with SQL Injection protection requirements, stop and ask the Systems or Network administrator to implement a packet filtering firewall.
• Always use session variables for critical security contexts in web applications. Passing variables in both the query string and the form are inadequately secure.
• Always run a security vulnerability test on your web application.
• Always include role-based security test cases with both your unit test and integration test cases so that your implementation’s security is verifiable.

This and a lot more come to mind. But again, this is a brief checklist to ensure that your implement is hack-proof, every good developer / architect should have his own threat-modeling checklist.

Construction

Based on the design, implement the changes in code into the code base using the Programming Language or IDE that the development process dictates.

It is advisable to segregate each atomic unit of business processing into its own container – usually a function / method. Modularity in code dictates that each independent action should be wrapped in an independent function to ensure that sections of code are non-redundant.

Commenting in code is critical to both the readability and the maintainability of the code. Comments should be used generously and should be as to-the-point and descriptive at the same time.Many IDE’s especially Visual Studio.NET for C# developers and the Java Net Beans IDE include very good support for commenting all classes and class members, along with built-in nDoc support to document Method inputs, outputs and function. This should be followed very strictly. When working with Tables and other Database objects, ensure that the code is commented appropriately and formatted for printable reading.

Depending on the type of implementation construction is a very diverse topic and it would be impossible to define it in a single document. This document is designed to take phases of various types and various aspects of construction and dictate the rigid platform of uniformity in everything that your enterprise will construct. This will enable others to understand your implementation and help you yourself to understand the implementation when you have to go back to study it, either for maintenance, extension or troubleshooting. Also, if you are reading this section, understand that you are, because you already know how to construct, or are on the path to becoming a constructor.

This document will entrench some basic principles into your development methodology, which will enable you to be part of a team that not only builds Applications, but Application Frameworks.

Unit Testing

Unit testing is the first logical phase of testing each building block of an application individually. More important than the testing itself is the ability to develop a Unit Test Case that can be used to test your specific unit of code when needed.

Platform tools like nUnit (C#.NET) and jUnit (Java) enable you to build the Unit Testing (Driver) Classes into your code so that when the application is being tested as a whole, each unit test case can be covered to ensure that all the bits and pieces that make up the process flow are working as intended.

This approach also readily integrates with automated testing initiatives, greatly reducing rework for test automation.

Source Control Reconciliation

New / Modified artifacts should be reconciled with the Source Control database on a periodic basis - Checkpoints.

In a team environment it is recommended that code that has been modified be checked in as frequently as possible, and only in a stage where the code is working. Checking in code that has a compilation error, or logical incompleteness that can affect application build ability, functionality or stability is NOT recommended, especially if multiple developers are working with the same artifacts in the repository.

For those specific modules that need to ensure that no parallel changes (branching of release) is being done, it is best to exclusively check out the artifacts and leave a note that indicates the reason for doing so, in order to communicate to other team members / developers the reason for the exclusive check-out.

There are multiple source control platforms out there, PVCS, CVCS, CVS, Subversion, Bazaar, Microsoft Visual SourceSafe, Microsoft Team Foundation Server, Rational Clearcase, PerForce, etc. Irrespective of platform, it is essential to have a user-guide to your specific platform (more like a cheat sheet - remember, quality not quantity) which you can share with developers to enable them to effectively use the repository.

Furthermore, it is extremely desirable to incorporate role-based security into your source control repository, by integrating with roles and users from Active Directory or LDAP / Samba to ensure that you dont spend half your time administering users.

Subversion is one of my favorites and integrates well with Apache's mod_ldap to provide an excellent Source Control solution, which is both time-tested and $CHEAP$. Integrates with Active Directory as long as your domain controller allowes anonymous LDAP queries, which is the default configuration.

Test Case Documentation

A Test Case is a defined set of steps that an application user or ‘Tester’ will execute to determine whether a requirement has been successfully met or not.

Test Cases begin with the developer. As mentioned earlier in section 1.2.13, the developer along with the release should include test cases. This will enable your Testing / Quality Assurance team to define Pass / Fail criteria and understand the functionality being implemented in more detail. (Remember the reference to the Traceability Matrix earlier.)

Test case documentation should include the following (at the very least):

• Requirement
• Description
• Test Case
• Pass Criteria
• Fail Criteria
• Results
• Release Notes
• Release Version #

Integration

Integration is defined as the act of enabling the execution of the functionality in a module or application in its true execution environment (TEE). Made that one up, I swear.

The true execution environment is the collective implementation of the module or application and all the other modules and applications that execute along with it as they would be in a production scenario.

• In a lot of cases, it may be impossible to produce an integration environment with all the integrating applications or modules.
• In such a case, clear mention has to be made in the test cases and release documentation, of the missing integration and how development and testing were completed and the pass/fail criteria were determined.
• Stubbing – It is a very good practice to write a proxy or an implementation skeleton of a real integrating application or module in its absence, with the ability to simulate application integration responses as would be in real time.
• A good example is payment / credit card processing. While it may not be possible to use real credit cards and integrate with the credit card processor during unit testing, or even integration testing, it makes sense to build a Proxy layer between the credit card processing API, so that your proxy layer can simulate standard success and failure responses, driven by configuration in the absence of the actual integrating layer.

The main goal of integration is to be able to obtain a build of the entire application with your changes, so that it is possible to test the functionality of the application’s execution in its true production state.

Integration Test Case Documentation

Integration Test Case Documentation is defined as the documentation of all applications and modules that integrate with the one that is being changed, and the scenarios in which these applications / modules integrate and criteria to determine if that integration scenario succeeded or failed.

The primary documentation responsibility of documenting the Integration Test cases is on the developer, though this responsibility extends to Integrating System Owners, Business Analysts and System Administrators that monitor the integration. Integration Test cases rely extensively on 3 C’s – Communication, Collaboration and Coordination between the three entities mentioned in the last sentence and the developers.

Integration test cases provide the first true means to identify and detect Impact of changes to one part of the Enterprise Application Framework on another.

The below list illustrates what an Integration Test Case Document should entail:

• Integration Scenario
• Integrating Application / Module Details
• Integration Test Case
• Pass Criteria
• Fail Criteria
• Results
• Release Notes
• Release Version #

Integration Testing

Integration testing is defined as the act of executing a module or application in its true execution environment (TEE) as described in the Integration section.

Integration Testing not only involves the testing of all the individual unit-test-cases in the order and context of processing execution, but also the testing of functionality in applications and the modules that depend on the module that is being changed.

Both the results of Unit Testing and Integration Testing should be provided to the Testing / QA Team along with the test cases themselves to facilitate the QA Process.

Integration Testing will also define the contract between external applications and process. The documentation of both the test cases and the results of the integration test cases will determine the overall success of the project / change within the enterprise.

Regression Testing

The testing of software from not only a process and load perspective, but also with the variance of data permutations and combinations to ensure that it will handle all data scenarios that it will encounter during its lifetime, especially when a part of the software has changed.

Regression, by definition means a repetition of the entire test effort. There is a common practice to do a regression test even if a small module within your application or application framework has changed. This is to ensure that not only the affected module or unit of code, but also all other pieces of the framework that are likely to interact with this module are tested, and that they are verified to work as they were before this change happened. One rule of thumb is that if you have a method in place with a specific signature and this method has been in the codebase for a while, do not change the method signature, even if you dont see a compilation error. This is a DLL world, and someone, somewhere has code that uses your method or class. Regression testing should not just test applications/modules in question, but the entire enterprise codebase (Intentionally redundant).

Another reason to regress, is that developers tend to work with very small units of arbitrary test data during their testing, and when the same unit of code is put through a much wider data regression, the code fails. A common example is a customer search screen which accepts first name, last name, city, state and/or zip to do a customer search. Unit testing was completed and even integrated testing was completed. When we did a test with a huge list of real customers, it broke when the customer "Jason O'Daniels" was passed to the module. Of course nobody anticipated a last name with a single quote - but that is the reason to regress - over and over, again and again.

They say - Try, Try, Try again till you succeed. We say - Try, Try, Try again till you fail.

Test Automation

Test Automation is defined as the ability to execute all the unit test cases and integration test cases in an automated manner with the ability to simulate a multi-user environment with appropriate regression to ensure quality.

Test Automation is a true reflection of the capability model of the development infrastructure. To get to this stage is definitely not an automated process.

Automating test cases depends on the fact that all test cases have been effectively defined, and have been programmed into the test codebase, either through driver programs written manually by the developer, the QA Engineer, or through driver programs generated using an automated test case validation framework like nUnit, jUnit and Rational Robot.

Isolating the Automated test environment from the production environment is crucial to prevent inadvertent modification to production data. This should happen in an isolated network where IP Traffic should be filtered from entering the production network at the WAN/LAN layer to prevent inadvertent damage due to incorrect configuration or artifacts that could point to production.

Most of today's automated test tools, Rational Robot and Silk Performer to mention a few, are capable of working with web applications, windowed applications (note, this is not just a windows world, Unix and Linux have applications too) and batch / service applications. These applications are designed to profile applications as they execute and expose Objects and methods in real time to the engineer so that they can inject data and invoke controls through code, simulating a real user's activity. This automation can then be parallelized to simulate multiple users with adequate data variance to achieve real world throughput.

When automated testing has been implemented, it can save a lot of time by eliminating a majority of the QA work, thus increasing the ability to identify success or failure at the enterprise integration level, with minimum scope for error. Most people undermine the importance of Test Automation, primarily due to the cost involved in developing the automation solution and then, having to maintain it everytime the code changes. However, this is by no means overkill.

Source Control – Release Labeling

At the point at which the source code has been released to QA, or after QA has successfully extracted a working build on which to perform QA, QA should take a snapshot of the Source code in the form of a labeled release.

This will enable development activity to continue uninterrupted and in parallel while QA efforts are under way in order to maximize development effort and the ability to utilize results. This also provides the ability to have checkpoints in the source codebase itself in the event that the enterprise may have to revert back to a previous release if failure is detected, post-deployment – Bad scenario, but definitely prevalent.

Imagine three modules A, B and C. On a specific date, say December 20, a specific version of A, B and C is handed over to QA, and they test it for three days. During these three days, developers continue to change A, B and C further, for future releases. At this point, on Christmas, if QA finds a small bug, and requests a re-build, it is mandatory to go back to the last known good state of the release instead of where it is now, so that the simple change can be made to get the current release working. It is also important that this simple change also be made to the current implementation, but not as important since that code has not been released yet.

Peer Code Review

Peer Code review includes a complete review of all the source code that is transitioned as part of the release, all the unit test cases, all the integration test cases and a demo of the integrated build of the application in its true execution environment by a team of peer developers, who preferrably are not involved directly in the development of that release, or at least have the opportunity to review modules that they did not develop.

Peer developers who are not directly involved in the development process, but have a complete understanding of the implementation and the implementation platform will perform the Code review. The goal is not only to ensure the quality of execution, but to ensure that consistency is maintained in the organization codebase in all aspects, from configuration, logging and event handling to naming conventions, indenting, commenting and even formatting of code.

Build Automation

Build Automation is the task of programmatically automating the following steps in SDLC:

• Extraction of a specific labeled release from the Source Control Repository.
• Labeling the release, if needed in the Source Control Database.
• Compilation of different components of the release to produce the executable binaries in one or all environments that the change will impact.
• Deployment of all binaries to their true execution environment, testing the Packaging and Deployment scripts themselves.
• Execution of the binaries in their True Execution environment to validate all the Test Cases (Unit, Integration and Regression).
• Documenting the results of the Deployment and Testing and automatically notifying the development team members in the event that failure is detected along with “What/Where/How” information regarding the test cases that failed.
• Automatically generating Documentation based on Code comments and the ability to generate release notes.

Packaging

Packaging is defined as the activity of organizing or bundling software that has changed in order to prepare it for deployment to a production environment.

Numerous tools and IDE’s enable packaging of all software artifacts into a deployable executable. There are three kinds of packaging based on Major releases and Minor releases depending on the functionality being released. The mode of release is decided, or governed by the implementation Experts.

• A Hot-Fix or Patch is used for a Minor release where the entire Implementation-base does not have to change in the production environment. In an existing production environment, this is the most frequent method of packaging a release. This kind of Package can be deployed without the need for downtime in most cases.
• A Service Pack or Major Release is performed when several components and their integrating counterparts have been changed and tested. This kind of Packaging will almost always require system downtime and is usually communicated and performed through a scheduled outage.
• A Full Release is the act of replacing the entire implementation with data conversion routines required at the Database layer in some cases. Like a Service Pack, this release definitely requires System Downtime and is usually communicated and performed through a scheduled outage.

Regardless of the kind of release, the following components are a mandatory part of the packaging process:

• Release Notes - A list of all fixes / features that are being implemented, outlined linearly indicating how this release is different from its predecessors.
• Bill Of Materials (BOM) – A complete list of all artifacts whose source code has been changed, the version number of that artifact in the Source Control database for the particular release and notes that clearly defining the reason for the change.
• Rollback Plan – A clear listing of the Version Numbers of all artifacts in the source control database to revert back to in the event of failure to restore the system to its last known good configuration (LKGC) and the steps to be performed in order to reach that LKGC. This may very often be the BOM pertaining to the Release that built the LKGC.
• Installation Plan – Documentation of the deployment process, as illustrated in the next section along with detailed instructions defining both the sequence and method of invocation of all scripts or steps required to deploy the changes into the pre-production (staging) area.
• Pass / Fail Criteria – Criteria to validate that the deployment was successful.

Distribution Documentation – Deployment Guide

The deployment guide is the documentation handed over to the System Administration, Network Administration and Database Administration team, elucidating the steps that need to be taken in order to release the changes from the Development or QA environment into either the Pre-Production (Staging) or Production environment.

The deployment team members are usually responsible for maintaining the production environment and transitioning verified changes from the QA Environment into the production environment.

The first step in the deployment process is usually to use the deployment guide to understand the deployment process, following which an attempt to deploy the software change from QA to the Pre-Production or Staging Environment will be made. At this point, if the deployment process is deemed defective or deficient in any manner, the build will be marked as a failure and rejected.

If the build is deployed successfully to the Staging area, the build is deemed successful, based on the pass/fail criteria provided and a date and time is provided for when the build will actually be deployed to production.

The responsibility of the deployment guide resides with both the developers and the build engineers who will aid in both constructing it and validating the release.

Quality Assurance

The QA Team (if there is one), or the development team will be in charge of ensuring that all the artifacts that pertain to this code release are functional and free of defects, with no deficiency in any of the steps covered earlier in this document.

QA is the final gateway before the developed product goes to the End User or Consumer. QA will be in charge of validating the Traceability matrix and ensuring that every single requirement has been matched with an implemented feature set in the application’s functionality. QA will also validate that the Deployment Scripts and documentation are intact by performing the actual deployment process from the development environment to the QA / Testing environment. Upon successful validation of the build, the QA Team or the application Test Team will pass on the application to the business analysts and business users for functionality validation and user acceptance testing.

User Acceptance Testing

In this step, the Application is provided to the user along with the traceability Matrix and the release notes so that the Business Users and Business Analysts can validate that the solution being implemented meets their needs.

Training

Depending on the functionality being released, after UAT, the Business Analysts and Business users performing UAT will also perform the task of training the end users / customers to utilize the functional changes that result in the application process as a result of these changes. This activity will very likely need to include a user guide.

Training Documentation - User Guide

Depending on the functional changes, the Business Analysts will have to construct documentation – User Guides that will enable to transition the system knowledge of the change in the business process to the application users.

This will be a modification or addition to the existing user / training guide and is a key part of the Application Knowledge Management Process.

Performance Monitoring & Testing

Performance Monitoring and Testing are the collective activity of constantly watching peformance metrics to detect anomalies, and testing the system by simulating users and user requests to match a production environment load to ensure that production needs can be met.

It is required that any kind of Performance Testing should be accompanied by adequate data regression to prevent false performance results that could result from cached transactions. From the Oracle Performance Analyzer for Oracle 10G to nMap to analyze network throughput metrics, there are a wide gamut of tools available to both measure and predict the throughput of a system to near-production numbers.

From an Database perspective, it is critical to monitor Transactions per day, transactions per hour and transactions per minute on a daily basis, along with the following metrics by day.

• Processor usage
• Database Locks
• Memory Usage
• Disk Size and Growth
• Database Size and Growth
• Database space usage by store
• Disk Activity, read-write queue length and saturation as pertinent to number of transactions

For windowed applications on Windows, the Windows Management Interface (WMI) provides several performance counters that are indicative of Application Server performance. Key counters include:

• Disk Throughput – Requests Queued, Reads, Writes, Bytes Per Sec and Errors
• Memory – Available Memory, Memory in Use – committed bytes and Page Faults
• Processor – Usage, peak and averages
• Other ASP.NET, SQL Server and application specific counters

On UNIX / LINUX / AIX, topaz and other monitoring utilities will provide the equivalent counters for performance. Simple Network Management Protocol applications can listen for specific SNMP events and threshold deviances.

It is required that a list of performance counters be documented prior to the test and results of the test be published with graphical representations of the performance results in order to be able to analyze and effectively assimilate the results. It is recommended that the Test criteria be determined by interpolating common results and business metrics to match the target criteria. For example, if we knew that each server processed X orders per day with an average of roughly Y orders per hour, 8:00 AM - 5:00 PM being the high-load period, we should be able to measure if the system that currently handles 20 orders is capable of handling 20 orders.

The performance test plan should clearly outline the parallelism of the test, the number of concurrent connections, the number of concurrent transactions and the reasoning behind the load. It is a good practice to increase the threshold after a successful test of the required criteria to determine the breaking point of the system. The disk being the slowest component in any database server is almost always the point of contention when available system memory has been exhausted. This is a direct result of paging. Therefore adequateness of system memory should be targeted and tested.
For Application Servers and Database Servers alike, the Processor is the next point of contention, after memory. SMP Implementations are recommended
Metrics like networking bandwidth, file system performance and application error logs are the last, but not least criteria to consider.

Turnover

Turnover is the process of successfully deploying a new application or application changes to a production environment and managing and completing the process of knowledge transfer of the new / changed environment with the Production Management Team.

Key aspects of this knowledge transfer include the technical documentation that will enable Production Processing to maintain the system from day to day.

The most important part of this knowledge is the information needed to determine normal system behavior and detect when there is an anomaly and determine the corrective action to be taken. This information transfer is achieved by sufficient documentation regarding how to monitor application performance checkpoints and configuration. Information about log files and the ability to detect error conditions with a complete documentation of all error codes and their descriptions along with the path to corrective action.

The ability to fix a problem by understanding the environment variables, configuration files and other factors that determine the application’s runtime characteristics. The Production support team should maintain an ever-growing troubleshooting guide that contains cause / effect documentation over time to be able to quickly establish corrective action from past experience.

Support, Troubleshooting and Monitoring Documentation

The Support, Troubleshooting and Monitoring Documentation is a collective effort by the Development Team, System Administrators, Network Administrators, Database Administrators and the Production Support team to document the system environment, the network environment, the business process environment and key metrics of system behavior in order to be able to manage, detect and correct system issues. This document is both a result of the SDLC and the day-to-day application / system infrastructure maintenance process – a living, breathing document that changes from day to day.

It is an ever-demanding task to monitor and keep a single document up-to-date with product support information. This is why, a web application that supports the ability to enter and review these day-to-day activities, thereby providing visibility into the application lifetime will play its role better here than a folder with numerous emails, documents and images.
This is why a knowledge management system becomes critical for incident management and recording. When an error is encountered in a system or infrastructure platform, it is extremely important that both the issue and its resolution be completely logged in the knowledge management system. The importance of logging to one of these systems is that if that issue were to ever occur again in the future, the person who encounters the error can search for the error code or symptom in one of these systems and obtain the knowledge to take corrective action without depending on another user’s intervention or availability.

Optimization & Maintenance

The task of utilizing monitoring, the troubleshooting guide, application logging and business process monitoring to constantly learn and optimize the existing implementation is key to any system and is an activity that occurs for the entire lifetime of the system. The key is to be able to upgrade and keep up this process, as the system transitions from one upgrade to another.

It is critical that the training and troubleshooting guides and knowledge management process is kept up-to-date with every change to the production platform. Also critical is to document all vendor contracts, licenses and product version numbers and update patches for every component of the enterprise to be able to detect any breach in software or infrastructure quality.

This step will go on throught the Software's Life Cycle.