NANC 389

Performance Test-Bed

Origination Date :10/16/2003

Originator:AT&T Wireless

Description:

Business Need:

Service Providers have expressed a desire to perform a performance volume test to mimic production behavior prior to “go-live”, and to “stress” and certify system readiness, but without having to use simulators to perform the NPAC role.  Simulators have been used because the test platform provided under SOW 34 does not support testing at performance volume load levels.  It is possible for a Service Provider to impact the overall stability of the SOW 34 test platform and negatively impact other NPAC users.  Even with the coordination and scheduling of performance tests in the off-hours, a single Service Provider still can negatively impact the NPAC test-bed, causing downtime to clear the inbound and outbound queues.

This change order defines system requirements for a separate NPAC test-bed suitable to meet the industry performance volume test needs.  Service Providers could use this test-bed at any time without support.  Testing support, including setup, would be provided as agreed.

Final Resolution:

Func Backwards Compatible:  YES

This will be explored during the Nov ’03 LNPAWG meeting.

Nov ’03 LNPAWG, discussion:

Still a desire to have a Test Bed that can handle volume test loads even though past go-live date for WNP.  As discussed during Oct ’03 meeting, configuration would be no failover site, and up to five simulators for SOA and LSMS sides.  Desire is to have an environment just like production, so it would mirror that configuration.

Some providers still bothered by the lack of definition on what will be tested, how often, number of SPs at same time, volumes at max, number of simulators, response time needs, assumptions, etc.  Just saying “production-like ” is not well defined.  We need to quantify the configuration.  It was also mentioned that we would want a separate Test Bed rather than just beefing up the SOW 34 Test Bed (which is used for unassisted functional testing).  The desire is to do end-to-end testing with volume, and not impact the functional Test Bed.  Additional input was for volume testing (in the 10s of thousands of TNs) to test end-to-end, so bottlenecks can be identified, and possibly implement flow control in one or more places along the end-to-end path.

It was finally agreed that since this started as a wireless issue, then the WNPO would work this as a group, then provide feedback/updates/definitions back to Working Group.  So, this change order will remain on the open list for now.

Apr ’04 APT, discussion:

The group discussed this.  A concern was raised about the name of this change order (“Production Equivalent Test Bed ”), yet there are specific performance volumes mentioned.  If this truly should be “Production Equivalent ” then it should mirror the production configuration, and not contain other performance requirements.  Since the desire was to meet certain performance levels, it was agreed to change “Production Equivalent ” to “Performance ”.  It was mentioned that the need for this test environment should be verified with the WNPO, in the context of something that is more cost effective, so the APT requested that the WNPO review this again, reconsider their specifications, and if still desired, resubmit to the APT for future discussions.

Dec 05 – moved to Cancel-Pending per LNPAWG discussion.

3/06 – Deleted after March LNPAWG meeting.

Related Release:

N/A

Status: Closed