NANC 384
NPAC Change Order Effectiveness Metrics
Origination Date :06/05/2003
Originator:ESI
Description:
Abstract:
This contribution proposes specific metrics for evaluating the operating characteristics of the NPAC RSMS, based on characteristics that have a direct impact on individual carriers cost of operations. It is expected that proposed change orders to NPAC RSMS could be evaluated based on projected improvements to the measurement of one or more of these metrics. Projected improvements in these measurements would be used by individual carriers to justify the cost associated with specific change orders.
Contribution:
As local number portability matures in its processes and supporting systems, and as telecommunications carriers continue to implement significant financial controls on their expenses, carriers are increasingly looking for justification for particular investments. The table below represents a list of 6 characteristic metrics that can be measured at the NPAC RSMS and have a direct impact on an individual carriers’ cost of operation. It is proposed that this set of metrics be used for regular reporting of NPAC RSMS performance capabilities, and that proposed change orders be evaluated by the potential improvement that the change may have on one or more of these metrics.
The second table represents an example of the measurements that should be captured to create a baseline measurement set and delta measurements for individual changes. These represent only estimates, and are included to illustrate the estimate or measurement data that could be provided going forward, for use in allowing businesses to make informed investment decisssions with respect to LNP capabilities.
Metrics:
Metric
|
Units
|
Measurement Technique
|
Throughput Capacity
Reflects the steady-state porting capacity of the NPAC without queuing (assuming infinitely fast LSMS and SOA systems)
|
TNs/Second
|
Test Technique 1, item 3
|
Individual Create Processing Time
Measurement in seconds of the time from receipt to SOA notification of create activity
|
Seconds
|
Test Technique 1, item 4
|
Individual Activate Processing Time
Measurement in seconds of the time from receipt to SOA notification of activate activity (assuming no late LSMS notifications)
|
Seconds
|
Test Technique 1, item 4
|
Individual Modify Processing Time
Measurement in seconds of the time from receipt to SOA notification of modify activity
|
Seconds
|
Test Technique 1, item 4
|
Query Response Rate
Measurement in Queries/Second that represent the steady-state capacity of the NPAC.
|
Query Requests/ Second
|
Test Technique 1, item 3
|
Individual Query Response Time
Measurement in seconds of the time it takes the NPAC to respond to a representative query
|
Seconds
|
Test Technique 1, item 4
|
Test Technique 1:
- Establish a representative traffic load that includes a production-like proportion of Create, Concur, Activate, Modify, and Query operations.
- Subject the NPAC to the representative proportions of traffic at increasingly high TN/seconds rates, and measure the output LSMS notification rate (the combined rate of SV Activate, SV Modify, and SV Disconnect requests, also in TNs/second).
- At sufficiently low rates, the NPAC will reach a steady-state where the input rate and the output rate are approximately equal. As the input rate increases, there will come a point where the input rate exceeds the output rate, indicating that the NPAC is queuing activities internally. The maximum input rate without queuing represents an effective through-put of the system, measured in TNs/second.
- When the NPAC loaded at its effective through-put rate, individual transactions each have a start and end time, the difference of which yields a duration calculation for the individual transaction. An average transaction processing time can be calculated for each transaction type from these individual records. The measurement of the start and end time are most accurately measured by a tool placed external to the NPAC. However, it may be acceptable to do initial measurements from transaction log records internal to the NPAC RSMS application software. This is measured in seconds.
Change Order Effectiveness Estimates
Metric
|
Units
|
Assumed Current Value
|
NPAC Prioritization of Notifications
|
NANC 179 - Ranged Notifications
|
NANC 347/350 - 15/60 minute abort timers
|
NANC 348 - BDD for notifications
|
NANC 351 - Send what I missed
|
NANC 352 - SPID recovery
|
NANC 368 - NPAC OBFC
|
Throughput Capacity
|
TNs/Second
|
25
|
+3
|
+20
|
+5
|
||||
Individual Create Processing Time
|
Seconds
|
1
|
No change
|
No change
|
No change
|
||||
Individual Activate Processing Time
|
Seconds
|
2
|
No change
|
No change
|
No change
|
||||
Individual Modify Processing Time
|
Seconds
|
2
|
No change
|
No change
|
No change
|
||||
Query Response Rate
|
Query Requests/ Second
|
12
|
+1
|
+14
|
+2
|
||||
Individual Query Response Time
|
Seconds
|
2
|
No change
|
No change
|
No change
|
Aug ’03 LNPAWG, discuss this change order in the Sep’03 APT meeting. Requirements will be worked in that forum.
Jan 06 – ESI discussed internally. Performance on both sides has been resolved. Agreed to move to Cancel-Pending.
3/06 – Deleted after March LNPAWG meeting
Final Resolution:
Func Backwards Compatible: YES
Jan ’06 – ESI discussed internally, and since perfor on both sides long ago significantly resolved, agree to cancel-pending.
3/06 – Deleted after March LNPAWG meeting.
Related Release:
N/A
Status: Closed