HomeMy WebLinkAboutC2019-609 - 10/23/2019 - NA TASK ORDER NO. 46
This Task Order pertains to a Master Services Agreement for Professional Services by and between City of Corpus
Christi, Texas (City)and LNV, Inc., (Consultant)dated August 16,2016(Agreement)by Motion No.M2016-093.
Consultant shall perform services on the project described below as provided in this Task Order and in the
Agreement. This Task Order shall not be binding until it has been properly signed by both parties. Upon execution,
this Task Order shall supplement the Agreement as it pertains to the project described below.
PROJECT: E16264 Water and Wastewater Treatment On-Call SuoDort
1. PROJECT DESCRIPTION: This project provides the Utility Department with on-call services to assist with
any technical,operational,maintenance,diagnostic and design concerns for the City's O.N.Stevens Water
Treatment Plant and six wastewater treatment plants.
2. SCOPE OF SERVICES: Water System Engineering Support, more fully described in Exhibit A.
3. COMPENSATION: The total amount not to exceed for Task Order No. 46 only is $144,427.00.
4. PROJECT DELIVERABLE(S): Deliverables will be received in 12 months upon issuance of Notice to
Proceed.
This Task Order is approved, and Consultant may proceed. All other terms and conditions of the Agreement
remain in full force and effect.
CITY OF CORPUS CHRISTI LNV,INC.
/pgENy4renn0 Edmonds
� / /
N cn,J.11 EdrorNs,o, �
• Dela 201910 Dena, !/9
Jeff H. Edmonds, P.E. 1;‘
3 SJ9�• � Date pi—D-n ',Leyendecker, P.E. Date
Director of Engineering Services President
801 Navigation, Suite 300
Corpus Christi, TX 78408
(361)883-1984 Office
danl@Invinc.com
Task Order No.46
Page 1 of 2
SCANNED
Fund Name Accounting Unit Account No. Activity No. Amount
Water Operating Fund 4010-31010-062 530000 E16264-01-4010-EXP $144,427.00
Task Order No.46
Page 2 of 2
Professional Services Contract
City of Corpus Christi
Water/Wastewater Treatment On-Call Support
City Project No. E16264
Task Authorization No. 46
AWIA Risk and Resilience Assessment and Emergency
Response Plan
October 9, 2019
LNif
Exhibit A
Page 1 of 9
`Oap U 5 c,
rlitlgese
LNVIHazen
SCOPE OF WORK
Water and Wastewater Treatment On-Call Support
Project No.E16264
Task Authorization No. 46—AWIA Risk and Resilience Assessment and Emergency
Response Plan
BACKGROUND & PURPOSE
The American Water Infrastructure Act(AWIA)requires all community water systems serving populations
greater than 3,300 persons to assess the risks to,and resilience of,its system(referred to hereinafter as the
Risk and Resilience Assessment or RRA). According to the AWIA,the RRA must include the following
elements:
• The risk to the system from malevolent acts and natural hazards.
• The resilience of the infrastructure,including SCADA/cyber resilience.
• The monitoring practices of the system.
• The financial infrastructure of the system.
• The use,storage,or handling of various chemicals by the system.
• The operation and maintenance of the system.
In the past, The City of Corpus Christi (City) has prepared a Vulnerability Assessment (VA) (2003), a
Water System Preparedness and Incident Response Plan (2018) and an Operation Contingency Plan for
Hurricane Conditions and Disasters (2019). These documents address many of the requirements for
assembling a RRA and an emergency response plan (ERP) and will be used as part of the project. The
purpose of this task authorization is to assist the City in developing an RRA followed by development of
the ERP that meets all regulatory requirements of the American Water Infrastructure Act of 2018.The 2003
VA will be used as a basis for the development of the new RRA.The scope of the 2003 VA was limited to
malevolent acts and did not include natural hazards. In addition,cyber security and financial infrastructure
evaluation were also not included in the scope since these were not required under the Bioterrorism Act
2002.The proposed RRA will include all these aspects.
The AWIA also recommends, but does not require, an implementation plan for capital and operational
needs for risk and resilience management of the system. The proposed scope of services does not include
development of an implementation plan but will provide a ranked list of measures that can be used to reduce
risk and improve resilience of the utility.
The assessment will be completed,and the completion certified to the Administrator of the Environmental
Protection Agency(EPA)by March 31,2020,as required by the AWIA.
The Emergency Response Plan(ERP)is required to include response protocols for any type of emergency
or event identified as a threat during the RRA. The suggested format for the ERP is based on the Federal
Emergency Management Agency (FEMA) Comprehensive Preparedness Guide (CPG101) and on
American Water Works Association guidance M19,with sections including the basic plan,hazard-specific
procedures,and communication/coordination information.The ERP must contain the following elements:
1
Exhibit A
Page 2 of 9
(`3,US C9
r
LNVIHazen AirAbr.'
IHti2
• Strategies and resources to improve the resilience of the system,including the physical security and
cybersecurity of the system
• Plans and procedures that can be implemented,and identification of equipment that can be utilized,
in the event of a malevolent act or natural hazard that threatens the ability of the community water
system to deliver safe drinking water
• Actions, procedures, and equipment which can obviate or significantly lessen the impact of a
malevolent act or natural hazard on the public health and the safety and supply of drinking water
provided to communities and individuals, including the development of alternative source water
options, relocation of water intakes, and construction of flood protection barriers; and
• Strategies that can be used to aid in the detection of malevolent acts or natural hazards that threaten
the security or resilience of the system.
The Engineer will assist the City in making necessary updates to the existing Incident Response Plan to
address the findings of the RRA. This updated plan will meet the Emergency Response Plan requirement
of the AWIA. It is understood that the ERP and certification of completion to the EPA must be completed
no later than six (6) months after the submission of RRA or September 30, 2020, whichever is first, as
required by AWIA 2018.
SCOPE OF WORK
TASK 046A—RISK AND RESILIENCE ASSESSMENT (RRA)
A.1 Project Initiation and Gap Analysis
The Engineer will conduct a kickoff meeting with City staff to review the scope of services and project
schedule. Prior to the meeting, the Engineer will compare the existing VA and ERP against the AWIA
requirements and will develop a preliminary gap analysis. The Engineer will also develop a questionnaire
to help receive required information from City staff. This questionnaire will be provided to City staff at
least two(2)days prior to meeting.
The following items will be discussed during the meeting:
• Identification of City staffto participate in the project—this should include senior management staff
from Communications, Engineering, Finance, Human Resources, Operations, Maintenance,
Customer Service, IT and Safety
• Identification of any external team members — this may include at the City's discretion,
representatives from local law enforcement,fire department and emergency management
• Additional information related to water system vulnerability, security, financial systems, and
resilience such as:
o Asset inventory
o Existing security plans and procedures,
o Business continuity plans(Continuity of Operations Plans),
2
Exhibit A
Page 3 of 9
PPus CA,
k
Lt4VIHazen
'e32
o SCADA system information,
o Business network information (system architecture, software, facility connectivity
information,email and password protocols,accounting system policies,etc.)
o Human resources policies,
o Door lock/key policies,
o Security camera use and policies,
o Source water protection plans,
o Local natural hazard mitigation plan(s),
DELIVERABLES:
o Questionnaire to City staff at least two(2)days prior to meeting
o Summary of the meeting(with action items)and a revised gap analysis following the meeting.
A.2 Critical Assets, Threat Characterization and Consequence Levels Workshop
The Engineer will meet with the City to identify critical assets and the most viable threats to the system.
The first part of the workshop will be to review the previously provided water infrastructure asset inventory,
identify critical assets and then rank in terms of criticality.
Prior to the workshop, the Engineer will compile an initial "threat list" that includes those previously
identified in the existing VA or described in the ERP,those identified during the cybersecurity audit,those
identified in guidance documents as well as those identified in regional hazard mitigation plans.The second
part of the workshop will be to develop a list of threat-asset pairs along with vulnerabilities using Hazen's
non-proprietary J100 spreadsheet tool.
During the workshop,the Engineer will work with City staff to establish the consequence levels associated
with the loss of a critical asset.These are typically categorized by:
1. Environmental impact
2. Sickness/Injury and loss of life
3. Cost to remediate and economic loss
4. Public perception
5. Loss of service
DELIVERABLES:
o Workshop summary and the resulting list of critical assets ordered by risk (from highest to
lowest)within two weeks of workshop completion.
3
Exhibit A
Page 4 of 9
Oapus c,
LNVIHazen � ►
,est
A.3 Physical Security and Network/SCADA Security Assessment
The network and SCADA security assessment will use the AWWA cybersecurity tool to provide a list of
best practices and security measures based on available information for the Corpus Christi system. The
subtasks below describe the information required to complete the basic review using the AWWA tools.
Additionally, a physical security review of existing documentation will be provided with limited time on-
site for a physical security professional to verify previously indicated security system policies and protocols.
Should additional on-site remediation strategy recommendations be required, optional Task 3.2 can be
added to the scope.
A.3.1 Business IT Network Evaluation
Network Configuration: The Engineer will review and evaluate documents and practices related to
the City's business network configuration for physical and cyber security with respect to local and
remote access,firewalls,robustness of communication links(fiber,copper,etc.),and data reliability
and security including:
• Network architecture and topology
• Communication link bandwidth
• Vulnerabilities associated with connection media(fiber optic cable,digital data service,etc.)
• Cyber security
• Hardware
• Software
A.3.2 SCADA Network Evaluation
Network Configuration: The Engineer will review and evaluate the City's SCADA network
configuration for physical and cyber security with respect to local and remote access,firewalls,
robustness of communication links(fiber,copper,etc.), and data reliability and security. The
Engineer will also review and evaluate the following aspects the SCADA network via document
review and on-site interviews(but not including system scan and system penetration test):
• Network architecture and topology
• Communication link bandwidth
• Vulnerabilities associated with connection media(fiber optic cable,digital data service,etc.)
• Cyber security
• Hardware
• Software
• Telemetry System
4
Exhibit A
Page 5 of 9
ij.
4
I ,rX,
LNVIHazen
7H,
A.3.3 Network Policies Review
Network: The Engineer will review and evaluate the access policies and procedures for the City's
business and SCADA networks, including password policies, procedures for remote access to
systems, granting of system administrator privileges, access to operating systems and other
applications on SCADA workstations,etc.
Email: The Engineer will review and evaluate the security policies and procedures for the City's e-
mail system to determine vulnerability to malware,phishing,and other typical security risks.
A.4 RRA Deliverable
EPA does not require a report to be submitted for the RRA.The Engineer will submit a spreadsheet showing
data collected,results of the assessment,summary of each critical asset and the mitigation measures chosen
by the City under Tasks 1.1 through 1.4. Hazen's non-proprietary spreadsheet will be used to catalog the
following:
• All Threat-Asset(T-A)pairs identified at the workshop
• A consequence analysis for the top 20 identified T-A pairs
• A list of potential resilience improvement strategies with order of magnitude costs for the top
T-A pairs
The Engineer will provide the draft RRA spreadsheet to the City for review. A brief letter report
summarizing key results and prioritized improvements will also be provided for review. It is assumed that
the City's comments will be received within two weeks after receipt of the draft. The Engineer will make
corresponding revisions to the spreadsheet and provide the final document in electronic format. Phone
meetings may be scheduled as needed to discuss findings with key City staff.
The City will certify by letter to the Administrator of the Environmental Protection Agency(EPA)that the
RRA has been completed and forward a copy to the Engineer. Such certification will remain valid for five
years. It is recommended that the City perform a review after four years, such that updates can be
incorporated, and the City can recertify to EPA that the RRA has been updated.
TASK 046B—EMERGENCY RESPONSE PLAN UPDATE
B.1 Task 2 Kick-off Meeting and ERP Coordination
The Engineer will conduct a kickoff meeting with the City to review the scope of services and project
schedule,review the existing ERP and the gap analysis conducted during Task 1,and identify any additional
information to be reviewed. The following items will be discussed during the meeting:
• Identification of City staff to participate in the emergency plan update
• Identification of any external team members and/or plan reviewers
• AWIA ERP requirements
• Recommended updates based on RRA
5
Exhibit A
Page 6 of 9
opcus
v r
LNV I Hazen 1,0 ,;„
,HS2
DELIVERABLES:
o Summary of the meeting(with action items)
o Outline of proposed updates to the emergency plan for City review and comment prior to
drafting the updated plan.
B.2 Emergency Plan Update(Development,Review and Final Version)
Using the existing ERP as the base document, the gap analysis, City input and industry guidance, the
Engineer will draft the updated emergency plan.The document will be submitted to the City for review and
comment. City staff will provide additional changes/comments on the draft plan within ten days after the
workshop and the Engineer will deliver a final version of the emergency plan to the City's designated Task
Authorization Manager in electronic format.
The City will then certify by letter to the Administrator of the EPA, that the ERP has been completed,
copying the Engineer on the correspondence. Such certification will remain valid for five years. It is
recommended that the City perform a review yearly and significant revision after four years, recertifying
to EPA following each revision.
NOTES ON CITY RESPONSIBILITIES:
• All existing documents will be provided by the City in electronic files which can be utilized in the
RRA.
• The City will coordinate with non-utility agency staff(such as from police or fire departments)who
are identified as part the RRA/Emergency Response Team.
• The City will ensure that required personnel from all necessary departments are present at all
meetings to provide necessary input in the development of the RRA and ERP.
• City's existing ERP/incident response plan will serve as the base document for the emergency response
plan.
• The City will file certifications by letter to the Administrator of the EPA, copying the Engineer,
following completion of the RRA and ERP.
SCHEDULE
The services for the RRA are anticipated to be completed within six months of Notice to Proceed and receipt
of all information requested from the City.
The services for the ERP are anticipated to be completed within four months of RRA completion and receipt
of all information requested from the City.
FEE
The services will be provided on a Time and Materials (T&M) basis as the project is moved towards
completion. The anticipated amount for this task authorization (Task 046A and Task 046B) is
$144,427. The Engineer will not incur cost in excess of this not-to-exceed amount without prior
authorization from the City. A fee summary and detailed man-hour breakdown for this task authorization
is provided in Attachment A.
6
Exhibit A
Page 7 of 9
`OQpus c,Rs
LNVIHazen '0 _Air:4
J �
1854
If the City decides to approve this task authorization for the amount of$144,427,the total amount remaining
under the project E16264-Water and Wastewater On-Call Support will be$258,653.90.
TASK AUTHORIZATION MANAGER
The Task Authorization Manager will be the primary contact person during the performance of this work.
All correspondence or inquiries should be addressed to the Task Authorization Manager. This work task
will be coordinated by:
Larijai Francis, P.E.,M.ASCE
Project Manager
City of Corpus Christi Engineering Department
Office: 361-826-1872
Email: LarijaiF@cctexas.com
7
Exhibit A
Page 8 of 9
ATTACHMENT A: MAN-HOUR AND FEE ANALYSIS
Table 1: Summary of Fees for TA46 -AWIA Risk and Resilience Assessment and Emergency Response Plan
Task No. Task Description LNV,Inc. Hazen and Sawyer Zivaro Expense Admin Fee Total Fees
0000 Project Management and QA/QC $ 6,175.00 $ 3,166.56 $ 1,848.00
$ 11,190.00
046A Risk and Resilience Assessment $ 14,015.00 $ 50,700.38 $ 24,256.00 $ 8,000.00 $ 96,980.00
0466 Emergency Response Plan Update $ 1,730.00 $ 20,725.95 $ 3,400.00 $ 25,860.00
TOTAL FOR TASK AUTHORIZATION= $ 21,920.00 $ 74,593.00 $ 29,504.00 $ 8,000.00 $ 10,410.00 $ 144,427.00
Table 2: Detailed Man-Hour and Fee Anal sis for TA46-AWIA Risk and Resilience Assessment and Ernes-.enc Res s onse Plan
SPC SPC SA PE SA AE : AE AE SPC PM Total Fee
Task Description $ 280 $ 280 $ 197 $ 145 $220 $126 $104 $ 95 $230 $135 $ 250 $175:$154 Hours Labor Expense Admin Fee Total
Project Management and QA/QC Plan 1 12 8 24 12 56 $ 8,663 $ 8,663
Kickoff Meeting/Call 3 31 3 3 12 $ 2,526 $ 2,526
A.1 !Document and Asset Discovery(Gap Analysis) 2 41 8 16 24 4 58 $ 6,851 $ 6,851
A.2 i RRA Threat,Asset,and Vulnerability Workshop
A.2.1 Workshop Preparation 4 21 2 2 8 4 4 _ 26 $ 4,348 $ 4,348
A.2.21 On-Site Workshop(2 Days) 16 161 16 16 16 16 16 112 $ 18,977 $ 5,000 $ 23,977
A.3 :Security Interviews and Walk-Throughs 1 40 40 4 84 $ 17,616 $ 3,000' $ 20,616
}
A.4 :Risk and Resilience Improvements Strategies
A.4.11 Consequence Analysis,Top 20T-A Pairs 4 8 8 24 24 4 4 4 80 $ 11,459 $ 11,459
A.4.21 Phone Call Review of Consequence and Ranking 2 2 2 2 2 10 $ 2,030 $ 2,030
A.4.3i Identify and Document Strategies,15 T-A Pairs4 4 30 16 30 8 8 100 $ 13,077 $ 13,077
A.4.4 Summary Memo and QA/QC 4 4 4i 8 16 8 44 $ 8,260 $ 8,260
A.4.51 Contingency for Site Visits 1 8 8 8 24 $ 2,675 $ 2,675
A.4.61 Review Workshop 3 3 3 3 3 3 18 $ 3,678 $ 3,678
B.1 Conduct Gap Assessment with Existing Documents 4 4 2 2 24 12 48 $ 7,199 $ 7,199
B.2 1DevelopERPwithupto10scenarios 4 121 2 8 40 40 2 4 8 8 128 $ 17,159 $ 17,159
B.3 ?Review Workshop(phone call) 2 2 2 2 8 $ 1,498 $ 1,498
8
Exhibit A
Page 9 of 9