NATIONAL TRANSPORTATION SAFETY BOARD OFFICE OF HIGHWAY SAFETY WASHINGTON, D.C. OPERATONS FACTORS GROUP CHAIRMAN’S FACTUAL REPORT A. CRASH INFORMATION Northbound Mill Avenue, approximately 380 40 feet south of the intersection Location: with Curry Road, in Tempe, Maricopa County, Arizona Vehicle 1: 2017 Volvo XC90, being operated by Uber Advanced Technologies Group in autonomous mode Pedestrian: 49-year old female, walking bicycle across Mill Avenue Date: March 18, 2018 Time: 9:58 p.m. Pacific Standard Time (PST) NTSB #: HWY18MH010 B. OPERATIONS FACTORS GROUP Michael Fox, Operations Factors Investigator, Group Chairman NTSB Office of Highway Safety 490 L’Enfant Plaza East, S.W., Washington, DC 20594 Noah Zych, Uber Inc., Group Member ATG Head of System Safety 50 33rd Street, Pittsburgh, Pennsylvania 15201 C. CRASH SUMMARY For a summary of the crash, refer to the Crash Summary Report in the docket for this investigation. D. DETAILS OF THE OPERATIONS FACTORS INVESTIGATION This investigative report addresses operations of Uber ATG a wholly owned subsidiary of Uber Technologies Inc. the owner and operator of Vehicle 1. This report documents the company history, operations, and safety culture of Uber. This report will also review the hiring practices, training procedures, and duties and responsibilities for the operator of Vehicle 1 who was Tempe, AZ – Operations Factors Factual Report Page 1 of 28 employed by Uber at the time of the crash. This report will also discuss Federal and state oversight of autonomous vehicle testing. 1. Uber History and Operations 1.1. Uber History Uber Inc. (Uber) was founded in March 2009 by creating a smartphone app called “Uber Cab” in San Francisco, California. In October 2009 the company removed the “cab” from the name and then simply called the company “Uber.” The purpose of the Uber app is to enable customers order a ride from their location to a specified location. The company business model provides alternative transportation to traditional taxi and limousine service in the concept of ride sharing. The company went international in 2011. The company business model is structured around the ride sharing app and has also branched out to other operations under the Uber parent company. Currently Uber has six divisions that include: 1. Ride sharing, 2. Uber Eats, 3. Uber Freight, 4. Jump, 5. Elevate, and 6. Advanced Technologies Group. The company mission statement states: “Bring transportation to everyone everywhere.” 1 The company surpassed 5 billion trips in May 2017. 1.2. Divisions At the time of the crash the company employed more than 16,000 worldwide and over 9000 employees in the US. Listed below is a summary of the various divisions or subsidiaries that comprise Uber. 1.2.1. Ride Sharing The Uber app automatically provides door to door transportation service. The app locates the closet Uber driver to the customer and dispatches the vehicle to the customer. The app provides the driver’s name, license plate number, photo, and driver’s rating. When customers enter their destination the Uber driver’s location is plotted by GPS. This information is visible to the customer and provides the driver’s location up to the pickup point and throughout the trip until the driver reaches the destination. Fares are automatically charged to the customers payment on file. The customer has the ability to rate the driver immediately upon completion of the trip. Trips can be “shared” with friends or family via an app in order to monitor if desired. Uber drivers are screened for driving positions and must meet specific criteria to qualify. Minimum requirements for Uber drivers include the following:2 1 2 • Must be at least 21 years old • Have a least one year of licensed driving experience in the US • Have a valid US driver’s license Retrieved from: www.Uber.com . See website for additional information concerning company history. Ibid Tempe, AZ – Operations Factors Factual Report Page 2 of 28 • Use an eligible 4 door vehicle • Pass criminal background check on and review of driving record After a driver becomes active, they are required to have proof of liability insurance and their vehicle must pass a safety inspection. Uber drivers are considered independent contractors for the company. At the time of writing this report, Uber had over 3 million drivers, operating in 65 countries in over 600 cities worldwide, and completed more than 15 million trips daily. The Ride Sharing division is the largest division in Uber and represents the main source of revenue for company. 1.2.2. Uber Eats Uber Eats started in August 2014 and is an online / smartphone app food ordering and delivery service. Uber Eats is headquartered in San Francisco, California. It partners with restaurants in the US and globally. Food can be delivered by car, bike, scooter or on foot. Uber Eats operates under the same business model as the Ride Sharing division. 1.2.3. Uber Freight LLC Uber Freight was founded in May 2017 is a subsidiary company that operates as a freight broker business. It is based in Chicago, Illinois and headquartered in San Francisco, California. Similar to the ride sharing operation, the Uber Freight app matches carriers with shippers.3 Carriers who are approved by the company can use the app and book a load. Carriers are paid within 7 days of proof of delivery. According to the Federal Motor Carrier Safety Administration (FMCSA) Motor Carrier Management Information System (MCMIS) database, Uber Freight LLC was issued USDOT number 2926893 and has MC # 987790 and shows “authorized for broker authority.” The MCS-150 shows date of September 8, 2016 and currently hauls general freight commodities. New motor carriers are vetted by Uber Freight by requiring the minimum qualifications: • Must have and maintain active DOT / MC number • No Conditional or Unsatisfactory Safety Rating4 • Proof of insurance ($1 million) Once a motor carrier is selected as an approved carrier, Uber Freight has a compliance team that monitors through “Safer Watch” and other software systems the carrier’s SMS scores, 3 Retrieved from: https://freight.uber.com/faq Safety Rating explain here: §385.9 Determination of a safety rating. (a) Following a compliance review of a motor carrier operation, the FMCSA, using the factors prescribed in §385.7 as computed under the Safety Fitness Rating Methodology set forth in appendix B of this part, shall determine whether the present operations of the motor carrier are consistent with the safety fitness standard set forth in §385.5, and assign a safety rating accordingly. 4 Tempe, AZ – Operations Factors Factual Report Page 3 of 28 roadside inspections or out-of-service infractions on a daily basis.5ʼ6 Motor carriers who fall out of good standing are removed from the active carrier list and prohibited from obtaining any additional loads until the infraction (i.e. insurance or operating authority) has been reinstated or corrected. 1.2.4. Jump Bikes Jump Bikes formally called “Social Bicycles,” is a bike sharing business that uses a “dockless” electric bike that operates in Washington, DC and San Francisco.7 The bikes are rented on an hourly basis and are located using a smartphone app. The Jump Bikes division was founded in 2010 and is headquartered in New York City, New York. Uber acquired the Social Bicycles in April 2018 and renamed the company Jump Bikes. Jump Bikes will be expanding into Europe in 2019. 1.2.5. Elevate Uber Elevate is an experimental division that plans on opening in the Dallas Fort-Worth and Los Angles markets in 2020. The program will consist of vertical take off and landing aircraft which Uber calls “flying cars.” Currently Elevate is in the initial stages of testing aircraft and is currently not a profit center. 1.2.6. Advanced Technologies Group The Advanced Technologies Group (ATG) was founded in 2015 and is headquartered in Pittsburgh, Pennsylvania. At the time of the crash ATG employed over 1000 employees. The operator of Vehicle 1 was employed by ATG. The ATG has core functions of supporting and developing the self-driving or autonomous vehicle platform. These requirements include: software, mapping, hardware, vehicle programs, operations, product and safety. At the time of the crash, ATG had four of five locations conducting testing of the autonomous vehicles. Examples of ATG autonomous vehicles owned and tested by Uber at the time of the crash are highlighted in Figure 1. [This space left intentionally blank] 5 SaferWatch is a third- party vendor that provides continuous updated motor carrier information concerning operating authority, inspections, accident, insurance and other FMCSA compliance data. Retrieved from: http://www.saferwatch.com/main/about/ 6 Driver and Vehicle Out-of- Service criteria is established by the Commercial Vehicle Safety Alliance (CVSA) and enforced by each state’s corresponding state motor carrier safety assistance program (MCSAP) law enforcement agency per 49 CFR 350. See https://cvsa.org/inspections/inspections/ for additional details. 7 Dockless means that the unit does not require it to return to a bike rack to recharge. The units operate off of a battery pack that is changed out every 3 days. Tempe, AZ – Operations Factors Factual Report Page 4 of 28 Figure 1. Uber Test Vehicles Equipped with Autonomous Hardware and Software Technology (Source: Uber Inc.) 1.3. Safety Culture The Uber corporate safety culture is framed by company policies and procedures. At the time of the crash, Uber Technologies had a safety team that focused on its Rides business. Separately, Uber ATG had a technical System Safety team, but did not have a standalone operational safety division or safety manager. Additionally, ATG did not have a formal safety plan, a standardized operations procedure (SOP) or guiding document for safety. Safety was however, guided by Uber’s core values or “Norms.”8 The Uber norms consist of the following elements: • We do the right thing. Period. • We build globally, we live locally. We harness the power and scale of our global operations to deeply connect with the cities, communities, drivers, and riders that we serve every day. 8 Uber Technologies, Inc. Stock Exchange Registration Notice retrieved from: https://www.sec.gov/Archives/edgar/data/1543151/000119312519103850/d647752ds1.htm Tempe, AZ – Operations Factors Factual Report Page 5 of 28 • We are customer obsessed. We work tirelessly to earn our customers’ trust and business by solving their problems, maximizing their earnings, or lowering their costs. We surprise and delight them. We make short-term sacrifices for a lifetime of loyalty. • Celebrate differences. We stand apart from the average. We ensure people of diverse backgrounds feel welcome. We encourage different opinions and approaches to be heard, and then we come together and build. • We act like owners. We seek out problems, and we solve them. We help each other and those who matter to us. We have a bias for action and accountability. We finish what we start, and we build Uber to last. And when we make mistakes, we’ll own up to them. • We preserve. We believe in the power of grit. We don’t seek the easy path. We look for the toughest challenges, and we push. Our collective resilience is our secret weapon. • We value ideas over hierarchy. We believe that the best ideas come from anywhere, both inside and outside our company. Our job is to seek out those ideas, to shape and improve them through candid debate, and to take them from concept to action. • We make bold bets. Sometimes we fail, but failure makes us smarter. We get back up, we make the next bet, and we go! 1.3.1. Policies and Procedures Uber utilizes a variety of company policies that mandate hiring practices, training requirements, human resources guidelines, and safety procedures. Failure to comply with company policies may result in disciplinary action up to and including termination. Examples of such policies and procedures are described in the sections below. 1.3.1.1. Minimum Hiring Standards At the time of the crash, ATG advertised for job openings on the company website www.uber.com, word of mouth, and through a 3rd party recruiting agency.9 Potential test vehicle drivers or vehicle operators (VO’s) were required to complete an application, undergo an in-person interview and pass a driving knowledge test. The minimum hiring standards / qualifications for drivers or vehicle operators (VO’s) for ATG include the following: • 9 21 or 22 years of age with three or more years of driving experience See Operations Attachment- Uber Job Announcement Tempe, AZ – Operations Factors Factual Report Page 6 of 28 • 23 years of age with one or more years of driving experience • No more than 3 minor traffic infractions (i.e. traffic light, speeding or moving violations) within the past 3 years • No major traffic violations (i.e. driving while suspended, uninsured violations) within the past 3 years • No serious traffic violations within the past 7 years (i.e. DUI, evading police, speeding +100 mph, or reckless driving) In addition to the minimum standards each applicant was required to successfully pass a criminal background check conducted by a third-party vendor and a written driving knowledge skills test. 1.3.1.2. Cell Phone Policy / No Texting Policy The ATG cell phone policy stated: “no driver shall use a hand-held mobile device while driving a test vehicle.” The policy further stated that an ATG driver may utilize vehicle mounted, hands free smart device for purposes of official duties such as navigation or scouting activities. Additionally, the policy stated: “an ATG Driver will not text or dial a phone while the driver is in motion or stopped at an intersection or in traffic.” Exception to the policy would be to communicate with law enforcement officials or other emergency services. Drivers were instructed that failure to comply with this policy could result in disciplinary action up to and including termination. The cell phone policy was reviewed during the initial phase of training for the accident driver the week of June 12, 2017. The cell phone policy was not however a standalone policy and ATG did not have a written record indicating that the accident driver had acknowledged (i.e. signed) the policy or received a copy of the policy. The accident driver did sign a document that stated: “I have read, understand, and will adhere to the rules, requirements, and guidance contained in the UATC Driver Program Policy and Procedures that govern the use and operation of UATC vehicles.”10 During the NTSB site visit at the Tempe, Arizona facility it was noted that located adjacent to the driver’s lounge were safety posters that stated: “It can wait. Cell phone usage is not allowed in the left seat” 11 1.3.1.3. Seat Belt Policy Per the ATG seat belt policy, all test vehicles that have been equipped with a seatbelt assembly must not be operated unless the driver has properly restrained himself/herself with the seatbelt assembly. The policy also stipulated that all passengers are also required to belted. 10 11 See Operations Attachment- UATC Driver Policy and Procedures Agreement Form. See Operations Photos 1-3 in the docket. Tempe, AZ – Operations Factors Factual Report Page 7 of 28 1.3.1.4. Drug Testing Policy Uber ATG has a zero-tolerance policy for the use of alcohol or illegal drugs while on duty. Although the vehicles used by ATG did not require a commercial driver’s license (CDL) all employees were subject to pre-employment, random, reasonable suspicion drug or alcohol testing and post-accident testing which modeled the USDOT drug and alcohol testing requirements under 49 CFR 382.12 Any operator who refused to be tested would be subject to disciplinary action up to and including termination. Illegal drugs were classified as of the following substances: • Cocaine • Opiates • Phencyclidine (PCP) • Alcohol • Marijuana Implementation of the drug testing program was conducted with a chain of custody procedure arranged through a third-party vendor with a certified laboratory. Employees that tested positive with a valid prescription from a licensed medical professional would qualify as an exception to this policy after an evaluation through the laboratory’s contracted Medical Review Officer (MRO). If applicable, Uber ATG required staff members to discuss the responsibilities of their job with the MRO to ensure that the prescription would not impair their ability to safely operate the ATG vehicle. Implementation of the drug testing policy was not consistent throughout the company and differed from location to location. Prior to the crash, the only employees who were pre-employment drug tested started on or before May 2017. The accident driver was not subject to any pre-employment, random, reasonable suspicion, or post-accident drug tests while employed at ATG. Additionally, the accident driver did not nor was required to submit to a post-accident drug or alcohol test by the Tempe, Arizona Police Department. See the Human Performance Group Chairman’s Factual Report for additional details. 1.3.1.5. Maximum Driving Time At the time of the crash, ATG established maximum driving limits which stated that the company shall not permit or require any driver to drive a test vehicle nor shall any driver drive a test vehicle under the following conditions: • More than 10 hours following 8 consecutive hours off duty; or 12 49 CFR Part 382 - Controlled substances and alcohol testing requirements of the Federal Motor Carrier Safety Regulations as required for operating a CDL required vehicle with a combination gross vehicle weight rating over 26, 001 lbs., or single vehicle with a GVWR of 26,000 lbs. or more, or a vehicle that is designed to transport 16 or more passengers including the driver. (Retrieved from www.fmcsa.dot.gov ) Tempe, AZ – Operations Factors Factual Report Page 8 of 28 • For any period after having been on duty 15 hours following 8 hours off duty. ATG drivers were also mandated to take required breaks of 20-40 minutes break after a maximum of 4.5 hours of sustained driving time. ATG drivers also had recommended rest breaks up to 20 minutes every 2.5 hours of sustained driving. 1.3.1.6. Fatigue Policy The company did not have a fatigue policy at the time of the crash. ATG provided NTSB investigators with an email memo dated Feb 7, 2018 entitled: “A R eminder on Fatigue” for flight managers. This email served as a reminder to managers that there were no disciplinary consequences for VO’s who felt fatigued, should not operate a test vehicle, and if necessary, should go back home to get adequate rest. 1.3.1.7. Rewards Program At the time of the crash, ATG had a rewards/bonus program. The bonus was graded on three criteria: professionalism, collaboration, and initiative/ownership. The accident driver had received this award for her overall performance for the 4th quarter of 2017. 1.3.1.8. Disciplinary Program The company had a tiered disciplinary program for infractions. 13 Levels 1 was classified as “negligence.” This infraction would be for behaviors that would be classified as avoidable. Example of avoidable infractions would include curb strike. Such infractions would result in remedial training. Level 2 was classified as “serious.” Such behaviors could result in risk to employees or other people due to a safety infraction such as speeding or running a red light. Such infraction might result in remedial training however further infractions within a 90-day period could result in discipline up to and including termination. Level 3 infraction was classified as “critical” which could put people in grave danger and risk. Such behaviors might include using drugs or alcohol while operating a company vehicle, any physical use of cell phone or electronic device (such as a smart watch or tablet) while operating a test vehicle, or willful or reckless driving behavior that resulted in an accident or near miss would result in termination. According to the ATG files, the accident driver did not receive any disciplinary actions while employed at ATG. Additionally, the files did not indicate the accident driver signed any document acknowledging have been trained on the ATG tiered discipline policy. The accident driver did sign a document titled “UATC Driver Policy and Procedures Agreement Form” as noted in section 1.3.1.2.14 1.3.1.9. Additional Policies The accident driver was trained on additional internal policies that included: attendance and time off policy, work place injuries policy, dress and appearance policy, vehicle security, public relations, traffic violations, and no competition policy. The training file indicated the accident driver had signed acknowledgement for the following policies: Non-disclosure 13 14 See Operations Attachment - Uber Tiered Infraction Policy. See Operations Attachment - UATC Driver Policy and Procedures Agreement Form. Tempe, AZ – Operations Factors Factual Report Page 9 of 28 Agreement, MVR Release Consent Form, Employee Agreement, and Authorization to take video or Photographs. 1.4. Training Program Applicants selected for a VO position were required to attend a three-week training program. Depending on scheduling, the first week of training typically took place at the corporate office in Pittsburgh, Pennsylvania. During this block of training, trainees received orientation to the company, were instructed on company policies/procedures and received a briefing on human resource items such as payroll and company benefits information. The first week consisted of instructor lead instruction and evaluation of operating a test vehicle. Trainees were tested at the completion of each block of instruction. After successfully completing the first week of training, trainees are sent back to their respective home city to receive an additional two weeks of training. During the second phase of training, trainees become integrated with an experienced VO. The experienced VO would coach the trainee during the final phases of VO responsibilities that included real world experience with transporting Uber passengers. Trainees were required to complete all phases of training and successfully pass all tests and evaluations and pass the final exam to complete the VO program. 1.4.1. Vehicle Operator Training The goal of the VO training program was to train new VO’s on the duties and responsibilities of the ATG autonomous test vehicles. Training and testing transpired on the ATG test track and on public roads. VO’s were trained on multiple traffic scenarios to include: noncompliant drivers, bicyclists, jaywalkers and animals. Additionally, VO’s were instructed to recognize when there were aggressive and erratic drivers and to “stay vigilant to recognize these behaviors before they happen so we can be prepared.” Furthermore, VO’s were trained to “hover” over the steering wheel/brake pedal and be ready to take control of the test vehicle at a moment’s notice for an intervention to include evasive steering or make an immediate brake application. Hovering over the steering wheel and brake pedal was a company requirement not only for training but also in autonomous mode. 2. ATG Operations 2.1. Operations Classification The accident vehicle involved in this crash and had a gross vehicle weight rating (GVWR) of 6635 lbs.15ʼ16 Because the vehicle did not exceed a GVWR of 10,001 lbs. it did not meet the definition of a commercial motor vehicle (CMV) and the driver did not require a CDL and was not subject to the Federal Motor Carrier Safety Regulations.17 Under Arizona state law, this type of 15 For-hire motor carrier – 49 CFR 390.5 -means a person engaged in the transportation of goods or passengers for compensation 16 Gross Vehicle Weight Rating (GVWR) -49 CFR 390.5 means the value specified by the manufacturer as the loaded weight of a single motor vehicle. 17 Commercial Motor Vehicle- §390.5 means any self-propelled or towed motor vehicle used on a highway in interstate commerce to transport passengers or property when the vehicle — (1) Has a gross vehicle weight rating or gross combination weight rating, or gross vehicle weight or gross combination weight, of 4,536 kg (10,001 pounds) or more, whichever is greater; or Tempe, AZ – Operations Factors Factual Report Page 10 of 28 operation would was not classified as a taxi, livery or limousine operation but was classified as a “transportation network company.”18 Arizona state law defines a "transportation network company" as an entity that has been issued a permit pursuant to this article, that operates in this state, that uses a digital network or software application to connect passengers to transportation network services provided by transportation network company drivers and that may but is not deemed to own, operate or control a personal motor vehicle of a transportation network company driver.19 Transportation network company does not include the following: (a) This state or a county, city, town or political subdivision of this state and any related entity, a nonprofit agency or any other public body that coordinates, operates, promotes or sponsors public transportation, carpool or vanpool services. (b) A program that is in place to meet federal air quality standards pursuant to section 49404. (c) Any individual, company or activity that meets the requirements of a rental car agent or rental company as defined in section 20-331 if all of the following apply: (i) Transportation is provided to another person or is arranged by the rental company but provided by another person. (ii) The route is predetermined. (iii) Any money exchanged between the provider of the transportation and the recipient does not exceed the cost of providing the transportation. 2.2. ATG Operations At the time of the crash, the ATG operations focused on the self-driving platform for both passenger vehicles as well as Class 8 trucks (truck-tractors).20 According to company officials, ATG started with 40 employees and at the time of the crash had over 1000. The ATG is responsible for creating and building the technologies that enabled the test vehicles to operate in autonomous mode. At the time of the crash, ATG had five locations: 1) Pittsburgh, Pennsylvania which focused on core development, 2) Toronto, Canada which conducted research and development, 3) Detroit, Michigan which worked on automotive improvements; 4) Tempe, Arizona which conducted AV testing; and 5) San Francisco, California was a core development center. The ATG produced the (2) Is designed or used to transport more than 8 passengers (including the driver) for compensation; or (3) Is designed or used to transport more than 15 passengers, including the driver, and is not used to transport passengers for compensation; or (4) Is used in transporting material found by the Secretary of Transportation to be hazardous under 49 U.S.C. 5103 and transported in a quantity requiring placarding under regulations prescribed by the Secretary under 49 CFR, subtitle B, chapter I, subchapter C. 18 Taxi, Livery Vehicle, and Limousine – defined under Arizona State Regulations §28-101, pertains to combinations of seating capacity, routes, fixed rates and other variables. See https://www.azdot.gov/motorvehicles/professional-services/vehicle-for-hire-licensing/taxi-livery-and-limousines/definitions for additional information. 19 Ibid. 20 Class 8 Heavy Trucks have a gross vehicle weight rating (GVWR) of 33,001. Retrieved from www.FHWA.dot.gov Tempe, AZ – Operations Factors Factual Report Page 11 of 28 software, hardware, and managed safety operations. The VO team was responsible for adapting to software upgrades and to ensure that the test vehicles performed safely and effectively to make the product ready for public distribution. 2.3. Tempe Operations The accident driver was assigned to the Tempe, Arizona testing facility. The Tempe, Arizona operations consisted of vehicle technology, software, and engineering. The purpose of the Tempe operation was to operate the test vehicles in autonomous mode and make continuous advancements to the software. At the time of the crash, the operation consisted of a site manager and 16 flight managers or supervisors. The staff consisted of 254 VO’s and garaged 40 test vehicles. 2.3.1. Tempe Flight Team Shifts The VO staff operated three shifts schedule. The VO’s were divided into 10 team shifts. The various teams and shifts are outlined in Table 1. The accident driver was assigned to the Juliet shift which she requested. Table 1. Team and Shift Schedule Team Name Weekly Shift Alpha Sun-Thu Bravo Sun-Thu Charlie Sun-Thu Delta Sun-Thu Echo Tue-Sat Foxtrot Tue-Sat Golf Tue-Sat Hotel Tue-Sat India Fri-Mon Juliet Fri-Mon Hours 0500-1330 1400-2230 0900-1730 1800-0230 0600-1430 1500-2330 1000-1830 1900-0330 0530-1600 1600-0230 2.3.2. Accident Driver Work Schedule The accident driver’s shift was from 4:00 pm to 2:30 am four days a week, Friday through Monday. The accident driver worked 40 hours per week and stated to NTSB investigators that at the time of the crash, ATG/Uber was her only employment.21 The accident driver was classified as an hourly worker and was paid bi-weekly. The driver received a 30-minute break and two 10minute breaks. For additional information concerning the accident driver’s 72 - hour work history prior to the crash, see the Human Performance Group Chairman’s Factual Report. 21 See Human Performance Attachment Tempe, AZ – Accident Driver Interview. – Operations Factors Factual Report Page 12 of 28 2.3.3. Tempe, Arizona AV Testing The testing procedures consisted of operating the test vehicles primarily on a designated route 11.2 miles in downtown Tempe, Arizona. At the time of the crash, the Tempe VO team had operated the test loop thousands of times. The accident driver had traversed the crash loop 73 times. The test loop had represented 18 percent of her total driving hours. The accident driver had operated a total of 234 hours in both automatic and manual mode and a total of 152 hours in autonomous mode. Upon completion of each shift drivers attended a debrief with a supervisor which usually lasted 10-15 minutes. The fleet was grounded on March 18, 2018. The Tempe facility ceased operations on May 23, 2018.22 2.4. VO’s Responsibilities The VO’s were responsible for operating the self-driving vehicles to demonstrate the safety and reliability of the systems and technology. The VO’s were responsible for the following tasks: • Operate the technology in Self-Driving Ubers on a shift basis • Complete driving report, procedures, and tests in order to collect data for the engineering team • Provide essential feedback on system performance to the team such as improper maneuver or issue identified • Interact with the public as an ambassador for Uber’s self-driving technology • Tag errors or near misses • VO’s were required to hover over the steering wheel and brakes at all times and be on the test vehicle’s tablet ready to take control of the vehicle at any time an intervention was necessary VO’s were trained to operate in both the driver position and co-pilot position. The duties for the pilot position required the VO to observe all driving performed by the test vehicle and be ready to take control of the vehicle for any reason. This would include: if the test vehicle ran a red light, encroached into another lane, deviated from the intended route or for any unsafe maneuver the VO was responsible to take the wheel, make an evasive maneuver, brake or do whatever was required to prevent an accident or mishap. The co-pilot position was to identify any abnormal driving behaviors, calls out road signs, help navigate the test vehicle, and communicated with the base operations center. During each shift, the VO’s attended a daily briefing on the latest testing or software upgrades. VO’s were required to evaluate the latest advancements in the software systems on the vehicle. VO’s were also trained and required to pick up paying Uber customers while in autonomous mode. The accident driver picked up numerous Uber customers during her employment at ATG. The accident driver’s last Uber customer pickup in a test vehicle was in 22 Retrieved from: https://www.usatoday.com/story/tech/news/2018/05/23/uber-shut-down-self-driving-caroperation-arizona-after-fatality/637122002/ Tempe, AZ – Operations Factors Factual Report Page 13 of 28 October 2017. In November 2017 ATG ended the program of picking up Uber customers in test vehicles. 2.4.1. VO’s Change of Responsibilities When the accident driver was hired, VO’s were trained to operate in both the pilot and copilot positions. In fall of 2017 ATG decided to phase out the co-pilot role and reduce the VO duty to a single operator in the driver’s seat only. The accident driver was trained and transitioned to single operator operations on October 25, 2017. 2.4.2. Accident Vehicle The original ATG test vehicles were Ford Fusion sedans called “Boron” and the newer test vehicles the Volvo XC90 called “Krypton.” For detailed information concerning the Volvo XC90 and the accident vehicle see the V ehicle Factors Group Chairm an Factual Report in the docket. 3. The Accident Driver 3.1. Accident Driver Driving and Employment History The accident driver was hired by ATG on June 23, 2017.23 Because of scheduling issues, the accident driver went directly to her home terminal in Tempe, Arizona to receive her initial training (Onboarding) from July 5-7, 2017. The accident driver then went to Pittsburgh, Pennsylvania for the second phase of training (Closed Course) from July 10-14, 2017. The accident driver conducted her last block of training (INDOX) back at the Tempe facility from July 17-28, 2017. According to the ATG files the driver passed all required training and started to operate as a VO. At the time of the crash the accident driver held an Arizona Class D driver’s license. There were no restrictions and 0 points assessed against the license. For additional information concerning the driver’s license history see the Human Performance Group Chairman’s Factual Report in the docket. The accident driver had worked in various positions in the computer, software, and social media before coming to Uber/ATG. For additional information on the accident driver’s employment history see Table 2. Table 2. Accident Driver Employment History Position Company Employment Dates VO Uber/ATG 6/2017- crash Moderation ICUC 9/2016-6/2017 Specialist Moderation Lithium 9/2012-7/2016 Specialist Technologies Inc Community Live World Inc 3/2012-6/2014 Content Telephone B&R 11/2011Mystery Shopper Technologies 1/2013 Driver Partner Uber 9/2015-4/2017 23 Reason for leaving NA Wanted fulltime work Contract ended Not provided Not provided Part-time See Operations Attachment- Uber Hiring Agreement. Tempe, AZ – Operations Factors Factual Report Page 14 of 28 Following the crash, all operations in Tempe, Arizona were grounded on March 18, 2018. The accident driver as well as the other employees at the Tempe facility was terminated on June 1, 2018. 4. Post-Crash Internal Audit Following the crash, Uber/ATG immediately grounded all of the self-driving fleets and conducted an internal audit and an independent external review of safety culture and opportunities for safety improvements.24 The internal audit was divided into three working groups: operations, technical, and organizational. As a result of the internal audit, numerous recommendations were made to improve safety and operational issues.25 Some of the recommendations included the following elements: 1. Implement Automatic Braking - While testing, incorporate an independent system that would automatically brake under a common collision – imminent scenario 2. Safety Culture - Develop a new and advanced training program. Refresh internal and external communications. Reinforce safety culture through recognition of safe behaviors. Measure and monitor 3. Risk Management - integrate risk management into all phases of the operations before returning to on the road testing. 4. Human Machine Interface (HMI)- Confirm use of industry best practices and guidelines for HMI design, specific to SDV supervision use case, and implement changes to current practices if appropriate. Formalize review process for changes to HMI design to be approved and tested prior to fleet wide implementation. 5. Testing & Triage- Augment the formal test track suite, and consistently require road testing to be preceded by successful passing of this suite. Consistently require testing to be preceded by successful passing of offline testing. Create guidelines to establish how, when, and where public road operation should be utilized. 6. VO Training- Continually assess the VO selection and training process and revise as needed to ensure an appropriate bar for safe driving performance is being set. Provide operators with additional training to improve their overall knowledge of the vehicle and self-driving system, including appropriate refresher training. Establish continuous improvement process for VO training program to expose operators to sufficient content for an operational domain and validate operator performance to the standards required of that domain, including appropriate refresher training and sufficient pass/fail criteria. 7. VO Guidelines- Research and implement new methods for effectively maintaining VO vigilance. Consider technical aids, training practices, operational procedures, 24 25 See Operations Attachment - Uber ATG Report Supplement Internal and External Safety Reviews 2018. See Operations Attachment - Uber ATG: A Principled Approach to Safety September 2018. Tempe, AZ – Operations Factors Factual Report Page 15 of 28 communication programs that continually build professionalism. Reinforce clear guidelines of the use of personal devices in vehicles for personal and work-related scenarios and monitor effectiveness. Revise VO performance monitoring and performance metrics to ensure on-road safety and attentiveness while incentivizing safety. Other recommendations included improvements to communications, hiring procedures, disciplinary policy improvements, and recruitment. Additionally, recommendations were addressed regarding improving software functions, engineering upgrades, and other operational systems improvements. 4.1. Post-Crash External Audit Uber ATG also hired an external consulting company to conduct an independent review and audit of Uber’s safety culture. The audit made numerous recommendations that addressed the ATG safety management system. These areas included: the ATG safety culture, ATG training, ATG safety policies, ATG processes, and ATG procedures. Examples of the recommendations included the following:26 1. Uber should develop a safety management system (SMS). 2. The head of ATG or his designee, should establish additional communications and feedback mechanisms within ATG. 3. The head of ATG should appoint a senior manager for operational safety. 4. ATG managers in cooperation with Human Resources, should take steps to verify that a complete and up-to-date job description is in place for every ATG position. 5. ATG senior leadership should take steps take further steps to communicate to all employees the importance of a robust safety culture as a core value. The audit also made recommendations to include updating the VO training program to incorporate proficiency testing and recurrent training. Other recommendations addressed areas of management and corporate leadership involvement as well as developing a safety management system.27 4.1.1. Post-Audit Safety Improvements As a result of the internal and external audits, Uber/ATG implemented multiple safety recommendations. For example, Uber redefined the ATG’s safety principles. The company also made several organizational changes to improve safety and safety culture that included:28ʼ29 26 See Operations Attachment- Independent Review of the Safety Culture of Uber Advanced Technologies Group Final Report. 27 Ibid. 28 See Operations Attachment- Uber Advanced Technologies Group A Principled Approach to Safety 29 Retrieved from: https://medium.com/@UberATG/learning-from-the-past-to-move-forward-f4af566f2c3 Tempe, AZ – Operations Factors Factual Report Page 16 of 28 • The title and duties of the Vehicle Operators were replaced with a new position call a “Mission Specialist.” Mission Specialists would have more advanced training and have more responsibilities. • ATG now required two Mission Specialists to operate the AV’s at all times. • New Core ATG Team would now consist of system engineering and a testing team. The team would focus on improve the link between development and testing. • Operational safety and training teams were moved from the operations department into the safety department. • Increased organizational focus and new foundational autonomy development process which would include a variety of steps to develop and test advancements in the software before implementing in real world applications. In addition to the stated changes, ATG created a new enhanced training program for the mission specialists. Such training would require 30 hours of training and evaluations before operating the AV on the road in manual mode. Additional new enhanced training would be required before the mission specialist could operate in self-driving mode. The basic training instruction would now include such topics as: driving dynamics and awareness, emergency maneuver exercises, fatigue management, distracted driving and additional blocks of instruction. Additional improvements to the ATG safety culture are detailed in the following sections. 4.1.1.1. Additional Training and Policy Changes New training requirements were added to the curriculum to improve overall safety. Mission specialists were now instructed on the ATG infractions policy and consequences for such actions. Additionally, mission specialists received new defensive driving drills to break poor driving habits and learn new defensive driving techniques. Mission specialists also received additional training on parking and reversing drills to sharpen driving skills. Moreover, the AV test vehicle, a Volvo XC90, that is equipped with the Advanced Driver Assistance System (ADAS); would now be activated at all times. Mission specialists would be required to be trained on the Volvo XC90 ADAS system and be knowledgeable of the collision avoidance, object detection, and autobraking system. 4.1.1.2. New Requirements for Mission Specialists Mission specialists would be subject to elevated hiring and screening procedures as well as new higher standards of levels of performance. For example, all mission specialists were now required to pass a medical examination similar to the DOT medical certification process under §391.41, as well as new standards for mandatory pre-employment and random drug and alcohol screening. Additionally, mission specialists were trained on fatigue management and new hours of service requirements. Such requirements limited operations in left seat not exceed 4 hours per shift and no more than 2 hours without taking a break. Vehicles were also modified with real time in cabin monitoring to detect if distraction was occurring and take appropriate action. Other advancements included more rigorous training for mission specialists and improved Tempe, AZ – Operations Factors Factual Report Page 17 of 28 communication procedures between during testing. Lastly, technological advancements have been made to the software for pedestrian and bike detection and to include jaywalking as a variable. 4.2. AV Testing After the Break in Service On December 20, 2018 ATG resumed AV testing. Multiple stipulations and new safety procedures were adopted from the internal and external audits. At that time some of the changes included: • The testing was limited to a one-mile loop in Pittsburgh near the ATG headquarters • The test vehicles would not exceed a speed of 25 miles per hour • AV vehicles now required to have two mission specialists (i.e. seated driver position and front passenger position) at all times • Furthermore, picking up passengers was prohibited • AV vehicles would have the Volvo ADAS activated at all times 4.3. ATG Organizational Changes ATG has made several organizational changes that includes a distinct and separate operational safety team, led by a new Head of Operational Safety that reports to the Head of Safety. This operational team is independent from the development and operational testing teams, in order to provide appropriate checks and balances for development operations. 4.4. ATG Safety Management System After hiring of the new Head of Operational Safety and reorganization of the Operational Safety Team, ATG has been working on implementing a Safety Management System (SMS). SMS is frequently associated with the aviation industry and is defined as: “SMS is a formal, top down, organization-wide approach to managing risk and assuring the effectiveness of safety risk controls. It includes systematic procedures, practices, and policies for the management of risk.” 30 On May 9, 2019 NTSB Investigators conducted a follow-up investigation at the ATG headquarters office in Pittsburgh, Pennsylvania. At that time ATG made a presentation concerning the organizational and safety culture changes that had been implemented since the crash. An overview of some of the implementation of the SMS was also reviewed. It should be noted however, that the SMS program was in its infancy and per the Head of Operational Safety, the complete rollout of the SMS could take four or five years to fully be implemented. 30 FAA Order 8000.369 Tempe, AZ – retrieved from: https://www.faa.gov/about/initiatives/sms/ – Operations Factors Factual Report Page 18 of 28 5. Federal Oversight of Autonomous Vehicles 5.1. Pre-Crash Federal Oversight At the time of the crash and the writing of this report there was no Federal oversight of the testing of autonomous vehicles. Under the US Department of Transportation, the Federal Highway Administration (FHWA), the National Highway Traffic Safety Administration (NTHSA) and the Federal Motor Carrier Safety Administration (FMCSA) each provide financial support for transportation systems and regulatory oversight to protect the motoring public. Specifically, the NTHSA has jurisdiction to enforce vehicle performance standards and works in cooperation with state and local government to reduce deaths and injuries and economic losses from motor vehicle crashes. NTHSA’s mission statement states: “Save lives, prevent injuries and reduce economic costs due to road traffic crashes, through education, research, safety standards and enforcement activity.”31 NTSHA is the primary regulatory authority that enacts the Federal Motor Vehicle Safety Standards (FMVSSs) that specify performance standards for new motor vehicles, trucks, and buses, and has the authority to take action with known safety defects. 32 NTSHA has broad enforcement authority to address existing and new automotive technologies and equipment. “NTSHA is authorized under Congress to protect the safety of the driving public against unreasonable risks of harm that may arise because of the design, construction, or performance of a motor vehicle or motor vehicle equipment, and to mitigate risks of harm, including, risks that may arise in connection with ADS.33 In September 2016 NTSHA released the Federal Automated Vehicle Policy called “A V ision for Safety 2.0” to promote improvements in safety, mobility and efficiency through automated driving systems (ADS).34 This policy which was in place at the time of the crash, was divided into two sections one for voluntary compliance and the other for technical assistance to the states. The policy was designed to encourage new entrants to the industry, improve regulatory process and to support industry innovation. These policy guidelines were intended to serve as voluntary guidance for entities involved in the testing of autonomous vehicles to encourage safety assessments of their systems to achieve safety and the deployment of ADS. This guidance was completely voluntary and had no compliance or enforcement component. The guidance also included best practices for state legislatures and best practices for highway safety officials. The Vision for Safety Policy 2.0 recognized the Society of Automotive Engineers (SAE) standards of automation. The five levels of automation are comprised of the following: 31 32 33 34 • Level 0 - no automation: driver performs all tasks • Level 1 - Driver assistance: vehicle is controlled by the driver, but some assist features may be included in vehicle design Retrieved from: https://www.nhtsa.gov/about-nhtsa/nhtsas-core-values NTSHA 3.0 page 8 NTHSA 2.0 page 3 Retrieved from: https://www.nhtsa.gov/press-releases/us-dot-releases-new-automated-driving-systems-guidance Tempe, AZ – Operations Factors Factual Report Page 19 of 28 • Level 2 - Partial Automation: Vehicle has combined automated functions, like acceleration and steering, but the driver must remain engaged with the driving task and monitor the environment at all times • Level 3 - Conditional Automation: Driver is a necessity but is not required to monitor the environment. Driver must be ready to take the control of the vehicle at all times • Level 4 - High Automation: The vehicle is capable of performing all driving functions under certain conditions. The driver must have the option to control the vehicle • Level 5 - Full Automation: The vehicle is capable of performing all driving functions under all conditions. The driver may have the option to control the vehicle 5.1.1. Voluntary Guidance According to the 2.0 Policy, entities engaging in ADS testing are “encouraged” to follow a program that takes 12 safety elements into consideration when developing and testing ADS. 35 These 12 safety elements consist of the following: 1. System Safety – entities should design ADSs free of unreasonable safety risks. For example, the process should take into consideration industry best practices, design principles established by accredited standards such as from SAE, International Organization for Standardization (ISO), the military, aviation, or space programs. 2. Operational Design Domain – entities should document testing of ADS on public roadways. 3. Object and Event Detection and Response (OEDR)- refers to the detection by the driver or ADS of any circumstance that is relevant to the immediate driving task. Entities are encouraged to have a documented process for assessment, testing, and validation of their ADS OEDR capabilities. 4. Fallback (minimum risk condition) – should have documented process for transitioning to a minimal risk condition when a problem is encountered or the ADS cannot operate safely. 5. Validation Methods – entities are encouraged to develop validation methods to appropriately mitigate the safety risks associated with their ADS approach. 6. Human Machine Interface – Considerations should be made for the human driver, operator, occupants and external actors with whom the ADS may have interactions, 35 NTSHA Automated Driving Systems, A Vision for Safety 2.0. Tempe, AZ – Operations Factors Factual Report Page 20 of 28 including other vehicles (both traditional and those with ADS), motorcycles, bicycles, and pedestrians. 7. Vehicle Cybersecurity – entities should take an engineering approach to minimize risks to safety, including those due to cybersecurity threats and vulnerabilities. 8. Crashworthiness – the occupant protection system should maintain its intended performance level in the event of a crash. 9. Post-Crash ADS Behavior – actions should be considered immediately following a crash such as shutting off fuel pump, moving the vehicle to a safe position off the roadway or removing power to the vehicle. 10. Data Recording – entities engaging in testing or deployment are encouraged to establish a documented process for testing, validating, and collecting necessary data related to the occurrence of malfunctions, degradations, or failures in a way that can be used to establish the cause of a crash. 11. Consumer Education and Training – Entities are encouraged to develop, document, and maintain employee, dealer, distributor, and consumer education and training programs to address the anticipated differences in the use and operation of ADSs from those of conventional vehicles that the public owns and operates today. 12. Federal, State, and Local Laws – Entities are also encouraged to document how they intend to account for all applicable Federal, State, and local laws in the design of their vehicles and ADSs. 5.1.2. Voluntary Safety Self-Assessment According to the 2.0 Policy, entities engages in ADS testing may demonstrate how they use their own best practices or industry best practices by following NTSHA’s voluntary safety self-assessment form.36 The assessment is designed to demonstrate to the public that entities are: 1) considering the safety aspects of ADSs, 2) communicating and collaborating with DOT, 3) encouraging the self-establishment of industry safety norms for ADS, and 4) building public trust, acceptance and confidence. 5.2. Post-Crash Federal Oversight of AV Following the Tempe, Arizona crash, in October 2018 NTSHA released new Federal guidance for AV called: Preparing for the Future of Transportation 3.0. The new guidance did not replace voluntary guidance as outlined in the original 2.0 Vision for Safety guidance however, incorporated additional guidance. AV 3.0 supports the safe development of automated vehicle technologies by: • 36 Providing new multi-modal safety guidance Retrieved from: https://www.nhtsa.gov/automated-driving-systems/voluntary-safety-self-assessment Tempe, AZ – Operations Factors Factual Report Page 21 of 28 • Reducing policy uncertainty and clarifying roles • Outlining a process for working with USDOT as technology evolves Specifically, the new AV 3.0 guidance provides initiatives relating to automated vehicles by: • Stating that the Department will interpret and consistent with all applicable noticed and comment requirements adapt the definitions of “driver” or “operator” as appropriate to recognize that such terms do not refer exclusively to human but may include an automated system. • Identifying and supporting the development of automation-related voluntary standards developed through organizations and associations which can be an effective non-regulatory means to advance the integration of automation technologies. • Affirming that the department is continuing its work to preserve the ability for transportation safety applications to function in the 5.9GHz spectrum. NTHSA further stated in AV 3.0 that the approach to shaping policy for AV will be based upon six principals: 1. We will prioritize safety 2. We will remain technology neutral 3. We will modernize regulations 4. We will encourage a consistent regulatory and operational environment 5. We will prepare proactively for automation 6. We will protect and enhance the freedoms enjoyed by Americans According to the press release from NTHSA regarding the 3.0 release, it stated: “USDOT’s role in automation is to ensure the safety and mobility of the traveling public while fostering economic growth. As a steward of the nation’s roadway transportation system, the Federal government plays a significant role by ensuring that automated vehicles can be safety and effectively integrated into the existing transportation system, alongside conventional vehicles, pedestrians, bicyclists, motorcyclists, and other road users.” 6. State Regulations for Autonomous Vehicles Uber/ATG conducted AV testing in several states. Specifically, Arizona represented the state that provided the AV testing oversight of the driver / vehicle at the time of the crash. Pennsylvania provides current oversight of AV testing since ATG resumed testing operations since the crash. Highlights of each states oversight programs are detailed in the following sections. Tempe, AZ – Operations Factors Factual Report Page 22 of 28 6.1. Arizona State Regulations for AV Nevada was the first state to authorize the operation of autonomous vehicles (AVs) in 2011. Since then, 21 other states—Alabama, Arkansas, California, Colorado, Connecticut, Florida, Georgia, Illinois, Indiana, Louisiana, Michigan, New York, North Carolina, North Dakota, Pennsylvania, South Carolina, Tennessee, Texas, Utah, Virginia and Vermont —and Washington D.C. have passed legislation related to autonomous vehicles. Governors in Arizona, Delaware, Hawaii, Idaho, Maine, Massachusetts, Minnesota, Ohio, Washington and Wisconsin have issued executive orders related to autonomous vehicles.37 6.1.1. Arizona Legislation for AV Arizona first introduced legislation for AVs in January 2013 called HB2167. The key features of the legislation defined AV’s and autonomous technology and it removed liability from the vehicle manufacturer when autonomous technology is fitted by a third party. 6.1.2. Arizona Executive Order In September 2015 Arizona Executive Order 2015-09 was enacted for autonomous vehicles. The executive order established state support for the development of technologies by allowing the testing and operation of self-driving vehicles on public roads in Arizona. 38 The Executive Order stated: 1. The Department of Transportation, Department of Public Safety, and all other agencies of the State of Arizona with pertinent regulatory jurisdiction shall undertake any necessary steps to support testing and operation of self-driving vehicles on public roads within Arizona. 2. Pilot programs will be enabled on campuses of selected universities in partnership with entities that are developing technology for self-driving vehicles, whereby an operator with a valid driver’s license may direct a vehicle’s movement, regardless of whether the operator is physically present in the vehicle or is providing direction remotely while the vehicle is operating in self-driving mode. 3. Testing and operation of self-driving vehicles in such pilot programs shall abide by the following rules: a. Vehicles may be operated by an employee, contractor, or other person designated or otherwise authorized by the entity developing self-driving technology. b. Vehicles shall be monitored, and an operator shall have the ability to direct the vehicle’s movement if assistance is required. 37 Retrieved from: http://www.ncsl.org/research/transportation/autonomous-vehicles-self-driving-vehicles-enactedlegislation.aspx#Enacted Autonomous Vehicle Legislation 38 See Operations Attachment – Arizona Executive Order 2015-09. Tempe, AZ – Operations Factors Factual Report Page 23 of 28 c. The individuals operating vehicles shall be licensed to operate a motor vehicle in the United States. d. The vehicle owner shall submit proof of financial responsibility, in an amount and on a form established by the Director of the Arizona Department (DOT) of Transportation. 4. The Director of the DOT may promulgate additional rules considered necessary to implement this Executive Order. 5. There shall be established within the Office of the Governor a Self-Driving Vehicle Oversight Committee to advise the DOT, the Department of Public Safety, the select universities, and other pertinent agencies how best to advance the testing and operation of self-driving vehicles on public roads.39 As a result of this crash, on March 26, 2018 ability to test and operate autonomous vehicles.40 the Governor of Arizona suspended Uber’s 7. Pennsylvania Department of Transportation AV Testing Oversight In June 2016, the Pennsylvania Department of Transportation (PennDOT) created the “Autonomous Vehicle Policy Task Force” to help craft policy recommendations for AV testing. The Task Force consisted of “stakeholders from federal, state and local governments, law enforcement, technology companies, higher education, manufacturers, motorists, trucking companies, and academic research institutions.”41 The Task Force created the policy and procedures for AV testing which falls under PennDOT’s oversight. The following sections describe the procedures for AV testing in Pennsylvania. 7.1. Notice of Testing Prior to any AV Testing an highly automated vehicle (HAV) tester must submit a Notice of Testing application. The application requires some of the following elements:42 1. Test information – name, address, contact information 2. Vehicle information model – license plate, VIN, proof of current registration, make and 3. Safety Driver information – driver’s information, license, training 39 See Operations Attachment- Arizona Executive Order 2015-09. See Operations Attachment- Letter from Governor Ducey dated March 26, 2018. 41 Retrieved from: https://www.penndot.gov/ProjectAndPrograms/ResearchandTesting/Autonomous%20_Vehicles/Pages/AV-TaskForce-.aspx 42 See Operations Attachment- Pennsylvania Department of Transportation Automated Vehicle Testing Guidance July 23, 2018. 40 Tempe, AZ – Operations Factors Factual Report Page 24 of 28 4. Location information – list counties AV testing will occur 5. Applicant Acknowledgements a. The vehicle complies with all federal laws and requirements. b. The AV meets the state safety and emissions requirements c. That the vehicle has been tested under controlled conditions (in simulation, closed track, or on-road) d. The AV is capable of operating in compliance with applicable traffic laws and can obey traffic control devices e. During testing a safety driver will be present in the driver’s seat and can take immediate control of the AV f. The HAV (Highly Automated Vehicle) can safely alter the driver, when applicable, that the safety driver can take control of the HAV. g. That the HAV has a mechanism to engage and disengage the ADS that is easily accessible to the HAV safety driver, the secondary safety associate, law enforcement and other emergency responder personnel. h. That the HAV has an audio signal or visual display inside the cabin to indicate when the ADS is engaged. i. That the HAV is equipped with a means to record operational data before a collision occurs. j. That the tester has considered and implemented reasonable measures, which may include industry best practices, company policies, or other methods, to mitigate cybersecurity risk. k. That all safety drivers and secondary safety associates have successfully completed the HAV tester’s safety driver training program. l. That quality controls are deployed and monitored to ensure the acknowledgements checked above are implemented, adhered to and measured to ensure safe operation. 6) Required Submissions – PennDOT requires the submission of either of the following: a) A Safety Risk Mitigation Plan (“Safety Plan”) that shall include at a minimum: i) A general overview of the intended ODD testing ii) A description of how the disengagement technology complies with industry standards Tempe, AZ – Operations Factors Factual Report Page 25 of 28 iii) A description of pre-testing and ongoing validation methods to ensure continued functionality of the software and hardware systems. iv) A criminal background check and records check for the safety drivers v) A description of safety driver training program that includes a description of: (1) How the tester informs HAV safety drivers of any HAV related operational issues, but not limited to, any operational limitations of the ADS and safe operating parameters for the given ODD for the conditions under which the HAV is tested on trafficways, but not limited to: a) Safety driver knowledge of HAV operations b) Safety driver’s disengages ADS system decision-making process when driver manually (2) Measures to prevent driver fatigue /inattentiveness / carelessness, including but not limited to: (a) Distracted Driving avoidance techniques (b) Fatigue and Wellness Awareness (c) Drug and Alcohol Impairment Awareness (3) A description of the safety measures in place in case of an ADS disengagement, regardless of the reason for the disengagement. (4) A description of the testers driver training program and how that program addresses: (a) Confirmation of Basic Driving Skills including but not limited to: i) Basic vehicle operation ii) Visual search iii) Night operations iv) Hazard perception v) Response to vehicle malfunctions vi) Response to sudden emergencies vii) Response to driver distractions viii) Responses to aggressive driver Tempe, AZ – Operations Factors Factual Report Page 26 of 28 b) Basic knowledge of vehicle controls c) Basic trip procedures to include i) Pre-trip inspection outside the vehicle ii) Pre-trip preparation inside the vehicle b) Alternatively, a company may provide PennDOT with a Voluntary Safety SelfAssessment as established by NHTSA in the 2017 Voluntary Guidance for Automated Driving Systems section I. 7) Operational Requirements a) For operations on trafficways posted at or under 25 mph, the tester must meet the criteria of Section 6 above. b) for operations on trafficways posted above 25mph, the tester must meet the criteria; of Section 6 and there must be a secondary safety associate present in the vehicle (in the forward passenger seat if installed in the vehicle.) c) Testers requiring use of a single safety driver for operations on trafficways above 25mph must present evidence of an enhanced performance driver training plan for Department review. E. DOCKET MATERIAL The following attachments and photographs are included in the docket for this investigation: LIST OF ATTACHMENTS Operations Attachment - Uber Job Announcement Operations Attachment - UATC Driver Policy and Procedures Agreement Form Operations Attachment- Uber Tiered Infraction Policy Operations Attachment- Uber Employment Agreement Operations Attachment- Independent Review of the Safety Culture of Uber Technologies Advanced Technologies Group Final Report Operations Attachment- Uber Advanced Technologies Group A Principled Approach to Safety 2018 Tempe, AZ – Operations Factors Factual Report Page 27 of 28 Operations Attachment- Arizona Executive Oder 2015-09 Operations Attachment- Letter from Arizona Governor dated March 26, 2018 Operations Attachment- Pennsylvania Department of Transportation Automated Vehicle Testing Guidance July 23, 2018 LIST OF PHOTOGRA PHS Operations Photos 1 -3 Tempe Operations Driver Safety Poster END OF REPORT Michael S. Fox Senior Highway Accident Investigator Tempe, AZ – Operations Factors Factual Report Page 28 of 28