在自動駕駛汽車研發(fā)與測試中，開發(fā)商通常會采用公共道路“影子駕駛”模式，也就是說車上會配備一名人類駕駛員，但這位駕駛員并不操控方向盤，僅觀察被測系統(tǒng)的運轉情況。但這一作法實際有兩個大前提，第一是車輛已經學會如何適當?shù)貞獙︸{駛過程中的各種事件；第二是人類駕駛員（觀察者）的反應足夠快，可以阻止任何不利結果的發(fā)生。
由于各種各樣的原因，看起來“影子駕駛”模式似乎是目前完全自動駕駛汽車研發(fā)與測試中不可或缺的組成部分。
事實上，自動駕駛汽車的測試并不輕松，每家制造商均需累積大約一萬億英里的自動駕駛里程，并覆蓋甚至多次覆蓋所有可能的場景。根據(jù)作者的保守估計，假設配置 23.4萬 輛汽車，每天以時速 50 英里 24 小時運行，那么累計一萬億英里自動駕駛里程需要 10 年，成本高達 3000 億美元以上。
除了成本，“影子駕駛員”還存在安全方面的隱患。舉個例子，為了訓練人工智能和 SAE L3級自動駕駛及控制交接功能，廠商必須讓車輛經歷一系列真實事故場景。小至剮蹭追尾，大到其他更加危險、更加復雜的場景，整個訓練過程可能涉及成千上萬個事故場景，極有可能造成不同程度的傷害，甚至傷亡。另外，此類事故場景還必須加上公共道路測試，這也會讓整個測試過程更加危險。無論是一些正在借助“影子駕駛員”進行研發(fā)測試的系統(tǒng)，還是已經投入使用的 SAE L3 級公共自動駕駛汽車，目前這些系統(tǒng)根本不可能留給“影子駕駛員”充裕的反應時間，以充分了解當下的駕駛環(huán)境，重新取得對車輛的有效控制，并保證安全。
正如最近幾起自動駕駛事故所掀起的風波一樣，“影子駕駛員”出現(xiàn)在公共道路上，有可能削弱消費者對自動駕駛汽車的信心與支持，甚至引來鋪天蓋地的負面媒體報道，催生更加嚴格的監(jiān)管，招致無休止的訴訟，還會造成投資者的信任喪失，給自動駕駛汽車的發(fā)展帶來沉重的后果，并最終將這項有潛力挽救成數(shù)十萬民眾的技術扼殺在搖籃之中。
值得慶幸的是，我們還有一個解決方案，那就是使用“完整模擬”完成主要驗證過程，降低行業(yè)對“影子駕駛員”的依賴。該解決方案采用“全系統(tǒng)工程”方法設計，以客戶的用戶需求和設計流程為基礎，并包含“最終狀態(tài)場景”。
合情合理的模擬手段
目前，汽車行業(yè)使用的模擬系統(tǒng)遠未達到航空航天行業(yè)的水平和復雜度（即美國聯(lián)邦航空管理局FAA的D 級規(guī)定），也沒有采用適當?shù)膶崟r架構?，F(xiàn)階段，車輛、輪胎和道路模擬的模型均不夠精確，特別是在模擬一些惡劣條件時。人工智能看起來似乎已經做好了學習準備，但事實并非如此。更可怕的是，直至真實場景發(fā)生之前，此類情況通常都很難發(fā)現(xiàn)，而一旦發(fā)生，則會給項目進展帶來沉重的時間和執(zhí)行壓力，甚至直接讓項目叫停。
注意，這些問題并不會暴露在一些常規(guī)測試場景下，只有當面對一些非常復雜，或對時間要求很高的場景，迫使車輛達到甚至超過性能極限時才會出現(xiàn)，而這通常也正是問題的開始。
假如不配備全動系統(tǒng)(full motion systems)，一些駕駛員在環(huán)（DiL）模擬器可能會讓開發(fā)人員的信心爆棚，但真實情況并非如此樂觀。運動系統(tǒng)設備可以配合真人模擬器使用，用于模擬自動駕駛行程，并允許開發(fā)人員評估車上人員的暈車感、舒適感及對自動駕駛汽車的信任水平。除了配備合適的運動系統(tǒng)外，開發(fā)人員還可以借助“航空航天/DoD/FAA”的仿真技術、最佳做法和測試方法，解決汽車行業(yè)面臨的自動駕駛模擬挑戰(zhàn)。這是因為，一些國防城市戰(zhàn)爭游戲的游戲場景與很多復雜駕駛場景高度重合，另外還有一些采用了專業(yè)模型并提供實時保真的效果，尤其可以發(fā)揮重要作用。
如圖所示，大多數(shù)自動駕駛汽車開發(fā)商均無法履行之前的承諾，即在有限范圍內推出真正具備SAE L4-L5級自動駕駛功能的自動駕駛汽車。(數(shù)據(jù)來源：Eric Paul Dennis/Center for Automotive Research)
數(shù)據(jù)方法論至關重要
并預先定義和構建最困難的場景，整個項目可能將最終遠超預計時間，才能做到所有模擬場景的執(zhí)行，包括在當下及未來無休止地重復修補這些復雜場景。
如果采用敏捷開發(fā)流程，可能浪費的時間將難以預估，而且從歷史經驗來看，一些較為復雜的元素通常很難完成，這只能為日后開發(fā)埋下隱患。此外，除非遭遇到一些最復雜和最困難的場景，否則這些設計缺陷通常很難暴露。最終，項目可能不得不進行“修修補補”，大量返工，而不是在一開始就在許多常見場景中設置妥當。
目前，“邊緣場景”和“角落場景”經常用于描述事故情景。但事實上，事故情景與任何其他情景并無差別，只是結果是沒有人想看到而已。一些真正的“邊緣場景”或“角落場景”是在任何情況下都不應該，也不可能發(fā)生的 — 例如要求搜索引擎尋找一張貓的圖像，但最終得到的是一張垃圾桶的圖像。工程師通常不會覆蓋所有可能的事故場景，也就是被他們劃分在“核心場景”之外的“邊緣場景”或“角落場景”。也正因如此，人們有理由進行必要的盡職調查。
模擬仿真的目標應集中在為 AI 堆棧提供可用于辨別不同物體的數(shù)字表達式，采用相同的輸入速率，并具備相同的模糊度，從而找到妨礙 AI 堆棧做出正確決定的問題。這些數(shù)據(jù)集中，最難實現(xiàn)的部分常被稱為“邊緣”或“角落”場景；然而，這些場景才是判斷 AI 堆棧是否具備成功決策能力的關鍵案例。為了清晰定義這些案例，并明確每個案例的預期結果，我們需要一種條理分明的可管理式遞歸數(shù)據(jù)方法。
最終狀態(tài)場景矩陣
除了提供影響上述系統(tǒng)工程方法的場景數(shù)據(jù)之外，所有各方（包括政策制定機構、驗證機構、保險公司和制造商等）都需要盡早了解模擬目標，也就是“項目完工”的定義。只有具備對實時變化的支持能力（從而及時修正任何 AI 感知錯誤），場景數(shù)據(jù)集才能真正稱得上全面，但相應的工作量幾乎與為仿真過程清晰定義“整合’’與“系統(tǒng)模型”一樣可觀。
從地理圍欄到 SAE L4 級和 L5級自動駕駛汽車，要成功實現(xiàn)這些目標，該測試數(shù)據(jù)集的建立需要依賴眾多數(shù)據(jù)源和數(shù)據(jù)域；要求全球汽車開發(fā)界開展最高水平的盡職調查；必須確保達到必要的安全水平并能夠證明這一點；還必須映射到上文提到的仿真系統(tǒng)，并與之同步。
在目前的 AV 測試范例做出改變之前，汽車行業(yè)永遠不會迎來可以挽救萬千生命的 SAE L4 級自動駕駛汽車，也不會迎來真正的全自動駕駛汽車。
Autonomous vehicle developers are widely using public “shadow” driving which involves a human in the driver’s seat letting go of the steering wheel and ceding control to the system under test to observe how it performs. The fundamental premise of this process is that the vehicle has learned the proper management of possible events which may occur during the maneuvers, and the human observer can react fast enough to stop any negative results from occurring.
It is a myth that public shadow driving is the best or only solution to create a fully autonomous vehicle, for several reasons.
To complete such an effort would require each AV maker to accumulate roughly one trillion miles in driving and re-driving all the potential scenarios. The estimated cost of such programs is over $300 billion [based on the author’s conservative calculation of 234,000 vehicles operating at an average of 50 mph, every day all day for 10 years, to arrive at one trillion miles].
Other problems with shadow driving involve safety, including the running of actual accident scenarios to train the AI and SAE Level 3/handover. The process of accident-scenario “training” has potential to cause thousands of accidents, injuries and casualties when efforts to train and test the AI move from the benign scenarios to more complex and dangerous ones. Thousands of accident scenarios will have to be driven multiple times on the public streets driving scenarios—whether in a system under development using public shadow driving or an SAE Level 3 vehicle in use by the public, it is impossible to provide the driver with a sufficient margin of time to regain situational awareness for safely executing effective vehicle control.
As we’ve seen in the aftermath of recent accidents, public shadow driving can weaken consumers’ support of AVs while bringing negative media coverage, increased regulations, endless litigation, and loss of investor trust. As a result, the industry could lose the opportunity to deliver true autonomous vehicles and thus save tens of thousands of lives and avoid hundreds of thousands of injuries.
There is a solution, however. It is to replace most of the AV public shadow driving as the primary validation process with complete simulation. Such a solution would be designed from full systems engineering, driven by a requirements definition and design process, augmented by an end-state scenario.
A proposal for proper simulation
The systems currently in use by the auto industry are inadequate, and nowhere near aerospace complexity, or FAA Level D competency. They do not have proper real-time architectures. The models being used for vehicles, tires and roads, are not precise enough, especially in degraded conditions. The AI will appear to have learned properly, when in actuality it has not. This is often not discovered until analogous real-world scenarios are experienced—and in the process, expose critical timing and execution gaps.
Keep in mind that the benign scenarios being run now will not encounter these problems. It is not until you run complex or time-critical scenarios that the performance envelope of the vehicle, tire or road models is reached. That’s typically when the problems start.
Driver-in-the-loop (DiL) simulators without full motion systems can cause a significant level of false confidence. A motion system device used in manned simulators would be used for the simulated autonomous driving. Motion systems permit evaluation of motion sickness and passengers’ feeling of comfort and trust with the autonomous vehicle management. In addition to having a proper motion system, simulation issues can be resolved by leveraging aerospace/DoD/FAA simulation technology, practices and test methodologies. Especially useful are those relating to DoD urban war games, which are directly analogous to complex driving scenarios, as well as those employing proper model and real-time fidelity.
Data methodology is key
Utilization of Agile processes, or a bottom-up engineering approach, is an inefficient if not debilitating process when it is employed in complex systems. Too much time is wasted time, by not developing components in parallel, as well as not defining and building to the most difficult scenarios up front. All of which require simulation to execute, including the immediate and endless repletion of these scenarios.
If the agile approach is taken the time lost will be extreme, and historically the less complex elements will be completed, leaving the more complex configurations for “later”. Also, flawed design assumptions will not likely be exposed until the most complex and difficult scenarios are encountered. That will usually drive a design and execution change that will need to be employed for many benign scenarios—with significant rework as a result.
Two popular and very flawed terms—“edge case” and “corner case”—are widely used today to describe accident scenarios. Accident scenarios are like any other scenario, but with outcomes that no one desires. A true edge or corner case is a scenario that should not happen in any possible scenario—such as asking a search engine to find an image of a cat, and then receiving the image of a garbage can. Engineers typically will not search out all the possible accident scenarios because they are deemed on the edges or corners of the core set. This then gives people an excuse to do the required due diligence.
The purpose of the simulation should be focused on presenting the AI stack the same digital representations it might experience with the same level of items to be discriminated, at the same input rate, with the same level of ambiguity to allow for determining where the AI stack has issues in making proper decision. The very hardest of these data sets to achieve have been called “edge” or “corner” cases; however, these are the key cases which define success for the AI stacks decision process. Defining those cases, along with defining the desired results of those cases, requires a structured, recursive and manageable data methodology.
End-state Scenario Matrix
Beyond providing the scenario data to affect the systems engineering approach mentioned above is the need for all parties—including policy makers, validators, insurance companies and manufacturers—to know what “done” looks like as early as possible. The scenario set would indeed be comprehensive, with support for real time variations, which will confirm any AI perception errors. The effort to do this needs to be just as significant as the definition of the integration and systemic models for the simulation.
From legitimate geofencing to legitimate SAE Level 4 and 5, this test set would need to be informed from a multitude of data sources and domains. It would need to reflect the highest level of due diligence the global vehicle-development community can muster. It must ensure that the requisite levels of safety are attained and proven. Finally, it would have to be mapped to and synced with the simulation/simulator system noted above.
The industry will never save nearly the lives it hopes to with Level 4 vehicles, nor will it get close to having a true autonomous vehicle, until the current paradigm of AV testing is changed.
Author: Michael DeKort
Source: SAE Aerospace Engineering Magazine
在一些關鍵的開發(fā)與駕駛場景中，系統(tǒng)根本不可能留給“影子駕駛員”充裕的反應時間，以充分了解當下的駕駛環(huán)境，重新取得對車輛的有效控制，并保證安全。
福特和Argo AI正在開發(fā)一套虛擬駕駛員系統(tǒng)，支持汽車制造商及其他潛在公司的自動駕駛汽車研發(fā)。
如圖所示，大多數(shù)自動駕駛汽車開發(fā)商均無法履行之前的承諾，即在有限范圍內推出真正具備SAE L4-L5級自動駕駛功能的自動駕駛汽車。(數(shù)據(jù)來源：Eric Paul Dennis/Center for Automotive Research)