ART Automated Testing Solution

ART Automated Testing Solution
for IVI HMI (In-Vehicle Infotainment System)

By replacing traditional manpower with AI intelligence and improving testing efficiency through automation, ART not only overcomes the time constraints of traditional manual testing, meeting the needs of continuous, 24/7 testing, but also significantly reduces human error, making testing accuracy more stable. It assists automotive industry customers in ensuring product quality and performance in a fast, efficient, and comprehensive manner, thereby creating a superior user experience.

Solution Inquiry

Current Status Analysis of the Intelligent Cockpit Application Ecosystem (IVI HMI)

With the rapid advancement of automotive technology, we have officially entered the era of the smart cockpit. Centered around Software Defined Vehicles (SDV), the smart cockpit is a next-generation in-vehicle experience platform that integrates Human-Machine Interfaces (HMI), In-Vehicle Infotainment (IVI), AI, sensing technologies, the Internet of Vehicles (IoV), and cloud services. Compared to traditional in-car systems, the core value of the smart cockpit has evolved drastically from pure ‘hardware functionality’ to a combination of digital experience, software services, and ecosystem integration. The competitive paradigm for automakers is shifting progressively from automotive manufacturing to mobile platforms, and ultimately toward application service ecosystems.

Under current trends, User Experience (UX) has inevitably become a critical indicator for major automakers to establish brand differentiation and stand out. However, when faced with such complex system integration, traditional manual evaluation methods are proving insufficient, leaving five major challenges that must be confronted and overcome:

  • Proliferation of Models and Configurations: Testing across varying vehicle models, screen sizes, and diverse hardware setups requires individual validation, resulting in high maintenance costs.
  • Massive Test Scenarios: Combining different features with diverse usage patterns generates highly complex use cases.
  • Prohibitive Regression Testing Costs: The continuous verification process demands a massive investment of labor and time, placing a heavy burden on automakers.
  • Inconsistent Testing Standards: Manual testing struggles to guarantee identical operating standards every time, making the process prone to variance and deviation.
  • Difficulties in Quantifying UX Behaviors: Users’ subjective perceptions are notoriously difficult to convert into objective, standardized data metrics.

In addition, testing bottlenecks are a challenge that automakers must face. Manual testing cannot fully cover all possible usage scenarios, and it is difficult to reliably reproduce and debug low-probability issues that may affect safety or quality.

Allion ART Automated Test Solution

To help automotive clients break through the bottlenecks of manual testing, Allion has developed the proprietary ART (Allion Robot Testing) automation solution. This comprehensive automation verification platform, engineered specifically for in-vehicle systems, is designed to deliver highly efficient and stable automated testing and evaluation within a real in-car environment. It not only drastically minimizes human testing errors but also brings you the following tangible benefits:

  • Comprehensive Elevation: Achieve a complete upgrade in testing accuracy, quality consistency, and verification efficiency.
  • End-to-End Simulations: Realize comprehensive end-to-end user scenario simulations.
  • AI-Driven Pass/Fail Judgment: Automatically determine test results utilizing advanced AI computer vision.
  •  Versatile Test Scenario Support: Support diverse testing environments, including robotic arm operations, voice assistants, and mobile device controls.
  • Deep IVI & Control System Integration: Deliver high-level integration across various in-vehicle control systems and IVI applications.
  • Programming-Free Test Creation: Build and deploy test scenarios rapidly with zero programming background required.
  • Scalable UX Verification: Provide scalable user experience validation capabilities that span across multiple vehicle models and geographical markets.
  • Accelerated Time-to-Market: Speed up product launches and effectively reduce manual testing labor costs through repeatable automated verification.

To help clients effectively address the validation demands of the smart cockpit at various ecosystem stages, Allion ART delivers a comprehensive, cross-phase automated testing platform solution:

Tier 1 Supplier “Component-Level” Validation

Conduct deep functional verification on the Test Bench to ensure the stability of individual systems and modules during the early stages of hardware development and software integration. ART’s standard testing platform is perfectly suited for this phase.

The Test Bench is a tailored solution developed by Allion specifically for Tier 1 suppliers, engineered for testing automotive electronics components such as In-Vehicle Infotainment (IVI) systems and clusters. Through the automated testing capabilities of the Test Bench, you can execute functional, interoperability, and other related test items to effectively guarantee product quality and performance.

ART Automated Testing Solution

Automaker (OEM) “Vehicle-Level” Testing

As projects advance to the full-vehicle verification phase, we have meticulously developed our “Vehicle-Level Testing Solution” for various system tests within the smart cockpit—including In-Vehicle Infotainment (IVI), climate control, steering wheel, and rear-seat entertainment systems. This solution assists OEMs in executing diverse scenario tests in a manner that closely replicates real-world vehicle operation environments, guaranteeing the stability, reliability, and interoperability of all in-car systems.

To satisfy the rigorous demands of full-vehicle testing, Allion engineered the “In-Car Robotic Arm Base” equipped with an adjustable mount. This base can be height-adjusted according to the specific vehicle models of different automakers, allowing the robotic arm to extend directly into the authentic cabin environment for real-car testing.

This phase can be integrated with customized fixtures and sensors to execute the following non-display control functions, achieving a comprehensive, automated audit through seamless hardware-software synergy.

Fixture

  • Operate the buttons on the side of the seat (side window, seat adjustment buttons).
  • Operating the stalk controls elements (turn signals, windshield wipers, etc.)
  • Supports lifting and placing the phone for wireless charging evaluation.
  • Automated operation of precision control components inside the vehicle

Sensor

  • Measure the distance the seat moves and the tilt angle.
  • Measuring the airflow intensity and temperature at the air conditioner outlet.
  • Detect changes in ambient light inside the vehicle
  • Measuring wireless charging coverage

Simulation of key user application scenarios
to fully meet the application requirements of in-vehicle systems.

With extensive experience in automotive system testing, Allion has categorized several critical user scenarios and common, high-impact market complaints associated with in-vehicle systems. These issues typically manifest only within specific critical scenes, requiring extended operational runtimes or high-frequency triggering. Consequently, traditional manual detection struggles to identify them, and rooting them out completely is virtually impossible.

By utilizing the Allion ART Automation Test Solution for IVI HMI to build scripts for these specific user scenarios, engineering teams can repeatedly simulate user operations to validate critical use cases. This approach effectively covers the blind spots and limitations of manual testing, significantly mitigating the risk of market complaints and relieving debugging pressure.

3 Core User Scenarios

Practical Applications

Users pair their mobile phones with the car’s infotainment system via Bluetooth to make calls while in the vehicle.

Market pain points (Manual testing)

  • Smartphones and the In-Vehicle Infotainment (IVI) system occasionally fail to pair successfully.
  • After turning off the engine and leaving the vehicle, the Bluetooth fails to automatically reconnect upon returning to the car.
  • The other party’s voice cannot be heard clearly during a hands-free phone call.
  • The system fails to synchronize the smartphone’s contact list successfully.

Practical Applications

Users can check the sound quality of music playback or calls while in the car; and operate the system via voice assistant while driving.

Market pain points (Manual testing)

  • Music streaming suffers from intermittent stuttering or noticeable background static.
  • In-call audio is choppy, resulting in fragmented speech and clipped words.
  • The system suffers from poor voice recognition rates, failing to execute user commands correctly.
  • Inconsistent or failing audio focus transitions between navigation prompts, music playback, and active voice calls.

Practical Applications

System responsiveness and smoothness during user touch operations

Market pain points (Manual testing)

  • The user interface lacks smoothness, touch inputs suffer from noticeable latency, and applications experience prolonged loading periods.
  • After starting the vehicle, it takes dozens of seconds or even up to a minute for the navigation map to open or for music playback to initialize.
ART Automated Testing Solution

Core System Architecture

The core architecture of the ART Automation Solution is comprised of the following proprietary technology stack. Through deep hardware-software integration, it delivers complete, high-fidelity automation for smart cockpit user scenario simulations. Below is the schematic illustrating the overall ART testbench hardware architecture and connection topology.

ART Automated Testing Solution

ART uses a high-precision robotic arm to simulate the actual user’s operation of the in-vehicle display screen, physical buttons and various control interfaces, ensuring that each test action is highly repeatable and accurate.

Supported operation types

Single-finger operation

Tap, Long Press, Double Click, Swipe, Handwrite, Multi – Point Click, Number Keys

Two-finger operation

Zoom In/Out, Rotate, Knob, Two-finger long press

The intelligent wizard guides users through various configurations easily and rapidly, significantly lowering the technical barrier to creating test scenarios. Even without a programming background, operator can quickly complete operational setups for the robotic arm.

ART Automated Testing Solution

After quickly recording robotic arm movements using the Smart Wizard module, creating a complete Test Case requires assembling them via the ‘Script Editor.’ Allion has modularized these robotic movements, allowing users to rapidly configure robotic arm actions, smartphone operations, and system parameters without writing any additional test scripts.

ART Automated Testing Solution

By comparing captured images with an image database, the system identifies objects, visuals, or text and executes corresponding responses. This vision recognition system is particularly suited for tasks that exceed the capabilities of human vision, such as long-term monitoring and identifying images that persist for only a few milliseconds.

Validation Capabilities

ART Automated Testing Solution

Audio analysis supports the following functions:

  • Speech to text
  • Audio Distortion Detection
  • Audio Volume Check
  • Audio Volume Compare
  • Voice Generator and Recorder
ART Automated Testing Solution

ART allows for smartphone operation (Android & iOS) via DCS, realistically simulating the interaction between a driver’s phone and the vehicle’s infotainment system. Functions include Bluetooth connectivity, music playback, phone calls, and voice assistant functionality.

ART Automated Testing Solution

Throughout the test execution process, ART provides complete recording and reporting functions to ensure that all test results are traceable and reviewable, including images and audio files.

ART Automated Testing Solution
ART Automated Testing Solution

Value-Added Services & Extensions

In addition to the standard package, Allion offers a diverse range of value-added services and optional extensions to satisfy the customized requirements of various IVI HMI testing scenarios.

Value-Added Services

  • Consulting Services: Delivering test scenario planning, along with test script design and authoring, to assist clients in establishing optimal testing strategies.
  • Custom Development: Providing bespoke functional module development and system integration tailored to the specific and unique requirements of clients.
  • Staff Augmentation: Offering professional on-site engineering placement services, where Allion’s expert engineers assist clients directly in executing test operations.
  • Equipment Leasing: Providing flexible leasing options for the ART system and its associated peripherals, enabling clients to fulfill testing missions with maximum agility.

Optional Extensions

Extension Modules
Function Description
Signal control box Integrated signal control, controlling multiple channels of signals such as power supply, voltage, current, and frequency.
Bluetooth packet capture
and integration
During testing, the Ellisys Vanguard Bluetooth analysis system (hardware/software sold separately) can be used to monitor and capture Bluetooth packets, and other Bluetooth tools can also be customized to support it.
Fixture development Customized fixtures include seat adjustment, wireless charging, air conditioning vents, steering wheel light/wiper stalk controls, etc.
HIL Integrated Control Supports CANoe /CANalyzer communication, and controls CAN signals through a Vector COM Server.
USB port switching
(USB Port Switch)
Supports USB-C 3.1 and USB-A 3.1, with one-to-four switching control.
High frame rate long recording 120fps camera recording, capable of recording the entire test process, supports up to 2TB of storage.
Contextual video surveillance 2MP/1920×1080/30fps full-process monitoring and recording, passively recording the test scenario and environment.
Safety Enclosure
with Optical Shielding
Provide security protection for the testing environment and establish a virtual security isolation barrier.

加值服務與擴充功能

除了標準套件之外,百佳泰還可提供多元的加值服務與不同的選配擴充功能,進而滿足不同 IVI HMI 測試情境的客製化需求。

加值服務

  • 顧問服務:提供測試情境的規劃、測試腳本的設計與編寫,協助客戶建立最佳測試策略。
  • 客製化開發:針對客戶的特殊需求,提供客製化功能模組開發與整合。
  • 人員派遣:包含人力派遣服務,由百佳泰專業工程師協助客戶現場執行測試任務。
  • 設備租賃:提供 ART 系統及相關配件的租賃服務,協助客戶彈性完成測試任務。

擴充功能

ART Automated Testing Solution