1
November 2009
Table of Contents:
Introduction ....................................................................................... 1
Importance of Desktop Testing ............................................................. 2
Test Tool Development ........................................................................ 2
Hardware Evaluation Test Development ....................................................... 2
Hardware and Software design evaluation tools .................................. 2
ETD ............................................................................................... 3
Software Image Development ................................................................... 3
Types of Testing ................................................................................. 3
System Board Validation Testing ............................................................... 3
Platform Validation Testing ....................................................................... 4
Mechanical Validation Testing .................................................................. 4
Module Functional Testing ........................................................................ 5
System Integration Testing ........................................................................ 6
Performance Testing ................................................................................ 6
Desktop Certifications .............................................................................. 7
Tested Configurations ......................................................................... 7
Hours of Testing ................................................................................. 8
Observations during Testing ................................................................ 8
On-going Qualifications ...................................................................... 8
For More Information .......................................................................... 9
Introduction
Testing is an important part of the value supplied by HP to our customers. Many companies can
assemble personal computers. HP has the engineering expertise and resources to perform the tens of
thousands of hours testing for every desktop platform we develop. This testing provides our customers
with peace of mind, knowing that hundreds of different tests have been performed to provide a stable
and reliable experience.
2
In our quest to continuously improve the customer experience, HP uses a disciplined and methodical
development and test process. We verify that each new desktop platform meets or exceeds HP quality
standards for structural integrity, reliability, and compatibility. This white paper describes the various
forms of testing performed by HP engineering when developing a new HP Business Desktop PC
platform.
Importance of Desktop Testing
Business Desktop PCs represent a large portion of the worldwide computer shipments and are often the
workhorses of business. As such, it is critical that they be durable and able to support a vast array of
operating systems, languages, applications, and add-in options. Helping to ensure this compatibility
and flexibility requires extensive, well-engineered testing.
In this industry, quality is a differentiator. Business customers recognize the value of a more reliable PC,
and that is a major factor in their purchase selection. The time and resources we invest in our testing
yield benefits to them and to HP. A well-tested product can lead to a more efficient factory, lower costs,
fewer failures in the field, and more satisfied customers.
The PC business is a dynamic industry in which new technologies are created and adopted. It is
imperative that we have a robust test process that can react to new technologies, verify their quality,
integrate them into our product lines, and get them to market quickly.
The following sections provide more details of the HP test tool development and capability and the
types of testing performed. These sections also provide a closer look at the investment of time in our
test process and how we track the results of tests conducted on HP Business Desktop PCs.
Test Tool Development
An essential element of the HP test process is our attention and dedication to continuously improving
the tests and processes used during product development. Two such areas of concentration are
Hardware Evaluation Test Development and Software Image Development. Hardware Evaluation Test
Development designs tests to help ensure that hardware components conform to published functional
engineering design specifications. Software Image Development architects image databases,
processes, and tools used to test and deliver software images on business desktop products.
Hardware Evaluation Test Development
HP’s Evaluation Test Development (ETD) team designs and builds specialized hardware and software
used to ensure that PCs meet HP’s high standards for design quality and functionality. These tools test
the hardware components for conformance to published functional engineering design specifications.
This engineering organization, established in 1988, is shared across other business units at HP. The
ETD tools have evolved over years of development, and are available to all HP design teams,
manufacturing sites, development partners, and HP field engineers for use exclusively on HP products.
This test capability is unique in the PC industry and enables HP to thoroughly evaluate computer
designs from firmware to silicon, as well as improve quality and reduce manufacturing costs.
Hardware and Software design evaluation tools
ETD validation tools include both low level functional tests which target particular subsystems in
isolation, and system stress tests which test nearly every subsystem in combination in order to discover
any undesirable interactions. The test tools are designed to support identification, isolation, and
debugging of issues so that the final system designs are verified to meet HP’s required functional,
electrical and mechanical standards.
3
ETD
Hardware and software tools are used to validate the following PC subsystems:
Microprocessors and chipsets, including cache memory, interrupts, and timers
System memory
Internal expansion interfaces: PCI, PCI-Express, mini-PCI-Express
Hard Drive and Removable Storage (Blu-ray, DVD, CD-RW) interfaces
Network adapters, Bluetooth and 802.11a/b/n Wireless adapters and Modems
Parallel, Serial, USB 1.1/2.0 and Firewire (IEEE 1394) ports
Trusted Platform (TPM) hardware
Flash memory readers / devices
PC Card / Express Card interfaces
Software Image Development
Software Image Development is optimized for time-to-market delivery, quality, and release integrity. A
dedicated tool development team was formed in 1998. Their charter was to design software image
databases, processes, and tools that encompass the following elements:
Test ManagementDatabase tools enable test teams to generate and manage test plans and test
procedures in a centralized, repeatable environment with assurances that the test configurations
conform to product definition and requirements.
Comprehensive test procedures within test plans contain pass/fail criteria that can be
leveraged by any workgroup. Rigorous database rules help verify that each test procedure
criteria is executed only when certain conditions are met.
Test configurations are tracked at the hardware revision level of each device, including
firmware (FW)/BIOS revisions, and at the software versions level for each component used
during software image download and test.
Automated Test EnvironmentThis enables test teams to download a products software image,
automatically install the software to replicate the manner in which a typical customer might install
it, automatically run test scripts, and then log results into the database. For a given product
configuration, hundreds of different software downloads, installations, and tests can be logged
with minimal time and effort.
Quality AssuranceSoftware image development databases and tools enable a complete, end-to-
end, repeatable image test process. All HW, FW, and SW changes are systematically managed.
System checks are in place to help make sure that what gets built in any factory worldwide and
sent to the customer is the same as what was tested and qualified in engineering.
Types of Testing
HP executes a vast array of testing on Business Desktop PCs to help ensure that systems meet important
industry standards. Multiple engineering teams conduct various types of testing throughout the product
development cycle. This helps mitigate risk by providing needed checks and balances and it helps to
ensure a high HP standard of quality. These tests include, but are not limited to:
System Board Validation Testing
This testing focuses on the signal quality of high-speed interfaces, power delivery to system
components, and ACPI power management. The System board is put through stress testing to help
ensure that it will operate normally while running under the range of operating conditions. Testing
includes:
4
Signal Integrity TestingThis helps ensure proper signal quality on all major interfaces on the
system board including CPU bus, memory bus, system clocks, USB (universal serial bus), SATA
(storage) bus, and all expansion buses.
Voltage Integrity TestingThis helps ensure that voltage regulators on the system board and in the
power supply remain within HP specifications under worst-case stress conditions. Additionally,
voltage stress testing is conducted and measured on the various supported memory and processor
configurations.
Power Management TestingThis helps ensure that the system will reliably enter and exit all
supported low-power states, including sleep (S3), hibernation (S4), and powered off (S5) . In
addition, power cycle testing is conducted to simulate many years worth of power-up/power-down
usage with thousands of consecutive no-fail cycles.
Fan controlThis helps ensure that the system fan(s) keep the system within the required thermal
window while minimizing fan acoustic levels.
Platform Validation Testing
This testing focuses on chipset validation, new architectural design features, the system board,
processor, system memory, system BIOS, and system-level hardware interoperability to help ensure
platform component stability prior to incorporation into an integrated desktop solution. Testing
includes:
System BIOS TestingThis is comprehensive validation and qualification of the system BIOS.
Testing includes BIOS calls and functionality, supported BIOS features, and expected BIOS
interoperability with the platform hardware and software.
System BoardThis is comprehensive validation and qualification of the main Printed Circuit
Board (PCB) and Printed Circuit Assembly (PCA) including all components and interfaces. Testing
includes all chipset features and functions, all PCA subsystems (i.e., CPU, memory, graphics,
audio, expansion buses, network adapter, USB, storage (SATA/RAID), input/output interfaces,
etc), and associated hardware components.
Processor QualificationThis is comprehensive validation and qualification of each unique
processor type, including all processor frequencies and revisions/steppings. Processor
qualifications are executed on every supported platform.
System Memory QualificationThis is comprehensive validation and qualification of each unique
memory module type. System memory module qualifications are executed on every supported
platform.
Mechanical Validation Testing
This testing focuses on environmental and Industrial Design factors for the chassis, power supply, and
system board. Mechanical testing includes form, fit and function tests; paint finish; durability tests; and
security tests to help ensure that the product can be assembled correctly. Additional testing includes:
Shock and Vibration TestingUnits are subjected to random vibration and shock to help ensure
that the system functions correctly in harsh environments.
Packaging TestingBoxed systems are subjected to resonance, random vibration, and multiple
drops from various heights to help ensure that the packaging can withstand rough handling and
protect the system.
Thermal TestingEach system board, power supply, and integrated subsystem (memory, graphics,
audio, NIC, etc.) is tested in elevated ambient, maximum power, and minimum power situations to
help ensure that no component temperature specifications are exceeded.
Environmental TestingTesting is conducted at various temperatures, altitudes, and humidity
settings for extended periods of time to help ensure that the system design is reliable in many
environments.
5
Acoustics TestingThis testing helps ensure quiet, reliable products. Acoustic evaluations are
performed with typical and power user applications: at idle, when the hard drive is being
accessed, and at tonal resonances (noises that could be perceived as adverse by the customer).
Safety/AgencySystems undergo extensive electromagnetic compatibility and safety testing to
help ensure that the product is safe to operate, is environmentally friendly, meets European
ergonomic fatigue standards and other various local regulations, does not interfere with radio or
TV reception, and is free from interference by external electrical effects.
Power Supply TestingThis testing focuses on the power supply; it includes full electrical
verification, voltage regulation, output ripple and noise, stability, input and output transients, stress
margins for voltage and current, high-voltage isolation, electro-static discharge, and harmonic
current emissions.
Reliability TestingSystem and component level testing and modeling is geared towards
mitigating risk such as Mean Time Between Failures (MTBF), Strife, High and Low temperature
power cycling, long-term reliability, cold/warm boot, AC power on/off, and various other tests.
Module Functional Testing
This testing focuses on a wide variety of module subsystems to help ensure HP Business Desktop PC
standards are met prior to integration into a desktop solution. Module functional testing is conducted
on the graphics, audio, network controller, hard drive, optical drive, and input/output device
subsystems, as well as on numerous software applications. Testing includes, but is not limited to:
Electric Signal Integrity TestingThis testing helps ensure that the proper signal quality is met on
all interfaces and layout routes through the specific module components.
Mechanical TestingEnvironmental and design ID testing helps to ensure the proper fit and
function and temperature variance tolerance for a module.
Electro-Static Discharge TestingThis testing helps ensure that a module component meets
specifications for continued operation after being subjected to electro-static discharges at various
points on the device.
Electro-Magnetic Conductance TestingThis testing helps ensure that a module is not susceptible
to and does not emit interfering radio waves.
Power Management TestingThis testing helps ensure that a module goes properly into and out of
defined low-power states.
Software Driver TestingThis testing helps ensure that the software drivers for a module perform as
required, deliver expected performance and features, and integrate into HP images to meet
customer needs and latest technology.
Software Application TestingAs part of the total PC solution, HP offers select software
applications, including hardware-enabling applications (such as DVD playback) and productivity
type applications (such as Microsoft Office). Testing is conducted to help ensure that these HP-
specific and industry standard applications offer the required features and integrate into HP
images.
Video Timing TestingThis testing helps ensure proper graphics controller output for image display
on a variety of monitors by complying with video timing specifications for all supported resolutions
and refresh rates. Additionally, commercial HP monitors are tested in combination with the HP
graphics controller solutions to better ensure proper image display.
Network Interface TestingThis testing helps ensure proper functionality of the network interface.
Testing includes, but is not limited to, the following: bit-error-rate, auto-negotiation, networking
protocol testing, and CRC (Cyclic Redundancy Check). Testing also verifies that packet loss
specifications are not exceeded and that network packet patterns are as designed.
6
System Integration Testing
This testing focuses on the system compatibility and integration test on overall quality of the hardware
and software module components assembled into a complete desktop solution. Testing is conducted
across a wide variety of configurations.
Image Integration Test helps verify:
integration of pre-installed operating systems and software in all supported languages
(currently 9 operating systems, 51 regions and 35 languages).
operation of the system after using the HPRM (HP Recovery Manager) solutions provided with
our platforms
System Compatibility Test helps verify:
hardware and software compatibility of supported modules on new products. (combinations of
interdependent modules)
functionality of integrated system hardware and its compatibility with alternate operating
systems (Linux, Novell, etc. if supported)
system hardware and pre-installed software compatibility with 3rd party hardware and
software products using primary and alternate operating systems. (over 500 3rd-party
products)
compatibility of the systems with commonly used accessibility products
physical form and fit of standard and aftermarket modules on new products
interoperability of HP Business Desktop PCs with other HP products
compatibility by participating in variety of plugfests sponsored by HP and other
component vendors
Localization Test
represents interests of geographic areas around the world to the development team
execute localization test plans to check for
o accurate translations
o appropriate language
o correct user interface
functionally tests translated versions of products
Module Integration Test
provides acceptance testing to help ensure selection of high quality options
verifies the functionality of supported peripherals and applications in a wide variety of
combinations of modules and platforms
Performance Testing
HP runs a number of popular, industry-standard and press-review benchmark tests to measure the
overall performance of new Business Desktop PCs. These tests target unique sub-systems as well as full
systems. Benchmarks are used to monitor the performance impact of design and configuration
changes. Representative system configurations are tested through several design phases up to the
production level. HP follows each benchmarks guidelines for preparing/configuring the operating
system and other configuration settings prior to testing. Through these benchmark measurements, we
can determine how well the systems performance in each design phase meets our product goals.
As benchmark updates are released, HP quickly transitions to using the latest versions. Archives for
previous test versions and the associated results are maintained for desktop products.
7
Benchmarks used include, but are not limited to the following:
Business Winstone
Multimedia Content Creation Winstone
SYSMark
PCMarkPRO
3DMark
WebMark
AquaMark3
Glaze3D
PassMark Performance
MS Velocity
Desktop Certifications
Certification testing is a unique type of testing that supports the goal of obtaining certain shipping and
logo rights for Business Desktop PC solutions. Specific certification tests must be passed in order to
successfully gain these rights. Testing is conducted prior to submission to resolve any issues that would
impede passing. These types of certification testing include:
Windows Hardware Quality Lab (WHQL) Testing and CertificationPassing this required
operating-system-level testing and certification allows HP to display the Designed for Windows
logo on a product. A Microsoft-controlled test suite is used as a validation tool prior to system
submission to Microsoft for certification.
Section 508 Testing and CertificationSection 508 is set of rules developed, maintained, and
enforced by the Access Board and General Services Administration (GSA) that helps to ensure that
electronic and information technologies provide or support certain accessibility features or
products for people with disabilities. For Section 508, HP conducts audit-level testing to verify
conformance. HP publishes the Voluntary Product Accessibility Template (VPAT) on the HP
Accessibility Web site for each product.
Linux Testing and CertificationSelect HP Business PCs are currently certified for Novell SUSE
LINUX and/or Red Hat Enterprise Linux. Additional or different Linux providers may be supported
on future products. Testing is conducted prior to submission using available industry tools and
utilities with the goal of helping to ensure a passing Linux configuration.
Novell Testing and CertificationSelect HP Business PCs are submitted for certification for use with
the Novell Netware Client.
Tested Configurations
With the multitude of components included in a PC, and the number of operating systems,
applications, and options offered, testing of every permutation becomes impossible. However, using
disciplined test-engineering methods, HP helps ensure good test coverage. Detailed understanding of
component interdependencies and testing technologies allows HP to create a set of test configurations
that are representative of many of our customers usage.
The hundreds of systems we use in the test process and the numerous well-controlled configurations we
build during development help maintain our high level of quality.
8
Hours of Testing
HP Business Desktop PCs are subjected to on average over 113,000 machine hours of testing. This is
strictly test hours and does not include time spent troubleshooting or reviewing design specifications.
The number of test hours varies as new system chipsets and system boards, chasses, video, audio and
other module components, and/or operating systems are incorporated into a desktop design. Table 1
shows the approximate test percentage allocation to qualify a business desktop PC with typical
architectural changes.
Table 1: Test coverage per area
Test area
Estimated Percentage of
total test hours
System Board Validation
17%
Platform Validation
28%
Mechanical Validation
28%
Module Functional Testing
12%
System Integration, Performance and
Certification Testing
15%
Observations during Testing
The result of this exhaustive testing is a detailed list of observations made by test engineers. The
observations are reported when the behavior of the PC is different than what is expected, based on
design specifications, product requirements, and an understanding of customer needs. These
observations are reviewed by the responsible HP staff and management team using robust database
tools and tracking processes, then HP determines the disposition for each observation.
After development is complete and the product is shipped to the end user, customer satisfaction is
closely monitored and customer-reported problems, requests, and comments are tracked. HP attempts
to resolve customer issues, and information is fed back to the development and test teams, thus
completing this closed-loop process. In this way, we help ensure that the Total Customer Experience
(TCE) is continuously improving.
On-going Qualifications
Testing does not stop after a new Business Desktop product completes initial testing and the new
Business Desktop computer is shipped. Rather, testing continues throughout the product lifecycle to
incorporate product updates including BIOS, module HW and SW, along with customer support
feedback. Additionally, test process improvements are constantly folded into the on-going qualification
process.
In conclusion, the HP Business Desktop product development team values testing and believes that the
HP customer values testing. We feel strongly that HPs level of testing leads to a higher quality product,
which benefits our customers. A Business Desktop PC is more than just an assembly of various modules
and components; it is the result of a concerted development and integration effort, with checks and
balances throughout the qualification cycle. It is a PC that the product development team is proud to
have carry the HP brand.
For More Information
Additional information can be obtained through the HP Web site at
© 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without
notice. The only warranties for HP products and services are set forth in the express warranty statements
accompanying such products and services. Nothing herein should be construed as constituting an additional warranty.
HP shall not be liable for technical or editorial errors or omissions contained herein.
372011-003, 11/2009
  • Page 1 1
  • Page 2 2
  • Page 3 3
  • Page 4 4
  • Page 5 5
  • Page 6 6
  • Page 7 7
  • Page 8 8
  • Page 9 9
  • Page 10 10

Ask a question and I''ll find the answer in the document

Finding information in a document is now easier with AI