Stereo Assessment: Part 1 - Accuracy

Roy Harris

  The following represent alternatives for evaluating stereo systems:

     I  Intrinsic (relating to the 'sound' of stereo systems)

        A  Accuracy/Neutrality

        B  Musicality

        C  Sonic Preferences

      II  Extrinsic (non sonic - the effects of listening or ownership)

        A  Satisfaction of Psychological Needs

        B  Creation Of Physiological States

      The audio equation includes the following variables:

     The Medium,   The Messenger,   The Message,   The Receiver

     (equipment)        (recording)           (music)             (listener)

Accuracy is based upon the (philosophical) premise that the purpose of a stereo system is to pass 'information' from component A to component B with minimal loss of content. This assumption, while accepted as 'gospel' by many in the audiophile community and by most audio professionals, can only be justified by opinion, not facts, albeit the opinion of experts. In theory, it is a valid approach for assessing the merits of stereo systems.

The selection of this criterion for evaluating stereo systems is not based upon logic. This criterion is not intrinsically correct and the others intrinsically wrong.  There is no rationale that can negate the validity of the other above-mentioned criteria.

What about other criteria that are either message or listener based?

The potential of the message is maximized when music is perceived as sounding natural and realistic (musicality), like the real thing, one's experience of live music.

The third criterion, tersely expressed, is 'please the listener', the nature of which depends upon subjective factors, such as personality (psychological needs and preference for physiological states) and sonic preferences.

Thus, while accuracy is one of several criteria possibly useful for evaluating stereo systems, it is no more or less valid than any other.

Here is a definition of accuracy provided by Webster's New Universal Unabridged Dictionary, Second Edition, 1983,  page 14:

  1. An exact conformity to truth or to a standard or a model

  2. Freedom from mistake; correctness, precision

With respect to an audio system, the truth is the signal (input). Conformity to the truth means that the reproduction of the signal (output) is achieved without error.

Here are three concepts which may help to assess the accuracy of stereo systems:

  Reference audio system

  Fidelity of reproduction

  Quantitative index

Currently an accurate (perfect) stereo system does not exist and no reference stereo system has been cited in any audiophile publication. Therefore, it is not feasible to expect to configure such a system to serve as a paragon for comparison to all other stereo systems.

Evaluation of a stereo system with respect to fidelity of reproduction requires verification of what is recorded, i.e., musical and non-musical information (content), the position of the performers in space (location) and the pitch, timbre, harmonics, dynamics and tempo of instruments (the 'sound'). The procedure for assessing fidelity of reproduction is not scientific. The methodology is imprecise, lacking controls, and the results are uncertain in nature and inconclusive. Let's see why this is so.

The first step is verifying the 'ingredients' - musical and non-musical sound sources. To pass this test, it is necessary to 'hear' what has been recorded. Liner notes and data from the recording company can identify what instruments and miscellaneous 'noises' have been recorded. The process seems simple. Just listen and compare what you hear to what you should hear. In fact, the process is not simple. It is an exercise in speculation and error. First there is the error of inclusion - hearing something that is not on the recording. The listener, the recording or the stereo system may be responsible for this error. If in fact the recording or the listener is the cause, the stereo system will have been misjudged.

From here on, verification is almost an insurmountable task.

The second task is specifying where musical and non-musical information resides within the soundstage. This phase is the test for location. While the recording company may provide a 'map' showing the precise position of each performer and the appropriate distances between them, trying to corroborate this data is a 'challenge', given the condition of an intangible soundstage.  How does one confirm concrete spatial relationships, microphone placement and depth of field when direct measurement is almost impossible and the dimensions of a listener's room differ from those of the recording studio?

The third test is the verification of timbre, pitch, harmonics, dynamics and tempo.

Can the recording company provide information about the above-mentioned musical qualities, which can serve as a reference for comparison to one's aural experiences? Some of the data provided by the recording company is qualitative, e.g., the description of timbre, while the balance is quantitative in nature, i.e., in units of dB or Hz.  Much of the quantitative data cannot be easily verified if at all by a non-professional, and the vocabulary used to describe timbre, subjectively based, is also very difficult to verify.

The results of testing for fidelity of reproduction are not definitive, possibly erroneous, unreliable and of questionable utility. The attempt to demonstrate a stereo system's fidelity of reproduction or lack thereof is laudable. However this process is not very effective. Some objectivity is needed to analyze the affects of the recording, the listener and the stereo system upon the evaluation of stereo systems. An objective measure of the accuracy of a stereo system does not exist. It would be almost impossible to design and implement such an index based upon the concept of accuracy as input=output (fidelity of reproduction). Instead, a simple error measurement technique in one of several forms based upon the frequency response curve, as indicated below, may be a suitable substitute:

Generate a frequency response curve, starting at a specific lower limit, e.g., the 3 dB point of a speaker and extending to some upper limit. Compute the sum of absolute deviations between the 0 dB point line, i.e., the x-axis, and the actual readings, using either specific frequencies or all frequencies within the range.

Alternatively, one may compute the sum of squared deviations in lieu of the sum of absolute deviations. In order to compare frequency restricted with extended frequency stereo systems, it is necessary to derive an average rating which can be accomplished by dividing the chosen 'index' either by the number of 'points' selected (a discrete function) or by the range itself if the domain is a continuum.

The accuracy indicator described above is a first step, perhaps a bit simplistic, to bringing objectivity into the discussion of accuracy of stereo systems. Hopefully this suggestion will stimulate the development of other useful and more complex quantitative indicators. Perhaps, of greater concern to audiophiles than measuring accuracy, is audibility of coloration and 'hearing what is on the disc'. Stereo systems that seem not to exhibit a sonic signature are deemed accurate. However, a more appropriate term is 'virtual' accuracy, as coloration exists even if (subjectively) undetected.

Coloration may be objectionable for two reasons. First, it is believed that coloration may be a barrier to 'hearing what is on the disc'. Secondly, there is an intrinsic aversion to a sonic signature because it distorts the efforts of the recording engineer and is inconsistent with the connotation of 'high fidelity'.

Note, distortion is indicative of inaccuracy but it may affect the 'sound' of instruments rather than other aspects of reproduction.

A good analogy to consider is the view using clear and rose colored lenses. With both, one may be able to correctly identify objects and their location. However, there is an obvious distortion in color perception using rose-colored glasses. Thus it may be possible to 'hear what's on a recording' and identify where musicians are located even if coloration is audible. However, pitch, timbre, harmonics and dynamics are likely to be distorted to some degree.

Want further proof? Consider the following 'logical' explanation:

  Let A=Stereo system is 'virtually' accurate

   Let B=One can 'hear what's on the disc'

  Hypothesis: If A then B

The hypothesis can be restated to its logical equivalent

  If not B then not A

However, the statement

  If B then A

cannot be deduced from the hypothesis and is not equivalent to it. This means that if one 'hears what's on the disc' the stereo system may or may not be 'virtually' accurate. Therefore, coloration, in and of itself, may not be an obstacle to correctly identifying the content of a recording, although it will impact perception of other aspects of the recording.

In the context of 'virtual' accuracy, there is a relationship between type 1 and type 2 errors and errors of inclusion and exclusion. A type 1 error is rejection of a null hypothesis when it is true and a type 2 error is accepting a null hypothesis when it is false.

  As an example, consider the null hypothesis shown below:

  H=A stereo system is not 'virtually' accurate

If a listener misperceived coloration, which was not generated by the stereo system (error of inclusion), a type 2 error would accrue. If a listener alleged that coloration was not present because of misperception (error of exclusion), a type 1 error would accrue. The propensity for erroneous judgment of a stereo system is always present when (subjective) audition is the sole means for evaluation. In addition, it is extremely difficult to prove that an error has occurred, i.e., how does one distinguish between accurate and inaccurate listener perceptions? Therefore, the term 'virtual' accuracy may not truly represent the performance of the stereo system, but instead depends on deficiencies in hearing, perception and memory and/or a consequence of source material.

Conclusion

Stereo systems are essentially inaccurate to some degree and ascertaining the magnitude of inaccuracy is problematic when relying solely upon listening tests. A rudimentary quantitative accuracy index has been proposed based upon the frequency response curve of stereo systems. While an objective measure may be a useful tool, its calculation is probably of little interest to the typical 'audiophile', who is more likely to rely on 'listening skills' to evaluate a stereo system.

The term 'virtual' accuracy was introduced to denote stereo systems which, based upon an audition process, seem not to exhibit a sonic signature. The quest for such a state is based upon the assumption that the absence of (audible) coloration means that a recording will 'sound' exactly as the recording engineer intended.

What's wrong with this picture?

Although a stereo system may be considered 'virtually' accurate, some 'information' may be hidden from the listener, possibly because of masking by noise. One may use the term 'veil' to describe this phenomenon. The effect may be subtle and not perceived as a sonic signature. Ascertaining the relevant facts about a recording will eliminate the problem.

The 'sound' of stereo systems can vary from day to day as a consequence of changes in temperature, humidity, the AC, tube usage, break-in, etc. In addition, the acuity of perception and the efficiency of the nervous system may also vary from time to time. This means that judgments may be subject to errors of inclusion and exclusion and may be inconsistent over time.

One may reasonably conclude that accuracy assessments based purely upon subjective listening are likely to be unreliable and invalid.