jlsem said:
It's quite trivial actually. It's been said a thousand times that putting a proper test in place is hard, but nobody has said how to put a proper on in place, or what's so improper about suggested strategies. They just claim it's improper, and leave it there. Any blind test that would satisfy scientific rigor would do. Maybe one of the experts who knows so much about testing could come up with a suitable blind test.
There is no doubt in my mind that the testing methods used to
evaluate HiFi gear are beyond reproach even though they give
answers that some people don't like.
My Story
Early on I worked as an electronics tech and believed that the
advancement of audio quality was a purely scientific endevour.
Then the subjectivists got in on the act. I remember going to a HiFi
shop and seeing the first Naim setup."where are the tone controls?"
I asked. I was told they are omitted because they ruin the sound.
They had a Linn LP12 setup along side the most expensive Direct
Drive TT available at that time both sporting identical arms and
cartridges. I thought they sounded the same until the salesman
said, "Listen to the chestiness in Tom Jones voice on the Linn."
Well I was amazed, Tom was definitely more chesty on the Linn.
Hey this was a lot more exciting than charts and numbers. After that
I was convinced that science didn't have all the answers but "HiFi
Answers" (UK magazine) did. I brought a Linn to drive my home
brew transistor amps. Then I rediscovered tubes. Measure worse
sound better. Then "cables" came along and I was floored again,
they all sounded so different, this was no illusion they definitely
sounded different. After awhile I settled on some cables and was
shocked a year later when I dragged out some of those cables to
find that I couldn't hear any differences. Oh dear my ears must be
going. Then I listened to a Sonic Impact, Oh dear again, that little
sucker is doing things my trick tube SE amp makes a mess of.
Maybe there is something in THD and IM measurements after all. . .
For the end user it is all about "audibility" and not whether there
are differences or not. Every stereo amp will measure differences
between channels. There are no speakers identical to each other.
Buy a pair of $5,000 interconnects and there will be a difference
between the left and right, the question is "can you hear that?"
DBT ABX are all about establishing what is audible and what is not.
So far it seems that the Golden Ears are "on average" pretty much
deluded.
"On average" however does allow the possiblity that some people
may have exceptional hearing. Obviously the golden ears on this
forum belong in this group and they are the very people that the
scientists would love to test.
I only wish that I was able to hear cable differences again, back
then Randi's one million dollar prize wasn't on offer.
evaluate HiFi gear are beyond reproach even though they give
answers that some people don't like.
My Story
Early on I worked as an electronics tech and believed that the
advancement of audio quality was a purely scientific endevour.
Then the subjectivists got in on the act. I remember going to a HiFi
shop and seeing the first Naim setup."where are the tone controls?"
I asked. I was told they are omitted because they ruin the sound.
They had a Linn LP12 setup along side the most expensive Direct
Drive TT available at that time both sporting identical arms and
cartridges. I thought they sounded the same until the salesman
said, "Listen to the chestiness in Tom Jones voice on the Linn."
Well I was amazed, Tom was definitely more chesty on the Linn.
Hey this was a lot more exciting than charts and numbers. After that
I was convinced that science didn't have all the answers but "HiFi
Answers" (UK magazine) did. I brought a Linn to drive my home
brew transistor amps. Then I rediscovered tubes. Measure worse
sound better. Then "cables" came along and I was floored again,
they all sounded so different, this was no illusion they definitely
sounded different. After awhile I settled on some cables and was
shocked a year later when I dragged out some of those cables to
find that I couldn't hear any differences. Oh dear my ears must be
going. Then I listened to a Sonic Impact, Oh dear again, that little
sucker is doing things my trick tube SE amp makes a mess of.
Maybe there is something in THD and IM measurements after all. . .
For the end user it is all about "audibility" and not whether there
are differences or not. Every stereo amp will measure differences
between channels. There are no speakers identical to each other.
Buy a pair of $5,000 interconnects and there will be a difference
between the left and right, the question is "can you hear that?"
DBT ABX are all about establishing what is audible and what is not.
So far it seems that the Golden Ears are "on average" pretty much
deluded.
"On average" however does allow the possiblity that some people
may have exceptional hearing. Obviously the golden ears on this
forum belong in this group and they are the very people that the
scientists would love to test.
I only wish that I was able to hear cable differences again, back
then Randi's one million dollar prize wasn't on offer.
Re: Re: Re: Re: Re: I believe...
I asked that a couple of times. No one added a thought or an experience.
I will rephrase:
Did you keep getting sighted listening illusions of cable differences during normal listening sessions AFTER some DBT or ABX test proved that you CANT tell cables apart?
salas said:
I asked that a couple of times. No one added a thought or an experience.
I will rephrase:
Did you keep getting sighted listening illusions of cable differences during normal listening sessions AFTER some DBT or ABX test proved that you CANT tell cables apart?
1. something can only perform poorly - but, as an esl, I might be wrongly....
2. How can a test perform poorly? It can only be designed poorly, so the test is open to criticism as to methodology, and the results from such a poorly designed test are invalid.
It can also be that the methodology might be designed to test the hypothesis, but the analysis of the results is flawed through errors in the statistical approach.
3. The test subjects performed poorly, being unable to discern differences.
Which is it?
So, what is the proper methodology?
None, and I repeat none, of the tests that have so far been conducted, following stringent scientific methodology or not, even hinted at the possibility of audibility of cables of different construction.
The general claim was and is - and I charge anybody who now comes forward and softens the claims to: "slight differences" as guilty of subterfuge, evasion and/or outright lying - that the differences are CLEARLY and UNEQUIVOCALLY audible no ifs, whens or buts.
If the influences are of a nature as stated, then even a scientifically non rigorous conducted A/B blinded test should reveal such CLEARLY and UNEQUIVOCALLY differences. But none has, not a single one, starting with Tom Nousaines test, tests by members of hydrogene audio, several test on german audio forums, etc. etc. has been able to reveal such audibility.
None, and I repeat none, of the tests that have so far been conducted, following stringent scientific methodology or not, even hinted at the possibility of audibility of cables of different construction.
The general claim was and is - and I charge anybody who now comes forward and softens the claims to: "slight differences" as guilty of subterfuge, evasion and/or outright lying - that the differences are CLEARLY and UNEQUIVOCALLY audible no ifs, whens or buts.
If the influences are of a nature as stated, then even a scientifically non rigorous conducted A/B blinded test should reveal such CLEARLY and UNEQUIVOCALLY differences. But none has, not a single one, starting with Tom Nousaines test, tests by members of hydrogene audio, several test on german audio forums, etc. etc. has been able to reveal such audibility.
The proper methodology means at least the inclusion of controls on a sufficient sensitivity level.
Then theres is no need to speculate about test conditions anymore, because you know that listeners reached the sensitivity level required and otoh that no external factors are the reason for positive results.
BTW, ABX isn´t my favourite test protocol but i think people can adopt to it and in the end there are controls.
Jakob2
Then theres is no need to speculate about test conditions anymore, because you know that listeners reached the sensitivity level required and otoh that no external factors are the reason for positive results.
BTW, ABX isn´t my favourite test protocol but i think people can adopt to it and in the end there are controls.
Jakob2
I don't wander in here often, but I am glad - yet not - that I did. I tend to get cold shivers when folks without research experience postulate about what is an appropriate 'methodology' for testing the audible differences between different components, which is a type of clinical research – changing the component is the intervention and the listeners are the subjects.
I have not the time to discuss my reasoning at the moment, but in all honesty, broad methodologies are available to suitable 'test' for audible differences between components (tho lateral thinking may be required to identify and implement them). However, the cost of researching and developing the protocol, implementing and monitoring it, and collecting, analysing, and reporting the results to determine the 'efficacy' of a component in a valid way would be beyond cost-prohibitive. Even then, the results would not be definitively valid, as our study would be essentially qualitative (hmmm, what validated instrument should I use?) and science, by definition, is constantly moving (assumingly forward). That said, they would be more likely to provide strongly relevant information. I have often thought of doing the literature review, developing the methodology, and documenting a research protocol for something like this; but then I wonder, WHY?..
So, 'valid' methods are out and the pseudo-scientifically valid methods do little more than provide misinformation, thereby fostering misunderstanding and confusion. So, where does that leave us... the case study perhaps - oh, we already do those (albeit often of less-than-great quality)... now we just need to recognise and understand their limitations, but then that goes for the pseudo-scientific methods also…
Ah, folks - this is AUDIO. Not life. Not death. It may be pleasurable, but if you are not gaining mere pleasure from something you should consider yourself fortunate enough to indulge in, then perhaps your fulfillment or sense of self-worth is dependent on it – why would you audio ‘ride’ you like that?
Oaky, gotta run.
I have not the time to discuss my reasoning at the moment, but in all honesty, broad methodologies are available to suitable 'test' for audible differences between components (tho lateral thinking may be required to identify and implement them). However, the cost of researching and developing the protocol, implementing and monitoring it, and collecting, analysing, and reporting the results to determine the 'efficacy' of a component in a valid way would be beyond cost-prohibitive. Even then, the results would not be definitively valid, as our study would be essentially qualitative (hmmm, what validated instrument should I use?) and science, by definition, is constantly moving (assumingly forward). That said, they would be more likely to provide strongly relevant information. I have often thought of doing the literature review, developing the methodology, and documenting a research protocol for something like this; but then I wonder, WHY?..
So, 'valid' methods are out and the pseudo-scientifically valid methods do little more than provide misinformation, thereby fostering misunderstanding and confusion. So, where does that leave us... the case study perhaps - oh, we already do those (albeit often of less-than-great quality)... now we just need to recognise and understand their limitations, but then that goes for the pseudo-scientific methods also…
Ah, folks - this is AUDIO. Not life. Not death. It may be pleasurable, but if you are not gaining mere pleasure from something you should consider yourself fortunate enough to indulge in, then perhaps your fulfillment or sense of self-worth is dependent on it – why would you audio ‘ride’ you like that?
Oaky, gotta run.
rdf said:
Then how about telling us why the test methods you are using to establish audible differences in your system are valid.
Instead of whining about the "flawed" ABX/DBT tests you've "seen", why not tell us exactly how the tests should be conducted to establish audible differences and then tell us which method you are using yourself.
This question is open to all hearing superheroes. What scientifically valid method are you using when "hearing" differences in the soundwaves impinging upon the ears?
cheers,
AJ
validity??
So, these uncontrolled tests being pursued are valid for exactly who?? Seems to me they're only valid for those undertaking them, since there are no controls and the response evaluation criteria seem to be nebulous. or am I missing something?
You left out lawyers, used car dealers, and real estate agents....
So, these uncontrolled tests being pursued are valid for exactly who?? Seems to me they're only valid for those undertaking them, since there are no controls and the response evaluation criteria seem to be nebulous. or am I missing something?
You left out lawyers, used car dealers, and real estate agents....
thanx Sy
Thanx for the clarification. Since many here seem to present their viewpoint as being "correct" w/o saying who it's correct for, I often wonder if I'm mis-interpreting their opinions and presentation as factual, since they often imply that this is so.
I'll take all the "data" presented from now on as being opinion only... since my research days are long past and audio is also "only" a hobby for me...
John L.
Thanx for the clarification. Since many here seem to present their viewpoint as being "correct" w/o saying who it's correct for, I often wonder if I'm mis-interpreting their opinions and presentation as factual, since they often imply that this is so.
I'll take all the "data" presented from now on as being opinion only... since my research days are long past and audio is also "only" a hobby for me...
John L.
nunayafb said:
Now that's the scientific spirit! No time until later for a proper reply, but no, it has to do with group sessions, pre-load speeches, the devices used, etc.. I need to confirm it but, for another example, its appears the DAC in a Yamaha player used in a recent, highly publicized, hi-rez vs. 16-bit test is only capable of 17-bit rez.
- Status
- Not open for further replies.