As it turns out, none of the research is very good -- relying heavily on poor retrospective surveys, misinterpreted data, and downright dishonesty. Doctors Harrison Pope (in His book Psychology Astray) and Richard McNally (in Remembering Trauma) have done excellent work in scrutinizing the many errors in recovered memory research, but I hope to highlight certain illustrative examples here on this blog. If you have a particular study you'd like me to look at, please let me know.
Bessel van der Kolk is something of a hero to recovered memory believers, but the deposition taken below (by Dr. Christopher Barden), as part of a trial in which van der Kolk was acting as an expert witness, reveals some grave questions regarding his scientific rigor. Following this deposition, van der Kolk vanished from the expert witness roster.
The below was previously available online elsewhere. I have retained the original introduction by Dr. Barden:
We are all glad that Dr. van der Kolk will no longer be testifying in court. It is important to understand why he has dropped out of the many cases in which he was to serve as an "expert." It is also important to note that attorneys who had overestimated their ability to understand psychology and psychiatry had been deposing Dr. van der Kolk for years without effect.
In the past year I have been able to eliminate approximately 90% of the "expert witnesses" that I depose in criminal, civil or family court matters. This is done by exposing their ignorance of basic scientific terms and concepts as well as their ignorance of their own field. I call this "science intensive litigation." The minimal requirement to conduct such a deposition is that the person asking the questions be as well trained or better in the relevant field as the "expert" being deposed.
The questions that eliminated van der Kolk were not so much questions about his old researchers or testimony in the Hungerford case but rather his shocking ignorance of his own profession. It is amazing to me that van der Kolk had been deposed and cross examined by dozens of lawyers over many years and not one ever asked him to specify the limitations on the testimony of experts included in the AMA and ethics codes. Not one lawyer ever asked him to discuss the differences between the concepts of reliability and validity. Not one lawyer asked him to discuss the actuarial prediction vs. clinical prediction research. Not one lawyer assessed his knowledge of testing, psychological measurement, the cross-cultural history of "dissociative phenomena" or other relevant matters (note for example, van der Kolk's stunning errors in discussing Greek literature). Not one of these attorneys engaged him in a dialogue about his own research methods including how he assessed the reliability and validity of his own measures and procedures. Not one of these attorneys apparently asked him about the Coping and Resiliency research literatures and not one even discovered that his old mentor was Bruno Bettleheim -- new believed to be one of the great frauds in psychology in the 20th Century. Van der Kolk's failure to answer these basic and critical professional questions about science, methodology and history doomed his credibility as an "expert" in this field and he knew it the moment those questions were asked.
Until attorneys stop using antiquated legal methods and begin to use science intensive litigation methods (experts or attorney/experts must prepare all of the questions and be present to follow up) experts like van der Kolk will continue to roam the courtrooms of America. Thousands of families have been
destroyed, innocent citizens have been sent to prison, and billions of dollars have been stolen from the public treasury because improperly trained and misinformed "experts" have been unchecked by a legal profession engaging in "business as usual."
Even the greatest of trial attorneys are simply not able to even begin to do this kind of highly specialized work. Even the greatest of trial attorneys need to have Loftus or Ofshe or Pope or Grove or Hudson or Barden or some other expert (whether they are also an attorney or not) asking the questions in such depositions in real time in the room. Lawyers, without very extensive training and experience in science, simply cannot do these "elimination" depositions --- multiple publications in premiere journals, a Ph.D. or M.D. from a top 10 school, peer review editorial experience, extensive philosophy of science training, and a national award in a relevant field are minimal prerequisites to do this properly. As the record indicates, one with such training can eliminate these "experts" in most cases.
Science marches on. I hope that the legal profession will follow. Science intensive litigation -- litigating cases using teams of lawyers and scientists -- is the best way to rid the legal system of junk science. Reforming the legal educational system to include much, much more training in science and the philosophy of science is also necessary. Improperly trained "experts" and overconfident attorneys remain the greatest dangers to the integrity of the legal system. Through the efforts of valiant citizens, well trained experts, and educated attorneys the rights of citizens will be protected and the mental health and legal systems will be reformed
Very best wishes,
R. Chris Barden
P.S. Feel free to distribute this material.
*
Following is the end of a deposition of Dr. Bessel van der Kolk conducted on December 28, 1996. [...]. I have been told that the court-order described by Dr. Barden was obtained and that Dr. van der Kolk is no longer listed as an expert witness in this case.
Dr. Christopher Barden and Mr. James Murphy served as lawyers for the defense, Mr Conlon for the plaintiff. Questioning by Dr Barden.
Line 19, Page 302:
Q. Back to the fragmentary memory study. You had mentioned that the subjects had received a number of questionnaires.
A. That's correct.
Q. So, then, you would have copies of the questionnaires that they had filled out.
A. I would. Confidential questionnaires, part of my agreement with my IRBs -- Mass. General Hospital; State of Massachusetts; Harvard -- is that these records cannot be released to any third parties.
Q. Certainly no records that contain patient identifying information can be released to third parties.
A. I don‘t believe I can release those records to anybody.
Q. Well--
A. Unless I would get specific release of information from every particular subject.
Q. Right. Well, that's a legal decision that the court's going to have to make. I'm just going to ask you about what kinds of documents you have or what kind of records you have.
A. I don't know exactly what records I have, but I would certainly do everything in my power to honor the spirit and the letter of my IRB obligations to my subjects .
Q. Certainly. But you would have to have copies of all of the data sheets from all of the questionnaires from all of the subjects, correct?
A. I do whatever needs to be done under the rulings of the IRBs that I was accountable to at that point.
Q. And you would have copies of these records confirming the accuracy of these memories: affidavits, medical records, court records. You would have those.
A. Whatever I have, I have preserved in locked files, yes.
Q. And you would have, I think you mentioned, some audiotapes of some of the discussions regarding scores of these tests.
A. There might be some; I would have to look. But, again, that's all confidential and not open to the prying eyes of anybody from the other side except for my own IRB.
Q. And then certainly something that would absolutely not be confidential would be you have copies of the manner in which the data was analyzed.
MR. CONLON: Objection. Copies of the --. Could you --. Do you understand?
THE DEPONENT: Yeah.
A. I don't know how confidential that is. I am not sure if the manner --
MR. CONLON: Let's not get into the confidential ——. He asked you if you have copies of the manner in which this stuff was analyzed. If you understand that, just answer that question "yes" or "no" and leave it at that.
A. I have copies of the raw data.
Q. Okay.
MR. CONLON: And, obviously, we're not agreeing to --
DR. BARDEN: No. That's --
MR. CONLON: We're going to leave confidentiality issues --
DR. BARDEN: For the court. Exactly.
THE DEPONENT: I think it would be highly unusual for an expert's studies that have nothing -- that have, are only peripherally relevant to be subpoenaed by people.
DR. BARDEN: That is an issue for the court, though. We don't need to go into that.
BY DR. BARDEN: (Continuing)
Q. Are there any other types of records from that particular study that you think you have at this point that we haven't talked about?
A. I don't think it's of any further relevance to this, to the deposition at hand.
Q. Relevance is for the court to decide.
A. Right.
Q. I'm asking you under oath: Do you have any other kinds of records from this study?
A. You know, I feel intimidated by your question.
Q. That's okay, you can feel any way you want, but you still have to answer it.
A. Right.
Q. What, if any, other kinds of data do you have in your possession from that study?
MR. CONLON: In his --. In your possession.
A. In my possession meaning here in my office?
Q. In your office; in your Trauma Clinic; anywhere.
A. In the research office we ought to have lengthy narratives of every trauma narrative that every one of these subjects gave us.
Q. would there be any other kinds of or data sheets that you would have relevant to this study that we haven't talked about yet?
MR. CONLON: Objection. Objection to the form of the question.
A. Not that I know of.
Q. Now, you talked yesterday
MR. JAMES MURPHY: Excuse me. Could I just clarify, when we're talking about "this study," are we referring to the study that was discussed in the article that was attached to the supplemental answer to interrogatory number 13?
DR. BARDEN: Yes, we are talking about Dissociation and the Fragmentary Nature of Traumatic Memories: Overview and Exploratory Study which is published in the Journal of Traumatic Stress, Volume 8, Number 4, 1995, pages 505 to page 525.
Q. Is that correct?
A. Can you ask the question again?
Q. That's the study that we've been talking about with regard to what you have in your office.
A. I think what --. You have to ask yourself that question because it was your question.
Q. Well, was it your understanding when I asked you questions about what data you have and what data sheets you have that I was referring to this particular study?
A. I have several file cabinets in accordance with my IRBs under which I have done these studies which contain the records of a series of studies that include this particular one.
Q. Now, --
MR. WILLIAM MURPHY: Excuse me. What is the definition of what "IRB" means?
THE DEPONENT: Yes. It's called Institutional Review Board, and this particular study that we're talking about here was done under the Institutional Review Boards of the Massachusetts General Hospital, Harvard Medical School, the Commonwealth of Massachusetts, and the Lindemann Mental Health Center.
MR. WILLIAM MURPHY: Thank you.
BY DR. BARDEN: (Continuing)
Q. Now, yesterday I asked you a series of questions about this study, which I'm going to call the fragmentary nature of memory study. All right? Just so we're clear on what we're talking about.
A. Fine.
Q. And I had asked you in particular who collected the data or did the interviews for the study.
A. Right.
Q. And we had mentioned the names of Michael Rater, Roslin Moore, Nan Herron, Ann Hostetler, and Joseph Rodriguez and Aminadav Zakai, and you had mentioned that they had been participating in the collection of the data for the study.
A. That's correct.
Q. Now, I had also asked you about a Danya Vardi, and you said she had not participated in the collection of data for this study; is that correct?
A. Correct.
Q. Is it not the case that in your published version of this study you acknowledge and thank Danya Vardi for her contribution in the collection of data for this study, sir?
A. She did not do patient interviews.
Q. What data did she collect?
A. She helped transcribe information from the, to my memory, inventory into the computer.
Q. And where did she do that?
A. At the Trauma Center.
Q. And who was with her when she did that?
A. Her work was checked afterwards by Rita Fisler, and after that by me.
Q. So during the time she was doing that she was by herself.
A. She may have been by herself part of the time; she may have been with Rita part of the time; she may have been with me part of the time.
Q. How did you check her data?
A. We checked whether what is on the sheets was in accordance with what showed up in the computer.
Q. so it's your testimony now that she transcribed data.
A. She transcribed data and did some of the, ran some of the computer runs of the correlation of relationship between various issues, yes.
Q. Is it your testimony that in fact she did not collect data of this study?
A. Well, I --
MR. CONLON: Objection.
A. When I give acknowledgment to people, I always acknowledge everybody who made a contribution to the study, particularly if I don't pay people for their, for their work.
Q. Is it your testimony that transcribing data in the scientific community is generally regarded as collection of the data?
A. I think --
MR. CONLON: Objection. A. I think that under the general rubric of acknowledging people for working on the study, that's a rough way of putting it.
Q. Isn't it in fact the case that Danya Verdi did much more than transcribe data and do computer runs --
A. No.
Q. -- for this study?
A. No. Because --
Q. Would you ever give a --
A. The reason for --. I imagine you're asking about her because she may know something about it, Danya Vardi. We threw out her, her interviews, actually, for the study.
Q. And how many interviews did she do that you threw out?
A. I don't know. We threw out every one that she did.
Q. would it have been one or two or five or --
A. It may be, I don't know, one or two or five.
Q. And at what time did you throw out her interviews? What was the date that you did that?
A. At the moment that we started to have some suspicion that she collected the data in a not careful fashion.
Q. And when was that?
A. I don't know the exact date.
Q. Would that have been before or after your testimony in the Hungerford trial?
A. Oh, way before.
Q. You mentioned yesterday that she had her Ph.D., and then clarified that later and said that she did not have her Ph.D.
A. That's correct.
Q. When did you come to learn that she did not have her Ph.D.?
A. Actually, I learned last week definitively that she would not be getting her Ph.D., because Judy Alpert happened to be in town, who was her thesis adviser, who told me.
Q. Do you work with a Dr. Pitman?
A. I have a colleague by the name of Dr. Pitman, (Deponent nods in the affirmative). I've done research with him.
Q. Did Ms. Vardi do research with him also?
A. Yes, she did.
Q. Are you aware of any problems Dr. Pitman had with Ms. Vardi?
A. I am.
Q. And what were the nature of those problems?
A. There was a suspicion that she falsifies data -- falsiļ¬ed data.
Q. And what was the outcome of those suspicions?
A. That it is confirmed and she was censured by the Harvard Medical School.
Q. She was basically, she was convicted of scientific fraud.
A. That's right.
Q. And when did you become aware of that?
A. I was aware of it from the very beginning. Actually, I was the person who alerted Roger Pitman to the fact that I did not trust the way that she was collecting her data.
Q. And when was that?
A. I don't know the exact date. But that's the reason why we threw out every piece that she collected, the interviews that she did.
Q. What are the nature of contributions that people have to do in order to be listed as a coauthor with you on a research study?
MR. CONLON: Objection.
A. I think people need to spend a number of hours working on some aspect of the study.
Q. Is it ethical to have someone listed as a coauthor if all they've done is essentially office work?
A. As a coauthor? I'm not quite sure --. What do you mean, with "office work"?
Q. Well, like transcribing data or doing computer runs. Would it be ethical to list someone that transcribed data and did computer runs as a coauthor on a study?
A. I'm not sure if it's a question of ethics. It's a question of how generous people want to be.
Q. Is it your testimony that there are no ethical guidelines or rules for who may or may not be listed as a coauthor of a research study?
A. There are some guidelines about it, yes.
Q. What are the guidelines with regard to listing someone who does nothing more than transcribe data and do computer runs, what would be the guidelines on using a person like that or giving a person like that coauthorship on a scientific study?
A. Well, as you know, she was not a coauthor of this study.
Q. That's not the answer to my question.
A. Right.
Q. What's your understanding about whether that would be permissible or not?
A. I think if somebody spends night after night doing very complicated computer runs, it would be reasonable to include him as a coauthor.
Q. How many complicated computer runs did Ms. Vardi do for you?
A. She was not a coauthor of this paper, as you know.
Q. This will go smoother if you just answer my question.
A. No, but --
Q. My question was: How many computer runs --
A. You asked me about coauthorship.
Q. I'm now asking you: How many computer runs did she do for you?
A. She actually did quite a few of them for me, yes.
Q. And is it your testimony that you checked her computer runs? Did anybody else check her computer runs?
A. Rita Fisler did and Jennifer Burbridge did in the second part of her study.
Q. These were after you threw out the subjects that she interviewed.
A. Both before and after.
Q. so after you threw out her subjects, did she continue to do computer runs for you?
A. I don't think so.
Q. But you're not sure?
A. I'm not totally sure. But at some point she was asked to leave the, all projects because we became increasingly distrustful about the quality of her work, yes.
Q. Now, did you have to redo all the computer runs, then, after you threw out her subjects?
A. We did. We did.
Q. And so, then, you had a small early number of subjects to run.
A. That's correct.
Q. Okay.
A. We lost about five Vietnam veterans who were included in the initial data and we had to omit.
Q. And that left you with a total of 46 subjects.
A. That's correct.
Q. And by that time Ms. Vardi had been asked to leave.
A. That's correct.
Q. And that was before you testified in the Hungerford case.
A. That‘s correct.
Q. Now, when you testified in the Hungeriord case, Dr. van der Kolk, did you submit a document to the court entitled "Dissociation and the Perceptual Nature of Traumatic Memories: Background and Experimental Confirmation," which was an earlier version of the fragmentary nature of memory paper?
A. You seem to have something in your possession that would indicate that.
Q. Do you recall submitting that to the court for your testimony in the Hungerford case?
A. That would make sense, yeah.
Q. Do you remember who was listed as the authors of that study that you submitted to a court of law in a criminal case in the Hungerford matter?
A. I think it's possible that Danya Vardi was still listed as a coauthor at that point.
Q. She was in fact listed as a coauthor.
A. Yeah.
Q. She was in fact listed as having a Ph.D.; isn't that correct?
A. I don't know.
Q. Does that refresh your memory?
(Document tendered to the deponent)
A. Well, since you --
MR. CONLON: Can I hand that over to him?
DR. BARDEN: Yes. I need it back, though, because it has notes on it.
MR. JAMES MURPHY: I have an unnoted copy, if you'd like.
DR. BARDEN: That would be great.
(Document tendered to the deponent)
MR. JAMES MURPHY: If I can have it back, though. Unless you want to mark it as an exhibit.
DR. BARDEN: Why don't: we mark that as van der Kolk Exhibit A.
MR. JAMES MURPHY: Do you want to just for clarity mark the fragmentary article as Exhibit B?
DR. BARDEN: Yes. That would be the actual published version as Exhibit B. The copy submitted to the court in the Hungerford matter will be listed as Deposition Exhibit A.
MR. JAMES MURPHY: Mine is highlighted, but I don't think the highlighting will carry through.
MR. CONLON: I'm not going to object.
No comments:
Post a Comment