"In contemporary research and clinical practice, Robert D. Hare’s Psychopathy Checklist, Revised (PCL-R) is the psycho-diagnostic tool most commonly used to assess psychopathy. Because an individual’s score may have important consequences for his or her future, and because the potential for harm if the test is used or administered incorrectly is considerable, Hare argues that the test should only be considered valid if administered by a suitably qualified and experienced clinician under controlled and licensed conditions. Hare receives royalties on licensed use of the test."
A paper and talk being given at Defcon 20 this week has gotten people all worked in in a lather within the news arena and has piqued my interest. The talk centers around the premise that one may be able to determine psychopathic traits (psychopathic and sociopathic behaviors) from of all things, the analysis of tweets.
Now, this may be a novel idea to some and it certainly seems the news has latched onto this, but, in the cold hard light of day, this premise has way too many failures to be actually applicable to gaining any insight into anyone’s psyche via Twitter.
In this article I am staking out my contention that this is not a suitable means of diagnostics of this type and in fact, were it to be followed up on and used, would lead to bad results and perhaps the citation of individuals online as being “Psychopathic” when they are not the least bit so.
As such, this talk may be an inquiry into whether or not this is possible, but, had the research been carried out to the extent of reading the materials and their ancillaries, one would quickly come to grips with some salient facts that make this method of detection untenable.
As the media hype has already started on this, I think it prudent to speak up on this here and now, as well as write an after piece once I have sat through the talk and had a chance to see exactly what they say they believe possible in the end.
A Flawed Premise:
Having read the original paper by Hancock, Woodworth, and Porter, (Hungry like the wolf: A word-pattern analysis of the language of psychopaths) the experiment clearly states that they had chosen a sampling of criminals convicted of murder (various degrees of which) and verbally interviewed them on their crimes.
The method of interview was strictly adhered to and was a known and well used process including blind interpretation (where the interviewer did not collate the data on psychopathy, just transcribed dialog and logged emotional states) Once the transcription was done (including disinfluences *uh and um*) this text was taken and run through the wmatrix and other tools to determine the languages affinities for psychopathy and other mental states.
This “text” is actual “dialog” and as such, is not the same as the “written word” that the speakers at Defcon are going to be assessing in their presentation, and this is a key difference that I am unsure they have taken into account. Writing is affected and not natural to many (i.e. fluid dialog in writing).
Add to this that you are talking about the emoting of data/emotion vis a vis Twitter at 140 characters at a time AND using quite a bit of word shortening and slang, and the premise of using “language” to determine psyche really falls apart.
A second key point is that the dialogs that are being used in the original paper are specifically stories of their crimes. This was a calculated effort on the part of the psychologists to elicit the emotional states of the subjects in relation to their crimes, and their victims.
This is a key factor in the determination of the language that the researchers were looking at, and as such, this, as far as I know, is not a part of the paper being presented at Defcon, and thus, misses a key data point… Making the premise suspect to start.
It is my opinion, just from the differences between the experimental inputs, that unless you have a larger dialectic to work from and a trained set of people to determine not only language, but also emotion (we all know how easy it is to misinterpret an email right?) of the poster, you cannot in any way, shape or form come up with a psychiatric profile, never mind an actual diagnosis, of psychopathy via online content, especially that which is culled from Twitter.
Background Data On PCL-R:
Another factor that I would like to address briefly is the use of the PCL-R test. This test, though being around for some time and used, is still not part of the DSMV as a diagnostic tool that they prefer. There are many papers and articles online that do not promote the use of this test as a lock on Psychopathy, nor is there really a consensus from DSM to DSM on exactly what Psychopathy is.
Psychology and Psychiatry is more of a plastic science due to the nature of the human brain. So, all of this supposition on trying to quantify an individual from their language written online at 140 characters at a time is being terribly kluged into an ideal. It is important to know the landscape here to understand that nothing is certain and even diagnosis of an ailment such as these can be countered by a second opinion by another doctor.
...And doctors should be involved in any of these experiments online as well. However, the bulk of what I have seen online and read elsewhere, as well as common sense, points to me the fact that even with a lot of online chatter, one must interview the subject in person to determine their illness.
Not All Problems Can be Solved With Big Data and Technical Solutions:
In the end though, I guess my biggest concern is that certain people out there (or government groups) might take this idea of sifting through big data online for such linguistic cues as something to run with. In fact, contextual searches already exist and often are used by agencies to determine where someone might live or have lived, gone to school, etc by the nature of their writing.
In fact, recently on Studio 360 I heard a report of a computer program being created for just such a thing. It however, was also an AI project to try and get the “Turing” effect to be so acute that a person online would not know the difference between computer and human communication.
Which, brings me to another idea... When will we see the first “psychopathic” AI out there? But I digress…
It seems to me that more and more we are being collectively mined not only for our habits, but now our emotions as well as our psychological makeups. All of this could potentially be collated from numerous sources (not just out of the context of language but also click behavior etc).
Remember those days in college when you took Psych 101 and thought the professor was just messing with you and taking notes? Well, I have the same feeling now with the internet in general and the companies and governments using it for contextual purposes.
I doubt though, we will ever be able to contextualize the human psyche just from internet datum… And that is where I think this talk is headed… And thus, I had to speak my peace. I will have another post on my thoughts after the talk.. Maybe they can change my mind a bit.
Cross-posted from Krypt3ia