WEBVTT NOTE This file was generated by Descript 00:00:11.116 --> 00:00:12.706 Avdi: This is the cash flush. 00:00:12.976 --> 00:00:17.986 A programmer's audio scrapbook, recorded May 8th, 2023. 00:00:19.516 --> 00:00:24.466 I'm Avdi Grimm, and today I have a selection from the 00:00:24.466 --> 00:00:29.536 introduction to "Computer Power and Human Reason: from Judgment to 00:00:29.536 --> 00:00:32.116 Calculation", by Joseph Weizenbaum. 00:00:40.684 --> 00:00:44.194 Most other programs could not vividly demonstrate the information processing 00:00:44.194 --> 00:00:45.524 power of a computer to visitors. 00:00:46.189 --> 00:00:49.039 Who did not already have some specialized knowledge, say of 00:00:49.039 --> 00:00:50.359 some branch of mathematics. 00:00:50.989 --> 00:00:54.709 DOCTOR, on the other hand, could be appreciated on some level by anyone. 00:00:55.279 --> 00:00:58.369 Its power as a demonstration vehicle was further enhanced by the fact 00:00:58.374 --> 00:01:00.829 that the visitor could actually participate in its operation 00:01:01.519 --> 00:01:04.849 Soon copies of DOCTOR constructed on the basis of my published description 00:01:04.849 --> 00:01:07.909 of it began appearing at other institutions in the United States. 00:01:08.509 --> 00:01:11.029 The program became nationally known, and even in certain 00:01:11.029 --> 00:01:12.439 circles, a national play thing. 00:01:13.459 --> 00:01:17.929 The shocks I experienced as DOCTOR became widely known and played were due 00:01:17.929 --> 00:01:20.059 principally to three distinct events. 00:01:20.749 --> 00:01:25.129 One, a number of practicing psychiatrists seriously believed 00:01:25.159 --> 00:01:28.459 the DOCTOR computer program could grow into a nearly completely 00:01:28.459 --> 00:01:30.319 automatic form of psychotherapy. 00:01:30.829 --> 00:01:32.239 Colby et al write, for example: 00:01:33.814 --> 00:01:37.144 further work must be done before the program will be ready for clinical use. 00:01:37.564 --> 00:01:41.164 If the method proves beneficial, then it would provide a therapeutic tool 00:01:41.314 --> 00:01:44.914 which can be made widely available to mental hospitals and psychiatric centers 00:01:45.094 --> 00:01:49.384 suffering a shortage of therapists because of the time sharing capabilities of modern 00:01:49.384 --> 00:01:53.254 and future computers, several hundred patients an hour could be handled by a 00:01:53.254 --> 00:01:55.024 computer system designed for this purpose. 00:01:55.534 --> 00:01:59.809 The human therapist involved in the design and operation of this system, would not 00:01:59.809 --> 00:02:04.129 be replaced but would become a much more efficient man since his efforts would 00:02:04.129 --> 00:02:08.779 no longer be limited to the one-to-one patient therapist ratio as now exists. 00:02:10.759 --> 00:02:14.209 I had thought it essential as a prerequisite to the very possibility 00:02:14.209 --> 00:02:18.049 that one person might help another learn to cope with his emotional problems, 00:02:18.049 --> 00:02:21.619 that the helper himself participate in the others' experience of those 00:02:21.619 --> 00:02:25.969 problems, and in large part by way of his own empathetic recognition of 00:02:25.969 --> 00:02:28.279 them, himself come to understand them. 00:02:28.759 --> 00:02:32.569 There are undoubtedly many techniques to facilitate the therapist's imaginative 00:02:32.569 --> 00:02:34.429 projection into the patient's inner life. 00:02:35.389 --> 00:02:39.859 But that it was possible for even one practicing psychiatrist to advocate that 00:02:39.859 --> 00:02:44.419 this crucial component of the therapeutic process be entirely supplanted by pure 00:02:44.419 --> 00:02:46.819 technique that I had not imagined. 00:02:47.359 --> 00:02:51.319 What must a psychiatrist who makes such a suggestion think he is doing 00:02:51.319 --> 00:02:55.039 while treating a patient, that he can view the simplest mechanical 00:02:55.039 --> 00:02:58.849 parody of a single interviewing technique as having captured anything 00:02:58.849 --> 00:03:00.409 of the essence of a human encounter. 00:03:01.039 --> 00:03:04.129 Perhaps Colby et all give us the required clue when they write: 00:03:04.714 --> 00:03:07.924 a human therapist can be viewed as an information processor and 00:03:07.924 --> 00:03:11.884 decision maker with a set of decision rules, which are closely linked to 00:03:11.884 --> 00:03:13.774 short range and long range goals. 00:03:14.044 --> 00:03:17.164 He is guided in these decisions by rough empiric rules. 00:03:17.284 --> 00:03:21.214 Telling him what is appropriate to say and not to say in certain contexts. 00:03:21.454 --> 00:03:24.754 To incorporate these processes to the degree possessed by a human 00:03:24.754 --> 00:03:28.744 therapist in the program would be a considerable undertaking, but we are 00:03:28.744 --> 00:03:30.304 attempting to move in this direction. 00:03:32.059 --> 00:03:36.799 What can the psychiatrist's image of his patient be when he sees himself, as 00:03:36.799 --> 00:03:42.559 therapist, not as an engaged human being, acting as a healer, but as an information 00:03:42.559 --> 00:03:44.719 processor following rules, et cetera? 00:03:46.099 --> 00:03:50.059 Such questions were my awakening to what Polanyi had earlier called a 00:03:50.209 --> 00:03:54.979 scientific outlook that appeared to have produced a mechanical conception of man 00:03:56.419 --> 00:04:01.099 two: I was startled to see how quickly and how very deeply people conversing 00:04:01.099 --> 00:04:04.339 with DOCTOR became emotionally involved with the computer and how 00:04:04.339 --> 00:04:05.939 unequivocally they anthropomorphized it. 00:04:06.829 --> 00:04:09.709 Once my secretary, who had watched me work on the program for many 00:04:09.709 --> 00:04:12.889 months and therefore surely knew it to be merely a computer program, 00:04:13.039 --> 00:04:14.089 started conversing with it. 00:04:14.599 --> 00:04:17.629 After only a few interchanges with it, she asked me to leave the room. 00:04:18.499 --> 00:04:19.159 Another time. 00:04:19.159 --> 00:04:23.569 I suggested I might rig the system so that I could examine all conversations 00:04:23.569 --> 00:04:25.669 anyone had had with it, say overnight. 00:04:26.209 --> 00:04:30.649 I was promptly bombarded with accusations that what I proposed amounted to spying 00:04:30.649 --> 00:04:34.519 on people's most intimate thoughts, clear evidence that people were conversing 00:04:34.519 --> 00:04:37.969 with the computer as if it were a person who could be appropriately and 00:04:37.969 --> 00:04:39.649 useful addressed in intimate terms. 00:04:40.204 --> 00:04:42.994 I knew of course, that people form all sorts of emotional bonds to 00:04:42.994 --> 00:04:46.444 machines, for example, to musical instruments, motorcycles, and cars. 00:04:46.624 --> 00:04:49.324 And I knew from long experience that the strong emotional ties 00:04:49.324 --> 00:04:52.624 many programmers have to, their computers are often formed after only 00:04:52.624 --> 00:04:54.184 short exposures to their machines. 00:04:54.634 --> 00:04:57.484 What I had not realized is that extremely short exposures to a 00:04:57.484 --> 00:05:00.964 relatively simple computer program could induce powerful, delusional 00:05:00.964 --> 00:05:02.254 thinking in quite normal people. 00:05:02.839 --> 00:05:05.869 This insight led me to attach new importance to questions of 00:05:05.869 --> 00:05:08.999 the relationship between the individual and the computer and 00:05:08.999 --> 00:05:10.519 hence to resolve to think about them 00:05:12.259 --> 00:05:12.709 Three. 00:05:13.579 --> 00:05:16.879 Another widespread and to me, surprising reaction to the ELIZA 00:05:16.879 --> 00:05:20.239 program was the spread of a belief, that it demonstrated a general 00:05:20.599 --> 00:05:23.929 solution to the problem of computer understanding of natural language. 00:05:24.859 --> 00:05:27.469 In my paper, I had tried to say that no general solution 00:05:27.469 --> 00:05:28.759 to that problem was possible. 00:05:29.029 --> 00:05:33.019 I e that languages understood only in contextual frameworks that even 00:05:33.024 --> 00:05:36.199 these can be shared by people to only a limited extent, and that 00:05:36.199 --> 00:05:40.339 consequently even people are not embodiments of any such general solution. 00:05:40.999 --> 00:05:42.679 But these conclusions were often ignored. 00:05:43.099 --> 00:05:46.099 In any case, ELIZA was such a small and simple step. 00:05:46.444 --> 00:05:49.744 Its contribution was, if any at all, only to vividly underline what 00:05:49.744 --> 00:05:51.274 many others had long ago discovered. 00:05:51.334 --> 00:05:55.774 Namely, the importance of context to language understanding . The subsequent, 00:05:55.774 --> 00:05:59.104 much more elegant and surely more important work of winograd in computer 00:05:59.104 --> 00:06:02.914 comprehension of English is currently being misinterpreted just as ELIZA was. 00:06:03.304 --> 00:06:07.014 The reaction to ELIZA showed me more vividly than anything I had seen hitherto. 00:06:07.264 --> 00:06:11.794 The enormously exaggerated a attribution, an even well educated 00:06:11.794 --> 00:06:13.414 audience is capable of making. 00:06:13.834 --> 00:06:17.674 Even strives to make to a technology it does not understand. 00:06:18.274 --> 00:06:22.024 If as appeared to be the case, the public's attribution are wildly 00:06:22.024 --> 00:06:26.364 misconceived, then public decisions are bound to be misguided and often wrong. 00:06:26.904 --> 00:06:30.094 Difficult questions arise out of these observations. 00:06:30.364 --> 00:06:33.784 What, for example, are the scientists responsibilities with respect to 00:06:33.784 --> 00:06:38.434 making his work public, and to whom or what is the scientist responsible? 00:06:45.476 --> 00:06:45.866 Again. 00:06:45.926 --> 00:06:50.756 That was a selection from the introduction to "Computer Power and Human Reason, 00:06:51.086 --> 00:06:57.776 from Judgment to Calculation" by Joseph Weizenbaum, published in 1976, 00:07:00.006 --> 00:07:01.596 that's what was in the cache today. 00:07:03.576 --> 00:07:07.696 If you enjoy this show it is brought to you by Graceful Dev. 00:07:08.716 --> 00:07:14.116 That's Graceful Dev, a Garden of Developer training where you can 00:07:14.116 --> 00:07:18.646 get two weeks free by using the code CACHEFLUSH all caps, all in one word. 00:07:20.791 --> 00:07:23.431 Thanks again for listening and don't forget to flush.