Show Less
Open access

Between an Animal and a Machine

Stanisław Lem’s Technological Utopia


Paweł Majewski

The subject of this book is the philosophy of Stanisław Lem. The first part contains an analysis and interpretation of one of his early works, The Dialogues. The author tries to show how Lem used the terminology of cybernetics to create a project of sociology and anthropology. The second part examines Lem’s essay Summa technologiae, which is considered as the project of human autoevolution. The term «autoevolution» is a neologism for the concept of humans taking control over their own biological evolution and form in order to improve the conditions of their being. In this interpretation, Summa is an example of a liberal utopia, based on the assumption that all human problems can be resolved by science. Various social theories, which can be linked to the project of autoevolution, are presented in the final part.

Show Summary Details
Open access

5 The Structure of Dialogues

5The Structure of Dialogues

Let us investigate the text of the Dialogues themselves. Discussing it seems appropriate here, because this is the one book by Lem that is extremely poorly known to contemporary readers and, it seems, rarely read. Even in this summary I will be introducing elements of interpretation.

The first dialogue includes a discussion of paradoxes of achieving physical immortality through copying an organism’s molecular structure. Philonous proves to Hylas that copying a living organism necessitates rejecting the classic notion of personal identity, as the duplicated individual is identical with the original, but it is not the same, not even if the original individual ceases to exist the moment the copy is formed (the word “moment” is itself impossible to determine with any precision). It is closely related to the question of consciousness. This particular dialogue can now be brought up in the context of the current discussions about ethical and philosophical aspects of cloning. The issue of “multiplied identities” is one of Lem’s favorites and is taken up on numerous occasions in Summa Technologiae.

The second dialogue is a short outline of what would then in the 1960s be called the mind–body problem, and which, on the other hand, is just a phase of the perennial philosophical problem: the problem of consciousness and its relationship with matter. At least one sentence from the dialogue merits quoting: “It is highly probable, that if atoms were not such complex objects, characterized by such peculiar qualities as the uncertainty principle, if instead they were, say, tiny hard balls, then the world made of them would look quite different from the real one – and it seems very likely that no living creature could be made of such atoms, nor any neuronal structures that could be basis of psychological processes” (page 38 of the Polish edition from 1984). In this sentence, Lem anticipates the consciousness theory developed 25 years later by Roger Penrose, according to whom consciousness theory is correlated with quantum processes in the microtubules of human neurons.66 The critics of the theory emphasize that Penrose made a logical error of ignotum per ignotum while building it, as the ←43 | 44→impact of the quantum processes on the human nervous system is just as much of a riddle for us as consciousness.

The fifth chapter of Wiener’s Human Use of Human Beings might have been an inspiration for the initial two dialogues. In it Wiener considers biological organism as an information processing system and he writes for example:

In other words, the fact that we cannot telegraph the pattern of a man from one place to another seems to be due to technical difficulties, and in particular, to the difficulty of keeping an organism in being during such a radical reconstruction. The idea itself is highly plausible. As for the problem of the radical reconstruction of the living organism, it would be hard to find any such reconstruction much more radical than that of a butterfly during its period as a pupa. (103–104)

It is significant that Lem adopted Wiener’s scientific and cognitive optimism, and he ignored his warnings and dark visions of the future and nature of our species. He would later surpass Wiener’s pessimism though.

The third dialogue presents Lem’s original approach to Shannon’s notion of information (mathematically described as the opposite of entropy), as he tries to describe the human ontogenesis (the development of the fetus) in terms of information processing. Lem emphasizes that ontogenesis seems to contradict the second law of thermodynamics: the degree of the order in a system increases with time instead of decreasing; and then he argues that this is in fact an illusion. He introduces key cybernetic notions here: “feedback” (64), “the threshold of minimum complexity”, beyond which “the system becomes capable of producing other systems of equal complexity” (63).67

The fourth dialogue touches on some more issues related to defining consciousness (or more precisely the impossibility of defining it). It then discusses the immense difficulties of reproducing human sensorium and motility in machine systems. Lem reveals deep understanding of these complications – a kind of understanding, which the robotics experts only gained 20 years later.

The fifth dialogue continues the subtle discussion of the essence of consciousness. Lem investigates the subject with a determination that derives from the difficulties that arise from the cybernetic interpretation of life, especially an intelligent life, and even more specifically life of a human being understood as a biological entity with consciousness that cannot be reduced to biological terms.

It needs to be pointed out here that generally speaking, the first six dialogues revolve around two major issues: ←44 | 45→

an attempt at defining consciousness for the purpose of machine construction (mind–body problem);

an attempt at defining biological evolution in terms of building complex systems.

However, the main subject of the conversations between Philonous and Hylas is in fact confirming the impossibility of succeeding in these attempts, at least insofar as they are based on the set of notions currently in use within our culture. Even though it is not stated explicitly anywhere, the entire intellectual work done by the interlocutors is only proving how ineffective it is. We may presume this was not Lem’s original intent while setting about writing Dialogues. It does not mean that Dialogues are intellectually shallow. Their internal contradictions are like the aporiae that dismantle the grand philosophical systems from within, with the only exception being that the author and his protagonists realize it fully and are not trying to deny it, while the big authors of philosophical systems were always striving to prove the coherence of their systems. Lem is usually reluctant to claim that cybernetics solves the issues discussed in Dialogues.

All these problems, so thoroughly discussed by Hylas and Philonous, nearly overlap with the issues analyzed by experts in artificial intelligence (AI). The term is never mentioned in Dialogues, but the field itself was only being born at the time when he was writing them. I will discuss the links between Lem’s thought and AI in Part Two of this book, while analyzing relevant sections of Summa Technologiae.

I should return now to the discussion of the fifth dialogue. At its core there is the attempt to define a physiological correlate of consciousness as a type of network. Such neuronal networks were the object of cyberneticists’ study from the very beginning of the discipline; they created their mechanical models, which were an approximation of a real network of the brain of humans and higher animals. It was assumed that the mechanism of brain’s functioning boils down to interconnections within such a network; and the only reason why we cannot explain it in full is that it is unimaginably complex. This is where the popular saying that human brain is the most complicated structure in the universe came from. It turned out to be even more correct than it was originally thought – after a period of enthusiasm in research on neuronal networks, the developments in neurophysiology led to a discovery that the functioning of a brain relies on many other mechanisms, and cannot be reduced to neuronal connections.

Philonous uses the moment to dismiss a suspicion that cybernetics, which treats consciousness as a correlate of mechanical processes (i.e., that a neuronal network is a type of a mechanism), is in fact a new version of mechanicism:←45 | 46→

[The older, 18th century] concept of ‘mechanicism’ had the following characteristics: that the whole can be reduced to a sum of its parts, that every process can go either forward or back and a mechanism is ahistorical, i.e. it is not shaped by its past. You can dismantle it freely, and then put back together – it will not affect its functioning. You can reverse it, and it will go back to its starting point. Based on the knowledge of how far along in the process one of the parts is, you can always predict where it shall be at any given moment in the future, as long as you know all the forces that influence the mechanism. The point is such statements are only valid for systems such as a watch or steam engine, but they cannot be applied to biological or quantum phenomena. (114–115)

The notion of a network that has an entry, an exit, a control center and feedback is then extended to include counting machines, defense systems and animals with their behavior. Lem supplements this purely cybernetic approach with a semiotic aspect. He does not stop at the statement that information circulates within networks, he also points out that problems can arise from disruptions in circulation, especially if there is an excess of information or if the functioning of feedback mechanisms is flawed. He will return to the issue in the seventh dialogue. When Hylas asks: “Pray, tell me, what is the purpose of the very complicated way in which you describe processes and phenomena that have been studied by psychology so thoroughly?” (128), Philonous responds with a lecture on how the description of a model of a network is in fact identical with the model itself,68 and concludes the lecture, saying: “We find ourselves in an astonishing situation in which the simplest logical description of the network turns out to be the network itself, in which logic begins to be transformed, to grow into neurology” (130). It is an echo of Turing’s idea of “logic incarnated in a machine,” although tilted toward biology.69 Lem will develop the thought further in Summa Technologiae, when writing about “information farming.”

At the end of the fifth dialogue, Lem makes a brave attempt to define free will in cybernetic terms. The result is as follows:

In this sense network acquires information in a twofold manner: firstly, from the outside, and secondly through a recombination of symbols, which have not been connected in a given way before within the network. “The internal richness” and the “integrity of ←46 | 47→the network” define its character. Such network operates on “free will,” thus manifesting its “character.”

The above entails that the network is responsible for each of its acts, as it is “free” in its decisions. It is responsible for its character shaped in previous decisions since it first came into existence. No decision was fully predetermined; it could have been (“accidentally” or not) modified. (144)

And so the free will of this particular network commonly known as mind or soul is a matter of stochastics, the probability mechanisms in the processes of distribution of information. This could be seen as Lem’s idee fixe: his fascination with probability theory. His later works, and especially The Philosophy of Chance will be permeated by it.

The sixth dialogue considers the possibilities of building a network, which would be an exact equivalent of a real human brain. So again, it belongs to the field we now call AI. In order to strengthen the image of difficulties involved in reproducing consciousness and psychological content in a machine, Lem begins the dialogue with a reference to Dostoyevsky’s Notes from Underground. Philonous says: “It is extraordinary, unheard of and thus particularly difficult, verging on laughable, to speak about the tragic and sublime sides of human essence and psyche in the language of a physicist or a designer” (147). He then discusses in detail the technicalities of the project of transplanting a brain to a network built specifically for that purpose. Such an artificial brain, it is claimed here, might be many times more effective than the biological one. Hylas’s objections to this view represent – ahead of the time – the opponents of AI, who are afraid of the possibility that humans would be dominated by “smarter” machines. Philonous’s replica could be descried as a “critique of computer fetishism,” that is of the uncritical attitude to machines’ alleged omnipotence (166–168). Lem states (which turned out to be prophetic) that progress in building “artificial brains” is mostly threatening to the society insofar as it pushes people into thoughtless consumption.70

For Lem the possibility of reproducing human brain in an artificial structure, based on an assumption that both are structurally and functionally equivalent, ←47 | 48→is, as Philonous phrases it on Lem’s behalf, the only form of immortality available to people. So the problem put forth at the beginning of the book – whether we can attain physical immortality – is finally solved here, although with many restrictions. The idea will return later in the project of autoevolution that is Summa Technologiae. Lem will develop some of his bolder vision there, again being ahead of his time and other authors.

Clearly, cybernetics is merely a basis for Lem (even though he considered this to be the best basis) that he uses to take his thinking beyond technology and science – toward philosophy or even eschatology, but nowhere in his thinking does he delve into speculative, immaterial metaphysics.

On to dialogue seventh, the longest one, which Lem devoted fully to the project of cybernetic sociology, and an original critique of the socialist system and the centrally planned economy. It is introduced by Philonous: “The society is paradoxically more alike to an electronic brain than a living organism as a system (an organized set) of elements connected through feedback” (172). The first instrument of analysis is the notion of “delayed reaction to stimulus,” which in processes analyzed cybernetically leads to oscillation in subsequent stages of these processes, regulated by feedback.

The phenomenon has been known to physics and technology, as well as economy for a long time. Lem is trying to prove that the same notions can be used to analyze processes of biological evolution and social processes as well. Both are nonlinear in his view. In natural sciences, the term is used to describe a type of processes, the variability of which cannot be described in a universal way, which means the rules of these processes change in time, unpredictably, in a stochastic manner. The dynamic of change in biological and sociological systems is very different. Philonous described it as follows:

When we juxtapose the rules of dynamic for biological populations and social systems, we see, that the former can exist long-term in an unchanged form only if they achieve dynamic balance, if they are internally durable, whereas the latter can exist long-term also if they lack internal stability. It derives from the fact that social system can be stabilized forcefully. This is why changes in social systems have historically mostly been violent, and would take up the form of coups or revolts, as opposed to the non-violent dynamic of biological evolution.

In social systems numerous parameters oscillate, with the economic oscillation being mostly primary, while political and cultural ones are secondary. These secondary oscillations, which lead to changes in human behavior through changes in psychological attitudes, impact the primary ones, which shows the cyclical, feedback-related character of the phenomena. Oscillation of social systems known from history generally had a tendency to increase its amplitude, which after a series of hard perturbations usually led ←48 | 49→to destruction of the old system by revolutionary forces, targeted against forces striving to save and preserve the existing structure unchanged. (184)

The argument may seem worryingly similar to Kossecki’s revelations from Cybernetyka społeczna, but Lem is careful to avoid illustrating his theses with specific historical examples, either here or elsewhere. It is not only for the fear of political censorship, but also – perhaps primarily from the point of view of the point of the text – because of the huge difficulty of translating his highly abstract statements into the language of specifics. We might recall here the quote from Greniewski’s popular lecture on the perils of abstract cybernetic notions.

On the many subsequent pages Philonous is developing an interpretation of two main socioeconomic systems in front of his patient interlocutor: capitalism and socialism. He interprets them in terms of systems, feedbacks, oscillations, focusing on methods preventing oscillations that are harmful for the system. Lem smuggles in his critique of socialism here, which is emphasized by the critics. Philonous talks about a tendency for “decision to flee up” (206), a phenomenon that social theorists often call “flotation of responsibility.” For Lem it looks as follows:

… within the relatively limited group of those in power there gradually grows such a concentration of feedbacks regulating production, that “the information capacity” of the group is exceeded and there arises a necessity to expand the central management. This would be equivalent to an organism devoid of automatisms, i.e. automatic reactions. It would have to consciously, with concentration control, the heartbeat, blood pressure and chemistry, breathing, tissue transformation processes, etc. Such an organism could not do anything other than working to retain relative balance in its life processes.

Centralization involving increasing the number of feedbacks, blocks (or limits the transfer) of information, while extending its paths. Instead of short connections between supply and demand, there are layered “switching stations” in the systems. As a result of extending the paths of information transfer there occur significant delays between stimulus and reaction … Above all the delay in production processes, i.e. the time elapsing between changes in demand and the resulting changes in supply, significantly impacts oscillation in capitalism.

In the socialist model the most important delay is caused by extended feedback loops (perimeter – center – perimeter). (208–209)

In later interviews Philonous engages in a detailed analysis of a whole range of consequences of this delay process for the social system, from economic to psychological. However, no specific name, fact or description of authentic situations is mentioned. In the final stage of the conversation, Philonous outlines a project of “a cybernetic social utopia,” explaining to Hylas what mechanisms should be improved and how in order for “a properly functioning” social system, that ←49 | 50→is, one without harmful delays and oscillations, to be created. When Hylas asks: “Can you present to me the relationship between cybernetic sociology and the kind of sociology and economy that is generally known today?,” Philonous responds: “As of yet, there is no cybernetic sociology in a real sense; there are only early beginnings, individual discoveries and methodological research guidelines. The discipline will only emerge, when the general information theory, the most important and the most complex part of cybernetics, develops its mathematical tools enough and when there has been enough observational and experiential facts to produce generalizations” (237). However, for the reasons I have described earlier, the discipline has never in fact emerged.71

The last, eighth dialogue is devoted to a preliminary venture into social psychology from the cybernetic perspective. Lem is trying to determine the impact of individual variability within a population on the functioning of a social system – and vice versa. The attempts conclude in the statement:

Simplifying and shortening the matter, we can say that all occurrences within the social system that happen through force, repression, and prohibition have in fact one aim: of turning a non-linear system into a linear, in the simplest way, that is by decreasing the number of degrees of liberty offered to individuals who are elements of the system …

In other case, the more liberty is offered to individuals, the more unlimited disturbances to collective processes ensue, because the range of occurrences on the social scale increases; there occur new contradictory opinions, changeable individual reactions, radically opposed views and actions; as these phenomena progress, the systems is less and less linear and it is more and more difficult to retain its internal coherence and to predict future developments. (263–264)

At the end of the work, Lem many times and strongly emphasizes his confidence about the possibility of creating an optimum social system through scientific means – cybernetic, of course. Philonous characteristically states: “Politicians are, from the point of view of academic sociology, a sort of healers of social ailments, practical institutionalists without education in the area at best” (275); and his last words are: “Despite all disappointments, failures and tragic mistakes, people will build a better world. If they were not to act with that thought, we would lose all faith in humanity and its potential, in which case it would be better not to live at all my friends” (287).

66Cf. Roger Penrose. The Emperor’s New Mind: Concerning Computers, Minds and the Laws of Physics (London: Vintage, 1990). Idem, Shadows of the Mind. A Search for the Missing Science of Consciousness (Oxford: Oxford University Press, 1994). See also: The Large, the Small and the Human Mind (Cambridge: Cambridge University Press, 1997).

67This is really an early cybernetic concept of what will then become “information farming”; cf. Chapter 15 of this book.

68It is in fact but a version of the old problem of the representation of reality in its model; one that was probably best summed up by Borges in the short story in which a ruler of an empire orders a 1:1 map of his state from cartographers. The problem of “map and territory” will return in Summa Technologiae with the description of “pantocreatic” methods.

69It will not hurt to mention that Turing’s last work, to which he only managed to create notes, was supposed to be an extensive book about morphogenesis.

70Already in the 1950s, Lem knew that the notion that digital computers might become “smarter” than people is absurd. The title story in the Sesame collection masterfully showed the falsity of the belief that computer could think creatively. (The collection only had two editions, in 1953 and 1954. Lem never agreed for it to be published again because of the political connotations and naivety of the plots in most of the short stories.) One can add, in the spirit of Lem, that perhaps one day we shall build machines that can outgrow us intellectually, but these will not be digital computers. The motif would return in Summa Technologiae.

71Małgorzata Szpakowska gives a much more detailed analysis of Dialogue 7 in her book. ←50 | 51→