There is a rich tradition in the West of describing the progress of the soul along the path to God in terms of stages of development, with certain characteristics proper to each stage. St. Teresa of Avila famously spoke of seven "mansions" corresponding to different levels of spiritual attainment; others divide the spiritual life into three phases: purgative, illuminative and contemplative. Medieval mystics such as Robert Grosseteste, Julian of Norwich and the author of the Cloud of Unknowing, following the tradition of Pseudo-Dionysius, take an apophatic approach to their theology, viewing the ascent toward God as a stripping away of assumptions and images about God in an attempt to contemplate the divine essence itself.
All these approaches have merit, and we should probably not insist on adhering to one too closely; they are all ultimately subjective expressions of what particular souls have experienced. Even if these experiences have been quite common in Church history, God ultimately works with each soul in a unique manner. No two souls take the same journey, even though all souls who seek God seek the same end. These descriptions are meant to be pedagogical, teaching the devotee what to expect on the way to holiness. They are not doctrines that we must insist on to the exclusion of other conceptual frameworks.
That being said, that does not mean there is not some commonality in mystical experiences. Most of the progressions described in the mystical writings of the Church, East and West, center on a particular shift in experience from the intellect to the affections—from the head to the heart. For example, in the West, meditation is commonly recommended as a sort of prayer for beginners in the spiritual life. Meditation consists in taking a certain episode of sacred history or truth of the faith and holding it before the mind's eye, drawing out different levels of meaning, making pious resolutions, and dwelling on the implications of the truths of what we are meditating on. It is primarily an intellectual activity utilizing the imagination.
Similarly, the Eastern tradition places great emphasis on the Jesus Prayer, intentionally choosing a particular expression and repeating it while meditating on a specific element of the Faith. This is again an active, intellectual work that depends on human activity.
But in both traditions, there is a transition that occurs. In meditation, one eventually moves to contemplation, while in the Eastern tradition, the repetition of the Jesus Prayer as an active prayer is supposed to yield to a more passive spirit of contemplative prayer. In both cases, the movement is from an active to a passive sort of prayer, from a field in which human activity predominates to one in which the soul is more responsive to the graces God wishes to bestow upon it. This transition cannot be forced; it does not yield to human effort. It may come on a soul suddenly, or gradually over many years, or perhaps never at all. No matter what schema we use to describe the transition, we get to a place where God is the dominant agent and the soul must be docile before Him.
This transition can be painful and disorienting. In some mystical traditions (most notably the Carmelite tradition exemplified by St. John of the Cross), the movement into these higher degrees of spirituality is accompanied by a painful episode that is known as a "dark night." The dark night is typically described as a period of deprivation, where the sensations, pious aspirations, consolations, and happy feelings that accompanied the individual in the lower stages of the spiritual life are withdrawn. This process of the dark night is part of the larger transition from the "head to the heart" that God affects in the spiritual life of docile souls.
It is commonly known that the dark night exists for the purpose of drawing souls closer to God, but why, specifically, is this the case? Why must a soul experience this deprivation of consolation in order to progress?
In this life, faith infused with charity is the only way a person can truly gain access to God in prayer. But when we begin, our faith and charity are weak and need to be propped up with other things: mental images, pious thoughts, spiritually pleasant feelings, imagination, etc. These are all objectively good; no matter how far we advance in the spiritual life, these will always have a certain place. But these things can never attain to God without a corresponding increase in faith, which is the key to prayer and union with the divine. That strengthening of faith we require to truly commune with God can only come about in a state of detachment, just as a person on crutches does not return to full use of their leg until the crutches are discarded and the muscles can be worked without the aid of the crutch. Similarly, growth in faith necessary to put a person into closer union with God requires that pious feelings, divine consolations, and the ability to approach Him through reason be set aside. This setting aside of all the active, human-based elements of the spiritual life is why persons in the dark night feel so incredibly helpless. Yet the dark night is extremely enriching, because by it faith is strengthened and prayer is transformed into a mutual exchange of love.
None of this is new to anyone who has even read a little bit of Catholic mystical theology, but it is something that is sadly misunderstood. There are plenty of counterfeit dark nights out there: experiences that people believe to be a dark night but are actually something other. For example, the dark night must be distinguished from a "period of dryness." All believers experience periods of dryness occasionally, during which prayer is difficult and spiritual consolations seem to be removed. This is what St. Ignatius refers to as the period of "desolation." These periods are usually briefer and are universal to all believers. These periods of desolation can be used by God, or they can be inflicted by evil one—God will typically use a period of desolation to turn someone towards him, while the devil's desolation is characterized by confusion and wavering in resolutions. A spiritually mature believer needs to be able to discern these cyclic periods of dryness from the greater "dark night" that the saints speak about. In other words, you are not "going through a dark night" just because you are spiritually dry or having a hard time.
Furthermore—and probably more common—we cannot mistake true dark nights with periods of confusion or disorientation that arise due to our own sinful activities. For example, about fifteen years ago, I experienced a profound period of dryness and dissatisfaction that lasted for about two years. Prayer was very difficult. I seemed to be making no progress in my spiritual life and had a very challenging time focusing on God. In my own limited understanding of things at the time, I imagined I was experiencing the dark night of the saints. What I did not consider was that I never prayed the Rosary, seldom went to Adoration, read the Bible only infrequently, attended Mass only on Sundays, and nurtured several bad habits and personal sins that I was unwilling to make the effort to overcome. In this case, was my dryness and difficulties really due to some dark night? Were they not rather due to my lukewarmness? Thank God I was roused from that slumber!
A true dark night comes not to souls who are tepid, but to those who are fervent and burning with charity. This is why it is so distressing for them; precisely because they are typically so inflamed with zeal for our Lord that the deprivation of His consolations is devastating to them.
It has become, in a certain sense, fashionable to speak of dark nights. People discuss their spiritual lives far too openly, and everyone who experiences some momentary setback in prayer or some cyclic lack of initiative wistfully speculates to their friends that they are suffering a dark night. Dark nights are not fashionable. They are not something casually discussed, and they are not something that come to those whose pursuit of God is not relentless; even among those who do pursue Him relentlessly may never pass through it. They are extremely distressing to the souls who undergo them, and even souls of exemplary holiness and clarity of mind may not understand what is happening to them.
If we feel ourselves in a period of dryness or desolation, rather than speculating about if we have been sufficiently holy to merit undergoing the trial of the dark night, let us turn to the much more practical advice of St. Ignatius Loyola: Consider that the dryness you experience is due to your own apathetic practice of the Faith. If you have noticed the dryness, however, God may make use of it to prod you on to a more fervent practice of the Faith. That fact that you recognize that you are dry is itself a grace. Ask God to bring you where He wants you to be and assent to whatever means He chooses to do this.
If you are already fervently practicing your faith, moving from good to better in the service of God, as St. Ignatius says, the dry spell may come from the evil one, who tries to place obstacles in the way of perfection. St. Ignatius calls this state "desolation." What can be done when this happens?
If you are in a state of desolation, do not make any changes to your spiritual routine. It is best to stay firm in our disciplines and resolves, focusing instead upon overcoming the desolation through prayer and meditation. Patience and fidelity to God are necessary here. Maintain faithfulness to the resolutions you made in the light. Changing your plan in the dark is never helpful because the desolation clouds your judgment. It will pass.
Besides our own slothfulness and tepidity, St. Ignatius says we sometimes go through periods of desolation because God wants to test us and try our faith, or because God wants to reveal to us our true state without the aid of His grace.
These periods of desolation are natural to all believers and are distinct from the dark night that is spoken of by the mystics and vouchsafed only to souls who have made exceptional progress in holiness. It is good to understand this and fortify oneself during a period of fruitful prayer and consolation by thinking how one will handle the desolation which will inevitably come.
If we made ourselves more familiar with these basic principles of spiritual life, we would do very well indeed. Through the successful navigation of these cycles of consolation-desolation, we in fact slowly come to master our spiritual life by God's grace and understand the movement of the Spirit. Thus, growing stronger, we eventually do come to the stage where our spiritual focus must shift from the head to the heart and we may in fact undergo the dark night. But if we have not mastered handling our periodic desolations, what will we do when God's consolations are utterly removed during that time of darkness?
Mysticism may be mysterious, but there is an inner logic to it, and without proper discipline and ascesis, we can't even get past our own periodic desolations, let alone the true dark night.