For this first post, I wanted to give a summary of how one might arrive at a language like Rust, starting with its peer system languages and then narrowing in on how Rust has achieved success in this space and what it does differently than these other mainstream languages.
While programming is frequently introduced as operating on data, most programmers soon discover that much of programming is about managing the resources available to the programmer. Informally, a resource could mean anything that there is only a finite amount of, such as memory, disk space, the display, or even time. While sometimes we pretend that computers have infinite resources, real machines have limits. It takes abstraction, from the machine, operating system, and programming language, to hide these limits. One such abstraction is Garbage Collection, largely ending the once traditional explicit management of memory. Another abstraction is files and folders over disk space. Protocols like TCP/IP and HTTP take one cable talking to the outside world and allow it to pretend there are large numbers of connections to an unimaginable number of potential destinations. The abstraction of a ‘window’ provides an organizing principle for what can be displayed on a screen.
However, these abstractions can only go so far. Files and connections still need to be opened and closed explicitly; garbage collection has not in practice improved that situation, and most garbage collected languages still require facilities for doing so that greatly resemble those that were once used for explicit memory management. No abstraction exists that lets a user interface programmer entirely ignore that there are only so many elements that can be on a screen at one time. Some abstractions can have adverse effects on other resources, most notably garbage collection uses unpredictable amounts of time. For some applications, these adverse effects can be unacceptable. A systems programming language, roughly speaking, is one that allows such resources to be managed explicitly when not doing so would have adverse effects. Lack of garbage collection is usually the defining characteristic of such languages, but most such languages are able to apply the same concepts for managing memory to other kinds of resources, a trait garbage collected languages generally do not share.
Early programming languages such as C required everything to be done explicitly, resource
management included. Generally, all resources would have a pair of subroutines: one to create or otherwise acquire the resource, and another to release or dispose of it. For memory, these subroutines are called malloc and free; for files, for instance, they might be called fopen and fclose. A common source of bugs in such languages is use-after-free, which as the name suggests is when some operation is performed on a resource handle after that handle has been released. The exact effects of such a bug can be undefined. Such bugs were one of the leading motivations for the use of garbage collection. As mentioned, this was very successful in eliminating such bugs in memory usage, but mostly unsuccessful for other kinds of resources. Garbage collection can ensure that resources like files will eventually be closed, but as
it turns out, eventually is not good enough for most resources, and even for memory there are applications where it is not good enough. Most garbage collected languages still have some sort of convention for designating the disposal subroutine of resources that need to be closed, like C#’s IDisposable interface, possibly with some syntactic sugar for creating a
resource and automatically calling this disposal subroutine within some block.
While C++ started out as an object-oriented superset of C, replacing malloc and free with new and delete, over the decades it has begun to veer in a different direction. One of the chief contributions is a pattern called Resource Acquisition Is Initialization or RAII for short. The essence of this pattern is that resource aquisition is associated with the definition and initialization of a variable of a certain type, and resource release is associated with automatic destruction of a variable when it goes out of scope. Unlike
with the syntactic sugar in other languages, these invocations of the constructor and
destructor respectively are not tied to an explicit block but to the implicit lexical scope of the variable itself.
Thus, a resource becomes associated with the lexical scope of a particular variable.
As it turns out, this approach has proven extremely flexible for a wide variety of resources.
If constructing and destroying resources was all we ever wanted to do with them, RAII would probably be enough. At some point, we also will want to either read from a resource or transform it. Often we will build separate subroutines to fulfill these tasks, and we will pass these subroutines the resource to use. In doing so, are we not giving the resource to a new variable, the function parameter, with a shorter lexical scope? RAII would then suggest that the resource be disposed when the subroutine ends! For numbers, this is fine, because every time we pass a number in to a subroutine we are creating a copy of it, which can safely be disposed independently of other copies. The abstraction of memory lets us pretend that we have space to store infinitely many copies of numbers, and numbers are small enough to be easily copied. But we lose the latter property even for relatively simple pieces of data like arrays. As we said, resources are defined by there being only finitely many of them available in practice, and for some resources like a user display, there is only one and copying does not make any sense. For others, like a file, the identity of each copy matters. Thus, what we find is that, more properly speaking, a resource is not necessarily copyable, or it may have an expensive copying mechanism that morally should only be done explicitly.
If we cannot copy a resource, then how are we supposed to pass it into a subroutine?
We do have abstractions like pointers and handles that allow us to create copies
of access to a resource without needing to copy the resource itself. This is sometimes called aliasing. In practice,
one such access must maintain responsibility for eventually disposing of the resource.
A subroutine taking on such a responsibility is said to have ownership of the resource. In contrast, subroutines accessing a resource without disposing of it are said to be borrowing
it. In a language like C, there is no formal distinction between these uses enforced by the language, but these concepts very much exist when designing working programs, though perhaps the terminology is different. In modern C++, what we are calling ownership of a pointer can be enforced by the
unique_ptr class, and borrowing is marked by the various reference types. However, there are usage patterns that are still unsafe; having a unique_ptr and references to the same resource at the same time is dangerous, because the unique_ptr could be passed
to a subroutine and destroyed early, thus
disposing its resource while references to that resource are still active. Unfortunately,
C++ does not have the tools for enforcing that all such usage patterns are safe, and due to
needing to preserve legacy behavior probably cannot ever have such tools built in to the language.
This is where a new language like Rust comes in to the picture. Rust was built from the ground up being able to enforce such usage patterns. The ownership and borrowing terminology I have been using actually comes from Rust. Aside from not all types being copyable, Rust enforces that while a resource is being borrowed, the original variable owning the resource is not available. It also enforces that borrowed references do not survive past the scope of the owning variable, through the use of what Rust calls its lifetime system.
Finally, it is worth nothing that if a resource cannot be copied, then the only way to transform it is to change the original resource, an action usually called mutation. In doing so, you unavoidably lose the original content of the resource. This is a significant enough consequence that most programming languages provide the option of marking that a variable or reference should not be mutated, C++ and Rust included, and newer languages, including Rust, actually make this the default. Rust, however, goes a step further, because mutating a resource while there are still references expecting to merely read from that resource is dangerous. Even worse is having multiple references expecting to mutate a resource at the same time. In fact, this is the same reason why locks, such as mutexes, are needed in multithreaded programming. Rust statically enforces that there is either a single mutable reference to a resource or multiple read-only references. A number of lock-like types are also available to push this burden from static checking at compile time to dynamic checks at runtime.